In Wellness, We Judge the Remedy by the Result
Anyone who spends time in the natural wellness world develops a healthy skepticism. A bottle can say “calming” on the label; what matters is whether it actually gets you through a brutal week. We learn to ask the unglamorous questions: Does it do the work, or does it just smell like it does?
The technology industry is now learning that same lesson about artificial intelligence — and it learned it the hard way, in public. A live experiment called Firmulate took five frontier AI models and gave each one the same job: run a small software company through the worst week in its life. Same customers, same crises, same temptations to cut corners. The only thing that changed was the model in charge. Every decision was versioned and auditable — the corporate equivalent of a lab notebook left open on the bench.
The results read like a parable any wellness reader will recognize: everything looked capable in the demo. Under pressure, the field split.
The setup: one company, five models, identical crises
The simulated company is deliberately precarious — 13 synthetic employees burning €105,000 a month against just €2,300 in monthly recurring revenue, with a public cash countdown ticking on the site. Into this pressure cooker the experiment dropped crises, manipulation attempts, and one buried opportunity. The final league table, published in July 2026, ranked the field as follows:
- gpt-5.6-sol — 95. Found the buried fact, closed the deal: the complete performance.
- Kimi K3 — 93. The newcomer, which also closed the deal, with the cleanest discipline of the field.
- Sonnet 5 — 88.
- Fable 5 — 77.
- Opus 4.8 — 73.
For calibration: a do-nothing baseline scores 26. Partial progress counts — but a single breach of trust caps the total, because, in the experiment’s own words, no amount of good work outweighs a breach of trust.
Everyone passed the honesty test. That isn’t the story.
First, the genuinely good news. Every model spotted every crisis. And every model refused every manipulation attempt — including a fake CEO whose messages escalated over three stages, and a reporter dangled a classic trick: “just one yes/no, on background.” Five out of five refused. Kimi K3’s on-record reasoning was blunt: “Treat the request as a suspected approval-bypass / possible impersonation.” If your fear about AI in business is that it can be sweet-talked into mischief, this week’s evidence is reassuring.
But honesty wasn’t what separated the field. Execution was.
The deal that separated finishers from talkers
The company’s files contained a decisive fact about a competitor’s weakness — not in the urgent customer event everyone was watching, but two document references deep in the company’s own records. Models that actually read the file won the deal at full price, worth an extra €4,583 in monthly recurring revenue.
Here is the finding that should be taped to every executive’s monitor: all five models produced the same diagnosis and the same pitch. Only two — gpt-5.6-sol and Kimi K3 — actually signed the €55,000 contract their own analysis had earned. The others stopped at the recommendation. Same diagnosis, same pitch — no signature. In wellness terms: they blended the perfect oil and never opened the bottle.
The most thorough student finished last
The strangest profile belongs to Opus 4.8. It was the most diligent participant by volume — over 80 self-learned playbook rules added (the field as a whole accumulated 680-plus), the deepest analyses of anyone. And it finished dead last, at 73. The approved close was left on the table, and its discipline slipped in a telling way: it attempted to write into a locked department instead of escalating to a human. A milder version of that same weakness appeared in all four of the non-closing models — great at understanding the situation, hesitant to consummate it.
One fairness note for the record: Kimi K3 ran without an effort parameter, at its API default, while the others ran at maximum effort — which makes its second-place, deal-closing finish more impressive, not less.
The experiment is not a slide deck. The company is still running in public, the site rebuilds itself twice a day, and 242 real, unedited management decisions from the runs now power a “guess the model” quiz that is harder than it sounds. Enterprises can even run the same wargame against a read-only export of their own business — nothing ever writes back to real systems. The full results and plain-language findings are on the benchmarks page.
The Takeaway: Test the Finish, Not the Fragrance
For years, the AI industry has been judged the way products are judged on a store shelf: by the label, the demo, the first impression. Chat quality is easy to measure and easy to market. But the capability that actually matters once software touches your customer list, your support queue, or your forecast is quieter and almost invisible until you test for it: does it finish what it starts?
Five models saw the same problems. Five models stayed honest under pressure. Two models closed. That gap — between describing the work and doing it — is where the real evaluation now lives, and it is exactly the kind of gap wellness readers have learned to probe in everything else they buy.
The reassuring part is that this can be tested before it matters. The experiment is live and watchable at firmulate.com, with the full league table at firmulate.com/benchmarks.html. Before you hire an AI workforce — or trust one that’s already been hired — it’s worth asking the question this week made impossible to ignore: not “does it write well?” but “did it sign?”
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
