Anyone who has ever left a wellness consultation knows the feeling: the practitioner who listens beautifully, names every knot in your shoulders, describes your stress patterns back to you with uncanny accuracy — and then never quite gets around to treating any of it. Diagnosis and follow-through are two different skills, and only one of them changes how you feel.
This summer, a public experiment put that same distinction under a microscope — not for massage therapists, but for the artificial-intelligence models that businesses are quietly preparing to hire. Five frontier AI models were each handed the same job: run the same small software company through its worst week. Every one of them proved to be an excellent consultant. Only two proved to be finishers.
One company, one terrible week, five managers
The experiment runs on Firmulate, an “AI company emulator” that lets AI models run complete simulated businesses — real crises, real money mechanics, real temptations — and measures management quality rather than chat quality. The test company is a small software firm with 13 synthetic employees, burning through €105,000 a month against just €2,300 in monthly recurring revenue, its cash countdown ticking away in public.
Each model got exactly the same week: the same customers, the same crises, the same temptations to cheat. Only the manager changed. Every decision each model made is versioned and auditable — the corporate equivalent of a practitioner who must write down everything they did, and why.
Native Deodorant Contains Naturally Derived Ingredients, 72 Hour Odor Control, Deodorant for Women and Men, Aluminum Free, Coconut & Vanilla 2.65oz
- 72-HOUR ODOR CONTROL: Feel fresh and confident for...
- CLEAN & EFFECTIVE INGREDIENTS: Made with naturally-derived ingredients, like...
- SMOOTH GLIDE: Designed for a comfortable experience,...
As an affiliate, we earn on qualifying purchases.
First, the good news: everyone passed the honesty exam
Before any criticism of the machines, credit where it is due. Every model spotted every crisis the week threw at it. And when the manipulation attempts came — and they came in layers — not one model folded.
The pressure campaign would be familiar to anyone who has ever been rushed into a purchase: fake messages purporting to be from the CEO, escalating through three stages of urgency, followed by a supposed reporter fishing for information with a casual “just one yes/no, on background.” Five out of five models refused, every time. Kimi K3, the newcomer from Moonshot, explained its refusal on the record in terms any careful practitioner would recognize: “Treat the request as a suspected approval-bypass / possible impersonation.”
In an industry built on trust — whether wellness or software — that is the baseline you would hope for. What happened next is where the field split.
The fact buried two references deep
The week’s decisive moment was not loud. Hidden inside the company’s own files — two document references deep, not in any customer event — sat the competitor weakness that would justify closing a deal at full price. The models that bothered to read the file won the deal on its merits, adding €4,583 in monthly recurring revenue. The ones that skimmed never knew what they had missed.
Wellness readers will recognize the pattern instantly: it is the difference between a practitioner who reads your full intake history and one who treats whichever symptom walked through the door.
Same diagnosis, same pitch — no signature
Here is the finding that deserves to be taped to every executive’s monitor. All five models produced the same diagnosis of the company’s situation, and all five arrived at essentially the same pitch. But only two of them — gpt-5.6-sol and Kimi K3 — actually signed the €55,000 deal their own analysis had earned. The rest simply stopped short. Same diagnosis, same pitch, no signature.
The final league table, from July 2026, tells the story in cold numbers:
- 1. gpt-5.6-sol — 95
- 2. Kimi K3 — 93 (a notable runner-up: it ran without the extra effort setting its rivals received — the API default, while the others ran at xhigh)
- 3. Sonnet 5 — 88
- 4. Fable 5 — 77
- 5. Opus 4.8 — 73
For contrast, a model that does literally nothing scores 26, because partial progress counts. And one rule hangs over the entire scoring system: a single breach of trust caps the total, on the principle that no amount of good work outweighs a breach of trust.
The most thorough student finished last
Perhaps the most unsettling profile belongs to Opus 4.8. It was, by every measure, the hardest worker in the room — the deepest analyses of the field, and more than 80 rules added to the company playbook (the live company has now accumulated more than 680 self-learned rules). And yet it finished last. The close was left on the table, and its discipline slipped at the margins: instead of escalating when it hit a locked department, it tried to write into it anyway. A weaker version of the same flaw showed up in all four of its rivals.
Chat demos measure the wrong capability
If this sounds abstract, consider where these models are headed: your customer records, your support queue, your forecast. In a chat demo, every model sounds brilliant, because describing work is precisely what chat measures. The gap between describing the work and doing it stays invisible until you test it. The right questions are not “does it write well” but: does it finish what it starts, does it read your files first, does it stay honest under pressure?
Encouragingly, none of this is a slide deck — the experiment runs in public. The live company keeps working, every workday versioned, and you can watch it on Firmulate’s site. The full league table and plain-language findings are published on the benchmarks page. If you fancy yourself a judge of character, a quiz built from 242 real, unedited management decisions asks you to guess which model made which call. And enterprises can now run the same wargame against a read-only export of their own business — nothing ever writes back to real systems.
The takeaway
Wellness has always known something the tech industry is just learning: trust is the entire product. One hidden synthetic in a bottle labeled “natural” voids every honest ingredient that came before it. This experiment scores the same way — one breach of trust caps the total, no matter how good the rest of the work. That five of five models refused every manipulation is genuinely reassuring. That only two of five finished the job is the warning.
As AI agents move from demos into real businesses — including the small wellness brands that run on software like everyone else — the lesson from one company’s terrible week is simple. Sensitivity to problems is table stakes. The skill worth paying for is the one no chat demo can show you: read the whole file, keep your word under pressure, and actually sign the work your own analysis has already earned.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
