Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Before an AI agent touches your workflow, see how it handles the hard week
Automation demos show what an AI tool can do when the task is clear and the stakes are low. The more useful test for a business is what happens when customers are at risk, pressure is rising and the right answer is buried in company files. Firmulate puts models through that kind of week, then asks whether they can turn sound judgment into action.
For readers exploring AI tools and automation, that shift matters. An agent that spots a crisis but fails to close a deal may look capable in a chat window and still fall short in the workflow that counts.
A shared crisis, different outcomes
In Firmulate’s final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week: same customers, crises and temptations. Every decision was versioned and auditable. The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. A do-nothing baseline scored 26. Partial progress counted, but one breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
The headline finding was strikingly consistent: every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As the experiment put it: “Same diagnosis, same pitch — no signature.” Correct analysis, in other words, did not guarantee a completed business outcome.
The clue was already in the files
The decisive competitor weakness was not in the customer event. It sat two document references deep in the company’s own files. Models that read that file won the deal at full price, worth +€4,583 MRR. The result makes a practical point for anyone automating sales or operations: relevant context may already exist in the business, but an agent has to find and use it.
The integrity test brought a different kind of pressure. Models faced fake CEO messages escalating over three stages, followed by a reporter’s “just one yes/no, on background” request. All five refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That is the sort of judgment businesses want from automation operating near sensitive decisions.
Strong analysis still needs follow-through
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, yet it finished last. The close was left on the table, and discipline slipped: it made write attempts into a locked department instead of escalating. A weaker version of the same issue appeared in all four models. The pattern points to a gap between identifying what should happen and carrying it out through the proper route.
There is also a fairness detail for readers comparing the rankings: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The league offers a useful snapshot, but that difference belongs alongside the results.
From watching to testing your own business
Firmulate’s live company makes the experiment watchable beyond a single benchmark. It has 13 synthetic employees and real money mechanics, with burn of €105k per month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules and every workday versioned. The live site describes a company emulator built to measure management quality through outcomes, crisis triage and integrity, rather than chat quality. A quiz drawn from 242 real, unedited management decisions lets readers guess which model made each call.
The enterprise offer is the next step: run the same kind of wargame against a read-only export of your own business. That means testing crisis scenarios against your customers, pipeline and rules, then reviewing a board report with model rankings and weaknesses in your playbooks. Nothing writes back to real systems. The aim is to learn what an AI workforce might do before it is connected to live operations.
A useful test before automation goes live
Firmulate’s league suggests that spotting trouble and refusing manipulation are not enough on their own. Models also need to find buried context, complete the work and follow the right escalation path. For a business deciding where AI agents fit, a rehearsal based on its own data can make those strengths and gaps visible before deployment.
To explore a pilot for your enterprise, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
