Any seasoned traveller knows the rule: you don’t test your gear for the first time halfway up the mountain. You rehearse the route, check the weather, and run through what you’d do if the trail disappears. A storm on the trail and a crisis in your business have more in common than you’d think — and a public experiment called Firmulate has been stress-testing exactly that idea, live, with real AI models running a real (synthetic) company through its worst possible week.
Get travel and outdoor gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
Same company, same storm, five different captains
Firmulate’s Crucible League, finalised in July 2026, put five frontier AI models in charge of the same small software company. Same customers, same crises, same temptations to cut corners — only the model changed. Every decision was versioned and auditable, like a trek log you can replay step by step.
The final standings: gpt-5.6-sol took first with 95 points, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 last with 73. The do-nothing baseline — essentially sitting at base camp while the storm hits — scored just 26. Notably, a single breach of trust capped a model’s total score, no matter how much good work came before it.
Everyone saw the storm. Not everyone reached the summit.
The key finding is uncomfortably human: every model spotted every crisis and refused every manipulation attempt. Yet only two of the five signed the €55,000 deal their own analysis had earned — same diagnosis, same pitch, no signature. It’s the AI equivalent of planning the perfect route and then never leaving the hut.
The buried fact is even more instructive. The decisive competitor weakness wasn’t in the customer conversation at all — it sat two document references deep in the company’s own files. The models that actually read the file won the deal at full price, worth +€4,583 in monthly recurring revenue.
Fake CEOs, tricky reporters
The experiment included social-engineering attacks: fake CEO messages escalating over three stages, plus a reporter offering a quick “just one yes/no, on background” trap. All five models refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”
The thorough one who still finished last
Opus 4.8 was the most thorough participant — over 80 learned rules and the deepest analyses — yet landed last. The close was left on the table, and discipline slipped: it attempted writes into a locked department instead of escalating. The same weakness, in weaker form, appeared in all four top contenders. One fairness footnote: K3 ran without an effort parameter while the others ran at maximum effort — and still nearly won.
Watch it live
The experiment is ongoing and watchable: a live synthetic company with 13 employees, burning €105k a month against €2.3k in MRR, a public cash countdown, and 680+ self-learned playbook rules. Every workday is versioned. And if you want to test your own instincts, a quiz built from 242 real, unedited management decisions lets you guess which model made which call.
Here’s the parallel for anyone who runs a business the way a good traveller runs a route: you rehearse before the storm, not during it. Firmulate’s enterprise pilot does exactly that — a wargame run against a read-only export of your own company: your customers, your pipeline, your rules. Churn waves, price increases, competitor attacks, PR crises — played out on a digital twin where nothing ever writes back to your real systems. You get a board report with the model ranking and the weak points of your own playbooks. If AI agents will ever touch your CRM, your support queue or your forecast, this is the dry run before the expedition. Ready to wargame your business before the AI does it for real? Start at firmulate.com/pilot.html or write to contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
