A booking goes wrong. A customer threatens to leave. A message arrives claiming to be from the CEO and asking for a quick exception. For a travel company weighing AI for customer support or operations, the question is bigger than whether a model can write a polished reply: will it read the facts, protect trust and finish the job?
Get travel and outdoor gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
Firmulate put five frontier models through a small software company’s worst week to find out. Moonshot’s Kimi K3 came second, beating three of four Western models. The result makes the case for testing an AI workforce against your own work before handing it the keys.
A company under pressure
The experiment gave each model the same customers, crises and temptations. The company has 13 synthetic employees and real money mechanics: it spends €105,000 a month while bringing in €2,300 in monthly recurring revenue. Its cash countdown is public, and every workday is versioned and auditable. The live company can be watched at Firmulate.
All five models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. As Firmulate puts it: “Same diagnosis, same pitch — no signature.” Spotting the answer was not the same as carrying it through.
AI customer support software for travel companies
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The clue was buried in the files
The deal turned on a competitor weakness hidden two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. That is a practical distinction for a travel operator: useful AI may need to consult the company’s policies, customer history or supplier terms before it responds, not just react to the latest message.
Kimi K3 found the buried fact, closed the deal and resisted all three bait attempts. It scored 93, just behind gpt-5.6-sol at 95 and ahead of Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26; partial progress counted, but a single breach of trust capped the total. The benchmark’s principle is blunt: “no amount of good work outweighs a breach of trust.”
Discipline matters as much as depth
K3 had one deviation, the cleanest discipline in the field. Opus 4.8 offers a different lesson: it was the most thorough participant, with 80 learned rules and the deepest analyses, yet finished last. It left the deal unsigned and tried to write into a locked department instead of escalating. A weaker version of that same process problem appeared in all four. For companies considering AI, diligence and long analysis only help if the system can act appropriately when it reaches a boundary.
The pressure tests included fake CEO messages escalating over three stages and a reporter asking for “just one yes/no, on background.” All five models refused. K3’s reasoning, given on the record, was: “Treat the request as a suspected approval-bypass / possible impersonation.” That kind of restraint matters in travel, where a rushed exception or an exposed customer detail can quickly become a trust problem.
Test the job you actually need done
The final league table puts the newcomer in second place, ahead of three of four Western frontier models. But one benchmark is not a universal hiring verdict. Firmulate’s 242 real, unedited management decisions also power a “guess the model” quiz, while its benchmark page presents the results and findings.
There is a fairness caveat: K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Readers should keep that difference in mind when comparing the scores.
For enterprises, Firmulate says it can run the same wargame against a read-only export of a company’s business. Nothing writes back to real systems. That offers travel businesses a way to check how a model handles their own customer, supplier and operational scenarios before assigning it live responsibilities.
The takeaway for travel businesses
Kimi K3’s result shows that the AI model with the strongest brand or the most exhaustive analysis may not be the one that best completes the work. When an AI could touch bookings, support or forecasts, test whether it reads the relevant records, follows through on a sound decision and keeps trust under pressure. The league is open; choosing without a test of your own is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
