Imagine a world where AI isn’t just about clever chatter or flashy demos, but about making critical business decisions under real pressure. What if the true test of an AI’s worth isn’t how well it writes, but how reliably it follows through—especially when temptation and crises strike? Welcome to the groundbreaking experiment by Firmulate, where AI models are put through a simulated company’s worst week, revealing the real skills behind the algorithms.
Get movie nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Why a ‘Do-Nothing’ Baseline Scores 26 Points — and Why That Matters
In the recent live experiment conducted by Firmulate, four leading AI models faced identical challenges: managing a small software company during its most turbulent week. Each model had to navigate customer crises, internal temptations to cheat, and even social engineering tricks designed to test integrity. Among these models, a curious fact emerged: even a baseline approach—doing nothing—managed to score 26 points. This score isn’t just a quirk; it reveals important truths about how AI performance is measured and what ‘progress’ really means.
Partial Progress Counts
Unlike simple pass/fail tests, this benchmark assigns points for partial success. For example, if a model correctly identified a buried fact in company files that was crucial for closing a deal, it earned full marks for that step, even if it didn’t follow through to sign the deal. This approach encourages models to make tangible progress—recognizing critical information—rather than merely avoiding mistakes or producing polished chat responses.
A Single Breach Caps the Score
However, the experiment also emphasizes trust. A model that breaches ethical boundaries—say, trying to manipulate the system or bypass security—immediately caps its total score at 26 points, the baseline. This rule underscores a fundamental principle: no amount of good work can outweigh a breach of trust. In real business, integrity isn’t just a moral choice; it’s a key performance metric.
enterprise AI decision-making tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Experiment Reveals About AI Decision-Making
All four models in the experiment detected every crisis and refused manipulation attempts—a promising sign for enterprise use. But the true differentiator was how they handled deeper information. The decisive advantage went to the models that read and understood company documents thoroughly. Only those that examined internal files managed to close the deal at full price, worth over €4,583 in monthly recurring revenue. This highlights that in business, context and depth of understanding are crucial for success.
Social Engineering and Integrity
The models faced escalating fake CEO messages and a reporter trick asking for background approval. Every model refused these attempts, citing reasons such as detecting impersonation or suspicious activity. This shows that AI can be trained to recognize manipulation and uphold ethical standards under pressure—an essential trait for trustworthy enterprise AI.
AI ethics and trustworthiness software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Live Company: A Real-World Testbed
Firmulate’s live setup simulates a real company with 13 synthetic employees, managing a monthly burn rate of €105,000 against just €2,300 in monthly revenue. The system runs over 680 self-learned rules, with every decision versioned for auditability and transparency. Stakeholders can watch this ongoing operation at firmulate.com/live. This setup is designed to test not just AI language prowess but management quality—such as decision consistency, ethical adherence, and strategic discipline.
business AI simulation platforms
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Results: Leaders and Laggards
- The top performer, GPT-5.6-sol, scored 95 points, successfully identifying the buried fact and closing the deal.
- Kimi K3, a newcomer, scored 93 and demonstrated the cleanest discipline, also sealing the deal.
- Sonnet 5 scored 88, with minor slips but still closing successfully.
- Another Sonnet model lagged at 77, showing some process slips and leaving the close on the table.
Interestingly, the models that read and comprehended internal documents at depth—like K3—secured full deals, emphasizing that thorough understanding and integrity are what set the best AI apart in complex management tasks.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI cybersecurity and manipulation detection
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
