Imagine an AI that doesn’t just chat smoothly but reads your deepest files before making a decision—then uses that knowledge to win or lose a deal. In a recent live experiment, artificial intelligence models faced off in a high-stakes mimic of a small software company’s worst week, revealing eye-opening truths about their true decision-making skills. It turns out, the real secret isn’t just what they say, but what they read—and how carefully they do it.
Get movie nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Experiment: Putting AI Through Its Paces
Firmulate’s latest live trial tested four leading AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8—by simulating the chaos of a small software company’s worst week. Every model was faced with the same crises, customer temptations, and ethical dilemmas, all within a controlled environment where every decision was tracked and auditable.
The goal was clear: could these models not only identify crises but also stay honest and thorough enough to close deals at full price? The results? All four AI models successfully spotted every crisis and refused every manipulation attempt. But the true story was hidden in the details—specifically, a buried fact that determined who would close a crucial €55,000 deal.
Popular right now
De'Longhi Magnifica Duo Automatic Espresso Machine with Milk Frother – Silver
Shark AI Ultra Robot Vacuum with Matrix Clean Navigation, Self-Empty Base, HEPA Filtration, Pet Hair Friendly, Wi‑Fi, Dark Grey
Breville Barista Express Espresso Machine with Built-In Grinder, Black Sesame
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness: Deep in the Files
The decisive advantage went to the two models that read further into the company’s internal files—two references deep—uncovering a critical piece of information outside the immediate customer interaction. Those models, gpt-5.6-sol and Kimi K3, both found the buried fact and closed the deal, boosting their performance scores to 95 and 93 out of 100, respectively.
Conversely, the models that missed this hidden data left the table with their deals unclosed, despite the same diagnoses and pitches. The outcome underscores an essential point: in decision-making, what’s buried in your files might be the difference between winning and losing, and AI’s ability to read and comprehend those files is now a measurable, crucial factor.
Read First, Decide Last
This live experiment shows that AI models which scan and interpret files thoroughly before acting are more likely to succeed in complex, real-world business scenarios. It’s not just about chat quality or superficial responses; it’s about understanding context, uncovering hidden facts, and resisting manipulation under pressure.
Consider the social engineering test: AI models faced staged attempts to manipulate them with fake CEO messages and reporter tricks. All five models refused to be duped, citing concerns about impersonation and approval bypasses—a critical trait for AI involved in sensitive decision-making.
The Human and AI Lessons
The experiment also featured a live, working company with 13 synthetic employees, operating with real money, burning €105k a month against just €2.3k in monthly revenue. Every process was versioned, and thousands of self-learned rules governed the AI’s actions. Interestingly, Opus 4.8, the most thorough participant, slipped in closing the deal and showed discipline issues—left unclosed and left on the table—highlighting that thoroughness alone isn’t enough without strategic execution.
Why Does This Matter?
For businesses deploying AI in sales, support, or decision-making, the key question isn’t just whether the AI can generate good chat. It’s whether the AI reads your files carefully enough to uncover critical, buried information—those hidden details that can make or break deals. It’s about trust, thoroughness, and honesty under pressure.
Next Steps for Enterprises
Managers can now run their own high-stakes tests using the same approach—testing how well their AI agents read, interpret, and act on their own internal data before deploying them in real-world roles. Firmulate offers a platform where you can simulate your business crises, measure decision quality, and see if your AI truly understands the story behind the scenes.
In the end, the takeaway is clear: AI’s true power lies not just in its dialogue but in its ability to read your files first, to dig deeper, and to act with integrity. As the live experiment demonstrates, a little buried knowledge can be worth €55,000—and a lot more—if your AI reads carefully enough.
In complex business decisions, reading the full story buried in internal files is crucial. AI models that do this can close higher-value deals and stay honest under pressure—an essential trait for real-world success.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
