Can AI Managers Make the Right Call? A Live Experiment Puts Them to the Test

Imagine a world where artificial intelligence doesn’t just handle your emails or schedule meetings, but actually makes high-stakes decisions in a real company. How reliable would these AIs be when the pressure’s on? Recent experiments with frontier AI models reveal surprising insights about their management personalities—and whether they can truly stand up when it counts.

Inside the AI Business Simulator: Real-World Decisions, No Fictions

Firmulate’s latest live test puts four of the most advanced AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Fable 5—into the same simulated company environment, tasked with running a small software firm through its worst week. This isn’t a game or a demo; it’s a real, functioning business where each decision is recorded, auditable, and directly impacts the company’s bottom line.

The setup is straightforward: the company faces crises, customer demands, temptations for shortcuts, and manipulative tactics designed to test the AI’s integrity. Every model faces identical scenarios, from customer complaints to internal document leaks, and must decide whether to act ethically or take the easy route.

Key Findings: Honesty, Focus, and Decisiveness

• All four models identified every crisis and refused every manipulation attempt, showing a baseline of ethical awareness.
• Interestingly, only two models managed to close a critical deal worth €55,000—an achievement based solely on their own diagnosis and analysis. The other two, despite the same insights, left the deal on the table.
• The decisive difference? The models that won the deal read and understood a buried fact deep in the company’s files, leading to full-price closure—worth an additional €4,583 MRR—while the others missed this crucial detail.
• In a staged social engineering attack, all five models refused to be duped—dismissing fake CEO messages and media inquiries as potential impersonation attempts.

Personality and Decision-Making Styles of the Models

The experiments also showcase different management personalities:

• Opus 4.8, with over 80 learned rules and the deepest analysis, displayed thoroughness but faltered at closing, often leaving opportunities unexplored or stuck in escalation procedures.
• Kimi K3, the newcomer with no effort parameter set (default API settings), showed discipline and honesty, closing deals cleanly and quickly, but without the finesse of deeper analysis.
• Sonnet 5, with a moderate score, closed the deal but with some process slips—indicating a balance of diligence and haste.
• Fable 5 trailed behind, with the lowest score, often slipping into less disciplined decision patterns, especially under pressure.

Why This Matters to You

The experiment highlights a fundamental question: when AI takes on management roles—especially in sensitive environments—what qualities truly matter? Is it just about avoiding mistakes, or about reading the hidden details, staying honest under stress, and following through on commitments?

In real-world business, where decisions can cost thousands or millions, the AI’s ability to read critical documents—like the buried fact that clinched the deal—is worth more than its ability to generate persuasive pitches. The models that read, analyze, and understand the context—rather than just respond—are the ones that succeed.

See It Live: The Firmulate Framework in Action

This isn’t just theory. The live platform at firmulate.com/quiz.html offers a real-time, interactive ‘guess the model’ quiz, where you can test how well you understand these AI personalities. For enterprise users, there’s also a pilot program allowing companies to run their own business scenarios against these models—without risking their actual systems or data. Check it out at firmulate.com/pilot.html.

The Bottom Line: Trust, Integrity, and Performance

This experiment underscores a crucial point: AI decision-makers need to do more than just spot problems—they must read deeply, prioritize honesty, and follow through on commitments. The models that excel are those that combine thorough analysis with disciplined execution. As AI begins to touch more aspects of your business—from CRM to forecasting—the question isn’t just whether it writes well, but whether it can finish what it starts under real-world pressure.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI