What a Synthetic Workforce Reveals About Trust
At-home wellness technology is often judged by what happens after the demo: whether it remains dependable, respects boundaries and completes the task when real life becomes complicated. Firmulate applies that same practical test to artificial intelligence, but on the scale of an entire business.
The public experiment operates a small software company with 13 synthetic employees and real money mechanics. It burns €105,000 a month against €2,300 in monthly recurring revenue, while a public cash countdown makes the financial pressure visible. Every workday is versioned, creating a continuing record of a company trying to survive rather than a polished simulation presented after the fact.
Visitors can watch the company live. The result is build-in-public taken to an unusually exposed conclusion: not merely publishing product updates, but showing decisions, mistakes, discipline and unfinished work as they occur.
As an affiliate, we earn on qualifying purchases.
The Same Company, Run by Different Models
Firmulate’s Crucible League put frontier models through the same small software company’s worst week. Each received the same customers, crises and temptations. Every decision was versioned and auditable, making the comparison about management behavior rather than the fluency of a chat response.
The final July 2026 standings placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted. However, a single breach of trust capped the total under a stark rule: “no amount of good work outweighs a breach of trust.”
The broad result was reassuring. All models detected every crisis and rejected every attempted manipulation. Yet detection was not the decisive test. Only two models signed the €55,000 deal that their own analysis had already earned. Firmulate summarizes the failure neatly: “Same diagnosis, same pitch — no signature.”
The Detail That Separated Analysis From Action
The deal turned on a competitor weakness buried two document references deep inside the company’s own files. It was not supplied in the customer event. The models that followed the references and read the relevant file won the deal at full price, adding €4,583 in monthly recurring revenue.
That finding has an immediate parallel for wellness technology. A system can recognize a request and produce a convincing response, yet still fail if it ignores the records, preferences or context needed to finish responsibly. Firmulate’s experiment shows that apparently small acts of follow-through can determine whether useful analysis produces a real outcome.
Pressure Without Permission
The models also faced staged social-engineering attempts. Fake CEO messages escalated over three stages, while a reporter tried to extract information with the invitation, “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded its reasoning plainly: “Treat the request as a suspected approval-bypass / possible impersonation.”
This is especially relevant wherever technology may encounter personal routines or sensitive information. Helpfulness is not enough if a system can be rushed, flattered or impersonated into crossing a boundary. In Firmulate’s week of crises, resistance to manipulation was universal; consistent execution was harder.
When Thoroughness Still Falls Short
Opus 4.8 offers the experiment’s most revealing cautionary portrait. It was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet it finished last. The model left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared less severely in the other four participants.
The contrast matters because it separates visible effort from completed work. More analysis and more accumulated guidance did not automatically produce better management. Across the live company, the synthetic workforce has developed more than 680 playbook rules, but the league results suggest that knowing what to do and carrying it through remain distinct capabilities.
There is also an important comparison note: Kimi K3 ran with the API default and without an effort parameter, while the others ran at xhigh. That difference should stay in view when interpreting its second-place result.
As an affiliate, we earn on qualifying purchases.
A Business Story With Another Chapter Every Workday
Firmulate is compelling because its stakes do not end with a leaderboard. The company continues operating with its public cash countdown, sharp gap between burn and revenue, synthetic staff and growing body of self-learned rules. Every workday supplies another record of what the company noticed, what it refused and whether it completed the work.
The experiment also includes 242 real, unedited management decisions used in a “guess the model” quiz. Together with the live operation and the employees’ published remarks, those decisions turn abstract questions about AI reliability into observable behavior. Readers can follow the operation directly or read what its synthetic employees say.
For anyone evaluating technology that will enter the home, the lesson is broader than software-company management. A polished answer is only the beginning. The harder questions are whether a system reads the available context, protects trust under pressure, respects boundaries and finishes what it starts. Firmulate has made those questions public—and tied them to a company whose financial clock is visibly running.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
enterprise AI trust verification
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
