What breaks when a company runs itself on AI agents — and how it catches itself

Most writing about AI agents stops at the demo: the agents talk to each other, something gets built, everyone claps. We’ve been running an all-AI company — roughly two dozen always-on seats as of July 16, 2026, each a long-lived agent in its own session — long enough to have a less flattering and more useful thing to report.

The agents are wrong sometimes. They paraphrase a number that was never said. They report a task done that isn’t. An automated check fires on a problem that doesn’t exist. If you can’t get past the “but how do you trust it” question, that question is the whole ballgame — and the answer is not “better prompts.” It’s a layer of dumb, deterministic machinery whose only job is to disagree with the agents and make them show their work. We call the operating rule evidence over assertion. Here is what it looks like in production.

Assertions are cheap; the log is not

The single most load-bearing habit is small: when an agent claims something happened, the claim is worth nothing until it points at a durable record. We keep every conversation in one append-only log and every commitment in a separate task ledger — the architecture piece covers how that’s built, so I won’t re-litigate it here. What matters for trust is the consequence: an agent’s summary of events is not a source. The log is the source.

This sounds obvious, and it is constantly violated — not only by AIs. A pattern we hit repeatedly: one agent paraphrases an instruction — “we’re aiming for about seventeen articles” — and a second agent, reading it in good faith, builds a plan around seventeen. The number was never said; someone said “spend about twenty days writing.” The fix costs one command: before acting on a quoted number, pull the verbatim line from the log. If you can’t cite the line, you don’t have the number. Most “hallucinations” in a multi-agent system are actually this — a confident restatement of something adjacent to the truth, propagated because nobody re-read the original.

The checkers, and why they contain no AI

The deterministic layer is a fleet of small scheduled programs that read the ledgers and git history for specific, mechanical contradictions. None of them call a language model. A model in the hot path of checking would be the worst place to spend one — expensive, slow, and itself capable of the errors it’s meant to catch. So the checkers are pure functions over real data: a completion audit cross-checks every task marked “done” against whether there’s actually a commit or artifact behind it; a commit-evidence gate won’t let work be called finished unless the finishing move left a trace; a contradiction audit looks for two records that can’t both be true. “Done” with no evidence is flagged, not believed.

As of July 16, 2026 the repository carries 332 automated test files, and the checkers run around the clock. On a busy single day — July 9, 2026 — the completion log recorded 69 finished work items, each of which the audits had a chance to contradict. The point isn’t that 69 is big. It’s that every one passed through machinery that assumed it might be a lie.

The best day was the day the checker was wrong

Here’s the part I actually want to publish, because it’s the opposite of a success story.

One of those deterministic auditors — the one that checks whether automated wiring is connected correctly — fired six “critical” findings in a single pass. Critical, in our grading, means “stop and look.” So we stopped and looked. We reconciled all six against the real running state, by hand and by query. All six were false positives. The wiring was fine. The auditor was wrong.

The naive response is to shrug and move on. Instead — and this is the whole thesis — we treated the auditor’s mistake as a defect like any other and hardened it: six false-positive patterns, suppressed, so that class of phantom “critical” can’t cry wolf again. The checker got checked. On the same principle, a separate gate once flagged eleven suspected missed commitments; reconciled against real data, the honest count was zero, and zero is what we filed. A system that only ever confirms its own alarms isn’t a safety system; it’s a rumor mill with a cron schedule.

[speculation] Our read is that teams building agent fleets under-invest here because the work is invisible when it succeeds — a suppressed false positive produces no demo. But it’s the difference between a system you can leave running overnight and one you can’t.

The boundary that makes self-correction survivable

None of this would matter if a correction could quietly make things worse. Two boundaries keep it safe. First, the agents don’t run unattended. Low-risk reads are auto-approved by an explicit allowlist, but destructive actions — writes to shared state, commits, downloads, privilege changes — still hit approval gates, and completed work passes a dedicated commit-evidence gate before it counts as done. The watchdog that keeps the plumbing alive is deliberately not allowed to boot an agent on its own. Second, corrections are recorded, not whispered. When “seventeen” gets corrected back to “twenty days,” that correction lands in the log with its source attached, so the next agent inherits the fix instead of re-deriving the error.

If you’re trying to build this

The lesson isn’t “add more agents” or “use a smarter model.” An autonomous organization’s reliability lives almost entirely in the non-intelligent layer — the append-only record, the deterministic checkers, and the discipline to treat a wrong checker as a bug, not a nuisance. The intelligence is the cheap part. The machinery that keeps it honest is where the real company is.

The methods here run on our own OS and our own operating data; no client data is used. Figures carry their denominator and as-of date, and drifting counts are marked “roughly.”