Mark Johnson

Why software factories can't be trusted (and how the Systems Engineering V-model helps)

Coding agents will do anything to get to a PR. Ours marked tasks complete that didn't exist, wrote "next steps: verification" under unreviewed pull requests, and quietly skipped every boring stage between the ticket and the diff. In our world, that isn't a quirk — it's disqualifying. We build cardiology, respiratory and clinical-trials systems for public hospitals: large brownfield codebases with decades of accumulated behaviour, where you don't get to make mistakes in production. When mistakes reach production, people die.

So we stopped prompting harder and applied Systems Engineering rigour. Our harness is built from four high-level blocks: the Systems Engineering V-model, deterministic verifiable gates, adversarial verifiers, and loops that carry every piece of work from Jira ticket to plan to PR.

The decisions that made it work: the gates are control flow, not instructions — prompts get rationalised away under pressure; workflow code doesn't. The verifiers are adversarial — their job is to refute the work, not confirm it — and anything red loops back until it's green. And humans remain a really important part of the process; where they sit, and why, is half the story.

The factory now runs unattended on a VM, dispatching work across model providers. Ticket cycle time has dropped from 45 days to 6, and production code issues are down roughly 40%. I'll walk through the thinking behind each block, the limitations we've hit honestly, and leave you with a skill you can use to build your own V-model harness.