Yulia Kuchina

How to Change an LLM System Without Guessing

We were running a production LLM pipeline that classified legal documents, and every change was a guess. Swap a prompt, change a model — better or worse? Nobody could say. The outputs looked plausible either way, and “plausible” is exactly how LLM systems hide their regressions.

This talk is how we went from operating on faith to changing the system on evidence — a systems story, not a model story. The layer, built in order: prompt fingerprinting so every output traces to the exact prompt and model; provenance on every document record; failure visibility so classification failures surface as data, not silence; and an evaluation harness wired into CI, scoring every change against a labelled baseline — report-only today, becoming a blocking gate once the corpus supports a threshold we trust. The corpus is the honest bottleneck — a starving eval set can’t gate anything — so we’re building a pipeline that mints eval fixtures from real production failures without PII ever entering git: labels in version control, documents in an erasable store inside the production boundary.

The trade-offs, honestly: why we grounded extraction in document citations the model has to resolve — fabricated evidence fails to resolve instead of producing a false highlight — why a lawyer’s override is becoming first-class state the system can re-check, and the decisions I’d make differently. I’ll show a regression the harness surfaced before it shipped, and a defect that sailed past every automated check: a legally-wrong output that was mechanically correct, in a dimension none of our instruments measured — where human judgment takes over.

The takeaway: the reliability of an AI feature lives in the harness around the model, not the model. Prompts, parsers, evals, telemetry, and release gates are one system — once you can measure it, you stop guessing.