Mukesh Singh

Building Agentic Memory for an AI SOC: Why Our Semantic Cache Was the Wrong Answer

We run a fleet of AI agents against production security detections at Atlassian. One tunes noisy detection rules, another reviews new ones, and more are coming for alert triage and hunting. Every one of them started blind and stateless, re-deriving the same context from Jira tickets, a Databricks lake, a rule repo and ATT&CK, or drowning in a raw dump of all of it. Nothing they learned survived the session, so they repeated each other's mistakes and contradicted each other.

Our first design was a semantic cache. It was wrong, and working out why reshaped everything. A semantic cache is keyed on input embedding and returns a cached conclusion, which in a domain with consequences means serving a stale security verdict to a look-alike question. What we built instead is an entity-keyed, facts-only store, each agent reads a bounded, per-action slice before it acts, then writes facts back. Conclusions are never cached. They are re-derived every run, deliberately.

I will walk through the schema, the four rules we enforce (facts only, provenance on every row, derived and never authoritative, bounded slices instead of whole-history blobs), and the point where native prompt caching stops paying and a store starts.

Then the measurements, on real tickets with human ground-truth labels that analysts produce as a byproduct of triage, input tokens per run, eliminating a 900-second lake query, false positives removed with zero true positives lost, and how consistent two agents stay when they share one memory instead of guessing separately.

Then what broke: entity resolution, write contention, memory poisoning, and the crossover where memory costs more than it saves.

You will leave with the schema, the eval design and the four rules, all of which transfer to any agent fleet that needs to remember something.