Fawaz Ahmad

Token usage is the new lines of code (and it's just as useless)

Canva hit mid-2026 with every AI metric a vendor could hand us and no answer to the only question the business actually cared about: is this working? Adoption flattened at around 85% of ~3,000 engineers, and the number stopped telling us anything. AI touched 43% of PRs, PR volume was up 44% year over year (and growing week-over-week), but none of us could say what any of that had actually bought the business. Token spend ran eight figures annually, and one engineer alone racked up a $75K bill. Ask five leaders what it meant and you'd get five different answers.

This is the story of what we built to fix that, and what broke along the way.

We ended up with a three-layer value architecture:

1. The bottom layer is spend and adoption telemetry. A necessary baseline, but worthless on its own.
2. The middle layer is team-level ROI: we join AI spend to DORA metrics, grade teams within cohorts of similar peers, use median-plus-20% as a talking point rather than a hard cap, and deliberately report a fuzzy signal ("bottom third of your cohort") instead of raw numbers, because raw numbers get gamed the same way PR counts did.
3. The top layer is an exec-facing layer any LLM can query we wrote our measurement methodology as a skill over the warehouse, so a CxO can ask any AI tool and get the same answer we'd give them directly.

Across it all, using analytics first principles to measure and experiment (A/B test) our developer platform’s AI capabilities to concretely show value, as well as something I don’t think anyone else is running at scale: measuring attentive human time per agent conversation, then having a post-merge agent estimate time saved against tokens spent. So ROI gets measured at the point where its value is created.

What you'll walk away with is the architecture, the failures behind it, and the one metric we're betting on: engineering capacity created