Marian Kamenistak · Speaker

5 Levels of AI Adoption: The KPIs You Should Be Measuring

From "should we buy seats" to "are we the best". One ladder, five rungs, two KPIs per rung, and the delivery curve that tells you which rung your company is on.

For: CTOs, VPs of Engineering, Engineering Managers, Heads of AIFormat: 45-60 min keynote or 90-min leadership workshop
Marian Kamenistak speaking at an engineering leadership event.

How do you measure whether AI adoption is working in your engineering team?

Most CTOs I mentor ask a version of the same question: we bought the AI seats, is it working? The ladder has five rungs above zero. Level 0, still debating. Level 1, licenses bought. Level 2, AI code in production. Level 3, the bill climbs. Level 4, you can say whether it is worth it. Level 5, you ask whether you are the best. Each rung has its own mistakes: the split team at level 1 (a few already paying out of pocket, most experimenting, the rest scared it replaces them), the review backlog at level 2, the $20k weekend agent at level 3. Navigara's delivery data across 500 open-source projects shows the shape of the climb: a 12% dip in engineering performance during the experiments, +104% per developer once it works, then a flat line while spend keeps rising. We go rung by rung and you place your own company on the ladder, with the two KPIs that prove it.

You walk out with my five-level KPI table: two numbers to track per level and four vanity numbers to stop reporting (seat utilisation, acceptance rate, share of AI-written code, self-reported speed), from the Head of AI transformation playbook on this site.

Why this talk, not a generic version of it

A generic AI adoption talk is a maturity model with five boxes and a vendor logo. This one uses measured delivery data: the dip, the payoff, the plateau, from Navigara's engineering performance index across 500 open-source projects and Microsoft's VS Code, shared by Jirka Bachel at ELC meetup #39, which I moderated. Level 5 is the part rooms argue about: 60% of all performance comes from the top 10% of engineers, and AI widened that gap instead of levelling it.

Built for the leader who has to defend the AI line in next year's budget. As a keynote you leave knowing your level and the two KPIs for it. As a workshop your leadership team leaves with the KPI set for the next level, agreed in the room, and the four vanity numbers crossed off the dashboard.

What you take home

  1. Place your company on the ladder: 0 debating, 1 licenses, 2 production, 3 spend, 4 ROI, 5 flying.
  2. Expect the dip. Delivery slows during the experiments, the same curve as onboarding a new hire.
  3. Track performance per developer against spend per developer. Flat performance with rising spend is level 3, however the dashboard looks.
  4. Measure only the two KPIs of your level. Drop seat utilisation, acceptance rate, AI-code share and self-reported speed from the board slide.
  5. Watch the top 10%. AI widened the spread, and a team average hides it.

The data behind the talk

Every number on the slides, with where it came from.

−12%
engineering performance at the dip, vs the pre-AI baseline
Source: Microsoft VS Code, Navigara ETV per month, ELC meetup #39
+104%
performance per developer since the AI push
Source: Navigara, 500 OSS Performance Index
60%
of all performance comes from the top 10% of engineers; the bottom 90% ship 40%
Source: Navigara, cumulative ETV share
6x
OpenAI outperforms Meta on the same index; a year ago the two were similar
Source: Navigara, 500 OSS Index
90%
of respondents already use AI at work, so seat utilisation proves nothing
Source: DORA, State of AI-assisted Software Development 2025

What the room walks out with:

Get the five-level KPI table →

Quick answers

Who is "5 Levels of AI Adoption: The KPIs You Should Be Measuring" for?
CTOs, VPs of Engineering, Engineering Managers, Heads of AI
What format is it delivered in?
45-60 min keynote or 90-min leadership workshop

More talks

← Back to all talks