AI operating governance · Field note

The people your AI review says gained nothing are teaching the tool

TL;DR

A year into an AI program, weaker staff have sped up and top performers show no gain. Suppose the tool learns mostly from your top performers' work and keeps learning. Then part of that gain is theirs, passed on through the tool, and the tool may be nudging their quality down. Moving people by measured gain can wear down that source. Before renewing, ask whose work the tool learns from. Judge those people on quality rather than speed, and protect what they give the tool.

What the paper develops

The decision

An AI program has run for a year and the funding review is coming. Weaker and newer staff are much faster with the tool. Top performers show little or no gain. The natural next step is to follow the gain: hire fewer seasoned people, leave departures unfilled, and move people with no measured gain to other work.

Where the gain comes from

A large field study looked at one support operation that gave its agents an AI assistant [1]. The least skilled fifth of agents resolved 36 percent more issues per hour. Top performers showed no significant change. The tool was trained on the firm's own conversations, with extra weight on top performers'. The authors say the gains are consistent with the tool passing top performers' practices on to everyone else. They also say productivity includes the training data people produce. It is one tool, one firm and one kind of job.

How the source wears down

Moves based on seniority, cost or measured gain can reach top performers. In the same study, top performers' resolution rates and customer satisfaction dipped slightly with the tool. The authors suggest the tool may lead top performers to take the easier option. If the tool keeps learning from current work, fewer top performers, or top performers leaning on the tool, mean fewer strong new examples. That step is a judgment, not a finding. And the risk builds slowly where the work is stable.

What to do before renewing

Ask the AI team or vendor whether the tool learns from your staff's work, mostly from whose, and whether it keeps learning. If it never learns from your staff, or learns evenly from everyone, this does not apply. If it does, take three steps. Record whose work it learns from. Judge those people on quality rather than speed. Protect what they give the tool. Turning the suggestions off for them may be a first step. Giving them time to review the tool's suggestions is another. Where the work keeps changing, that time is worth paying for.

What to do next

Before the review, ask whose work the tool learns from, and judge those people on quality, not on their own speed gain.

Inside the white paper

  • Where the gain shows
  • Where the gain comes from
  • How the source wears down
  • What the renewal should do

Sources and notes

  1. Erik Brynjolfsson, Danielle Li and Lindsey Raymond, The Quarterly Journal of Economics — Generative AI at Work (2025)