The talent review list is rated by managers with something to lose

What the paper develops

Suppose a leader in an enterprise project management office has to fill one role that will decide whether a late program recovers: the lead for the cutover to a new system. Few people in the company have done the job before. The leader opens the talent review list, which names the employees whose managers rate them ready for bigger roles. Three names are on it. The list reads like a neutral record of who is ready. Who rated each name, and what does that manager lose if the person moves?

What one company's records show

Ingrid Haegele studied a large German manufacturer where each manager gives a worker two ratings at the same time. Performance stays inside the team. Potential, the rating for bigger jobs, goes into lists that HR circulates across the company. Managers with more of their pay tied to team results, and managers with smaller teams, were more likely to rate someone's potential lower than performance would predict. Asked how often a leader might need to talk a team member out of exploring another department because of team needs or goals, three-fourths of managers said at least sometimes. And 55 percent agreed that developing talent is a conflict of interest, because developed people are more likely to leave.

So the list can already carry a staffing decision. In that firm, the managers with the most to lose when someone left were more likely to rate their people's potential lower than performance suggested, and the potential rating is the one the rest of the company sees.

People also stay out of view by not applying. In the quarter a manager moved to another team, workers' applications for jobs elsewhere in the company rose by 78 percent. The people who applied only because their manager was leaving were hired 49.1 percent of the time. The average applicant was hired 27.6 percent of the time. That is one company, and its measure counts applications to posted jobs. A scarce role filled by appointment skips that step but still starts from the list.

Why pay and ratings are not an easy fix

The leader typically does not set the pay of managers on other teams, and a working paper on manager pay finds that fixing it can cost more than it returns. Fairer ratings face a limit too. In a study of developmental ratings of more than 2,000 managers, each rated by seven people, the raters' own habits explained more than half of the differences in ratings. The person being rated explained about a fifth to a quarter. A calibration meeting puts managers in one room to compare ratings, and it leaves each manager's stake where it was.

A staffing rule for scarce roles

For the few roles that are hard to fill and costly to get wrong, treat the list as leads. Ask for delivered work the current manager did not write, such as results recorded by the program that received it. Let someone outside the person's reporting line decide whether the person can move, once the person says they want the role. Help the manager replace the person, and credit the manager for what the person does next. No study has tested these four steps together. They are a recommendation built from the studies.

What to do next

Take the last three scarce roles you filled from a list. For each, write down who was on the list, who rated each name, and how much of that manager's pay depends on team results. Then ask the programs that received work from people across the company for the last two years of delivered results, and write down anyone with strong results who was not on the list. If that second list is not empty, the list has been missing people for the roles you can least afford to get wrong. The white paper has the six studies, the limits of each, and a made-up case.