WASHINGTON– Credit union leaders, especially those in middle management, are as concerned as workers everywhere over which jobs artificial intelligence might replace, and a new study finds even AI itself is uncertain where the job cuts will come.
The new study, by economists affiliated with the National Bureau of Economic Research, raises questions over how reliably artificial intelligence can predict which jobs are most vulnerable to automation, finding that leading AI models often disagree sharply on which occupations are most exposed.
According to the working paper posted last month by the NBER, economists Michelle Yin and Hoa Vu of Northwestern University and Claudia Persico of American University examined how three prominent AI systems ranked occupations for potential exposure to AI-driven disruption.
Use of ‘Exposure Scores’
The researchers said economists increasingly use so-called “exposure scores” to estimate which occupations could be affected by AI. Those scores are typically built using a task-based framework that relies on Labor Department data detailing what workers in various occupations actually do.
Under that framework, researchers evaluate which tasks AI could significantly accelerate or perform. Jobs with a larger share of AI-capable tasks are considered more exposed to automation.
The economists said exposure scores have become common in research reports, consulting analyses and policy papers.
Three Methods
According to the study, researchers generally rely on three methods to generate those scores:
- Human evaluators who manually assess AI’s ability to complete tasks
- Surveys of workers using AI tools
- AI models themselves evaluating occupational exposure
The economists said each method comes with limitations. Human evaluations can be subjective, while worker surveys may reflect only the experiences of users of a single platform rather than the broader labor force.
The study focused on the third method — using AI systems to judge exposure to AI — and found significant inconsistencies among the models.
Yin, Vu and Persico asked OpenAI’s ChatGPT-5, Google DeepMind’s Gemini 2.5 and Anthropic’s Claude 4.5 to rank occupations according to AI exposure.
‘Widely Differing Results’
According to the researchers, the models frequently produced widely differing results.
The economists found, for example, that Claude ranked accountants as highly vulnerable to AI disruption, while Gemini assigned accountants a much lower exposure score. The models also diverged in their assessments of occupations including advertising managers and chief executives.
The researchers reported that ChatGPT and Gemini produced the most similar rankings but still disagreed roughly one-quarter of the time.
According to the study, some of those differences stemmed from variations in the underlying AI models themselves. But the economists also found evidence that model rankings may be influenced by which occupations are already making heavy use of AI technology.
The paper said early adopters such as financial analysts generate large amounts of AI-related training data through frequent use of the systems, potentially shaping how future models assess those professions.
A Caution is Shared
The researchers cautioned that policymakers and employers may not fully appreciate the uncertainty embedded in AI-generated exposure scores.
The economists noted that the study is currently a working paper and has not yet undergone peer review.
Still, the authors said disagreement among rapidly evolving AI systems is not necessarily unexpected, and it remains unclear whether AI-generated rankings are more or less reliable than other exposure-scoring methods.
As a possible improvement, the economists recommended that future researchers compare results across multiple AI models rather than relying on a single system. They also said researchers should be transparent about the uncertainty surrounding AI-generated exposure estimates.
One Researcher’s Advice
Ultimately, the authors suggested that surveys examining how AI is actually being used across the economy — and which workplace tasks are being automated in practice — may provide more reliable insight into job vulnerability.
“I personally would not rely on just one measure to say, ‘Oh, I should change my job,’ or ‘I should change my kid’s major,’” Yin said, according to the NBER paper.
