I merged Karpathy's AI jobs scores that people are calling AI slop with Eloundou et al. data from their Science (2024) paper and the new Anthropic’s measure that captures AI displacement risk.
The rank correlation of Karpathy’s scores was .82 vs human expert raters (from the Science paper) and .72 vs Anthropic's observed exposure metric.
Low scores on Karpathy index → near-zero real AI usage. High scores → 40–64% observed exposure.
Yes, this doesn't tell you whether you are going to lose your job in the future (that was not the point of his project).
But this is also not some AI generated slop or noise (even if the scores are based on a single generic prompt and derived with Gemini Flash, not even a frontier model).
To me, this is the most remarkable part of his project - that the scores are even picking something real about where LLMs are being used at work (the observed exposure metric) and track surprisingly well with the human benchmark.
This shows the practical usefulness of these tools.
One person, one lightweight LLM, and a single prompt can now produce over a weekend (as a side project) something that picks a real signal and would have taken a well-funded research team months to do a few years ago.
The rank correlation of Karpathy’s scores was .82 vs human expert raters (from the Science paper) and .72 vs Anthropic's observed exposure metric.
Low scores on Karpathy index → near-zero real AI usage. High scores → 40–64% observed exposure.
Yes, this doesn't tell you whether you are going to lose your job in the future (that was not the point of his project).
But this is also not some AI generated slop or noise (even if the scores are based on a single generic prompt and derived with Gemini Flash, not even a frontier model).
To me, this is the most remarkable part of his project - that the scores are even picking something real about where LLMs are being used at work (the observed exposure metric) and track surprisingly well with the human benchmark.
This shows the practical usefulness of these tools.
One person, one lightweight LLM, and a single prompt can now produce over a weekend (as a side project) something that picks a real signal and would have taken a well-funded research team months to do a few years ago.