Benchmarks are the rulers of AI, in both senses of the word. As measuring sticks, they work according to an extremely simple logic: If you want to know what an AI model can do, devise a standardized test to gauge it. But AI benchmarks can also dominate the behavior of the entire field.
In 1987, the Defense Advanced Research Projects Agency’s first benchmark on automatic speech-to-text translation showed that state-of-the-art AI could not outperform so-called “dumb” statistical approaches. The poor performance helped to usher in an AI winter. For 25 years, interest in AI research waned and funding dried up.
Then, in 2012, another benchmark called ImageNet swung the pendulum back. An eight-layer neural network trounced other algorithms at classifying images and kicked off a “deep learning revolution.”
Nowadays, new AI benchmarks, geared at large language models such as OpenAI’s GPT-6 Astra, get created and solved with clockwork regularity, purporting to measure everything from domain-specific expertise to “general intelligence.” But rather than hail this trend as proof of AI’s skyrocketing potential, many in the field — skeptics and boosters alike — take it as evidence that benchmarking itself has become fatally flawed.
Moritz Hardt, a computer scientist and a director at the Max Planck Institute for Intelligent Systems, agrees with this diagnosis. Shortly after the ImageNet result, he began to think that machine learning systems could game leaderboards “without doing anything intelligent.” “There’s no two ways about it,” he said. “The benchmarking ecosystem is broken.”
In his 2026 book The Emerging Science of Machine Learning Benchmarks, Hardt makes the provocative case that these measuring sticks have been broken for decades already: Benchmarks, he writes, are “the mistake that made machine learning.” Notably, he argues, instead of creating a replicability crisis, they drove legitimate progress anyway — “mostly by sheer luck,” in his view. By understanding why, we can explain how benchmarks have since lost this accidental credibility in the age of LLMs, and how they might regain it.
Quanta spoke to Hardt about what went wrong, and right, with benchmarks, and where to take them next. The interview has been condensed and edited for clarity.
