Richard Sutton, Father of RL, Thinks LLMs Are a Dead End

Note: This post is a summary of an interview with Richard Sutton. For more information, you can refer to the original video.

While the entire tech world is captivated by the impressive capabilities of Large Language Models (LLMs), one of the field’s true giants is offering a radically different perspective. Richard Sutton, known as the “father of reinforcement learning” and a recipient of 2024‘s Turing Award, believes that LLMs are fundamentally on the wrong path. 
In his view, these models are merely mimicking humans rather than truly understanding the world, and they lack the most essential component of intelligence. Sutton is no lightweight; he’s an inventor of key techniques, such as TD learning and policy gradient methods. When a figure like him says we might be in a technological cul-de-sac, it’s worth listening.

The Core Flaw: Mimicry, Not World Modeling

Sutton’s central argument is sharp and to the point: the essence of LLMs is to mimic human language, not to build a model of how the world works. They excel at predicting “what a person would say” but are utterly incapable of predicting “what will happen next”. “To mimic… what people say is not really to build a model of the world at all.”

An actual world model allows you to predict the consequences of your actions. LLMs lack this ability. When they encounter unexpected feedback from the world, they don’t feel “surprised,” nor do they adjust their internal model as a result. They are just generating text, not interacting with and learning from the real world.

The Essence of Intelligence: A Goal

Sutton cites AI pioneer John McCarthy’s definition: Intelligence is the computational part of the ability to achieve goals. This is precisely what LLMs are missing.

• LLMs have no substantive goals: You might argue that “next token prediction” is their goal. But in Sutton’s view, this isn’t a goal related to the external world. It doesn’t change the world; it’s just passively predicting a data stream and feeling “happy with itself that it’s predicting accurately”.
• Without a goal, there is no “right or wrong”: If there is no goal to achieve, actions are neither good nor bad, and there is no “ground truth”. Without a goal, continual learning is impossible because the system can’t determine if its actions led to a good or bad outcome. The Real Way to Learn: From Experience, Not Data Sutton argues that the way we, and all animals, learn is entirely different from how LLMs do. An infant doesn’t learn primarily through imitation but through an active process: they try things and see what happens.
• The Stream of Experience: True intelligence should learn from a continuous stream of “sensation-action-reward”. The knowledge you acquire is about this stream and can be continuously tested and updated by it — that’s continual learning.
• Learn from the Squirrel: Sutton even asserts that we should pay less attention to what makes humans special. “If we understood a squirrel we’d be almost all the way there” to understanding human intelligence. That’s because a capability like continual learning is something all mammals possess, yet our most advanced AI systems today do not.

“The Bitter Lesson” Revisited

Many people cite Sutton’s own famous essay, “The Bitter Lesson,” to defend the massive scaling of LLMs, arguing it’s a prime example of leveraging massive amounts of compute. But Sutton offers a surprising interpretation: LLMs are not a pure manifestation of “The Bitter Lesson.” While they use massive computation, they also inject massive amounts of human knowledge (i.e., internet text). He predicts this will ultimately be a trap. Systems that can learn directly from experience, rather than from human data, will be more scalable and will eventually supersede LLMs. When that happens, the story of LLMs will become just another instance of “The Bitter Lesson”: methods that relied heavily on human knowledge were eventually surpassed by more general methods that relied only on computation and experience.

Conclusion

Sutton’s perspective is a radical challenge to the current AI development paradigm. He argues that we shouldn’t be satisfied with building an ever-more-powerful imitation machine, but should instead return to first principles to create an agent that can truly learn by interacting with the world. He is content, even happy, to be “out of sync with my field,” believing that, in the long run, simple, general principles will prevail. So, the question is: Are we on the path to AGI, or are we speeding down a beautifully decorated but ultimately limited dead end?