Richard Sutton thinks LLMs are a dead end. His arguments are valid but hackers and engineers have pragmatic solutions.

I just watched the podcast episode by Dwarkesh with Richard Sutton, a famous AI researcher who is widely recognised as one of the founding fathers of modern computational re-inforcement learning.

The TLDR is that Richard Sutton believes that LLMs, trained on vast amounts of text data, are a dead end in the pursuit of true artificial intelligence. While this could be true, I think his arguments to back up this claim are really surprising.

Here is a short summary of what he says:

• Lack of world model - LLMs do not build true model of the world. They only mimic what humans say.

• No Ground Truth - LLMs lack ground truth because they don’t have a way to verify their predictions against reality. In contrast RL has a clear definition of a right action - one that leads to a reward.

• Absence of Goals: A core tenet of Sutton’s argument is that intelligence is about achieving goals. He contents that LLMs do not have goals in meaningful sense and they are “simply” predicting the next token

• Imitation vs Experience: True learning outcomes come from active learnings - trying and seeing what happens - not from imitation. He argues that even children learn primarily through trial and error, not by copying.

He isn’t the first with Yann Lecun (Meta AI Chief Scientist), Gary Marcus (Cognitive scientist and psychologist) already being in the camp against LLMs creating human like intelligence.

I find Richard Sutton’s take to be quite surprising, while I don’t claim to be an AI researcher or scientist I am a power user of all kinds of AI.

Firstly, there are very few applications that are useful which involve just using an LLM besides maybe AI girlfriend or chatbot. Even chatgpt is no longer a pure LLM but a system of LLMs and routing. Additionally, each of these limitations has a strong engineering solution.

And the real question is that how far can each of these engineering solutions go?

In context learning, tool calling, system prompts are all ways to overcome these challenges.

For example

Goal Definition & System Prompts

The goal of an AI system can be defined with a system prompt. Popular applications typically use system prompts approaching 20K tokens in length. In contrast, most human tasks don't require such extensive goal descriptions.

Grounding & RAG & Tool Calling

RAG, Tool Calling, and increased Context Lengths all help ground AI systems. When seeking answers from a corpus, the system becomes grounded in what should be verifiable facts. This capability extends to web searches as well.

Continual Learning

This argument carries the most merit, but it's not insurmountable. Current training runs for continual learning create new models that incorporate recent information from the internet and other data sources gathered since the last training cutoff. While inefficient, AI applications can develop token-efficient methods to feed this information into the context window, thereby improving their systems.

When it comes to the argument of LLMs just do imitation. The question comes to mind, that does it matter if the core LLM is just imitation, whereas the rest of the AI system is helping using the above engineering solutions to architect every problem into a next token prediction problem.

This actually reminds me of a time when Ilya said in a March 2023 GTC conference -

"Say you read a detective novel. It's like complicated plot, a storyline, different characters, lots of events, mysteries like clues, it's unclear. Then, let's say that at the last page of the book, the detective has gathered all the clues, gathered all the people and saying, 'okay, I'm going to reveal the identity of whoever committed the crime and that person's name is'. Predict that word.”

This analogy became great at explaining why large language models trained on next-token prediction can exhibit what appears to be reasoning and understanding - because accurately predicting what comes next in complex contexts requires building an internal model of the world that generated that text.

Now the question is will the engineers find a way to make LLM powered system AGI before the researchers come up with a world model? The market is betting trillions of dollars on the hackers. I think the philosophy of an engineer is the same of a hacker - getting your way when the system isn’t in your favor. It’s kind of poetic that an ex president of the hackers Sam Altman is leading the AGI race.

I think Richard Sutton is right that these engineering solutions to LLMs limitations are limited but it’s going to be some time till we get a world model.

Till then us hackers got to ship!