DeepMind researcher argues modern LLMs lack true reasoning capabilities

A researcher who recently left Google DeepMind has argued that current large language models (LLMs) fail to demonstrate true reasoning, contrasting their performance with the architecture of AlphaGo. While AlphaGo defeated Lee Sedol 4-1 in 2016 by using a search mechanism to evaluate thousands of possible future game states, modern LLMs primarily rely on next-token prediction, which the researcher categorizes as a fast, associative System 1 process rather than deliberative System 2 reasoning.

According to the researcher, three primary shortcomings prevent chatbot chains of thought from qualifying as scientific reasoning. First, models lack a persistent, inspectable epistemic state that tracks hypotheses and confidence levels. Second, there is no separation between the model's knowledge and its manipulation of that knowledge, as both are interwoven in neural network weights. Third, research indicates that chains of thought are often generated after the fact, with the model reporting a reasoning path that does not reflect its actual decision-making process.

For high-stakes fields such as medicine, engineering, and scientific research, the researcher emphasizes that understanding how a system arrives at a conclusion is as important as the conclusion itself. The researcher advocates for a new approach to machine reasoning that incorporates AlphaGo's architecture, specifically a data structure like a game tree that records what the system holds as settled, doubts, or rules out. This structure would allow for a systematic update of knowledge as new information arrives, rather than relying on the iterative token prediction currently used in language models.

What to watch: The development of epistemic-based reasoning architectures in future AI systems.

Editor's note: The draft comprehensively captures the researcher's critique of LLMs, the comparison to AlphaGo, and the specific limitations of current chain-of-thought reasoning.