Former DeepMind expert argues LLMs lack genuine reasoning capabilities

In an analysis of machine intelligence, a former Google DeepMind specialist highlights that despite the fluency of today's Large Language Models (LLMs), they lack the genuine reasoning powers demonstrated by earlier systems like AlphaGo. The 2016 match in Seoul, where AlphaGo defeated professional player Lee Sedol 4-1, centered on the famous 'Move 37.' While commentators initially viewed the move as a glitch, it was the result of a deliberate search process that weighed future consequences by constructing a game tree with thousands of branches. This differs fundamentally from LLMs, which operate primarily through next-token prediction.

The report distinguishes between two modes of thought: System 1 (fast and intuitive) and System 2 (slow and deliberative). While AlphaGo utilized a policy network for intuition and a search mechanism for deliberation, LLMs function almost entirely as System 1 pattern completion engines. Capability claims for 'chain of thought' processing in models like ChatGPT have shown real gains in coding and mathematics. However, these are not verified as independent reasoning mechanisms; rather, they are the same next-token prediction process iterated for a longer duration before the model commits to an answer.

Three primary shortcomings prevent current chatbots from achieving true reasoning. First, they lack an explicit, persistent epistemic state to track hypotheses and evidence. Second, there is no separation between the system's knowledge and how it manipulates that knowledge, as both are interwoven in neural network weights. Third, research indicates that the 'chains of thought' produced by bots are often concocted after the fact, meaning the model may reach an answer by one route but report another. These flaws make current AI difficult to use in high-stakes fields like medicine and engineering where the process of conclusion is as vital as the result.

AlphaGo's architecture is proposed as a superior template for future AI development. By maintaining a game tree, the system could update its judgments as reasoning progressed. A similar approach for general reasoning would require a system to maintain an epistemic state representing what is settled, doubted, or ruled out. This structural transparency would allow human experts to pinpoint whether a failure resulted from faulty reasoning, invalid evidence, or incorrect assumptions. The author suggests that a fresh approach to machine reasoning is necessary to produce trustworthy results in scientific research.

What to watch: Development of new AI architectures that incorporate explicit epistemic states and search machinery.

Editor's note: Faithfully and clearly translates the technical and analytical arguments from the source text.