LLMs Don't Run on Facts or Logic

Source # LLMs Don’t Run on Facts or Logic: Leading AI researchers explain the limitations of large language models David Truog 10 Mar 2026 7–9 min read Problems such as hallucination and opacity are inherent to the way LLMs operate — not fixable defects. This brief contains reflections from some of the field’s seminal thinkers about why. ## Purpose The purpose of Source briefs such as this one is to aggregate source material that I refer to in other briefs. ## Dario Amodei [Dario Amodei](https://en.wikipedia.org/wiki/Dario_Amodei) is the CEO of Anthropic. He was formerly VP of Research at OpenAI where he led the development of GPT-2 and GPT-3. He is also the co-inventor of reinforcement learning from human feedback. Before joining OpenAI, he worked at Google Brain as a Senior Research Scientist. ### On opacity Amodei wrote the below in a post about interpretability on his personal website.[^1] > People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. They are right to be concerned: this lack of understanding is essentially unprecedented in the history of technology. > > \[…\] > > Modern generative AI systems are opaque in a way that fundamentally differs from traditional software. If an ordinary software program does something — for example, a character in a video game says a line of dialogue, or my food delivery app allows me to tip my driver — it does those things because a human specifically programmed them in. Generative AI is not like that at all. When a generative AI system does something, like summarize a financial document, we have no idea, at a specific or precise level, why it makes the choices it does — why it chooses certain words over others, or why it occasionally makes a mistake despite usually being accurate. > > As my friend and co-founder Chris Olah is fond of saying, generative AI systems are grown more than they are built — their internal mechanisms are “emergent” rather than directly designed. It’s a bit like growing a plant or a bacterial colony: we set the high-level conditions that direct and shape growth, but the exact structure which emerges is unpredictable and difficult to understand or explain. Looking inside these systems, what we see are vast matrices of billions of numbers. These are somehow computing important cognitive tasks, but exactly how they do so isn’t obvious. > > Many of the risks and worries associated with generative AI are ultimately consequences of this opacity, and would be much easier to address if the models were interpretable. For example, AI researchers often worry about misaligned systems that could take harmful actions not intended by their creators. > > Our inability to understand models’ internal mechanisms means that we cannot meaningfully predict such behaviors, and therefore struggle to rule them out; indeed, models do exhibit unexpected emergent behaviors, though none that have yet risen to major levels of concern. More subtly, the same opacity makes it hard to find definitive evidence supporting the existence of these risks at a large scale, making it hard to rally support for addressing them — and indeed, hard to know for sure how dangerous they are. > > \[…\] > > AI systems’ opacity also means that they are simply not used in many applications, such as high-stakes financial or safety-critical settings, because we can’t fully set the limits on their behavior, and a small number of mistakes could be very harmful. > > Better interpretability could greatly improve our ability to set bounds on the range of possible errors. In fact, for some applications, the fact that we can’t see inside the models is literally a legal blocker to their adoption — for example in mortgage assessments where decisions are legally required to be explainable. > > Similarly, AI has made great strides in science, including improving the prediction of DNA and protein sequence data, but the patterns and structures predicted in this way are often difficult for humans to understand, and don’t impart biological insight. Some research papers from the last few months have made it clear that interpretability can help us understand these patterns. ## Andrej Karpathy [Andrej Karpathy](https://en.wikipedia.org/wiki/Andrej_Karpathy) was a founding member of OpenAI, where he served as a research scientist contributing to deep learning and generative model research. He was head of AI at Tesla from 2017 to 2022, focused on language model research. ### On hallucination Karpathy wrote the below in a post on X/Twitter.[^2] > I always struggle a bit when I'm asked about the “hallucination problem” in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. > > We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful. > > It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does. > > At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" — it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem. > > All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallucinations in these systems — using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All active and very interesting areas of research. > > TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it. ## Yann LeCun [Yann LeCun](https://en.wikipedia.org/wiki/Yann_LeCun) is a Turing Award–winning pioneer of deep learning. His work on convolutional neural networks (CNNs) helped establish modern neural-network–based AI. He was Chief AI Scientist at Meta from 2013 to 2025. ### On cognition LeCun said the below in a podcast interview.[^3] The interviewer said: “You’ve said that autoregressive LLMs are not the way we’re going to make progress towards superhuman intelligence. These are the large language models like GPT-4, like Llama 2 and 3 soon and so on. How do they work and why are they not going to take us all the way?” LeCun’s response was: > \[Autoregressive LLMs are not the way we’re going to make progress towards superhuman intelligence\] for a number of reasons. > > The first is that there are a number of characteristics of intelligent behavior. For example, the capacity to understand the world, understand the physical world; the ability to remember and retrieve things, persistent memory; the ability to reason; and the ability to plan. Those are four essential characteristics of intelligent systems or entities, humans, animals. > > LLMs can do none of those; or they can only do them in a very primitive way, and they don’t really understand the physical world. They don’t really have persistent memory. They can’t really reason and they certainly can’t plan. And so if you expect the system to become intelligent just without having the possibility of doing those things, you’re making a mistake. > > That is not to say that autoregressive LLMs are not useful (they’re certainly useful), that they’re not interesting, \[or\] that we can’t build a whole ecosystem of applications around them. Of course we can. But as a path towards human-level intelligence, they’re missing essential components. > > And then there is another tidbit or fact that I think is very interesting. Those LLMs are trained on enormous amounts of texts — basically, the entirety of all publicly available texts on the internet. That’s typically on the order of 1013 tokens. Each token is typically two bytes, so that’s 2×1013 bytes as training data. It would take you or me 170,000 years to just read through this at eight hours a day. So it seems like an enormous amount of knowledge that those systems can accumulate. But then you realize it’s really not that much data. If you talk to developmental psychologists they tell you a four-year-old has been awake for 16,000 hours in his or her life, and the amount of information that has reached the visual cortex of that child in four years is about 1015 bytes. > > And you can compute this by estimating that the optical nerve carries about 20 megabytes per second roughly. And so 1015 bytes for a four-year-old versus 2×1015 bytes for 170,000 years worth of reading. What that tells you is that through sensory input, we see a lot more information than we do through language, and that despite our intuition, most of what we learn and most of our knowledge is through our observation and interaction with the real world, not through language. Everything that we learn in the first few years of life, and certainly everything that animals learn has nothing to do with language. [^1]: Dario Amodei, [“The Urgency of Interpretability”](https://www.darioamodei.com/post/the-urgency-of-interpretability) ([darioamodei.com](https://www.darioamodei.com/), April 2025). In the excerpt in this brief, I inserted a few additional paragraph breaks to improve legibility. [^2]: Andrej Karpathy, [“On the "hallucination problem"”](https://x.com/karpathy/status/1733299213503787018) (*X,* 8 December 2023). In the excerpt in this brief, I corrected a few typographical errors that are present in the original. [^3]: Yann LeCun in an interview with Lex Fridman, [“Yann LeCun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI”](https://lexfridman.com/yann-lecun-3/) (*Lex Fridman Podcast,* episode #416, 7 March 2024). The interview is also available as a [transcript](https://lexfridman.com/yann-lecun-3-transcript/); for the excerpt featured in this brief, I corrected a few transcription errors and adjusted the formatting for clarity.