An AI hallucination is a confident, fluent answer that is false or unsupported, such as a made-up fact, citation, statistic or API function. Large language models hallucinate because they are trained to generate likely-sounding text, not to look up verified truth. You can't eliminate hallucinations today, but you can reduce them sharply by grounding answers in retrieved sources, requiring citations, validating outputs in code and evaluating systematically.
Why do language models hallucinate?
An LLM learns the statistical patterns of language from huge amounts of text. When it answers, it is predicting a plausible continuation, token by token. Usually plausible and true coincide, because the model saw the true information many times. Hallucinations appear when they don't:
- Missing or rare knowledge: the fact appeared rarely, or never, in training data.
- Outdated knowledge: the world changed after the training cut-off.
- Pressure to answer: training often rewards helpful-sounding answers, and a guess can look more helpful than "I don't know".
- Ambiguous prompts: the model fills gaps with assumptions.
- Long or complex contexts: details get mixed up across a long document or conversation.
- Sampling randomness: higher temperature makes less likely, sometimes wrong, tokens more probable.
What forms do hallucinations take?
| Type | Example | Risk |
|---|---|---|
| Invented facts | A wrong founding year or a non-existent rule | Misinformation |
| Fake citations | A realistic-looking paper, URL or court case that doesn't exist | Serious in legal, academic and medical use |
| Fabricated numbers | Precise-sounding statistics with no source | Bad decisions |
| Unfaithful summaries | A summary that adds a claim not in the document | Hard to notice |
| Code hallucinations | Calling a library function that doesn't exist | Broken builds; security issues if a fake package name is installed |
| Reasoning errors | Correct steps leading to a wrong conclusion | Subtle mistakes in analysis |
Why are hallucinations so convincing?
Because fluency and accuracy are produced by the same mechanism. The model writes a fake citation in exactly the format of a real one, with the same confident tone it uses for correct answers. People tend to trust well-written text, which makes hallucinations more dangerous than obvious errors. That is why systems, not just prompts, must catch them.
How do you reduce hallucinations?
1. Ground answers in sources (RAG). Retrieve relevant documents and instruct the model to answer only from them. See what is RAG and embeddings explained.
2. Require citations and verify them. Ask for the passage or document ID behind each claim, then check in code that the passage exists and supports the claim.
3. Allow "I don't know". Say explicitly that it is acceptable not to answer when the context doesn't contain the information.
4. Use structured outputs and validation. If the answer must be a valid product ID, date or amount, validate it against your database rather than trusting the text.
5. Lower temperature for factual tasks. Less randomness means fewer improbable tokens.
6. Use tools for exact work. Let the model call a calculator, a database query or a search API instead of recalling numbers from memory. See LLM tool use.
7. Add a verification step. A second pass compares the draft with the sources and flags unsupported sentences.
8. Keep a human in the loop for high-stakes outputs: legal, medical, financial and anything sent to customers without review.
How effective is each method?
A qualitative mapping to the six causes listed earlier (missing, outdated, pressure, ambiguity, long context, randomness), not a benchmark. Measure on your own test set.
The pattern is clear: system-level fixes (retrieval, verification, tools) address more causes than wording tweaks. Prompting still matters, but it can't supply knowledge the model doesn't have.
How do you measure hallucination?
- Build a question set with known answers from your documents, including questions whose answer is not in the documents.
- Score faithfulness: is every claim supported by the retrieved context? Automated checkers and human review both help.
- Score abstention: does the system say "I don't know" when it should?
- Track over time: re-run after every change to prompts, retrieval or model.
Are hallucinations always bad?
In creative work (brainstorming names, writing fiction, generating varied examples) the same tendency to produce novel combinations is useful. The problem is hallucination where facts matter. Good product design makes the mode explicit: creative tools can be imaginative; question-answering tools should be grounded, cite sources and admit uncertainty.
Which tasks are highest risk?
| Task | Risk level | Minimum safeguard |
|---|---|---|
| Brainstorming, first drafts | Low | Human edits before use |
| Internal document Q&A | Medium | RAG with visible citations |
| Customer-facing answers | High | Grounding, validation, escalation to humans |
| Legal, medical, financial advice | Very high | Expert review of every output |
What should users do?
When you use AI assistants yourself: ask for sources and open them, be suspicious of precise numbers and named references you haven't seen elsewhere, cross-check important facts, and give the assistant the documents you want it to rely on. Treat output as a well-written first draft from a fast but unreliable colleague.
What does a hallucination-resistant design look like?
Consider an assistant that answers questions about a company's leave policy. A fragile design sends the question straight to a model and returns whatever it says. A robust design retrieves the relevant policy sections, passes them to the model with an instruction to answer only from them, asks for the section number behind each statement, checks in code that each cited section exists and contains the key terms, and falls back to "I couldn't find this in the policy; please contact HR" when retrieval returns nothing relevant. Every answer shows its source so employees can verify it in one click. None of these steps is exotic, but together they turn an unreliable chatbot into a useful tool.
Do bigger or newer models hallucinate less?
Generally, newer and larger models answer more factual questions correctly and are better at admitting uncertainty, but no current model is free of hallucination, and rates vary a lot by task, domain and language. Performance on obscure facts, recent events and low-resource languages is typically weaker. The only reliable way to know how a model behaves on your task is to measure it on your own test set, and to keep measuring after every upgrade.
What is the difference between hallucination and bias?
They are related but different problems. A hallucination is a factual error the model invents. Bias is a systematic skew in what the model says or assumes, for example associating certain jobs with one gender or describing some regions less accurately. Both come partly from training data and both need evaluation, but the fixes differ: grounding and verification address hallucinations, while bias needs diverse test cases, careful review of outputs across groups and sometimes changes to data or fine-tuning.
A useful habit is to keep separate test sets for each, so a change that reduces one problem doesn't quietly make the other worse.
How do engineers learn to build reliable LLM systems?
Reducing hallucination is an engineering skill: retrieval pipelines, evaluation sets, validation code, tool use and monitoring. Program Zero's generative AI and LLM course teaches these after transformers and fine-tuning, and LLM evaluation is part of the same phase. Read what is generative AI for the broader context.