LLM Hallucination Research

What Is Hallucination

Hallucination is when LLMs generate fabricated content not grounded in training data or context. It’s one of the most studied problems in current LLM research.

Causes

• Training data noise and inconsistency
• Maximum likelihood training doesn’t directly optimize factuality
• Randomness introduced by decoding strategies like top-p sampling

Mitigation

RAG. Retrieval-Augmented Generation is currently the most effective mitigation — see RAG Retrieval-Augmented Generation in Practice.

Self-Reflection. Have the model generate an answer, then self-check for factual errors.

Factuality Evaluation. Use NLI models or specialized benchmarks to measure hallucination rates.