Hallucination (artificial intelligence)

Hallucination (artificial intelligence) is the generation by an artificial intelligence system of information that is inaccurate, unsupported, fabricated, or inconsistent with the available evidence, while being presented as if it were factual or reliable. The term is commonly used in discussions of generative AI, particularly large language models.

AI hallucinations can include fabricated facts, nonexistent sources, incorrect calculations, invented quotations, misleading descriptions, or false statements presented with a high degree of confidence.

Characteristics

An AI hallucination may have one or more of the following characteristics:

factual information that is incorrect;
fabricated or nonexistent sources;
invented citations or references;
incorrect attribution of statements or events;
logically inconsistent responses;
unsupported conclusions;
inaccurate descriptions of data, images, or documents;
confident presentation of information that lacks evidentiary support.

A hallucinated response may be grammatically correct and appear plausible even when its underlying information is incorrect.

Large language models

Large language models generate text by predicting sequences of tokens based on patterns learned from training data and other available inputs.

Because this process does not inherently guarantee that every generated statement corresponds to a verified fact, a model can produce plausible-sounding information that is unsupported or incorrect.

Hallucinations can occur even when a model produces fluent and coherent text.

Causes

AI hallucinations can arise from several factors, including:

incomplete or ambiguous information;
limitations in training data;
conflicting information in training data;
insufficient context;
statistical generation of plausible sequences;
model uncertainty;
errors in retrieval or external data;
limitations in reasoning or inference;
requests that require information unavailable to the system.

The precise causes vary according to the architecture, training process, data, prompting method, and application.

Types

Hallucinations can occur in different forms.

Factual hallucination

A factual hallucination occurs when an AI system provides information that is factually incorrect.

Examples include:

incorrect dates;
incorrect names;
false numerical values;
nonexistent events;
inaccurate descriptions of real entities.

Fabricated citation

A fabricated citation occurs when an AI system generates a citation, publication, URL, or other reference that does not correspond to a real or relevant source.

This can be particularly problematic in academic research, journalism, legal research, and other applications requiring verifiable sources.

False attribution

False attribution occurs when an AI system incorrectly attributes a statement, quotation, discovery, publication, or action to a person or organization.

Logical hallucination

A logical hallucination occurs when an AI system produces a conclusion that does not follow from the information or reasoning presented.

The individual statements may appear plausible while the overall conclusion is unsupported or inconsistent.

Hallucinations in generative AI

Generative artificial intelligence systems can produce hallucinations across different forms of generated content.

They may occur in:

text;
images;
audio;
video;
computer code;
structured data.

The specific characteristics of hallucination depend on the type of generative system.

Retrieval-augmented generation

Retrieval-augmented generation (RAG) is an approach in which a generative AI system retrieves information from external sources before generating a response.

RAG can reduce some forms of hallucination by providing the model with relevant source material. However, retrieval does not guarantee factual accuracy.

Hallucinations can still occur if:

the wrong information is retrieved;
relevant information is missing;
the retrieved information is ambiguous;
the model misinterprets the retrieved material;
the generated response goes beyond the available evidence.

Grounding

Grounding refers to connecting an AI system's output to identifiable information, evidence, data, or external sources.

Grounding techniques may include:

retrieval from trusted databases;
document-based generation;
citations;
structured data;
tool-assisted verification;
human review.

Grounding can improve factual reliability but does not completely eliminate hallucinations.

Detection

AI hallucinations can be detected through verification and evaluation methods.

These may include:

checking claims against authoritative sources;
verifying citations;
comparing generated information with source documents;
automated fact-checking;
consistency checks;
human review;
evaluation against benchmark datasets.

The appropriate verification method depends on the application and the potential consequences of errors.

Mitigation

Methods used to reduce AI hallucinations may include:

improving training data;
providing more relevant context;
using retrieval systems;
requiring source citations;
restricting generation to verified datasets;
using structured outputs;
applying automated validation;
incorporating human oversight;
instructing systems to acknowledge uncertainty;
using external tools for calculations and verification.

No single technique completely eliminates hallucinations.

Human verification

Human verification can be particularly important when AI-generated information is used in high-stakes contexts.

Examples include:

medical information;
legal analysis;
financial decisions;
academic research;
journalism;
government decision-making.

The level of review required depends on the potential consequences of inaccurate information.

Evaluation

AI systems can be evaluated for hallucination rates using task-specific benchmarks and datasets.

Evaluation may measure:

factual accuracy;
source attribution;
consistency;
completeness;
faithfulness to provided documents;
citation accuracy.

Different evaluation methods may produce different results because hallucination is not defined identically across all AI applications.

Hallucination and confidence

An AI system may produce an incorrect statement with a high degree of linguistic confidence.

Fluent language or a confident tone does not necessarily indicate that the underlying information has been verified.

Consequently, users should distinguish between:

fluency;
confidence;
factual accuracy;
evidentiary support.

Hallucination and misinformation

AI hallucination and misinformation are related but distinct concepts.

AI hallucination generally refers to an AI system generating information that is inaccurate or unsupported.

Misinformation generally refers to false or inaccurate information, regardless of whether it was produced by an AI system.

An AI hallucination can therefore become a source of misinformation when it is distributed to other users.

Hallucination and disinformation

Disinformation refers to false or misleading information that is deliberately created or distributed to deceive.

An AI hallucination is not necessarily intentional because the system may generate inaccurate information without an intention to deceive.

However, humans or organizations may deliberately use AI-generated hallucinations as part of a disinformation campaign.

Impact

AI hallucinations can have consequences including:

inaccurate decision-making;
reputational damage;
financial losses;
dissemination of false information;
incorrect academic or professional work;
legal or regulatory problems;
reduced trust in AI systems.

The potential impact depends on the application and the importance of the generated information.

Importance

AI hallucination is a significant reliability challenge for generative artificial intelligence systems. The ability of AI models to produce fluent and plausible outputs does not guarantee factual accuracy. Reducing hallucinations therefore requires appropriate system design, reliable information sources, verification, evaluation, and human oversight where necessary.