View source for Hallucination (artificial intelligence)

'''[[Hallucination (artificial intelligence)]]''' is the generation by an [[Artificial intelligence|artificial intelligence]] system of information that is inaccurate, unsupported, fabricated, or inconsistent with the available evidence, while being presented as if it were factual or reliable. The term is commonly used in discussions of [[Generative artificial intelligence|generative AI]], particularly [[Large language model|large language models]].

AI hallucinations can include fabricated facts, nonexistent sources, incorrect calculations, invented quotations, misleading descriptions, or false statements presented with a high degree of confidence.

==Characteristics==

An AI hallucination may have one or more of the following characteristics:

* factual information that is incorrect;
* fabricated or nonexistent sources;
* invented citations or references;
* incorrect attribution of statements or events;
* logically inconsistent responses;
* unsupported conclusions;
* inaccurate descriptions of data, images, or documents;
* confident presentation of information that lacks evidentiary support.

A hallucinated response may be grammatically correct and appear plausible even when its underlying information is incorrect.

==Large language models==

[[Large language model|Large language models]] generate text by predicting sequences of tokens based on patterns learned from training data and other available inputs.

Because this process does not inherently guarantee that every generated statement corresponds to a verified fact, a model can produce plausible-sounding information that is unsupported or incorrect.

Hallucinations can occur even when a model produces fluent and coherent text.

==Causes==

AI hallucinations can arise from several factors, including:

* incomplete or ambiguous information;
* limitations in training data;
* conflicting information in training data;
* insufficient context;
* statistical generation of plausible sequences;
* model uncertainty;
* errors in retrieval or external data;
* limitations in reasoning or inference;
* requests that require information unavailable to the system.

The precise causes vary according to the architecture, training process, data, prompting method, and application.

==Types==

Hallucinations can occur in different forms.

===Factual hallucination===

A '''factual hallucination''' occurs when an AI system provides information that is factually incorrect.

Examples include:

* incorrect dates;
* incorrect names;
* false numerical values;
* nonexistent events;
* inaccurate descriptions of real entities.

===Fabricated citation===

A '''fabricated citation''' occurs when an AI system generates a citation, publication, URL, or other reference that does not correspond to a real or relevant source.

This can be particularly problematic in academic research, journalism, legal research, and other applications requiring verifiable sources.

===False attribution===

'''False attribution''' occurs when an AI system incorrectly attributes a statement, quotation, discovery, publication, or action to a person or organization.

===Logical hallucination===

A '''logical hallucination''' occurs when an AI system produces a conclusion that does not follow from the information or reasoning presented.

The individual statements may appear plausible while the overall conclusion is unsupported or inconsistent.

==Hallucinations in generative AI==

[[Generative artificial intelligence]] systems can produce hallucinations across different forms of generated content.

They may occur in:

* text;
* images;
* audio;
* video;
* computer code;
* structured data.

The specific characteristics of hallucination depend on the type of generative system.

==Retrieval-augmented generation==

[[Retrieval-augmented generation]] (RAG) is an approach in which a generative AI system retrieves information from external sources before generating a response.

RAG can reduce some forms of hallucination by providing the model with relevant source material. However, retrieval does not guarantee factual accuracy.

Hallucinations can still occur if:

* the wrong information is retrieved;
* relevant information is missing;
* the retrieved information is ambiguous;
* the model misinterprets the retrieved material;
* the generated response goes beyond the available evidence.

==Grounding==

'''Grounding''' refers to connecting an AI system's output to identifiable information, evidence, data, or external sources.

Grounding techniques may include:

* retrieval from trusted databases;
* document-based generation;
* citations;
* structured data;
* tool-assisted verification;
* human review.

Grounding can improve factual reliability but does not completely eliminate hallucinations.

==Detection==

AI hallucinations can be detected through verification and evaluation methods.

These may include:

* checking claims against authoritative sources;
* verifying citations;
* comparing generated information with source documents;
* automated fact-checking;
* consistency checks;
* human review;
* evaluation against benchmark datasets.

The appropriate verification method depends on the application and the potential consequences of errors.

==Mitigation==

Methods used to reduce AI hallucinations may include:

* improving training data;
* providing more relevant context;
* using retrieval systems;
* requiring source citations;
* restricting generation to verified datasets;
* using structured outputs;
* applying automated validation;
* incorporating human oversight;
* instructing systems to acknowledge uncertainty;
* using external tools for calculations and verification.

No single technique completely eliminates hallucinations.

==Human verification==

Human verification can be particularly important when AI-generated information is used in high-stakes contexts.

Examples include:

* medical information;
* legal analysis;
* financial decisions;
* academic research;
* journalism;
* government decision-making.

The level of review required depends on the potential consequences of inaccurate information.

==Evaluation==

AI systems can be evaluated for hallucination rates using task-specific benchmarks and datasets.

Evaluation may measure:

* factual accuracy;
* source attribution;
* consistency;
* completeness;
* faithfulness to provided documents;
* citation accuracy.

Different evaluation methods may produce different results because hallucination is not defined identically across all AI applications.

==Hallucination and confidence==

An AI system may produce an incorrect statement with a high degree of linguistic confidence.

Fluent language or a confident tone does not necessarily indicate that the underlying information has been verified.

Consequently, users should distinguish between:

* fluency;
* confidence;
* factual accuracy;
* evidentiary support.

==Hallucination and misinformation==

AI hallucination and [[Misinformation|misinformation]] are related but distinct concepts.

'''AI hallucination''' generally refers to an AI system generating information that is inaccurate or unsupported.

'''Misinformation''' generally refers to false or inaccurate information, regardless of whether it was produced by an AI system.

An AI hallucination can therefore become a source of misinformation when it is distributed to other users.

==Hallucination and disinformation==

[[Disinformation]] refers to false or misleading information that is deliberately created or distributed to deceive.

An AI hallucination is not necessarily intentional because the system may generate inaccurate information without an intention to deceive.

However, humans or organizations may deliberately use AI-generated hallucinations as part of a disinformation campaign.

==Impact==

AI hallucinations can have consequences including:

* inaccurate decision-making;
* reputational damage;
* financial losses;
* dissemination of false information;
* incorrect academic or professional work;
* legal or regulatory problems;
* reduced trust in AI systems.

The potential impact depends on the application and the importance of the generated information.

==Importance==

AI hallucination is a significant reliability challenge for generative artificial intelligence systems. The ability of AI models to produce fluent and plausible outputs does not guarantee factual accuracy. Reducing hallucinations therefore requires appropriate system design, reliable information sources, verification, evaluation, and human oversight where necessary.

==See also==

* [[Artificial intelligence]]
* [[Generative artificial intelligence]]
* [[Large language model]]
* [[Natural language processing]]
* [[Machine learning]]
* [[Knowledge representation]]
* [[Retrieval-augmented generation]]
* [[Misinformation]]
* [[Disinformation]]
* [[Fact-checking]]
* [[Artificial intelligence safety]]

[[Category:Artificial intelligence]]
[[Category:Generative artificial intelligence]]
[[Category:Large language models]]
[[Category:Natural language processing]]
[[Category:Machine learning]]
[[Category:Information quality]]
```