What is AI hallucination and why does it happen?

Ask a language model for sources and it may write the title of a paper that does not exist, a legal clause that was never passed or a product feature that was never released. The text is fluent and the tone confident, but the information is wrong. That is hallucination. There is no error message, which is what makes it hard to spot.

In 2023 a US lawyer was sanctioned for filing a brief that cited court decisions ChatGPT had invented, one of the best known examples of the risk.

Large language models do not look records up in a database; they generate the most likely next words. Training teaches them to complete text plausibly, and stopping at the edge of their knowledge to say "I don't know" does not come naturally out of that training. When a question touches something rare in the training data or something that happened after training, the model still builds a fluent answer.

Questions with false premises raise the risk. Ask "how did company X's 2019 lawsuit end?" about a lawsuit that never existed and the model is likely to invent an outcome.

• Feed the model the right information: putting the source documents into the prompt with RAG is one of the most effective methods.
• Ask it to show which document each claim comes from, so claims can be checked.
• Allow the model to say when it does not know, and state that rule clearly in the system prompt.
• In critical areas such as law, health, finance and pricing, verify output with guardrail checks or human approval.

Lowering temperature makes answers consistent but does not fix wrong information.

Not with today's technology. Newer models hallucinate less, but the rate is not zero. Systems that talk to customers or make decisions are therefore designed on the assumption that hallucinations will happen: the model answers only from approved information and hands the conversation to a person when it is unsure.