Chunking Strategies for Documents

AI Hallucinations

When AI Sounds Confident — But Is Totally Wrong

One of the biggest limitations in Generative AI today is the problem of hallucinations.

An AI hallucination happens when a model gives an output that is factually incorrect, made up, or misleading — but sounds perfectly confident.

This is especially dangerous in situations where people trust the answer blindly, like in healthcare, law, or education.

---

🧠 Why Do Hallucinations Happen?

LLMs don’t “know” facts like humans do. They generate responses by predicting the next word based on patterns they’ve seen during training.

They:

• Don’t access the internet in real time (unless specifically built to)

• Can’t verify facts

• Will try to complete any prompt — even if they don’t know the answer

If unsure, a human says: “I don’t know.” A language model might say: “Here’s a confident answer... that’s wrong.”

---

🧪 Examples of Hallucinations

Input Prompt	Hallucinated Output
“What is the capital of Australia?”	“Sydney” (Incorrect — it’s Canberra)
“Summarize this PDF” (with no file given)	AI makes up a fake summary
“Who won the Nobel Prize in Physics 2024?”	AI gives a made-up name if trained before 2024
“List 5 books by Elon Musk”	AI invents fake book titles

---

🧭 Real-World Risks

Domain	Example Problem
Legal	AI invents fake court cases (known as hallucinated citations)
Medical	AI gives incorrect drug dosage or diagnosis
Finance	AI provides wrong investment advice
Education	AI explains scientific facts inaccurately

---

🔍 How to Reduce Hallucinations

Strategy	Description
RAG (Retrieval-Augmented Generation)	Use a vector DB to ground answers in real documents
Function Calling	Let AI ask tools instead of guessing (e.g., search, calculator)
Prompt Engineering	Use clearer instructions: “If unsure, say 'I don’t know.'”
Output Validation	Use tools like Guardrails AI to catch false outputs
Model Choice	Use more accurate models (e.g., GPT-4 instead of GPT-3.5)

---

🧠 Summary

• AI hallucinations = confident but wrong answers

• They’re common, especially in older or ungrounded models

• Always combine LLMs with retrieval, validation, or human review in sensitive applications

---