Chunking Strategies for Documents
AI Hallucinations
When AI Sounds Confident — But Is Totally Wrong
One of the biggest limitations in Generative AI today is the problem of hallucinations.
An AI hallucination happens when a model gives an output that is factually incorrect, made up, or misleading — but sounds perfectly confident.
This is especially dangerous in situations where people trust the answer blindly, like in healthcare, law, or education.
🧠 Why Do Hallucinations Happen?
LLMs don’t “know” facts like humans do. They generate responses by predicting the next word based on patterns they’ve seen during training.
They:
Don’t access the internet in real time (unless specifically built to)
Can’t verify facts
Will try to complete any prompt — even if they don’t know the answer
If unsure, a human says: “I don’t know.” A language model might say: “Here’s a confident answer... that’s wrong.”
🧪 Examples of Hallucinations
“What is the capital of Australia?”
“Sydney” (Incorrect — it’s Canberra)
“Summarize this PDF” (with no file given)
AI makes up a fake summary
“Who won the Nobel Prize in Physics 2024?”
AI gives a made-up name if trained before 2024
“List 5 books by Elon Musk”
AI invents fake book titles
🧭 Real-World Risks
Legal
AI invents fake court cases (known as hallucinated citations)
Medical
AI gives incorrect drug dosage or diagnosis
Finance
AI provides wrong investment advice
Education
AI explains scientific facts inaccurately
🔍 How to Reduce Hallucinations
RAG (Retrieval-Augmented Generation)
Use a vector DB to ground answers in real documents
Function Calling
Let AI ask tools instead of guessing (e.g., search, calculator)
Prompt Engineering
Use clearer instructions: “If unsure, say 'I don’t know.'”
Output Validation
Use tools like Guardrails AI to catch false outputs
Model Choice
Use more accurate models (e.g., GPT-4 instead of GPT-3.5)
🧠 Summary
AI hallucinations = confident but wrong answers
They’re common, especially in older or ungrounded models
Always combine LLMs with retrieval, validation, or human review in sensitive applications
Last updated