Why ChatGPT Makes Mistakes: Fake Citations

Why ChatGPT makes mistakes is easier to see in one very specific failure mode: fake citations and fake-looking source links. The strange part is not that the model says nonsense. The strange part is that the nonsense often looks formatted, scholarly, and calm enough to pass a first glance. That is the useful curiosity gap: why would a machine that can explain a transformer architecture also invent a paper title, a DOI, or a URL that never existed?

TL;DR

ChatGPT makes fake citations because a language model is trained to continue patterns, not to maintain a live library catalog in its head. Citations have strong surface patterns: author names, years, journal titles, volumes, pages, and URLs. When the model is uncertain, standard training and evaluation can still reward a plausible guess unless the system is explicitly pushed toward abstention, verification, or tool-grounded retrieval.

The short answer: fake citations happen when fluent pattern completion outruns grounded checking. A model has learned what references usually look like, and it may have partial memory of real authors, journals, topics, or URLs. If it is asked for a source and does not have enough grounded information, it can stitch those fragments into something that feels source-shaped without being source-real. OpenAI's own hallucination explainer frames the broader mechanism as confident falsehoods encouraged by systems that often reward guessing more than saying "I don't know" (OpenAI, 2025).

Why ChatGPT Makes Mistakes When a Citation Looks Right

A citation is a tiny costume of authority. It has names, punctuation, years, journal titles, volume numbers, page ranges, and sometimes a DOI or URL. That costume is easy for a language model to imitate because it is made of recurring text patterns. The hard part is whether the dressed-up string corresponds to an actual record in Crossref, PubMed, a publisher site, or a library database.

That gap showed up clearly in a Scientific Reports study of ChatGPT-generated bibliography entries. The authors asked GPT-3.5 and GPT-4 to generate short papers with scholarly citations, then checked 636 references. Their methods section says they evaluated fabricated citations, errors in non-fabricated citations, APA-format adherence, and hyperlink characteristics (Walters & Wilder, 2023). The important lesson is not "ChatGPT bad." It is smaller and more practical: a reference can be syntactically convincing while still failing the one test that matters, which is whether the source exists and says what the model claims.

Why ChatGPT Makes Mistakes From Next-Token Prediction

The simplest mental model is this: ChatGPT is very good at predicting the next useful piece of text in context. OpenAI's GPT-4 technical report describes GPT-4 as a transformer model pre-trained to predict the next token in a document, and the same report warns that GPT-4 can still hallucinate facts and make reasoning errors (OpenAI GPT-4 Technical Report). Those two sentences belong together. The capability and the failure mode share a root.

Next-token prediction does not mean "random autocomplete." It can produce deep reasoning, careful summaries, and useful explanations because the model has compressed enormous patterns of language and knowledge. But when the task asks for an exact external identifier, such as a DOI, page range, obscure title, or URL, pattern fluency is not enough. Exactness requires a lookup or a memory trace precise enough to survive verification.

This is why fake citations are such a good microscope. A model can know that a paper about "curiosity and memory" should probably include author names, a psychology journal, a year around the 2010s, and a scientific title. That gets you close to a plausible reference. It does not get you to a real reference. The answer has the rhythm of knowledge without the closure of verification.

Why ChatGPT Makes Mistakes When Guessing Is Rewarded

OpenAI's 2025 hallucination explainer makes a second point that matters for everyday users: many evaluation setups reward a wrong guess more than an honest abstention. If a model is graded only on whether it lands on the exact answer, a guess has some chance of being right, while "I don't know" gets no credit. OpenAI argues that this creates pressure toward confident answers when uncertainty would be the better behavior (OpenAI, "Why language models hallucinate").

That incentive structure helps explain the emotional texture of a hallucinated citation. It often does not look hesitant. It looks finished. That finished feeling is what traps people. Human readers are trained to treat formatted references as evidence that some checking has already happened. In a generated answer, formatting may only prove that the model knows the genre.

This is the same curiosity pattern MillionWhys is built around: the satisfying part is not the confident sentence; it is the closure when the sentence is checked against the world. A good AI answer should invite that last step. It should make you more curious about the source, not less.

Why ChatGPT Makes Mistakes More Often With URLs

URLs look precise, but they are also text patterns. A model can learn that a university article might live under /news/, that a publisher DOI page has a particular shape, or that a government report probably sits on a familiar domain. None of that guarantees the path exists. A URL is not true because it resembles a URL. It is true because a server returns the right page and the page supports the claim.

This is why source verification should be boringly mechanical. Open the page. Check the title. Check the author or institution. Check the date. Search inside the page for the claim. If the model gives a DOI, resolve it. If it gives a quote, find the quote in the source. The more polished the answer looks, the more valuable this small ritual becomes.

A 2026 arXiv preprint makes the scale problem concrete by auditing 111 million references across major research repositories. The authors report a conservative estimate of 146,932 hallucinated citations in 2025 alone, especially in areas with rapid AI uptake (Zhao et al., 2026). Because that paper is a preprint, the exact claims should be read with normal preprint caution. But the object it studies is highly checkable: whether a cited work exists.

Why ChatGPT Makes Mistakes Even When It Is Useful

The wrong takeaway is "never use ChatGPT." The better takeaway is that language models are strongest when the task benefits from fluent synthesis, and weakest when a tiny external identifier must be exactly right. Asking for an explanation of why fake citations happen is a good use. Asking it to invent a bibliography and trusting the result without opening the sources is a bad use.

There is a nice human parallel here. People also remember the shape of a source before they remember its exact details. You may know that you read a study about curiosity and memory, remember the rough finding, and forget the title. The difference is that a careful person says, "I need to look that up." A language model can sound as if it already did.

So the practical rule is simple: use ChatGPT to create questions, draft search terms, explain mechanisms, and compare interpretations. Do not use it as the final authority for citations unless it is connected to retrieval and you verify the retrieved source yourself. The curiosity-first version of AI literacy is not cynicism. It is learning where the gap is.

What People Usually Miss

People often describe hallucinations as if the model is "lying." That metaphor is emotionally satisfying and technically muddy. Lying requires knowing the truth and choosing to hide it. A fake citation is usually stranger: the model has learned the surface of truth-telling and may produce it without the grounded object underneath.

The other missed point is that hallucinations are not just random mistakes. They are predictable enough to design around. Ask for uncertainty. Ask for source titles and links separately. Verify URLs. Prefer retrieval-backed answers when exact citations matter. Treat any obscure, perfect-looking reference as a hypothesis until it survives a real lookup.

That turns the mistake into an "aha" moment instead of a culture-war take. The model is useful because it can compress patterns. It fails because compression is not the same as contact with reality. The closure comes when the pattern touches a source that actually exists.

Related Videos

• Computerphile: Ch(e)at GPT?
• How to Solve the Biggest Problem with AI

FAQ

Why ChatGPT makes mistakes even when it sounds confident?

Because confidence in the wording is not the same as verification. The model can generate the most likely answer-shaped text while still being wrong about an exact fact, citation, date, or URL.

Are ChatGPT citations always fake?

No. Many are real, and newer systems can use retrieval tools to ground answers. The risk is that fake citations can look real, so the only safe rule is to open and verify each source before relying on it.

Why does ChatGPT invent URLs?

URLs have patterns. A model may generate a path that resembles a real site structure without checking whether that page exists. A real URL must return the right page and support the claim.

How can I reduce ChatGPT mistakes?

Ask for uncertainty, ask the model to separate claims from sources, use retrieval or browsing when available, and verify important links yourself. For high-stakes work, use human review and primary sources.

What does this have to do with AIgneous Million Whys?

MillionWhys treats AI as a spark for curiosity, not a permission slip to stop checking. The useful loop is question, answer, closure, and a better next question. AI is strongest when it helps you notice the gap and then close it against the real world.

Sources

OpenAI: Why language models hallucinate

OpenAI: GPT-4 Technical Report

Scientific Reports: Fabrication and errors in bibliographic citations generated by ChatGPT

arXiv: LLM hallucinations in the wild: Large-scale evidence from non-existent citations