The mechanism
Large language models predict tokens. When asked something they don’t know, they don’t have a ‘say I don’t know’ instinct — they just keep predicting plausible-looking tokens.
Why it sounds confident
Plausible text is what they optimize for. Confident-sounding wrong answers ARE perfectly-optimized output from the model’s perspective. The training reward shaped this.
The fix
Retrieval augmentation (give the model real sources to cite). Asking ‘are you sure?’ doesn’t help — it just generates new plausible text. Tools like web search grounding actually move the needle.