The language model is trained to predict the next word in the most linguistically plausible way. It optimizes for fluency, not accuracy. When it doesn’t know the answer, the most linguistically plausible response is still a coherent sentence—one that sounds very convincing but is completely wrong.
Three situations that are prone to giving rise to false rumors
- Rare events: There is so little information about them in the training data that the model can only grasp their general pattern, not the details
- Questions with false premises: Asking “why did X happen” when X has never actually happened; the model tends to offer an explanation rather than refute the premise
- High accuracy requirements: numbers, dates, citations, identification codes—elements that cannot be inferred from context
Ways That Really Help
Put the data into context. Instead of asking the model what it remembers, find relevant documents and include them with your question. The model shifts from simply recalling information to reading and understanding—something it’s much better at.
Cite your sources. Each claim must specify which passage in the provided material it refers to. Any statement that does not do so is suspect.
Allow the model to say "I don't know." If the prompt does not explicitly state this, the model must respond by any means necessary.
Verify using machines. For code, run tests. For numbers, recalculate. For quotes, cross-check. Don’t let people have to manually catch errors that machines can detect.
Something that doesn't help much
The "be accurate" or "don't make things up" rule has very limited effectiveness. The model has no way of knowing when it is making things up—to it, fabricated sentences and true sentences are generated by the same mechanism.
Thảo luận