Online, level 13
💬 How Chatbots Write
How does a chatbot come up with its answers?
- How does a chatbot come up with its answers? Many chatbots today are built on large language models, which are computer programs trained on enormous amounts of text. During training, a model is shown vast collections of writing and gradually adjusts a huge number of internal settings, so that it becomes better and better at one particular task: predicting what comes next. The National Institute of Standards and Technology describes the basic mechanism plainly, noting that large language models predict the next token or word in a sentence or phrase. A token is a small chunk of text, often a whole word or a piece of one, so a long word may be split into two or three tokens while a short word is usually just one. After predicting a single token, the model adds it to the text and then predicts the following one, building an entire answer gradually, step by step, from many small predictions.
- Because a model generates outputs that approximate the statistical patterns of its training data, as NIST puts it, the result is usually fluent and frequently accurate, but not always. NIST calls a confidently stated but false answer a confabulation, and it notes that people often call such answers hallucinations. A model can produce a made-up quotation, an invented source or an incorrect date in the same smooth, assured tone that it uses for true statements, and it may even supply logical-sounding steps that lead to an answer that is wrong. Common Sense Education also points out that chatbots are designed to sound confident and certain, a feature that can make what they say feel more credible than it actually deserves. Fluent writing, in other words, is not the same thing as accurate information, and a wrong answer can read just as well as a right one, so you cannot tell them apart by how they sound.
- So what should you do? The sensible approach is to treat a chatbot's answer as a starting point rather than a final authority. Check important facts against reliable sources, such as a textbook, a library or the official website of an agency, using the same lateral reading habits that professional fact-checkers use. Be especially careful with numbers, names, quotations and anything involving health or safety. Just as important, never share personal information with a chatbot, such as your full name, address, school, passwords or photographs, because what you type may be saved, and NIST notes that these systems can sometimes reveal information that was included in their training data. A chatbot is a tool for generating text, and like any powerful tool, it is most useful in careful hands, ideally with a trusted adult nearby who can help you decide when to use it and when to look somewhere else.
Try it at home: Ask a chatbot together about something you already know well, check its answer against a book or an official site, and talk about anything it got wrong.
Questions and answers
How Chatbots Write
- What are large language models trained on? Enormous amounts of text
- 🔮 According to NIST, what do large language models predict? The next token or word (NIST says large language models predict the next token or word in a sentence.)
- A small chunk of text, often a whole word or a piece of one, is called a… token (A token is a small chunk of text, such as a word or part of a word.)
- 🧱 How does a model build a whole answer? One predicted token after another
- What does NIST call a confidently stated but false answer from an AI system? A confabulation (NIST's term is confabulation; people often call it a hallucination.)
- 🎭 A chatbot writes a smooth, confident answer. What does that tell you about accuracy? Nothing by itself; fluent isn't accurate (Models use the same assured tone for wrong statements as for true ones.)