How does ChatGPT actually write its answers?

The short answer

ChatGPT writes its answer one small piece at a time. It predicts a likely next token, adds it and repeats, and nothing in that loop checks whether the result is true.

Animated diagram: How does ChatGPT actually write its answers?

Next-token prediction

ChatGPT, Gemini and Claude are chatbots built on an LLM (large language model). An LLM writes its answer one small piece at a time, and each piece is a prediction. Google describes these models as essentially autocomplete mechanisms. The small piece is called a token, and the method is called next-token prediction.

A model that only predicts text is not yet a helpful assistant. So after its first training, it gets a second round called fine-tuning. People write example chats that show good answers. Then people rank the model’s own answers, and the model is trained on those rankings. That step is called RLHF (reinforcement learning from human feedback). It’s what makes the model answer like an assistant.

What next-token prediction actually does

First, your words are cut into tokens. A token can be a whole word, part of a word or a single character. The exact split depends on each model’s tokenizer. Each token is turned into a number, because the model only works with numbers.

Long before you chat, the model trained on huge amounts of text by guessing the next token. Every guess was compared with the real next token in that text. A method called gradient descent then nudged the model’s internal numbers, its parameters, toward the right answer. GPT-3 had 175 billion parameters. Notice what the training compares against: the text, not the truth.

How the model writes a reply

  1. Your message becomes tokens. Your question is split into tokens and turned into numbers.
  2. Attention weighs earlier tokens. Every token weighs the earlier ones, a step called attention. Chatbot models only look backwards, and this is how the model works out what a word like “it” refers to once enough of the sentence has appeared.
  3. Every possible token gets scored. The model gives each possible next token a probability, a score for how likely it is to come next.
  4. Pick, add and repeat. The app picks the highest-probability token, or a random one above a threshold. It adds that token to the reply and runs the whole loop again for the next one.

That loop is the whole writing process. Here’s the catch: no step in it checks whether the reply is true. A chatbot predicts what sounds right. It doesn’t check what’s true. When a prediction sounds right but is false, it’s called a hallucination.

A chatbot is not a search engine

Many people think a chatbot looks up its answers. The model itself doesn’t look facts up. ChatGPT, Gemini and Claude can add web search, but even then the reply is still predicted token by token. In 2023, in Mata v. Avianca, lawyers in a New York federal court cited court decisions that ChatGPT had invented. On June 22, 2023, the court fined the law firm and the attorneys $5,000. Anthropic’s own documentation says techniques to reduce hallucinations “don’t eliminate them entirely” and advises: “Always validate critical information, especially for high-stakes decisions.”

What to do

  • Before acting on money, health or legal answers, ask the chatbot for its sources.
  • Open each source yourself and find the exact sentence that backs the claim.
  • Tell the chatbot it may answer “I don’t know” instead of guessing, and ask it to quote the passage it relies on.
  • Treat a confident tone as style, not proof. The reply was predicted, not checked.

#HowAIWorks#LargeLanguageModels#LLM#AIHallucination#ChatGPT#AIExplained#GenerativeAI#MachineLearning#ArtificialIntelligence

Watch it in 60 seconds