How does an AI like ChatGPT really work?

The short answer

An LLM (large language model) like the one behind ChatGPT doesn't pull its answer from a database. It splits your text into tokens and predicts the likely next part, one token at a time.

Animated diagram: How does an AI like ChatGPT really work?

Large language model (LLM)

An LLM (large language model) is the engine that produces the text of assistants like ChatGPT. Google compares it to a giant autocomplete. Starting from what you write, it predicts what comes next, and it can string together thousands of tokens. A token is a piece of text. It can be a word, part of a word, or even a single character.

The important point is where the answer comes from. The model itself doesn’t look it up in a database of facts. It calculates it, piece by piece, with probabilities. Some products add a web search around the model. But the prediction calculation itself checks nothing. It produces a likely continuation.

What an LLM actually does

First, your question is split into tokens. Next, each token is turned into numbers, and the model works only with those numbers. Then comes attention, also called self-attention. For each token, the model asks itself a question: how much does each of the other tokens influence the meaning of this one?

In French, the word “avocat” means both “avocado” and “lawyer.” Attention is what lets it become a fruit in “une salade à l’avocat” (an avocado salad), or a lawyer in “mon avocat a appelé” (my lawyer called). The word is the same. It’s the tokens around it that change its meaning in the calculation. These examples are illustrations. The actual split into tokens depends on each model.

How an answer is built token by token

  1. Your question becomes tokens. The text is split into words or pieces of words. Those pieces are then turned into numbers.
  2. Attention weighs the context. For each token, the model measures the influence of all the other tokens.
  3. The model calculates probabilities. It gives a probability to every possible next token. Then it picks one.
  4. The token is added, and it starts again. The chosen token joins the text. The whole calculation then runs again for the next token.

The result is likely text, not necessarily true text. Google says it clearly: LLMs hallucinate, and their predictions often contain mistakes. A September 2025 study by OpenAI explains part of the problem. Training and evaluation reward guessing over admitting uncertainty. Without the right information, the model guesses anyway. That’s what a hallucination is.

A confident tone is not proof

Many people think an answer given with confidence must be right. That’s wrong, because nothing in this calculation checks the facts. The Air Canada case shows it. In November 2022, the airline’s chatbot told a customer he could claim a bereavement fare after the fact, within 90 days. That was false. The airline argued that the chatbot was responsible for its own answers. A tribunal in British Columbia, Canada, rejected that defense. It held the airline responsible.

What you can do

  • For legal, health, or money questions, check the official source before you act. For a question about your lease, for example, read the text of the law on Fedlex.
  • Don’t take a confident tone as proof. A clear, assured answer can be a hallucination.
  • Ask yourself where the information comes from. On its own, the model predicts text. It doesn’t consult a database of facts.
  • If your company uses a chatbot, keep in mind that it is still responsible for what the chatbot tells its customers.

#GenerativeAI#LLM#ChatGPT#Hallucinations#ArtificialIntelligence#CyberSecurity#InfoSec#AI