What's the difference between an LLM, RAG and an AI agent?

The short answer

An LLM (large language model) predicts text, RAG (retrieval-augmented generation) pastes looked-up pieces into its prompt, and an AI agent is a loop where the model picks each step and the software around it acts.

Animated diagram: What's the difference between an LLM, RAG and an AI agent?

LLM vs RAG vs AI agent

A large language model (LLM) is a program trained to predict the next piece of text, called a token. It’s the engine under a chatbot and under an artificial intelligence (AI) agent. When a chatbot becomes an agent, the model doesn’t change. It still only writes text, and the software around it acts.

The easiest way to keep the terms apart is to picture five layers. The LLM is the bottom. Above it sit the prompt, RAG (retrieval-augmented generation), the agent and AI automation. The stack is a teaching device, and real products mix the layers. AI automation also has no formal standard definition. Still, each layer answers a different question: what does the model read, what can it look up, and who chooses the next step?

What each layer actually does

The prompt is what the model reads: the app instructions, your question and the chat so far. RAG adds a lookup. Documents are split into pieces, and OpenAI’s default maximum for a vector store piece is 800 tokens. Each piece becomes a list of numbers called an embedding. Your question becomes numbers too, and the closest pieces get pasted into the prompt.

The top two layers differ in who chooses the path. In an agent, the model chooses each step. In AI automation, the path is fixed in advance: new email, AI summary, text message. The line can blur, because Zapier says an AI step with tools can act autonomously.

How an AI agent works in a loop

  1. The app lists the tools. The software tells the model which tools it can ask for. The model never runs anything on its own.
  2. The model makes a request. In Anthropic’s documentation, the model’s answer carries stop_reason set to tool_use, plus a JSON (JavaScript Object Notation) request.
  3. The app runs the request. The software does the work and returns the outcome to the model as a tool_result.
  4. The loop repeats or ends. The normal finish is end_turn. The exact rule is that the loop exits on any stop reason other than tool_use, including max_tokens, stop_sequence and refusal.

The result looks like a model acting by itself. In fact, the model only asks, and permissions decide what happens.

An AI agent is not a smarter model

An agent is the same kind of model inside a loop, with software that carries out its requests. That brings a catch. The model gets one token sequence, so hidden text can act like instructions. That’s prompt injection. In a proof of concept Brave disclosed on August 20, 2025, a hidden Reddit comment made Perplexity’s Comet browser take the email address from the Perplexity account page, read a one-time password from logged-in Gmail, and post both in a reply. It wasn’t an attack on real users. RAG isn’t the cure either, because OWASP (the Open Worldwide Application Security Project) says it doesn’t fully stop prompt injection.

What to do

  • Open myaccount.google.com/linkedapps, choose Agent access, and stop what you don’t use. Google says Stop using affects only the agent.
  • Before you connect an agent to an account, check what it would be allowed to do. Permissions decide what happens.
  • Remember that text an agent reads can act like instructions, even a hidden comment on a web page.
  • Ask how a tool works: does it follow a path fixed in advance, or does the model choose each step?

#aiagents#llm#rag#promptinjection#aiautomation#generativeai#aisecurity#cybersecurity

Watch it in 60 seconds