---
title: "What is an LLM?"
summary: "A large language model predicts the next chunk of text. It is not a database and it does not 'know' your app."
track: "The model"
day: 1
minutes: 18
author: "Akash Panchal"
url: https://ai-sdk-patterns.dev/learn/fundamentals/day-1/what-is-llm
dateModified: 2026-08-29
---

# What is an LLM?

A large language model predicts the next chunk of text. It is not a database and it does not 'know' your app.

*The model · Day 1: How AI apps work · ~18 min. Written by [Akash Panchal](https://github.com/akashp1712).*

Canonical: https://ai-sdk-patterns.dev/learn/fundamentals/day-1/what-is-llm

## After this topic

You will be able to explain, in your own words, what happens between a user pressing send and a reply appearing on screen — including what the model can see, what it cannot, and what you are paying for.

If someone asks “does the AI know our customers?”, you will have a precise answer: only if you put that data in the request.

## Start from zero

If you have never built with AI before, start here: an LLM is just a program that continues text.

An LLM is a program that writes text. That is the whole product, at the lowest level. You give it some text. It writes more text. ChatGPT, Claude, Gemini, and the model behind this course all do that one job.

The name is just a description. Large: the file is huge, trained on a huge amount of writing. Language: it works on words, code, and other text. Model: it is a learned function, not a database of facts and not a set of if/else rules you wrote.

You do not run this program on your laptop in a product. A company (OpenAI, Anthropic, Google, …) hosts it. Your server sends them the text so far. They send back the continuation. The next two topics are how that request looks, and why we wrap it in the AI SDK instead of raw HTTP.

## The only operation: next token

> A token is a small chunk of text — sometimes a word, sometimes part of a word, sometimes punctuation. An LLM’s only job at runtime is: given the tokens so far, pick the next one.

When you type a question into a chat box, it feels like the model “answers”. Internally it does not retrieve a stored answer. It looks at every token you sent — the question, any system instructions, any previous turns — and asks one question: what token is most likely to come next?

Then it asks again, with that new token now part of the input. Then again. The reply you read is that sequence, decoded back into characters. It stops when it emits a stop token, or when you cap the length.

This is why a long answer takes longer than a short one: the model cannot skip ahead. Token two cannot be chosen until token one exists. Streaming (Day 3) is just showing you those tokens as they are born, instead of waiting for the last one.

It is also why the model can sound sure and still be wrong. “Likely English” is not the same as “true”. Your job as an engineer is to constrain the guessing — with the prompt, with tools, with a schema, with checks — so the likely text is also useful text.

- Prompt — everything you send in: instructions, the user question, chat history, maybe retrieved documents
- Token — the unit the model reads and writes (not the same as a word)
- Completion — the tokens it writes back; decoded, that is the reply
- Context window — the maximum number of tokens it can see at once. Overflow and it forgets the beginning. We go deep on this in Context.

## Tokens, slowly

English is not what the model sees. A tokenizer splits your string into a list of integers. Each integer is a token id. “Hello” might be one token. “TypeScript” might be one or two. A space, a newline, and `{` are tokens too. Code is often more tokens than the same idea in prose, which is why dumping a whole file into a prompt is expensive.

You will never write the loop below in an app. It exists to make the idea physical: the model extends a string one chunk at a time. The provider runs the real loop on GPUs. You send a prompt and read a completion.

Billing is almost always: tokens in plus tokens out, times the model’s price. A long system prompt costs you on every call, even if the user only said “hi”. A long reply costs more than “OK”. That is not a footnote. It is the unit of every production decision later — how much history to keep, whether to retrieve documents, when to summarize.

### tokens-idea.ts

```ts
// Illustration only — not how you call a model.
// The model always asks: "what token comes next?"

let soFar = "The cat";
soFar += " sat";   // next token
soFar += " on";    // next token
soFar += " the";   // next token
soFar += " mat";   // next token
// soFar === "The cat sat on the mat"
```

- **This is the idea, not an API.** There is no npm function called nextToken in your app. generateText and streamText are the APIs. Underneath, the provider is doing this loop.
- **Tokens are not words.** “TypeScript” might be one token or two, depending on the tokenizer. That is why token count ≠ word count, and why cost calculators ask for tokens.
- **The past is the only input.** Each new token is chosen from everything already in the string — prompt plus tokens written so far. The model has no separate scratchpad unless you put one in the text (a later pattern: tool results, retrieved docs).

## A mental model that survives contact with production

Treat the model as a very fast autocomplete that has read a lot of public text. It is extraordinarily good at continuing the kind of writing it has seen: answers, code, emails, JSON that looks like JSON. It is not looking at your Postgres. It did not attend yesterday’s standup. It cannot hit an HTTP API unless you give it a tool and run that tool yourself.

Statelessness follows from the same idea. Each HTTP call is a fresh one. If you want a conversation, you send the previous messages again. If you omit a turn, that turn never happened as far as the model is concerned. “Memory” in a chat app is your array, your database, or your summary — not a session living inside the model.

Your product therefore has three jobs, and only three: gather the right text (instructions, history, maybe documents or tool results), call the model on the server, and show or store what came back. The rest of this course is those three jobs, done carefully, in TypeScript.

## You are not training a model

Training is the expensive, one-time (or rare) process that produced the weights: billions of parameters nudged so that “predict the next token” works on a huge corpus. You will not do that in this course. Almost nobody building a product does.

Inference is the cheap-per-call process: load those weights, feed them your tokens, sample the next ones. That is what you buy from a provider. Fine-tuning exists, and it is a later, optional lever. For everything in this track, the model is a black box with a prompt in and tokens out.

That split is freeing. You do not need a GPU cluster. You do need to get good at what goes into the box, and at not trusting the output blindly.

## What you actually control

You pick the model (a snapshot with a name — we will use a Gateway id like anthropic/claude-sonnet-4-5). You pick the text it sees. You cap how many tokens it may write. You can ask it to be more or less random (temperature). You can stop it early. You can require a JSON shape. You can let it call your functions.

You do not control the internal “thoughts”. Chain-of-thought you see in a product is still tokens — text the model was asked to write before the answer. Useful, sometimes. Not a debugger for the weights.

The practical rule: if it is not in the request, the model does not have it. That one sentence explains most “the AI forgot” bugs, most hallucinated prices, and most leaking of secrets (because someone put the secret in the prompt).

## What it is not

It is not your database. Yesterday’s orders exist only if you query them and put the rows in the prompt, or expose a tool that does.

It is not a search engine. It cannot browse your docs unless you retrieve chunks first (RAG — a later chapter) or give it a search tool.

It is not a source of truth. It will invent APIs, package names, and citations that look real. For facts that must be right, you fetch them in code and then ask the model to write in terms of that data.

It is not your auth layer. Never put the provider API key in the browser. The model call belongs on the server. The next topic is that HTTP call.

## Trade-offs you will keep meeting

A bigger, slower model is usually better at hard instructions and long context. A smaller, cheaper model is enough for classification, routing, and short copy. There is no single right model — there is a budget and a failure mode.

More context (history, docs, tool output) often helps until it does not: you pay for every token, and past a point the model dilutes. Context engineering is the name for that problem. We will treat it as its own chapter.

You do not need to memorize this list. You need the reflex: every extra token is cost and noise; every missing token is a fact the model will guess.

## Where this course is going

We stay on one stack so the code you copy is the code you ship: AI SDK v6, Next.js on Vercel (Fluid Compute — streaming does not need Edge), and Vercel AI Gateway for models. Keys stay on the server.

First you will see the raw HTTP shape (next topic), then the SDK that hides it. Then a full reply, then a stream, then JSON you can trust, then embeddings. After that: the agent loop — the model calling your code until it can answer.

eve exists for durable, filesystem-shaped agents (sessions, sandboxes, channels). We mention it when a Route Handler is the wrong shape. For these lessons, generateText / streamText plus getModel() is the layer.

## Common questions

### What is an LLM?

A large language model predicts the next chunk of text. It is not a database and it does not 'know' your app.

### What will I be able to do after this lesson?

You will be able to explain, in your own words, what happens between a user pressing send and a reply appearing on screen — including what the model can see, what it cannot, and what you are paying for.

### How long does this lesson take?

About 18 minutes of reading. It is a free chapter in the AI SDK Patterns TypeScript course.

### When should I not use this?

A bigger, slower model is usually better at hard instructions and long context. A smaller, cheaper model is enough for classification, routing, and short copy. There is no single right model — there is a budget and a failure mode.