How large language models work becomes much easier to understand when you break the process into a few simple steps.
Tools such as ChatGPT and Microsoft Copilot may appear to understand normal human language, but underneath they are working with tokens, numbers, vectors, attention, and probabilities.
The basic process is:
Text → Tokens → Token IDs → Embeddings → Attention → Prediction → Next Token
Let’s see how it works.
Step 1. You Give the LLM a Prompt
Everything starts with a prompt.
For example:
I heard a dog…
The large language model tries to predict what should come next.
Possible words might include:
- bark
- run
- sleep
- eat
Because the words heard and dog strongly relate to a sound made by a dog, bark is likely to have a high probability.
At its core, this next-token prediction is an important part of how large language models work.
Step 2. Text Is Converted Into Tokens
An AI model cannot directly perform neural-network calculations on normal text.
First, the text is broken into pieces called tokens.
For example:
I heard a dog bark
might be simplified as:
I | heard | a | dog | bark
However, one token does not always equal one word.
Tokens can also contain:
- parts of words
- punctuation
- numbers
- common character combinations
For example, a long word might be split into several tokens.
Step 3. Tokens Get Integer IDs
Every token in the model’s vocabulary has an integer ID.
A simplified example might be:
| Token | ID |
|---|---|
| I | 1 |
| heard | 2 |
| a | 3 |
| dog | 4 |
| bark | 5 |
If the token dog has ID 4, the number 4 itself does not mean dog.
It simply tells the model:
Use token number 4 from the vocabulary.
The token ID is therefore an identifier, not the actual meaning of the word.
Step 4. Token IDs Become Embeddings
The neural network needs more than integer IDs.
Each token is converted into a numerical vector called an embedding.
A very simplified example could be:
dog → [10, 3, 2]
puppy → [5, 3, 2]
cat → [10, 3, 1]
car → [-2, -2, 1]
Real embeddings contain many more dimensions.
These vectors allow the model to represent relationships mathematically.
Because dog, puppy, and cat are often used in similar contexts, their vector representations can have similarities.
This is one of the most important concepts for understanding how large language models work.
Embeddings are also widely used in:
- semantic search
- Retrieval-Augmented Generation (RAG)
- AI knowledge bases
- recommendation systems
Step 5. Attention Finds Important Relationships
Next comes one of the most important parts of the Transformer architecture: attention.
Consider:
I heard a dog bark.
When processing bark, not every previous word is equally important.
The words:
heard
and:
dog
provide much stronger clues than words such as:
I
or:
a
Attention helps the model mathematically determine which tokens are most relevant to other tokens.
Modern Transformers use multi-head attention, which allows the model to examine different relationships at the same time.
This helps the model build a richer understanding of the context.
Step 6. The Model Predicts the Next Token
After processing the context, the model calculates probabilities for possible next tokens.
For:
I heard a dog…
a simplified result might look like:
| Token | Probability |
|---|---|
| bark | 60% |
| run | 15% |
| growl | 10% |
| jump | 5% |
| others | 10% |
The model then selects a token based on these probabilities and its generation settings.
Suppose it selects:
bark
The sequence becomes:
I heard a dog bark
The model now runs the process again.
Step 7. The Process Repeats
This is the final key to understanding how large language models work.
The LLM normally does not create an entire paragraph in one step.
It generates text token by token.
For example:
I heard a dog
↓
bark
↓
I heard a dog bark
↓
loudly
↓
I heard a dog bark loudly
↓
at
↓
I heard a dog bark loudly at...
This process repeats very quickly until the response is complete.
The Complete LLM Process
The complete journey looks like this:
Human Text → Tokens → Token IDs → Embeddings → Transformer + Attention → Next-Token Probabilities → Select Token → Add Token → Repeat
That is the core idea behind how large language models work.
Final Thoughts
Understanding how large language models work does not require advanced mathematics.
Remember these seven ideas:
Prompt → Tokens → IDs → Embeddings → Attention → Prediction → Repeat
Once you understand this process, other AI concepts become much easier to understand.
For example:
- Tokens help explain context windows and API costs.
- Embeddings help explain semantic search and RAG.
- Attention helps explain how an LLM uses context.
- Next-token prediction helps explain both AI-generated text and hallucinations.
This same foundation helps you better understand technologies such as ChatGPT, Microsoft Copilot, Copilot Studio, Azure AI, AI agents, and modern generative AI applications.
Further Reading
- Attention Is All You Need — Original Transformer Paper
The original 2017 research paper that introduced the Transformer architecture. Best for readers who want the mathematical and architectural details behind attention.
Read the paper on arXiv - Hugging Face LLM Course — How Transformers Work
A much more approachable deep dive into Transformers, attention, encoder/decoder architectures, tokenization, and modern LLM concepts.
Read the Hugging Face LLM Course