Deep Dive · AI & Machine Learning
Inside every AI assistant — ChatGPT, Claude, Gemini — is the same remarkable architecture. This guide pulls it apart layer by layer: tokens, transformers, attention, and training. No PhD required.
Contents
- What is an LLM? The big picture
- Step 1 — Tokenization: how text becomes numbers
- Step 2 — Embeddings: numbers become meaning
- Step 3 — The Transformer architecture
- Step 4 — Attention: the secret ingredient
- Step 5 — Training on the internet
- Step 6 — How LLMs generate text (temperature)
- Step 7 — RLHF: teaching the model to be helpful
- Limitations & hallucinations
- Knowledge check
Section 01
What is a Large Language Model?
A Large Language Model (LLM) is a neural network trained to predict the next word (or token) in a sequence of text. That’s it. The remarkable thing is how much intelligence emerges from doing this one task at massive scale.
When GPT-4 writes a poem, Claude explains a legal contract, or Gemini summarizes a research paper — all of them are doing the same fundamental thing: given a sequence of tokens, predict what comes next, one token at a time.
💡 Core Insight
An LLM isn’t “thinking” in the way humans do. It’s a very sophisticated pattern-completion engine — but trained on so much text that the patterns it learns encode reasoning, facts, language, and style.
Scale
GPT-4 has ~1.8 trillion parameters. That’s 1,800,000,000,000 numbers that define its behaviour.
Data
Trained on hundreds of billions of words scraped from the web, books, and code repositories.
Compute
Training large models costs tens of millions of dollars in GPU time and energy.
Architecture
Almost all modern LLMs use the Transformer architecture, invented at Google in 2017.
Section 02
Tokenization — How Text Becomes Numbers
Computers can’t process raw text. The first step is breaking text into tokens — small chunks that the model understands. Tokens aren’t always full words; they can be parts of words, punctuation, or even single characters.
Modern LLMs use Byte Pair Encoding (BPE) tokenization. Common words like “the” become single tokens; rare words like “tokenization” might split into [“token”, “ization”]. Each token gets mapped to a unique integer ID.
# Using OpenAI's tiktoken library (same tokenizer as GPT-4) import tiktoken encoder = tiktoken.get_encoding("cl100k_base") text = "Large Language Models are fascinating!" tokens = encoder.encode(text) print(tokens) # → [35, 18, 33, 5400, 46, 18, 13, 1234, ...] print([encoder.decode([t]) for t in tokens]) # → ['Large', ' Language', ' Models', ' are', ' fas', 'cinating', '!'] print(f"Text length: {len(text)} chars → {len(tokens)} tokens") # A useful rule of thumb: ~4 chars per token in English
Type any text below to see how it gets split into tokens. Each color = one token.
⚠️ Why Tokens Matter
LLMs have a context window — a maximum number of tokens they can process at once (e.g., 128,000 tokens for Claude 3). Long documents must be chunked to fit. Knowing this helps you understand why very long conversations or documents can cause issues.
Section 03
Embeddings — Numbers That Carry Meaning
Once we have token IDs, each ID gets mapped to an embedding — a list of hundreds or thousands of decimal numbers called a vector. Think of it as coordinates in a very high-dimensional space.
The magic: similar words end up close together in this space. The word “king” and “queen” are near each other. “Paris” is to “France” what “Tokyo” is to “Japan”. The model learns these relationships entirely from predicting text — nobody hand-labels the geometry.
# Each token becomes a high-dimensional vector # GPT-4 uses 12,288-dimensional embeddings! token_id = 5400 # "fascinating" # The embedding table has shape: [vocab_size, d_model] embedding = embedding_table[token_id] # embedding is now a vector like: # [0.23, -0.51, 0.87, 0.12, ..., -0.34] # 4096 numbers capturing the "meaning" of this token
Section 04
The Transformer Architecture
The Transformer, introduced in the landmark 2017 paper “Attention Is All You Need” by Google researchers, is the engine powering almost every modern LLM.
A Transformer is a stack of identical layers. GPT-4 has ~96 of them. Each layer has two main components: a Multi-Head Attention block and a Feed-Forward Network. Information flows through all layers sequentially, each one refining the representation of the input.
Input Embedding + Positional Encoding
Each token is converted to a vector. Since attention has no built-in sense of order, a positional encoding is added — a pattern that tells the model where in the sequence each token sits.
Multi-Head Self-Attention
The most important layer. Every token “looks at” every other token and decides how much to attend to it. We’ll explore this interactively in the next section.
Feed-Forward Network
Each token position passes through an independent MLP (two linear layers with a non-linearity). This is where most of the model’s “factual knowledge” is stored.
Residual Connections + Layer Norm
After each sub-layer, the original input is added back (residual connection) and then normalized. This is what allows training of very deep networks without gradients vanishing.
Section 05
Attention — The Secret Ingredient
The attention mechanism lets the model understand that in “The cat sat on the mat because it was tired”, the word “it” refers to “cat”, not “mat”. This kind of long-range dependency was the main weakness of older models (RNNs).
For each token, the model computes three vectors: a Query (what am I looking for?), a Key (what do I contain?), and a Value (what do I contribute?). Attention scores come from matching Queries against Keys across all positions.
import torch import torch.nn.functional as F def scaled_dot_product_attention(Q, K, V): d_k = Q.shape[-1] # dimension of key vectors # 1. Compute attention scores: how much does each query match each key? scores = torch.matmul(Q, K.transpose(-2, -1)) / (d_k ** 0.5) # 2. Normalize scores into a probability distribution (0–1, sum to 1) weights = F.softmax(scores, dim=-1) # 3. Weighted sum of Value vectors output = torch.matmul(weights, V) return output, weights # "Multi-Head" = run this 8 or 16 times in parallel with different Q,K,V projections # Then concatenate and project back down. Each head can "look" at different things.
Click a word below to see which words in the sentence it “attends to” most strongly.
← Click a word to see its attention pattern
🔬 Multi-Head Attention
Modern LLMs run 32–128 attention heads in parallel, each looking for different relationships. One head might track pronouns, another tracks verb-subject agreement, another tracks named entities. The outputs are concatenated and projected.
Section 06
Training — Learning from the Internet
Training an LLM means adjusting billions of parameters so the model gets better at predicting the next token. The training data is a massive corpus: web pages, books, Wikipedia, code, scientific papers.
The training loop is elegant: take a sequence of tokens, hide the last one, ask the model to predict it, measure the error (cross-entropy loss), and adjust all parameters slightly in the direction that reduces the error (backpropagation + gradient descent). Repeat this hundreds of billions of times.
| Model | Parameters | Training Tokens | Est. Cost |
|---|---|---|---|
| GPT-3 (2020) | 175B | 300B | ~$4.6M |
| LLaMA 2 (2023) | 70B | 2T | ~$3M |
| GPT-4 (2023) | ~1.8T (est.) | ~13T | ~$100M |
| Claude 3 Opus (2024) | Undisclosed | Undisclosed | Undisclosed |
for batch in dataloader: tokens = batch['input_ids'] # shape: [batch, seq_len] inputs = tokens[:, :-1] # all tokens except last targets = tokens[:, 1:] # all tokens except first # Forward pass — predict next token at every position logits = model(inputs) # [batch, seq_len, vocab_size] # Measure how wrong we are loss = F.cross_entropy( logits.reshape(-1, vocab_size), targets.reshape(-1) ) # Backward pass — compute gradients loss.backward() # Update parameters (Adam optimizer) optimizer.step() optimizer.zero_grad()
Section 07
Temperature — Controlling Randomness
When an LLM predicts the next token, it produces a probability distribution over the entire vocabulary (~50,000–100,000 tokens). Temperature is a dial that controls how peaked or flat this distribution is.
Low temperature → the model almost always picks the most probable token (deterministic, repetitive). High temperature → the model samples more randomly from lower-probability tokens (creative, unpredictable, sometimes nonsensical).
Adjust the temperature to see how it affects the output for the prompt: “The weather today is…”
Section 08
RLHF — Teaching the Model to Be Helpful
A pretrained LLM predicts text — but that doesn’t mean it’s helpful, safe, or honest. A model might complete “How do I make a bomb?” with a recipe, because that’s what a human might write on the internet.
Reinforcement Learning from Human Feedback (RLHF) — used by OpenAI, Anthropic, Google, and others — aligns the model with human preferences. The process has three stages:
Supervised Fine-Tuning (SFT)
Human contractors write ideal responses to thousands of prompts. The model is fine-tuned on these examples. Now it knows the format of helpful answers, but not yet which answers are better.
Reward Model Training
Humans rank pairs of model outputs: “Response A is better than B”. A separate neural network — the Reward Model — is trained to predict these rankings, scoring any response from 0 to 1.
Reinforcement Learning (PPO)
The LLM generates responses. The Reward Model scores them. The RL algorithm (Proximal Policy Optimization) nudges the LLM’s parameters to produce higher-scoring responses. This loop runs for thousands of steps.
📌 Beyond RLHF
Newer approaches include Direct Preference Optimization (DPO), which skips the reward model and directly trains on preference data — simpler and often equally effective. Anthropic’s Claude uses a variant called Constitutional AI (CAI), where the model is trained to critique its own responses against a set of principles.
Section 09
Limitations & Hallucinations
LLMs are powerful but imperfect. Understanding their failure modes makes you a much better user (and builder) of AI systems.
Hallucinations
Models generate fluent text even when they don’t “know” something. They can fabricate citations, facts, and names with full confidence.
Knowledge Cutoff
Training data ends at a fixed date. The model has no awareness of events after that point unless given context.
Math & Reasoning
LLMs are next-token predictors, not calculators. They can struggle with precise arithmetic and multi-step logical chains.
Context Window
They can only “see” a fixed window of tokens at once. Very long inputs require chunking strategies like RAG.
✅ Best Practices
Always verify factual claims from LLMs against primary sources. Use them for drafting, brainstorming, and reasoning assistance — not as authoritative sources of truth. Provide grounding context in your prompts, and use tools like search or code execution to extend their capabilities.
Section 10
Knowledge Check
Test what you’ve learned. Five questions covering the key ideas from this article.
