Large Language Models: Under the Hood, in Plain English

The Most Important Mental Model for Using LLMs Well

Large language models (LLMs) - ChatGPT, Claude, Gemini, Llama - are the AI tools most people interact with every day. Most people use them with a flawed mental model: a knowledgeable assistant that knows things and can reason about them.

The accurate mental model is different: a highly sophisticated text prediction system. It predicts the most statistically likely continuation of any text it is given, based on patterns in an enormous amount of training data.

This shift - from "knowledgeable assistant" to "sophisticated text predictor" - explains every important LLM behavior: why it is often brilliant, why it hallucinates, why it sounds confident even when wrong, and why prompting it better actually works.

What Training Data Does to an LLM

An LLM is trained by showing it vast amounts of text - books, articles, web pages, code, conversations - and teaching it to predict the next word (technically, the next token) in each piece of text. After processing trillions of tokens, the model's parameters encode the statistical relationships between words, phrases, ideas, and concepts across that entire corpus.

The result is a model that has internalized the patterns of how knowledgeable people write about nearly every topic. It can produce text that sounds like a lawyer, a doctor, a programmer, or a poet - because it has been trained on text written by lawyers, doctors, programmers, and poets.

What it has not internalized is the underlying knowledge those experts have. It has the patterns of expert language, not expert understanding.

Tokenization: Why LLMs Struggle With Some Obvious Things

Before processing text, an LLM converts it into tokens - chunks of text that are usually subword fragments, not full words.

"Unbelievably" might become ["un", "believ", "ably"]. "ChatGPT" might be a single token. "Don't" might be ["Don", "'t"]. The model sees tokens, not words or letters.

This is why LLMs sometimes fail at tasks that seem trivially easy: counting letters in a word (they never see individual letters, only tokens), doing arithmetic (numbers are tokenized inconsistently), or handling unusual proper nouns that are split in unexpected ways.

These are not signs of stupidity. They are artifacts of a specific architecture that was not designed for character-level or arithmetic reasoning.

How LLMs Generate Text

After you send a prompt, the model generates a response one token at a time. For each new token, it considers everything that came before (your prompt plus whatever it has already generated) and outputs a probability distribution over all possible next tokens. It samples from that distribution and appends the result. Then it does it again. And again. Until it produces a stop token or reaches the length limit.

The temperature setting controls how this sampling works. At temperature 0, the model always picks the most probable token - deterministic, consistent, but sometimes bland. At higher temperatures, it samples more widely - more creative and varied, but sometimes incoherent.

The critical implication: the model has no plan for the full response when it starts generating. Each token is chosen based on what came before, not based on an intended complete answer. The model is not writing an essay from an outline - it is generating the next word, then the next, then the next. This is why LLM responses sometimes contradict themselves, trailing into a different conclusion than they started building toward.

Why LLMs Hallucinate

Hallucination - confidently stating false information - is a direct consequence of the text prediction architecture.

The model has been rewarded for producing text that looks like accurate, knowledgeable text. It has not been rewarded for checking whether the text is actually accurate. When asked about something it has limited training data on, it fills in the gaps with statistically plausible text - text that looks like what an accurate answer would look like. It has no way to distinguish between "I know this" and "I am pattern-matching to what an answer to this would look like."

This is not dishonesty. The model has no concept of honesty or dishonesty. It is doing exactly what it was trained to do. The failure is architectural: text prediction and factual accuracy are not the same objective.

The Context Window: Memory Without Understanding

Every LLM has a context window - the maximum amount of text it can consider at once. Older models had context windows of a few thousand tokens (roughly a few pages of text). Current models have context windows of 100,000 to over a million tokens.

Within the context window, the model can use any information you have provided. Outside it, the model has no memory. It does not remember your previous conversation from last week. It does not "know" things you told it yesterday unless you include that text in the current context.

This has practical implications:

• For long documents, only what fits in the context window is accessible
• Every new conversation starts fresh - the model has no memory of past sessions unless you provide it
• Putting the most important instructions at the start and end of a prompt makes them more likely to influence the output

What Fine-Tuning Does (and Does Not Do)

You have probably seen claims that a model has been "fine-tuned" for a specific task - customer support, coding, medical questions. Fine-tuning continues training a base model on a smaller, domain-specific dataset.

Fine-tuning adjusts how the model responds: more helpful, more formal, more cautious, more focused on a specific domain. It does not give the model new factual knowledge it did not have before, and it does not eliminate hallucination. A fine-tuned medical model is better at medical language and medical-style responses; it is not better at knowing whether a specific claim is medically accurate.

What This Means for How You Use LLMs

• Treat every factual claim as an assertion to verify, not a known fact. The model sounds confident regardless of accuracy.
• Use specificity to reduce ambiguity. The model predicts the most likely continuation of your text. Vague prompts produce average outputs. Specific prompts produce more targeted predictions.
• Longer context helps - but does not guarantee attention. Models sometimes "forget" important instructions buried in a long context. Put critical instructions prominently.
• Temperature matters for your task. Factual extraction tasks should use low temperature. Creative tasks benefit from higher temperature.

Where to Go Next

The final module of Phase 1 explains what data does in an AI system - why data quality determines model quality, and what this means for the AI products you encounter at work.

What to Practice Next

• Tokenize the same sentence with three different tokenizers (tiktoken for GPT-4, transformers BPE for LLaMA, and the bert-base-uncased WordPiece tokenizer) and compare token counts and boundary decisions - this makes tokenization concrete.
• Step through a minimal transformer forward pass in a Jupyter notebook using Andrej Karpathy's nanoGPT - modify the number of heads or layers and observe how output shapes change.
• Experiment with sampling parameters: generate the same prompt 10 times at temperature 0.0, 0.7, and 1.5, then describe in one sentence each how the outputs differ in diversity and coherence.

A Small Transformer Flow

For one input sentence, an LLM roughly moves through these stages:

Stage	What happens	Why it matters
Tokenization	Text becomes token IDs	The model works with token chunks, not human words
Embeddings	Token IDs become vectors	Similar tokens and contexts become comparable numeric patterns
Attention	Tokens exchange information with other tokens in context	The model decides which earlier words matter for the next prediction
Feed-forward layers	Each position is transformed through learned nonlinear functions	Patterns become more abstract across layers
Logits	The final vector becomes scores over possible next tokens	The model has not chosen text yet; it has produced a distribution
Sampling or decoding	A token is selected from that distribution	Temperature and decoding settings affect determinism and variety

This is why the same prompt can produce different answers. The model is not retrieving a single stored paragraph. It is repeatedly turning context into a probability distribution, choosing a token, appending it, and doing the process again.

Tightening the Mental Model

"Text predictor" is useful, but it can sound too small. A modern LLM is a text predictor whose internal representations encode a great deal of structure about language, code, facts, style, and reasoning patterns. That is why it can be useful.

The caution is that those representations are not the same thing as verified knowledge, intent, or responsibility. When the output matters, pair the model with evidence, tests, retrieval, tools, or human review.