A large language model (LLM) like ChatGPT or Claude is a transformer neural network trained to predict the next piece of text. It learns language patterns from huge amounts of text during pretraining, is then tuned to follow instructions and be helpful and safe, and answers by predicting one token at a time.

Step 1: text becomes tokens

Models don't read words; they read tokens, which are whole words or pieces of words. A tokenizer splits text into tokens and maps each to a number. Common words are usually one token; rare words are split into several.

Step 2: tokens become vectors

Each token number is turned into a list of numbers called an embedding. Tokens with related meanings end up with similar vectors. The model also adds information about each token's position in the sentence.

Step 3: attention connects the context

The key idea of the transformer is self-attention. For every token, the model asks: which other tokens in this text matter for understanding me? In "The bank of the river", attention helps "bank" look at "river" and pick the right meaning. Many attention layers stacked on top of each other let the model capture grammar, facts and reasoning patterns.

Step 4: predicting the next token

At the end, the model outputs a probability for every possible next token. It picks one, adds it to the text, and repeats. A whole answer is produced one token at a time.

How are LLMs trained?

StageWhat happens
PretrainingThe model reads a very large amount of text and learns to predict the next token. This is where most of its knowledge comes from.
Supervised fine-tuning (SFT)It is trained on examples of good instructions and answers, so it learns to follow requests.
Preference alignmentPeople compare answers, and methods such as RLHF or DPO teach the model to prefer helpful, honest and safe responses.
ServingThe trained model runs on optimised inference systems so it can answer quickly and affordably.

Why do LLMs sometimes make things up?

An LLM generates likely-sounding text; it doesn't look facts up unless it's connected to a source. That's why good AI applications add retrieval-augmented generation (RAG) to ground answers in real documents, and why checking outputs still matters.

Can you learn to build one?

Yes. The same maths and code that power large models work at small scale. In Program Zero's Phase 8 you build a GPT-style model from scratch in PyTorch and fine-tune an open model; in Phase 9 you run a small distributed pretraining run with an alignment pass and serve the result. The difference from frontier labs is compute and team size, not the ideas.