Large language models (LLMs) are AI systems trained on enormous amounts of text to predict the next small piece of text, called a token. By repeating that prediction thousands of times, they write fluent answers. They are then fine-tuned to follow instructions and be helpful, but they still generate likely text rather than looking up verified facts.
The core idea: predicting the next token
At its heart, a large language model does one thing: given some text, it predicts what comes next. Not a whole answer at once, but one small piece at a time. Those pieces are called tokens, which are usually whole words, parts of words or punctuation. "Understanding" might be split into "under", "stand" and "ing".
When you ask a chatbot a question, the model reads your message as tokens, calculates the probability of every possible next token, picks one, adds it to the text, and repeats. A few hundred repetitions later, you have a paragraph. It feels like conversation, but underneath it is very fast, very sophisticated prediction.
How a model learns to predict
Stage 1: pre-training
The model is shown a vast amount of text from books, websites, code and other sources. For each passage, it tries to predict the next token, checks the real answer, and adjusts its internal parameters slightly. After trillions of these tiny adjustments, it has absorbed grammar, facts, styles of writing, reasoning patterns and much more, all stored as numbers rather than as a database of sentences.
Stage 2: fine-tuning and alignment
A pre-trained model is good at continuing text but not at being a helpful assistant. So it is further trained on examples of good conversations and on human feedback about which answers are better. This teaches it to follow instructions, answer clearly, refuse harmful requests and admit uncertainty more often.
Stage 3: tools and retrieval
Many modern assistants can also search the web, read uploaded files, run code or call other tools. This helps them answer with current information, but the core is still the same predictive model deciding what to write.
The transformer and "attention"
Most LLMs use an architecture called the transformer, introduced by Google researchers in 2017. Its key innovation is attention: a way for the model to weigh which earlier words matter most when predicting the next one. In "The trophy did not fit in the suitcase because it was too big", attention helps the model link "it" to "trophy". This ability to track relationships across long passages is a big reason LLMs write so coherently.
Context windows: the model's working memory
A model can only consider a limited amount of text at once, called its context window. Everything in the conversation, plus any documents you upload, must fit inside it. Newer models have very large context windows, but when a conversation gets very long, earlier details may be summarised, dropped or given less attention. Practical lesson: for important tasks, restate key requirements rather than assuming the model remembers everything.
Why LLMs hallucinate
Because a model generates the most plausible continuation, it can produce text that sounds right but is false: a made-up statistic, a non-existent book, a wrong date. It does not "know" it is wrong, because it is not checking facts against a source unless it has been given one. Our article on AI hallucinations covers how to reduce and catch them.
What this means for how you use AI
Understanding the mechanics leads to better habits:
- Give context. The model only knows what is in its training and your prompt. Tell it who you are, what you need and what good looks like.
- Ask for reasoning on hard problems. Asking the model to work step by step often improves accuracy on maths and logic.
- Provide sources. Paste or upload the material you want it to use, and ask it to quote from it.
- Verify important facts. Especially names, numbers, dates, citations and anything medical, legal or financial.
- Do not paste private data into tools unless you know how that data is handled.
| Myth | Reality |
|---|---|
| The AI looks up answers in a database | It generates text from learned patterns, unless connected to search or documents |
| If it sounds confident, it is right | Confidence and accuracy are unrelated in LLM output |
| It remembers all my past chats | Only if the product has a memory feature; otherwise each chat is separate |
| It understands like a human | It models language patterns very well, but has no lived experience |
Want to go deeper?
If you are curious about building with LLMs, our prompt engineering course teaches reliable prompting, and generative AI development teaches how to build apps on top of model APIs with Python.
Frequently asked questions
Are ChatGPT, Claude and Gemini the same technology?
They are all large language models built on transformer-style architectures, made by different companies (OpenAI, Anthropic and Google). They differ in training data, size, safety approach and features, so their strengths vary.
Do language models understand what they say?
They capture deep statistical relationships in language, which lets them reason in useful ways, but whether that counts as understanding is debated. Practically, treat their output as capable but unverified.
Why do answers change if I ask the same question twice?
Models usually include some randomness when choosing tokens, so wording and sometimes content vary. For consistent output, give precise instructions and examples.
