What a language model really is

01How a language model is made

A large language model, LLM for short, is trained on huge amounts of text: books, websites, documentation, forums. It does not learn facts like a database. It learns patterns: which words and ideas typically follow each other. From these patterns it later builds new text.

When you ask a question, the model does not search an archive for the right answer. It calculates which continuation fits most probably, word by word. The result reads like an answer, but technically it is a prediction.

02Why it hallucinates

Hallucination means: the model writes something that sounds plausible but is wrong. Invented sources, wrong numbers, court rulings that never existed. This is not a defect that some update will eventually fix. It is a direct consequence of the principle: a system that generates likely continuations optimises for plausibility, not for truth.

The tricky part: the model sounds just as confident about an invented detail as about a correct one. You cannot hear the difference in its tone, only find it by checking.

03What a language model does well

• Rewriting: turning bullet points into an email, shortening a text, adjusting the tone.
• Structuring: sorting messy notes into sections, tables or categories.
• Summarising: pulling the key points out of ten pages, which you then double-check.
• Pattern spotting: grouping similar customer requests, making recurring topics visible.
• Explaining: translating jargon into plain words and making unfamiliar topics accessible.

04What it does not do well

• Exact facts: names, dates, legal sections and prices need checking, every time.
• Arithmetic: numbers belong in a calculator or a spreadsheet, the model only estimates.
• Current events: without a connection to search or a database it only knows its training data.
• Your data: what is not in the training or in your request, it cannot know.

05The most important practical takeaway

Treat a language model as a drafting machine, not a truth machine. It hands you a rough draft in seconds, which you check, correct and finish. Used like that, it saves real time. Taken over unchecked, it only moves the work to a later point: the moment the error surfaces. And by then it is usually more expensive.

Rule of thumb: the more expensive a mistake would be, the more thorough the check. An internal email gets a quick read, a number in a quote gets recalculated, a legal question gets verified at the source.