It learned patterns, it does not look answers up
Nobody programmed the model with rules like “if someone asks X, say Y.” Instead it was shown a staggering volume of text — books, articles, code, conversations — and adjusted over and over to get better at guessing missing or upcoming text. What is left behind is not a lookup table with addresses you could go check. It is billions of internal settings that encode patterns of language, style, facts, and reasoning-shaped moves. A useful analogy: someone who has read millions of books and absorbed how ideas fit together, but has no card catalog to go check an exact page.Training happened once (and is refreshed periodically), and it was frozen in place before you ever opened a chat. The model is not learning live from your conversation.
What happens when you hit enter
The model does not search a database for “the answer.” It computes: given everything so far — your prompt plus whatever it has already written in this reply — what is the single most likely next token (a chunk of text)? It produces that token, appends it, and asks the same question again. Thousands of times per reply. It is the same instinct as finishing “Once upon a…” or “The early bird gets the…” — language is full of predictable patterns. The model runs that instinct, just far sharper and at a scale no person can match.Why it is fluent — and why it is confidently wrong
Both come from the same mechanism.- Fluent is a direct product of pattern volume. The more thoroughly a pattern showed up across the training text, the more reliably the model reproduces it. That is why it excels at summarizing, drafting, translating, boilerplate, and well-trodden facts.
- Confidently wrong is the flip side. When the true pattern is rare, ambiguous, or simply was not in what it saw, the model still has to produce a next token — there is no built-in “I don’t know” halt. So it fills the gap with the most plausible-sounding continuation, delivered in the exact same confident voice as everything else.
What it remembers between sessions
Each new conversation starts fresh, unless something is explicitly handed back to it as context — pasted text, an uploaded file, a re-attached thread. Inside a single session, everything said so far sits in the context window and is “in mind.” Start a brand-new session and that window is gone.Do not assume the model remembers “what we discussed last time” unless you re-paste or re-attach it. Some products layer an explicit memory feature on top of this — that is a deliberate, separate feature, not the default behavior.
Pattern-matching, not understanding
The model does not hold beliefs, intentions, or a model of truth the way a person does. What looks like understanding is extremely sophisticated pattern recognition and reproduction across a huge cross-section of human writing. That is not a put-down — pattern-matching at this scale is genuinely powerful, good enough to draft, summarize, and reason through most everyday tasks well. But it explains the limits: novel situations poorly represented in training, precise arithmetic or lookup, and knowing when it does not know.Hype versus fear
Two extremes are both wrong. Hype says “it basically understands, it is basically a person.” Fear says “it is a toy, it can’t be trusted at all.” The accurate middle: an extremely capable pattern predictor, fluent and useful, that needs verification for anything that actually matters.The five ideas in one glance
Where to go next
Capabilities & limits
Maps exactly where this strength-and-weakness pattern shows up in real tasks.
Prompting & getting good results
Turns “plausible, not verified” into a practice: prompt well, spot a bad answer, recover.