How ChatGPT is made: the story of teaching a machine to talk
No database of answers, no written replies. How next-word prediction, a mountain of reading, and thousands of human rankings produced the fastest-adopted product in history.
On November 30, 2022, a San Francisco company put a chat box on the internet and called it a research preview. The team expected a few thousand curious visitors. Five days later a million people had signed up. Two months later it was a hundred million, the fastest adoption of any consumer product in history to that point. Most of the people typing into ChatGPT that winter had no idea what was answering them, and honestly, the explanation is stranger than the guesses.
There is no database of answers inside. Nobody wrote the replies. The system was never taught grammar, facts, or manners in any direct way. It was built by making one simple bet at an absurd scale: if a machine gets good enough at predicting the next word, something that looks like understanding falls out.
What is the machine actually doing?
One thing. Given some text, it predicts what word probably comes next. That is the whole trick, repeated forever. Feed it "The capital of France is" and it computes that "Paris" is overwhelmingly likely. Then it appends "Paris" and predicts the next word after that, one word at a time, each choice shaped by everything before it.
Everything else — the essays, the code, the apologies, the jokes — is that loop running fast. Which raises the real question: how does a next-word guesser learn that Paris is the capital of France, that code needs matching brackets, and that an email to your landlord should sound different from a birthday card?
Where does the knowledge come from?
Reading. An amount of reading no human could do in a thousand lifetimes. Training begins with a huge slice of the public internet plus books and code, trillions of words. The model starts as a blank network, guessing randomly. It reads a fragment, guesses the next word, gets told the truth, and adjusts millions of internal dials a tiny bit. Then again, trillions of times, on thousands of specialized chips running for months, at a cost measured in tens of millions of dollars and up.
Nobody tells it that Paris is a capital. It sees Paris follow "capital of France" so many times, in so many contexts, that the pattern gets burned into the dials. Grammar happens the same way. So does the difference between how a lawyer writes and how a teenager texts. The architecture that made this scale possible is called a transformer, published by Google researchers in 2017 in a paper titled "Attention Is All You Need". The T in GPT stands for transformer, and nearly every modern AI model runs on some version of that design.
Why did raw training not produce ChatGPT?
Because a machine trained only to continue internet text is a mirror of the internet: brilliant, unhelpful, and frequently vile. Ask an early model a question and it might respond with three more questions, because online, questions often follow questions. Ask something dangerous and it would answer, because somewhere in the training text, someone did.
The fix is the part most people never hear about. After the reading phase, humans enter. The lab hires people to write examples of good answers, and to rank the model's attempts from best to worst, thousands upon thousands of times. Another system learns from those rankings what humans prefer, and the model gets tuned toward it. The technique is called reinforcement learning from human feedback, RLHF, and it is the difference between a machine that continues text and a machine that answers you. It is also why ChatGPT says "I can't help with that" — not morality, but training. Thousands of ranked examples taught it that refusing certain requests is what the preferred answer looks like.
Does it think?
Here is where honest people disagree, and it is worth sitting with the discomfort instead of picking a slogan.
The deflating view: it is statistics. A very large autocomplete. It has no goals, no memory of yesterday, no idea that you exist between messages. When it says "I understand how frustrating that must be", no understanding and no frustration occurred anywhere.
The unsettling view: to predict the next word in a physics explanation, the network had to compress something about physics into its dials. To predict dialogue, it had to model how people talk and, in some functional sense, what they want. Researchers keep finding structures inside these networks that look like maps, like logic, like little internal models of the world. Prediction, pushed hard enough, starts to resemble comprehension from the outside.
Both views are defensible. What is not defensible is treating it as a database of verified facts, because it is not one, and it will state a wrong answer with the same confident fluency as a right one. It is producing plausible text. Plausible and true are cousins, not twins.
What changed after 2022?
Scale, then variety, then price. Models got bigger and then, interestingly, smaller and cheaper as labs learned to squeeze similar ability out of less computing. Chat became one interface among many: models that read images, generate video, write and run code, and take multi-step actions. Meta released models with open weights that anyone can download and run. A Chinese lab called DeepSeek showed in early 2025 that near-frontier ability did not require frontier budgets, which rattled markets for a week and quietly changed everyone's math.
But the core of every one of them is still the same strange object: a next-word predictor, taught by the internet, sanded into politeness by human rankings. Knowing that will not make the answers better. It will make you a sharper user of them — impressed in the right places, suspicious in the right places, and never confused about whether someone is home.