How LLMs Actually Work: Neural Nets, GPUs and One 2017 Paper

By John published: August 6, 2026

Here is a thing that should bother you more than it does.

The AI you have been talking to is a file full of numbers. That is it. There is no code in there that understands English. Nobody sat down and wrote the rules of grammar, or a database of facts, or a module that handles sarcasm. It is billions of numbers, and a fairly boring program that multiplies them together over and over until words come out the other end.

And it works. It writes decent code. It explains things. It picks up on tone. Nobody, including the people who built it, can point at any particular number in that file and tell you what it is for.

I find that genuinely wild, and I think it is more interesting than most of the stuff written about AI, which is either breathless nonsense about robot gods or dismissive nonsense about fancy autocomplete. The truth sits in the middle and it is a much better story. It involves a bit of 1950s biology, an accident involving video game hardware, and one research paper from 2017 that quietly rearranged the entire industry.

No maths in here. You do not need any.

It starts with a very stupid neuron

The idea goes back to the 1940s and 50s, and it starts with someone looking at a brain cell and thinking, how hard can that be?

A brain cell, roughly, takes in signals from other cells, and if the incoming signals add up past some threshold, it fires and passes a signal on. That is the cartoon version and neuroscientists will wince, but it is the version that got copied.

So the artificial one works like this. It takes a bunch of numbers coming in. Each of those inputs has a weight, which is just a number saying how much this particular input matters. Multiply each input by its weight, add them all up, and if the total is big enough, output a signal.

That is the whole neuron. It is a weighted average with an attitude problem. On its own it does nothing interesting at all.

The trick is that you do not build one. You build billions, arranged in layers, where the output of one layer feeds the input of the next. Text goes in one end as numbers, gets multiplied and added through layer after layer, and numbers come out the other end. That stack is a neural network.

Drawn out, the simplest useful version looks like this:

That arrangement has a name: feed-forward. Information enters on the left, moves in one direction through the layers, and leaves on the right. Nothing loops back. It is the plainest form a neural network takes, and it is worth having the picture in your head, because language models are built out of exactly this plus one extra idea that we will get to shortly.

Two things the diagram is lying about, in the interests of fitting on a screen. A real model has dozens of layers rather than two, and thousands of neurons per layer rather than five. And the picture shows fifty-odd connections, where an 8 billion parameter model has eight billion of them. Scale is doing a lot of work in this subject.

When people say a model has “8 billion parameters”, those parameters are the weights. Eight billion little numbers, each saying how strongly one thing should influence another.

Nobody programs it, it gets tuned

This is the part that trips people up, especially people who write software for a living, because it is the opposite of how we normally build things.

Nobody chooses those eight billion numbers. You could not. There is no meeting where someone decides that weight number 4,502,119,388 should be 0.0071.

Instead the numbers start out random, and the model gets trained. Picture a mixing desk the size of a football pitch, covered in billions of faders, all set to random positions. You play the music through, it sounds like a car crash, and then some tireless machine goes along nudging every fader very slightly in whichever direction made it sound marginally less awful.

Then you do that again. And again. Trillions of times.

In practice the process is:

1. Show the model some text with the next word hidden.
2. Ask it to predict the hidden word.
3. It gets it wrong.
4. Measure exactly how wrong, and work backwards through the network figuring out which weights contributed to the error and in which direction each should move.
5. Nudge them all a tiny amount.
6. Repeat, across a chunk of the internet, for weeks, on thousands of GPUs.

Step 4 has a name, backpropagation, and it is the single idea that makes any of this possible. It is calculus, it is from the 1980s, and you do not need to understand it beyond this: the system can work out who to blame for a mistake, and adjust accordingly.

What comes out the far end is not a program that knows things. It is a set of weights that happen, through billions of tiny corrections, to be extremely good at predicting what comes next.

”It’s just predicting the next word” is true and also misleading

You will hear this a lot, usually said dismissively. It is technically correct. It is also one of those statements that is true in the same way that “chess is just moving pieces on a board” is true.

Think about what it actually takes to reliably predict the next word.

“The capital of France is ___”. Fine, that is lookup. But now: “The detective realised the killer must have been left-handed, because the wound was ___”. To predict that word you need to have tracked who is speaking, what a wound is, what handedness implies about angles, and what kind of sentence this is. There is no lookup table for that.

Push prediction hard enough, across enough text, and the only way to keep getting better at it is to build internal machinery that represents how things relate to each other. Not because anyone asked for it, but because it is the cheapest way to reduce prediction errors.

Whether that adds up to “understanding” is a genuinely open argument and I am not going to settle it in a blog post. What I will say is that “just predicting the next word” undersells what the prediction requires.

The accident: why gaming hardware runs AI

Now for my favourite part of the story, because it is a total accident of history.

Your CPU is a small number of extremely clever cores. Four, eight, sixteen. Each one is brilliant at following a complicated sequence of instructions with lots of decisions in it. Think of a handful of Michelin-starred chefs. Give them a complex recipe and they will nail it.

Your graphics card is thousands of much simpler cores. Individually, each one is fairly dim. But there are thousands of them, and they can all do the same simple operation at the same time on different data. Think of ten thousand line cooks who can each only chop an onion, but who can all chop onions simultaneously.

Graphics cards are built like that because drawing a 3D scene means doing the same bit of arithmetic to millions of pixels and vertices at once. Nobody designed them for AI. They were designed so that Night City looks good at 60 frames per second.

And here is the thing about neural networks: all that multiplying and adding is exactly the same operation, repeated across enormous grids of numbers. It is matrix multiplication. Which is, precisely, the thing graphics cards were built to do a stupendous number of times per second.

Two pieces of kit designed for different purposes turned out to be the same shape.

But that raises a better question. Why did we have thousands of tiny parallel cores lying around in consumer PCs in the first place? Nobody builds that speculatively. Somebody had to want it badly enough to pay for it, in volume, for years.

That somebody was teenagers who wanted to shoot demons.

The road from Doom to CUDA

In December 1993, id Software released Doom. You know what it did culturally. What matters here is what it did to the hardware market: it turned the PC from a spreadsheet machine into a games machine, on a scale nobody had seen. It shipped as shareware, spread across university networks and office LANs like a virus, and got itself banned from more workplaces than any program before it. Microsoft cared enough about it to produce a promotional video with Bill Gates composited into the game to sell Windows 95 as a gaming platform. That is how central one shareware title had become.

Here is the wrinkle, and it is my favourite detail in the whole story: Doom itself did not use a graphics card. There were none to use. Every pixel was drawn by the CPU, using a pile of extraordinarily clever tricks that let a 486 fake a 3D world it had no business rendering. John Carmack’s genius at that point was making mediocre hardware do the impossible.

What Doom created was the appetite. Millions of people now wanted their PC to draw fast 3D, and were willing to spend money on it. That is a market. Markets get served.

The hardware answer arrived in 1996 with the 3dfx Voodoo, a card that did nothing but 3D. It could not even display your desktop; you ran a cable from your normal graphics card into it. And what made people buy it was, again, Carmack. Quake shipped in 1996 with true 3D geometry, and in 1997 he released GLQuake, a version rewritten to hand the drawing to an accelerator card through OpenGL.

The difference was not subtle. Same game, transformed. Reviewers described it as looking like a different generation of hardware, because effectively it was. People bought expensive add-in cards to make one game look better, which at the time was a genuinely new consumer behaviour, and the graphics accelerator market was born fully formed.

After that it is an arms race, and every step of it edges closer to the machine you would later run a language model on:

The 2001 row is the quiet turning point. Once shaders were programmable, the card was no longer a machine that only drew triangles. It was a machine that ran your code across thousands of cores at once, and it happened to have a monitor attached. Academics started smuggling physics simulations and linear algebra onto graphics cards by disguising their data as textures, which is as ridiculous as it sounds and worked far too well to ignore. One of the people building tools for that, Ian Buck, ended up at NVIDIA leading the project that became CUDA, which finally let you write parallel code for the card without pretending any of it was graphics.

Which brings us to 2012, when a neural network called AlexNet won an image recognition contest by an embarrassing margin, trained on two consumer gaming GPUs sitting in a desktop. Everyone noticed. NVIDIA in particular noticed, having spent five years building CUDA and quietly seeded it through every university that would take it. That head start is a large part of why NVIDIA is now one of the most valuable companies on earth.

So the honest through-line runs: kids wanted faster demons, which funded parallel hardware, which got programmable, which academics hijacked, which trained the neural networks, which needed one more idea to work on language. That idea arrived in 2017, and we will get to it. But the machine it needed was already sitting in millions of bedrooms, bought for entirely different reasons.

Before 2017: reading through a straw

By the mid 2010s neural networks were doing well at images. Language was harder, and the reason was architectural.

The best language models of the time were recurrent networks. They read text the way you read a sentence: one word at a time, left to right, carrying a running summary of everything so far. Read a word, update the summary, read the next word, update again.

Two problems with that, and they were both brutal.

They forgot. Everything from earlier in the text had to survive by being squeezed into that running summary, getting slightly more mangled with every step. By the end of a long paragraph, the beginning was mush. There were clever fixes, but it was a losing battle.

They could not be parallelised. Word five could not be processed until word four was done, which needed word three. That is a sequential chain by definition. So you had thousands of GPU cores sitting idle while the model shuffled along one word at a time. All that beautiful parallel hardware and the architecture could not use it.

That was the ceiling. Not compute, not data. Architecture.

The 2017 paper that broke the ceiling

In June 2017, eight researchers at Google published a paper called “Attention Is All You Need”. It is thirteen pages long. It is named after a Beatles song. It is, by a distance, the most consequential computing paper of the last twenty years, and if you have used ChatGPT, Claude, Gemini, Llama or anything like them, you have used the thing it describes.

It introduced the Transformer. The T in GPT. That is what the letter has been standing for the whole time.

The idea, stripped of the maths, is this: stop reading one word at a time. Put the whole passage in front of the model at once, and let every word look at every other word simultaneously and work out which ones matter to it.

That mechanism is called attention. When the model processes “it” in the sentence “the server crashed because it ran out of memory”, attention is what lets “it” look back across the sentence and work out that it refers to the server rather than the memory.

The scores are the important part. It is not a yes or no decision about which word matters; it is a distribution across all of them, worked out fresh every time. And the picture only shows one word’s worth of it. Every other word in the sentence is doing the same thing simultaneously, so the real calculation is that diagram repeated nine times over, and then again at every layer in the stack.

That is also why this is such a good fit for a graphics card. Nine words scoring nine words is a grid of numbers multiplied all at once, which is precisely the operation those thousands of tiny cores were built for.

Compare the two approaches:

That last row is the one that changed the world. Suddenly the limit on how big a language model you could train was not the architecture. It was how many GPUs you could afford to point at the problem.

And it turned out, when people did point more GPUs at the problem, that the models did not just get a bit better. They got better in ways nobody predicted, picking up abilities they were never explicitly trained for. Translation. Arithmetic. Writing code. Explaining a joke. The recipe of “same architecture, more data, more compute, more parameters” kept working long past the point where people expected it to stop.

Everything since 2017 has essentially been that recipe, plus a great deal of refinement about what data to use and how to make a raw text predictor behave like a helpful assistant.

What happens when you press enter

So here is the whole pipeline, end to end, when you type a question into a model running on your own machine.

1. Your text gets chopped into tokens. Chunks of roughly four characters. “Unbelievable” might become “un”, “believ”, “able”.
2. Each token becomes a vector. A long list of numbers positioning that token in a space where related meanings sit near each other. This is an embedding, and it is where “king minus man plus woman lands near queen” comes from.
3. The stack does its work. Layer after layer of attention (every token looking at every other token, working out what is relevant) followed by ordinary neural network layers that do the processing. Dozens of layers deep.
4. Out comes a probability distribution. Not a word: a score for every token in the vocabulary. Maybe “the” at 12%, “a” at 8%, and so on across 100,000 options.
5. One gets picked. How adventurous that pick is comes down to a setting called temperature. Low temperature always takes the most likely option and sounds flat and predictable. High temperature takes more chances and eventually produces nonsense.
6. Everything repeats. The chosen token gets appended to the input and the whole process runs again for the next one. Every single word you see streaming out is a full pass through billions of parameters.

That last point is the one that connects to the hardware. Producing one token means reading essentially every weight in the model. Which is why the number that governs speed is not how clever your graphics card is, but how fast it can read its own memory. And why the model has to fit in that memory in the first place.

Why it confidently makes things up

Once you know how the pipeline works, the failure modes stop being mysterious.

The model is not looking anything up. There is no database in there, no search step, no fact table. It is generating text that is statistically plausible given everything before it. Most of the time plausible and true overlap nicely, because it was trained on a great deal of text where they did.

But when they come apart, nothing in the system notices. There is no little module checking claims against reality. Asked for a citation it has never seen, it produces something that looks exactly like a citation, because that is what fits. That is hallucination, and it is not a bug in the sense of something that got coded wrong. It is the machinery working as designed, applied to a question where plausibility and truth diverge.

Which also explains the fix. If you want a model to be accurate about your documents, you do not argue with it. You put the actual text in front of it, so that the plausible continuation and the true one are the same thing. That is what retrieval systems do, and it is why they work.

Nobody fully knows what is in there

I want to end on the bit that gets glossed over.

We know how to build these things. We know how to train them, measure them, make them bigger, and make them behave. What we cannot do is open one up and explain it.

You cannot point at a region of the file and say, that is where it keeps French, or that is the part that does sarcasm. There is an entire research field, interpretability, dedicated to prising these things open, and it has made real progress finding structures inside that correspond to recognisable concepts. But it is closer to neuroscience than to reading source code. People are poking at a system they built and running experiments to find out what it does.

That is a strange situation for engineering to be in. We built a thing out of arithmetic, at a scale where the arithmetic produces behaviour nobody explicitly designed, and now we study it empirically to find out what we made.

Whatever else you think about all this, that is a hell of a thing to be true.

Frequently asked questions

What is a neural network, in plain English?

Layers of very simple units, each one multiplying its inputs by a set of numbers called weights, adding them up, and passing the result on. Stack enough of them together, tune the weights on enormous amounts of data, and useful behaviour comes out. No rules are written by hand.

Why do LLMs need graphics cards?

Because running a neural network means doing the same arithmetic across huge grids of numbers, which is exactly what graphics cards were built to do for 3D rendering. A CPU has a handful of clever cores; a GPU has thousands of simple ones that all work at once. It was an accident of history that gaming hardware turned out to be the right shape for AI.

Did Doom lead to modern AI hardware?

Indirectly, yes, and the chain is real. Doom created mass demand for fast 3D on ordinary PCs in 1993, though it was rendered entirely on the CPU. That demand funded the first consumer 3D accelerators from 1996, Carmack’s GLQuake proved they were worth buying in 1997, shaders made them programmable by 2001, researchers hijacked them for general computation, and NVIDIA formalised that as CUDA in 2007. Five years later AlexNet trained on two gaming cards and the AI boom had its hardware.

What was the 2017 transformer paper?

“Attention Is All You Need”, published by eight Google researchers in June 2017. It introduced the transformer architecture, which lets a model look at an entire passage at once rather than reading word by word. That made training massively parallel, which made scale possible, which produced every large language model since. The T in GPT stands for transformer.

What does “attention” mean in an AI model?

A mechanism that lets every word in a passage look at every other word and decide which ones are relevant to it. It is how a model works out that “it” refers to the server rather than the memory in a sentence mentioning both.

Do LLMs actually understand anything?

Genuinely contested, and anyone claiming certainty in either direction is overselling. What is clear is that predicting text well at this scale requires internal structure representing how things relate to each other, which is more than a lookup table and less than a settled philosophical question.

Why do LLMs hallucinate?

Because they generate plausible continuations rather than retrieving facts. Nothing in the system checks claims against reality, so when plausible and true diverge, the model has no way to notice. Putting real source material into the prompt is the practical fix.

Related guides

• Best GPU for Running Local LLMs: A VRAM-First Buyer’s Guide - which card to buy, by memory tier
• Local LLM Jargon Explained: Tokens, Weights, VRAM and Quantisation - the vocabulary, and what your hardware can actually run
• Build a Headless Ubuntu LLM Server with Ollama on 8 GB VRAM - the practical build

This article may contain affiliate links to Amazon, eBay and other retailers. I may earn a small commission from qualifying purchases made through these links at no additional cost to you.