Von Neumann Architecture: How a Computer Works, Explained

One space for both data and instructions

When a computer runs a program, it needs to keep both the instructions to follow and the numbers to work on somewhere. In the von Neumann architecture, that “somewhere” is one and the same: a single memory holds both, and the processor reaches them through the same bus, the channel linking memory and the compute unit.

It’s a subtler idea than it sounds. The report that first describes it, John von Neumann’s “First Draft of a Report on the EDVAC,” published in 1945, notes that it would be “tempting to treat the entire memory as one organ, and to have its parts even as interchangeable as possible” — rather than dedicating one piece of hardware to data and another to instructions. Everything else follows from that choice: where a program sits, where its intermediate results go, how the processor finds them. Readers who want the story leading up to this point can start with the history of computers, which covers the machines that came before the EDVAC.

The organs of the EDVAC report

The 1945 document divides the machine into organs, a term that sounds almost biological today but was von Neumann’s own word choice. Three sit at the center of the architecture: the arithmetic unit (CA), which performs addition, subtraction, multiplication and division; the control unit (CC), which sequences the operations; and memory (M), which holds both the numerical data and the program’s instructions. Input and output organs round out the design, carrying information in and out of the system.

The instructions of a complex program, the report says, “may constitute a considerable material” that “must be remembered” — exactly like the intermediate results of a calculation. Treating both the same way, inside the same memory, is where the whole architecture starts.

Practical example: when a machine runs “add 2 and 3,” the processor first goes to memory to fetch the instruction (the “add” order), then goes back to memory to fetch the two numbers, then writes the result. Three separate trips, but one path: the same bus that carries instructions also carries data. An algorithm is exactly this sequence of steps that the control unit runs one after another.

The shared bus and the von Neumann bottleneck

Sharing memory and bus makes the machine simpler to build, but it comes at a price: if data and instructions travel the same channel, they can’t travel together. The processor has to wait its turn to read what it needs, and the bigger memory gets relative to bus speed, the heavier that wait becomes. Wikipedia calls this limit the “von Neumann bottleneck”: the limited data transfer rate between processor and memory compared with the amount of memory available. The phrase doesn’t appear in the original 1945 report; it’s a label that came later, popularized by computer scientist John Backus in a 1977 lecture, where he asked whether there wasn’t “a less primitive way of making big changes in the store than by pushing vast numbers of words back and forth through the von Neumann bottleneck.”

Comparing with the Harvard architecture

An alternative exists, with an equally historic name: the Harvard architecture. Instead of one bus, it uses two, one for data and one for instructions, with physically separate paths to memory. The advantage is that the processor can read an instruction and access a piece of data at the same moment, without either one waiting in line on the same channel.

The Harvard architecture is common in microcontrollers — Microchip Technology’s PIC and Atmel’s AVR are two examples — and in processors dedicated to audio or video, where read speed matters more than flexibility. In personal computers, though, the model that won out has stayed von Neumann’s.

Why it still underlies today’s computers

It isn’t just a matter of habit. IBM Research, explaining why this architecture also weighs on computing for machine learning and neural networks, notes that its strength lies in flexibility: separate but connected memory and compute units can be configured in almost any way, and that, IBM writes, is why the von Neumann architecture remains the most widespread choice for general-purpose computing. For very specific tasks, like processing huge amounts of numerical data in parallel, the shared bus becomes a bottleneck again, and that’s one reason research into new kinds of processors for artificial intelligence continues.

Who the idea really belongs to

The 1945 report carries only John von Neumann’s name; he had been brought on as a consultant to the EDVAC project in 1944. But the Wikipedia entry on the subject is blunt on one point: crediting the invention of the architecture to von Neumann alone would be incorrect, because John Mauchly and J. Presper Eckert, at the University of Pennsylvania’s Moore School of Electrical Engineering, had already laid out much of the stored-program computer design before discussing it with von Neumann and his colleague Herman Goldstine. It was Goldstine who circulated the still-unfinished document with only von Neumann’s name on the cover — to the consternation of Eckert and Mauchly, the same source reports. One fact anyone can check by reading the original document: the 1945 report carries no other signature.

Readers curious how these principles carried over into the operating systems of modern computers can continue with the history of Linux, which describes how the kernel manages memory and the processor on this same architecture today.