The History of Computing & AI (Part 6): The Architecture of Memory — The Von Neumann Concept, Shannon’s Entropy, and the Manchester Baby

For all its blistering electronic speed, the ENIAC was crippled by a fatal architectural flaw: it had no memory for programs. To change a calculation from artillery ballistics to nuclear fission, technicians spent days physically pulling, untangling, and reconnecting hundreds of heavy patch cables. In 1945, Hungarian-American polymath John von Neumann articulated the conceptual paradigm that solved this crisis: store the program instructions inside the very same electronic memory that holds the data.

1. The Stored-Program Concept: The EDVAC Synthesis

While serving as a consultant on the Manhattan Project and observing the construction of ENIAC, John von Neumann engaged in intense daily discussions with J. Presper Eckert and John Mauchly regarding the successor machine: the EDVAC (Electronic Discrete Variable Automatic Computer). In June 1945, von Neumann synthesized these ideas into a 101-page draft: First Draft of a Report on the EDVAC.

Although intended as an informal working document for the project team, Herman Goldstine mimeographed and distributed copies across the global scientific community bearing only von Neumann’s name on the cover—permanently cementing the design in computer history as the Von Neumann Architecture.

The Five Core Organs

Von Neumann modeled the computer after the human nervous system, decomposing the machine into five interconnected functional subsystems:

• Central Arithmetical (CA / ALU): Executes elementary arithmetic operations ($+$, $-$, $\times$, $\div$) and bitwise logical comparisons.
• Central Control (CC / Control Unit): Fetches instructions sequentially from memory, decodes operation codes, and orchestrates gating control signals across all organs.
• Memory (M): A unified, linearly addressable storage holding both numerical data operands and coded program instructions without physical distinction.
• Input (I) & Output (O): Transducers converting external media (punched cards, paper tape, magnetic wire) into internal electronic words, and vice versa.

“` +——————————————————-+ | CENTRAL PROCESSING UNIT | | +——————–+ +——————–+ | | | Control Unit | | Arithmetic Logic | | | | (PC, IR, Decoder) | | Unit (ALU, Acc) | | | +——————–+ +——————–+ | +——————————————————-+ ^ ^ | Address Bus | Data / Control v v +——————————————————-+ | UNIFIED MEMORY | | [ Instructions: ADD R1, R2 | Data: 0x4F ] | +——————————————————-+ ^ ^ | | +—————+ +—————+ | Input Device | | Output Device | +—————+ +—————+ “`

The Von Neumann Bottleneck

By routing both instructions and data across a shared system bus, the architecture introduced an intrinsic physical limitation: the Von Neumann Bottleneck. Because the CPU can fetch only one word (either an instruction or a data operand) per memory cycle, computational throughput is fundamentally constrained by memory bus bandwidth—a problem modern computer architects still combat using multi-level SRAM caches, branch prediction, and prefetching.

2. Claude Shannon and the Birth of Information Theory

Simultaneously in 1948, mathematician and cryptographer Claude Elwood Shannon published his foundational paper at Bell Telephone Laboratories: A Mathematical Theory of Communication. Shannon did for communication what Isaac Newton did for mechanics: he established its fundamental physical laws.

Shannon defined the fundamental unit of information as the bit (binary digit, a term suggested by John Tukey). He proved that information is not semantic meaning, but the reduction of uncertainty. He formalized the mathematical measure of information—Information Entropy ($H$)—mirroring the statistical thermodynamics of Ludwig Boltzmann:

$$H(X) = -\sum_{i=1}^{n} P(x_i) \log_2 P(x_i)$$

If an event is completely predictable ($P = 1$), its entropy is $0$ bits; if a coin flip is perfectly unbiased ($P = 0.5$), it yields exactly $1$ bit of information. Shannon went on to derive the theoretical limit of error-free data transmission over a noisy channel: the Shannon-Hartley Theorem:

$$C = B \log_2 \left(1 + \frac{S}{N}\right)$$

where $C$ is channel capacity in bits per second, $B$ is bandwidth in Hertz, and $S/N$ is the signal-to-noise power ratio. Shannon proved that with optimal digital encoding, information could be transmitted across noisy physical channels with arbitrarily low error.

3. The Memory Problem: Mercury Delay Lines vs. Williams Tubes

In 1945, the major obstacle to building a stored-program computer was not logical architecture, but physics: how do you build an electronic memory that can store thousands of bits and access any random bit in microseconds?

Two rival technologies emerged:

1. Acoustic Mercury Delay Lines: Used in Maurice Wilkes’s EDSAC (Cambridge, 1949). Bits were converted into ultrasonic sound pulses traveling through a five-foot steel tube filled with liquid mercury, detected by quartz crystals at the other end, amplified, reshaped, and recirculated. Memory was serial, dynamic, and fragile to temperature fluctuations.
2. The Williams-Kilburn Tube: Invented by Frederic Williams and Tom Kilburn at the University of Manchester. It utilized a standard commercial cathode-ray tube (CRT). By firing an electron beam at the phosphor screen, secondary electron emission left localized electrostatic charges (+ for 1, neutral for 0) that persisted for a fraction of a second, refreshed dynamically by sweeping the beam 60 times a second.

4. The Manchester Baby: The First Stored Program

To test whether Williams-Kilburn tubes could reliably hold data while a CPU executed instructions, Tom Kilburn and Freddie Williams constructed a stripped-down testbed: the Manchester Small-Scale Experimental Machine (SSEM), affectionately known as the Manchester Baby.

On Monday, June 21, 1948, the Baby made history. Kilburn wrote a 17-instruction program to find the highest proper factor of $2^{18} = 262,144$ by testing every integer descending from $262,143$ using repeated subtraction:

“` Manchester Baby Machine Code Program (June 21, 1948): Memory: 32 words of 32 bits (1,024 bits total) Word 0: Variable (Running Difference) Word 1: Divisor Under Test Word 2: Fixed Constant 1 Instruction Set: 7 primitive opcodes (LDN, STO, SUB, CMP, JMP, JRP, STP) “`

Kilburn flipped the switches to enter the program into the Williams tube, initiated the run, and walked away. The cathode-ray tube flickered through 3.5 million operations. Fifty-two minutes later, the machine halted. The display showed the correct answer: $131,072$.

For the first time in human history, an electronic machine had loaded a program into internal memory, executed it autonomously without human intervention, and halted on the correct mathematical truth.

---