The AI Spotlight Follow
A Forgotten 1940s Science Is Hiding Inside Every AI You Use. Its Founder Warned Us About Exactly This
Big Ideas

A Forgotten 1940s Science Is Hiding Inside Every AI You Use. Its Founder Warned Us About Exactly This

Before 'AI' existed, a group of mathematicians, engineers, doctors and anthropologists invented cybernetics: the science of feedback, control and information. It gave us the computer's blueprint, the first artificial neurons and the math behind chatbots. Its founder also predicted the problem AI labs now call alignment.

Advertisement

Cybersecurity, cyberspace, cyberpunk: almost nobody asks where "cyber" came from.

It came from a science born in the 1940s that most people have never heard of: cybernetics. It's worth knowing, because it quietly planted the seeds for a startling amount of the modern world. The basic design of nearly every computer. The information theory inside every phone and network. The first artificial neurons, the distant ancestors of today's chatbots.

And its founder, a child prodigy turned MIT mathematician, saw something coming that today's AI labs are still struggling with. He gave it a warning. We now give it a name: alignment.

Here's the story, in plain language, and why it suddenly matters again.

Advertisement

The steersman

Start with the word itself, because it already tells you where this is going.

Kybernētēs is ancient Greek for steersman, the person who steers a ship. Through Latin, the same root gave English governor, governance and government. From steering a boat to steering a state, the word has always been about one idea: keeping something on course.

The man who turned that word into a science was Norbert Wiener. He earned his PhD from Harvard at 18 and spent his career at MIT.

The problem that started it: hitting a plane that's trying not to be hit

During World War II, Wiener and the engineer Julian Bigelow worked on a very concrete military problem: how do you aim an anti-aircraft gun at a moving plane?

You can't shoot where the plane is. You have to shoot where it will be when the shell gets there. And the pilot knows he's being shot at, so he dodges.

Wiener and Bigelow realized that gun, gunner, plane and pilot form one connected system, each reacting to the others. The size of a miss gets fed back to correct the next shot. Information runs in a circle.

That loop has a name: feedback.

When feedback goes wrong

They noticed something odd. If the correction is too strong, the system overshoots, corrects back too far, overshoots again, and the swings get bigger and bigger.

They asked a physiologist, Arturo Rosenblueth, whether humans ever do this. He said yes, and it has a name: intention tremor. Some patients with damage to a part of the nervous system don't shake at rest. They shake when they reach for something.

A gun and a nervous system had the same problem. In 1943, Rosenblueth, Wiener and Bigelow described this in a paper called "Behavior, Purpose and Teleology," often treated as the birth certificate of cybernetics.

The thermostat version

If that sounds abstract, think of a thermostat. It measures the room, compares it to your setting and corrects. Actual value, target value, correction. That's the whole loop. Wiener's big claim was that living things and machines steer themselves by the same principle: take in information, compare where you are with where you're supposed to be, and adjust.

Feedback comes in two flavors: negative feedback dampens and pulls a system back to target; positive feedback amplifies, which is how you get panics, explosions and microphone screech.

Information becomes the most important thing

Once you see steering as a loop of information, something follows. If information flows, the system can steer. If the flow is disturbed, the system steers blind.

Apply that to a society. News and media become the sensors through which a society learns how it's doing. If the important facts reach people, the society can react. If the flow is polluted, by propaganda for instance, it acts on false readings. Like a thermostat that believes it's a comfortable 72°F while January blows through an open window.

In 1948 Wiener pulled all of this together in a book whose title is its thesis: Cybernetics: Or Control and Communication in the Animal and the Machine. For Wiener, control engineering, communications, brain science and computing machines were all one field.

And right at the start of that book he made a prediction that reads very differently in 2026. Roughly: just as the first industrial revolution devalued human muscle, the next one would devalue the human brain.

One table, many sciences: the Macy conferences

No single genius made cybernetics. It took a place where people from completely different fields discovered they'd been talking about the same thing all along.

That place was the Macy conferences: ten meetings in New York between 1946 and 1953, funded by the Josiah Macy Jr. Foundation. Their official title was a mouthful: Circular Causal and Feedback Mechanisms in Biological and Social Systems. The young Austrian physicist Heinz von Foerster suggested using Wiener's shorter word in the title instead.

What was radical was who sat at the table: mathematicians next to psychiatrists, engineers next to anthropologists. The goal was a common language for steering, whether in a brain, a machine, a family, a company or a whole country.

Three building blocks came out of that room. Understand them, and you understand cybernetics.

Building block 1: brains and machines run on the same logic

The first came mainly from Warren McCulloch, a neurophysiologist who chaired the conferences. In 1943, he and the young logician Walter Pitts showed that a neuron could be described, in a heavily simplified way, as a switch: it fires or it doesn't. One or zero.

Real neurons are far more complex, but the point was that networks of such switches can carry logic, the same AND and OR circuits that are the building blocks of computers.

For the first time, the brain could be described as a kind of computing machine. Those simplified "McCulloch-Pitts neurons" are the distant ancestors of the artificial neural networks behind today's AI.

Building block 2: feedback, and the human version

You already know Wiener's feedback. The human side came largely through the anthropologist Gregory Bateson. In his fieldwork he'd watched groups escalate against each other: one side brags, the other brags harder, until it blows up. He called it schismogenesis.

At the Macy table he realized what he'd been describing all along: a feedback loop that doesn't dampen but amplifies. Positive feedback, in human relationships.

Building block 3: the observer is part of the system

The third block came a bit later and took the field a step further.

Wiener's theory described systems from the outside: the thermostat, the gun, the nervous system. That's now called first-order cybernetics. Von Foerster asked: what about the observer?

People and societies watch themselves, and watching changes them. A weather forecast doesn't change the weather. An economic forecast changes the economy, because people react to it. Von Foerster called this second-order cybernetics, "the cybernetics of cybernetics." Once you include yourself in the system you're analyzing, you're also responsible for what you do.

Working alongside that idea, the biologists Humberto Maturana and Francisco Varela coined autopoiesis: the capacity of living systems to continuously produce themselves. Your body keeps replacing its own parts. You're less a fixed thing than a pattern that keeps rebuilding itself, like a flame. The concept became central to systems theory, especially in the work of sociologist Niklas Luhmann.

Legacy 1: the computer

Two names matter here.

John von Neumann, one of the 20th century's greatest mathematicians and a Manhattan Project veteran, sat at the Macy table from the start. In 1945 he described the design that now carries his name, the von Neumann architecture: program and data share the same memory while a central unit works through the instructions.

Von Neumann even described his computer's building blocks in terms of McCulloch-Pitts neurons. Brain and computer were thought about together from the start.

The second name is Alan Turing. He never sat at the New York table, but he moved in the same intellectual world in Britain. He belonged to the Ratio Club, a small cybernetics discussion group, and corresponded with Wiener. His 1936 paper described the Turing machine, the theoretical model behind every computer, and showed that such a machine could in principle carry out any step-by-step logical procedure.

In 1950 he proposed what we call the Turing test: if you chat with a machine by text and can no longer tell whether a human or a machine is answering, you should grant that it thinks.

Seventy-five years later, it happened. In a 2025 study at UC San Diego, GPT-4.5, told to adopt a particular human persona, was judged to be the human 73% of the time in five-minute, three-way conversations, more often than the actual humans it was up against. But the test only measures whether a machine behaves like a person in conversation. It says nothing about whether anything is going on inside.

Legacy 2: information

If steering runs on information, what is information?

Wiener offered a bold answer: information is information, not matter or energy, a third fundamental quantity in its own right.

The mathematical measure arrived the same year, 1948, from Claude Shannon, who founded information theory in essentially one paper. His theory is at work in every digital transmission and every zip file.

Shannon deliberately set meaning aside. To his math, a love letter and a random jumble of letters are the same kind of thing: what counts is the statistical structure of the signal. That was argued over at the Macy conferences too. Bateson later pushed back with a famous line: information is "a difference that makes a difference," which means it depends on the receiver.

Ever since, there have been two ways of talking about information: Shannon's technical one and the one about meaning. That split is more relevant now than ever.

Around 1950, Shannon ran experiments in which people guessed the next letter in a piece of English text, to measure how predictable language is. That is, almost exactly, the task today's large language models are trained on, using networks descended from McCulloch and Pitts.

These models work purely with statistics, with no built-in meaning, and yet they produce text that is meaningful to us. So does meaning arise inside the system, or only in the person reading it? Cybernetics framed that question decades before ChatGPT made it urgent.

Legacy 3: self-regulation and the law of requisite variety

The last legacy may be the most important for anyone who runs anything.

In 1948 the British psychiatrist W. Ross Ashby built a machine called the homeostat: four interconnected boxes, each with a moving needle, each influencing the others. Whenever a needle drifted out of its allowed range, the machine randomly rewired itself and kept trying until the whole system was stable again. A machine that found its way back to balance without anyone telling it how.

In his 1956 book An Introduction to Cybernetics, Ashby stated the principle he's still famous for, the law of requisite variety. "Variety" just means how many different states something can be in. The law says: a system can only stay in control if it has at least as many ways to respond as there are disturbances that can hit it.

Picture a goalkeeper facing penalties who can only ever dive left. He'll lose to any shooter who sometimes aims right. Only variety can absorb variety.

So anyone trying to manage complexity has exactly two options:

  1. Increase your own variety: more ways to respond.
  2. Reduce the variety of your environment: make things simpler so fewer disturbances get through.

Parents do the second when they baby-proof a room. That option has a darker version.

The bold experiment: running a country like a nervous system

The next generation tried to apply these tools to organizations, and in one extreme case, to an entire state.

Stafford Beer, a British management consultant, founded management cybernetics. He took Ashby's law to its conclusion: a system only stays viable if its parts have as much autonomy as possible and can organize themselves. From that he built the Viable System Model, modeled on the human nervous system and laid out in his 1972 book Brain of the Firm.

Beer said every organization that wants to survive needs five functions. Take a bakery chain:

  1. The units doing the work. The individual shops, largely independent.
  2. Coordination that smooths conflicts between them, like a shared delivery schedule so two shops don't fight over the same van.
  3. Here-and-now management that allocates flour, staff and money, and drops in unannounced to see what's really going on.
  4. An outward-looking function: what is the competition doing, and do customers suddenly want spelt?
  5. Identity: what kind of bakery do we want to be? This is where today is balanced against tomorrow.

Two features are especially clever. First, recursion: each shop is itself a complete viable system with all five functions, and the chain is part of something bigger, like cells inside organs inside bodies inside societies. Second, Ashby's law: a head office can never have enough variety to steer everything itself, so it must push complexity downward. For Beer, autonomy wasn't a moral nicety. It was a precondition for control.

Project Cybersyn

In 1971, Chile's newly elected socialist government under Salvador Allende invited Beer to help steer a large, freshly nationalized industrial sector. The result was Project Cybersyn .

Through a network of telex machines, factories were to report their figures to the government daily. In an operations room with a ring of futuristic chairs, officials could watch the economy almost like the bridge of a starship. Beer later described bringing a large share of the economy's major firms into the model, using recursion to break the economy into branches and sub-branches.

It was never finished. The United States had worked for years to destabilize Allende's government, including through the CIA. On September 11, 1973, the military under Augusto Pinochet staged a coup. Allende died in the presidential palace, Pinochet's dictatorship killed and tortured thousands, and the control room was destroyed.

The dark side: same tools, different hands

So far, cybernetics sounds hopeful: autonomy, self-organization, participation. But the tools of steering don't care who holds them.

In 1950 Wiener published a second, more popular book, The Human Use of Human Beings. The title says it all. People should do human things, not serve as parts in a machine.

Shortly after his first book appeared, a French priest imagined a "governing machine" that would run a state by cybernetic principles. Wiener's answer, in essence: the real danger isn't that machines seize power on their own. It's that people or small groups use such machines to extend their control over everyone else, or start doing politics as if the decisions had been worked out by a machine.

That's the heart of it. Power hides behind machines. A decision looks objective because "the algorithm calculated it," and responsibility disappears into thin air.

Remember Ashby's darker option? If a system can't handle people's variety, it can try to reduce it, making people more predictable.

In September 2024, Oracle co-founder Larry Ellison told analysts that with AI analyzing constant footage from security cameras, police body cams, doorbells and dash cams, "citizens will be on their best behavior because we are constantly recording and reporting everything that's going on." He framed it as accountability for police too. Critics heard a description of exactly the machine Wiener feared.

Control doesn't need cameras, though. Whoever shapes the flow of information steers too. Cambridge Analytica harvested data from up to 87 million Facebook profiles through a harmless-looking personality quiz and used it to target voters, including in the 2016 U.S. election. How well that targeting worked is disputed. The logic was pure cybernetics: measure each person, nudge, measure again, adjust.

God and Golem: Wiener's warning about AI

Wiener's last book, God & Golem, Inc., came out in 1964, the same year he died in Stockholm at 69. It won the National Book Award after his death.

In Jewish legend, the golem is an artificial servant made of clay that does what it's told. In many tellings it grows, slips out of control and must be deactivated. Wiener asked: is God to the golem as man is to the machine?

In the book he discussed a checkers program that learned to beat its own creator. He reflected on machines that reproduce themselves. And he warned against "gadget worship," using images like the sorcerer's apprentice who can't stop the broom, and "The Monkey's Paw," the horror story about a talisman that grants every wish, but always in the most terrible way. The machine fulfills the wish literally, not as it was meant.

In the Cold War, Wiener had automated nuclear systems in mind. Today the problem has its own field. Alignment: how do you get a system to do what we mean, not just what we say?

Wiener wasn't warning about machines that secretly want something against us. He was warning about machines that do exactly what we tell them, at a speed and scale where we no longer notice our mistakes in time, let alone fix them. A control loop where feedback arrives too late no longer controls anything.

And he warned about people using machines to dodge responsibility. There's always someone who formulated the wish. The machine doesn't take responsibility off anyone's shoulders. It makes visible who carries it.

Why this matters now

  • Feedback loops are the hidden engine of modern AI. Training a model is a loop: predict, compare to the target, correct. Recommendation feeds are loops: show, measure, adjust.
  • Information quality is a safety issue. If AI systems shape what billions of people see, the "thermostat reading" a society steers by can be distorted at scale.
  • Requisite variety explains AI's appeal and its danger. Organizations adopt AI to increase their capacity to respond. The tempting shortcut is the other option: make people more predictable.
  • Alignment is an old problem with a new urgency. The monkey's paw problem is now an engineering problem.

Practical takeaways

  • When you deploy an AI system, design the feedback loop first. How will you know it's wrong, and how fast?
  • Watch for "the algorithm decided" language. Somebody chose the target value. Ask who.
  • Push complexity down. Beer's lesson for managers still holds: central control can't match the variety of the real world.
  • Be careful what you wish for, literally. Write goals for AI agents that include what you mean, not just what you can measure.

Counterpoints

  • Cybernetics overreached. Its grand claim to unify all sciences didn't survive, and some of its social applications were naïve.
  • Cybersyn's promise is debated. It was never completed, so judgments about whether it would have worked are speculative.

FAQ

What is cybernetics in simple terms? The science of how systems, living or mechanical, steer themselves using feedback: measure, compare to a goal, correct.

Who founded cybernetics? Norbert Wiener named and defined it in his 1948 book, building on 1943 work with Arturo Rosenblueth and Julian Bigelow, and the field grew through the Macy conferences.

How is cybernetics connected to AI? Directly. McCulloch-Pitts neurons inspired neural networks, Shannon's information theory underpins language modeling, and Wiener's warnings anticipated today's alignment problem.

What is the law of requisite variety? Ashby's principle that a controller needs at least as many possible responses as the disturbances it must handle. Only variety can absorb variety.

The bottom line

As a named field, cybernetics faded by the 1970s. Its ideas won so completely they became invisible, dissolved like sugar in tea: you can't see it, but you taste it everywhere. Feedback, information, self-regulation: they're baked into every computer, every network and every AI model you use.

But the founder of the field left two warnings alongside the blueprints. Machines will do exactly what we say, faster than we can catch our mistakes. And whoever holds the steering wheel matters more than the wheel.

After all the talk of control, there's one more lesson worth keeping. The most human move is sometimes to loosen your grip, trust your judgment, and remember that you're not a component in anyone's machine.

Sources

Gemini 4 Neon Played a Live Piano Duet and Rebuilt a Villa in 21 Minutes. Then It Couldn't Find a FrogUp next · 12 min readGemini 4 Neon Played a Live Piano Duet and Rebuilt a Villa in 21 Minutes. Then It Couldn't Find a FrogGoogle's 'middle' model, Gemini 4 Neon, went through ten hard hands-on tests: a two-piano duet, video effects reverse engineering, an animated biology lesson, a full song, an Unreal Engine sword fight, an Airbnb rebuilt in Blender, brain scans and more. It was faster than its rivals almost everywhere. It was not better everywhere.Keep reading →