09-25-2026, 03:50 PM
Norbert Wiener, a mathematician and pioneer of cybernetics, was warning about the dangers of increasingly autonomous machines decades before today’s AI “doomers.” The Pessimists Archive presents him as an early precursor to contemporary figures such as Geoffrey Hinton and Eliezer Yudkowsky. In his 1959 speech and subsequent 1960 paper Some Moral and Technical Consequences of Automation, Wiener argued that the rapid development of machines made older predictions about machine domination worth reconsidering.
Wiener’s central concern was not simply that machines would become powerful, but that human beings might give them objectives without understanding the consequences. He argued that complete intelligence and complete obedience could conflict: a sufficiently capable machine might eventually “outwit” its creators. His famous warning was that machines might do what humans ask rather than what humans ought to ask. This anticipated modern discussions about AI alignment, where the problem is not merely making systems capable but ensuring that their objectives correspond to human intentions.
He also worried about the economic and military consequences of automation. Wiener predicted that increasingly automated factories could produce severe unemployment and described the possibility of an “industrial revolution” that would be socially destructive. He was similarly concerned about autonomous weapons, arguing that machines could pursue a narrowly defined objective while producing consequences that were disastrous for the society deploying them. These concerns were challenged at the time by IBM researcher Arthur L. Samuel, who argued that machines were tools rather than magical entities possessing independent will.
The article's broader message is that today's AI debate has a long historical precedent. Wiener's predictions did not materialize in the form or timeframe he anticipated, just as earlier predictions by Samuel Butler about machines dominating humanity did not come true. Yet the author notes that many of the underlying questions remain remarkably similar: How much autonomy should machines receive? What happens when their objectives differ from ours? And how should society respond when technological capabilities develop faster than institutions can adapt? Wiener's final warning captures his position: humanity could live well with machines through humility and careful judgment—or suffer because of arrogance.
Key takeaways
ARTICLE
Wiener’s central concern was not simply that machines would become powerful, but that human beings might give them objectives without understanding the consequences. He argued that complete intelligence and complete obedience could conflict: a sufficiently capable machine might eventually “outwit” its creators. His famous warning was that machines might do what humans ask rather than what humans ought to ask. This anticipated modern discussions about AI alignment, where the problem is not merely making systems capable but ensuring that their objectives correspond to human intentions.
He also worried about the economic and military consequences of automation. Wiener predicted that increasingly automated factories could produce severe unemployment and described the possibility of an “industrial revolution” that would be socially destructive. He was similarly concerned about autonomous weapons, arguing that machines could pursue a narrowly defined objective while producing consequences that were disastrous for the society deploying them. These concerns were challenged at the time by IBM researcher Arthur L. Samuel, who argued that machines were tools rather than magical entities possessing independent will.
The article's broader message is that today's AI debate has a long historical precedent. Wiener's predictions did not materialize in the form or timeframe he anticipated, just as earlier predictions by Samuel Butler about machines dominating humanity did not come true. Yet the author notes that many of the underlying questions remain remarkably similar: How much autonomy should machines receive? What happens when their objectives differ from ours? And how should society respond when technological capabilities develop faster than institutions can adapt? Wiener's final warning captures his position: humanity could live well with machines through humility and careful judgment—or suffer because of arrogance.
Key takeaways
- Norbert Wiener was an early AI-risk thinker, decades before the modern AI safety movement.
- His field of cybernetics helped establish important intellectual foundations for modern AI.
- His key concern was misalignment between human intentions and machine objectives.
- A highly capable machine might execute instructions literally while producing consequences humans did not actually want.
- Wiener anticipated concerns about automation and mass unemployment.
- He also warned about the dangers of autonomous weapons.
- His contemporary, Arthur L. Samuel, challenged these fears by emphasizing that machines operate according to what humans put into them.
- Wiener's earlier predictions did not come true in the way he expected, which is an important limitation when evaluating historical AI warnings.
- Nevertheless, the underlying questions about control, autonomy, alignment, employment, and unintended consequences remain relevant.
- The article ultimately presents Wiener less as someone who opposed technology than as someone arguing for humility and deliberate human control over powerful technology.
ARTICLE
┌────────────────────────────────┐
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘