In the mid-1960s, the mathematician and Bletchley Park cryptographer I. J. Good proposed a thought experiment that has since become the secular gospel of Silicon Valley. If we were to build an “ultra-intelligent machine,” it could then design even better machines, sparking an intelligence explosion that would leave human cognition far behind, he said, adding that the first such machine would be “the last invention that man need ever make.”
Today, that prophecy, once the stuff of science fiction, has become the core objective of the world’s most powerful institutions. Google DeepMind’s chief strategy officer Demis Hassabis, for example, speaks of “solving intelligence” to “solve everything else.” It is a seductive story. However, even if we assume, for the sake of argument, that future systems can learn, experiment and generate genuinely novel solutions far beyond today’s models, the last-invention thesis still rests on multiple questionable assumptions.
The first is that innovation resembles a frictionless sprint from idea to impact. It does not. The discovery process is more like a chain, only as strong as its weakest link.
Illustration: Yusha
These weak links define much of human progress. In 1986, the space shuttle Challenger broke apart 73 seconds after launch, not because of a failure in its world-class engines or software, but because a small rubber seal failed when subjected to cold atmospheric temperatures (as the Nobel laureate physicist Richard Feynman brilliantly exposed at hearings into the disaster). The “O-ring” has since become a metaphor for the kinds of critical bottlenecks that could sink even the most sophisticated systems.
Discovery works the same way. Artificial general intelligence (AGI), generally understood as a model that can perform any cognitive task, might dramatically accelerate early-stage medical research, but if it cannot navigate clinical trials, manufacture at scale, or secure regulatory approval, the “breakthrough” never becomes an invention that improves lives. When the early stages of discovery are automated, the human role does not vanish; it simply migrates toward the remaining bottlenecks, where judgment, tacit knowledge and practical know-how are what matter.
This complication points us to an even bigger one: AGI would not just have to outperform humans; it would have to outperform humans using AGI. For the last-invention story to hold, people would have to become unnecessary even as partners or supervisors to AIs.
Intelligence is not a quantity: “More” does not simply replace “less.” Even a very capable AGI might be different in kind than a human: exceptional at speed and pattern-finding, but fragile when confronted with rare cases. Different strengths imply different blind spots, and when those do not overlap, combining human and machine judgment continues to beat either one alone.
The game of go offers a useful reminder. After AlphaGo beat Lee Sedol 4-1 in 2016, its superiority to human players seemed settled. However, in 2023, researchers showed that by steering top engines into unusual positions outside their training, a human amateur with modest computing skills could reliably defeat the best programs. Apparent supremacy can still hide systematic weaknesses and that is often where human input adds the most value.
A third problem concerns knowledge itself. The last-invention thesis assumes that all relevant information can be codified, but this is usually not the case. Few inventions changed the world more than the Ford Model T, which transformed the automobile into a mass-market product. However, Henry Ford’s achievement lay not just in a new design. More important was his approach to organizing production.
That is why delegations from Italy, Germany, the Soviet Union and elsewhere traveled to study Ford’s factories firsthand. The crucial know-how could not be gleaned from any blueprint. It was embedded in routines, sequencing, tooling and day-to-day problem-solving by those on the shop floor. Similarly, Toyota’s lean-production system was difficult to replicate because it is embedded in human routines and culture, not hardware.
More intelligence does not automatically overcome the “knowledge problem” — the fact that what makes complex systems work is dispersed, local, often unspoken information. If knowledge were frictionlessly portable, industries would not cluster so intensely, as in Silicon Valley or the City of London.
AI enthusiasts might respond by saying, “Fine, put sensors, cameras and microphones everywhere, and we would codify the missing knowledge.” However, this strategy assumes that people being monitored would openly communicate and share the knowledge they generate, and it assumes away politics and the law. Recording “everything, everywhere” would collide with the EU’s General Data Protection Regulation, which has become a blueprint for privacy regulation worldwide.
Moreover, the EU’s AI Act does not give a free pass to the surveillance-heavy deployments that would be necessary to harvest human know-how at scale. Even if it did, one cannot assume that all human know-how, let alone judgment, is so easily digitized.
Ultimately, AGI could automate intelligence, but the process of invention depends on something more. Often, the hard part is not thinking up a solution but translating it into practice. You need local know-how, trusted routines, supply chains and institutional capacity to make something work reliably in the real world. More intelligence does not automatically produce those complements.
AGI might change discovery by making expertise cheaper and experimentation faster. However, “humanity’s last invention” is a much stronger claim. For it to be true, we would need a world where practical know-how is fully transferable through digital channels and where responsibility can be automated along with cognition. That is not the world we live in.
As intelligence gets cheaper, the assets that command the highest value could change. The advantage would go to those who can deliver outcomes. Humans are not becoming redundant; they are becoming the world’s most decisive bottlenecks.
Carl Benedikt Frey, associate professor of AI & Work at the Oxford Internet Institute and director of the Future of Work Program at the Oxford Martin School, is the author of How Progress Ends: Technology, Innovation and the Fate of Nations.
Copyright: Project Syndicate
Taiwanese pragmatism has long been praised when it comes to addressing Chinese attempts to erase Taiwan from the international stage. “Taipei” and the even more inaccurate and degrading “Chinese Taipei,” imposed titles required to participate in international events, are loathed by Taiwanese. That is why there was huge applause in Taiwan when Japanese public broadcaster NHK referred to the Taiwanese Olympic team as “Taiwan,” instead of “Chinese Taipei” during the opening ceremony of the Tokyo Olympics. What is standard protocol for most nations — calling a national team by the name their country is commonly known by — is impossible for
China’s supreme objective in a war across the Taiwan Strait is to incorporate Taiwan as a province of the People’s Republic. It follows, therefore, that international recognition of Taiwan’s de jure independence is a consummation that China’s leaders devoutly wish to avoid. By the same token, an American strategy to deny China that objective would complicate Beijing’s calculus and deter large-scale hostilities. For decades, China has cautioned “independence means war.” The opposite is also true: “war means independence.” A comprehensive strategy of denial would guarantee an outcome of de jure independence for Taiwan in the event of Chinese invasion or
A recent Taipei Times editorial (“A targeted bilingual policy,” March 12, page 8) questioned how the Ministry of Education can justify spending NT$151 million (US$4.74 million) when the spotlighted achievements are English speech competitions and campus tours. It is a fair question, but it focuses on the wrong issue. The problem is not last year’s outcomes failing to meet the bilingual education vision; the issue is that the ministry has abandoned the program that originally justified such a large expenditure. In the early years of Bilingual 2030, the ministry’s K-12 Administration promoted the Bilingual Instruction in Select Domains Program (部分領域課程雙語教學實施計畫).
Former Fijian prime minister Mahendra Chaudhry spoke at the Yushan Forum in Taipei on Monday, saying that while global conflicts were causing economic strife in the world, Taiwan’s New Southbound Policy (NSP) serves as a stabilizing force in the Indo-Pacific region and offers strategic opportunities for small island nations such as Fiji, as well as support in the fields of public health, education, renewable energy and agricultural technology. Taiwan does not have official diplomatic relations with Fiji, but it is one of the small island nations covered by the NSP. Chaudhry said that Fiji, as a sovereign nation, should support
RSS