The multi-trillion-dollar artificial intelligence (AI) boom was built on certainty that generative models would keep getting exponentially better. Spoiler alert: they are not.
In simple terms, “scaling laws” said that if you threw more data and computing power at an AI model, its capabilities would continuously grow. However, a recent flurry of press reports suggests that is no longer the case, and AI’s leading developers are finding their models are not improving as dramatically as they used to.
OpenAI’s Orion is not that much better at coding than the company’s last flagship model, GPT-4, Bloomberg News said, while Alphabet Inc’s Google is seeing only incremental improvements to its Gemini software. Anthropic, a major rival to both companies, has fallen behind on the release of its long-awaited Claude model.
Illustration: Mountain People
Executives at OpenAI, Anthropic and Google all told me without hesitation in recent months that AI development was not plateauing. However, they would say that. The truth is that long-held fears of diminishing returns for generative AI, predicted even by Bill Gates, are becoming real.
Ilya Sutskever, an AI icon who popularized the bigger-is-better approach to building large language models, recently told Reuters that it had leveled off.
“The 2010s were the age of scaling,” he said. “Now we’re back in the age of wonder and discovery once again.”
“Wonder and discovery” puts quite the positive spin on “we have no idea what to do next.” It could also, understandably, spark anxiety attacks for investors and businesses, who are expected to spend US$1 trillion on the infrastructure needed to deliver on AI’s promise to transform everything.
Wall Street banks, hedge funds and private equity firms are spending billions on funding the buildout of vast data centers, a recent Bloomberg News investigation showed.
Does this all add up to a terrible gamble? Not exactly.
There is no question that the main beneficiaries of the AI boom have been the world’s largest tech companies. Quarterly cloud storage revenue for Microsoft Corp, Google and Amazon Inc has been growing at a steady clip and their market capitalizations, along with those of Nvidia Corp, Apple Inc and Meta Platforms Inc, have soared by US$8 trillion in aggregate over the past two years. Returns on investment for everyone else — their customers — are taking longer to show up.
Yet a break in the market hype around AI could be useful, just as it has been for previous innovations. That is because technology typically does not hit a brick wall and die, but goes through an S-curve. The idea of the S-curve is that initial progress takes years before rapidly accelerating, as we have seen over the last two years with generative AI, before it starts to slow again and, crucially, evolve.
Critics over the years, for instance, regularly declared Moore’s Law dead just before a manufacturing breakthrough for chips pushed it forward again. The development of airplanes progressed at a glacial pace until the transition from propellers to jets in the late 1950s led to a leap forward — before the technology seemed to plateau. Just like chip manufacturing, aviation’s development did not stall, it transformed; passenger planes have become far more fuel efficient, safer and cheaper to operate, even if they are only nominally faster than they were in the 1960s.
A similar plateau for AI and its scaling laws might also mean a new approach to development and measuring success, which until now has focused too much on capability, and not enough on other areas such as safety. Some of the most advanced generative AI models fall short on critical areas such as security and fairness, according to a recent academic study that measured how well they followed Europe’s upcoming AI law.
For much of this year already, AI researchers have been looking at new paths for improving their models that do not just involve throwing more data and computing power at them. One approach is to focus on enhancing a model after it has been trained, in the so-called inference phase. This can involve giving a model extra time to process multiple possibilities before settling on an answer, and it is why OpenAI described its most recent model, o1, as being better at “reasoning.”
The beauty of the S-curve is that it can give everyone else some breathing room, instead of clamoring for the latest tech that would give them an edge over their competitors. Companies that have been experimenting with generative AI and grappling with ways to boost their productivity, now have some time to redesign their workflows and business processes to better capitalize on current AI models, which are already powerful. (Remember, it took years for businesses to reorganize themselves around computers in the 1980s.)
Stanford University professor Erik Brynjolfsson’s writing on the “productivity paradox” said that output often appears to stall or drop when major new technologies arrive, before surging. A pause for AI gives businesses more space in that all-important investment phase.
It also gives regulators time to design more effective guardrails. The EU’s AI Act, which companies would be subject to from 2026, needs to be more specific in how it defines harms. As standards bodies do that work, it helps that newfangled models leading to a batch of unexpected problems are not about to flood the market.
Generative AI has been on a bullet train during the past two years, and the momentum has clearly been lucrative for tech giants. A slowdown at the station offers a much-needed break for everyone else.
Parmy Olson is a Bloomberg Opinion columnist covering technology. A former reporter for the Wall Street Journal and Forbes, she is author of Supremacy: AI, ChatGPT and the Race That Will Change the World.
Taiwanese pragmatism has long been praised when it comes to addressing Chinese attempts to erase Taiwan from the international stage. “Taipei” and the even more inaccurate and degrading “Chinese Taipei,” imposed titles required to participate in international events, are loathed by Taiwanese. That is why there was huge applause in Taiwan when Japanese public broadcaster NHK referred to the Taiwanese Olympic team as “Taiwan,” instead of “Chinese Taipei” during the opening ceremony of the Tokyo Olympics. What is standard protocol for most nations — calling a national team by the name their country is commonly known by — is impossible for
China’s supreme objective in a war across the Taiwan Strait is to incorporate Taiwan as a province of the People’s Republic. It follows, therefore, that international recognition of Taiwan’s de jure independence is a consummation that China’s leaders devoutly wish to avoid. By the same token, an American strategy to deny China that objective would complicate Beijing’s calculus and deter large-scale hostilities. For decades, China has cautioned “independence means war.” The opposite is also true: “war means independence.” A comprehensive strategy of denial would guarantee an outcome of de jure independence for Taiwan in the event of Chinese invasion or
A recent Taipei Times editorial (“A targeted bilingual policy,” March 12, page 8) questioned how the Ministry of Education can justify spending NT$151 million (US$4.74 million) when the spotlighted achievements are English speech competitions and campus tours. It is a fair question, but it focuses on the wrong issue. The problem is not last year’s outcomes failing to meet the bilingual education vision; the issue is that the ministry has abandoned the program that originally justified such a large expenditure. In the early years of Bilingual 2030, the ministry’s K-12 Administration promoted the Bilingual Instruction in Select Domains Program (部分領域課程雙語教學實施計畫).
Former Fijian prime minister Mahendra Chaudhry spoke at the Yushan Forum in Taipei on Monday, saying that while global conflicts were causing economic strife in the world, Taiwan’s New Southbound Policy (NSP) serves as a stabilizing force in the Indo-Pacific region and offers strategic opportunities for small island nations such as Fiji, as well as support in the fields of public health, education, renewable energy and agricultural technology. Taiwan does not have official diplomatic relations with Fiji, but it is one of the small island nations covered by the NSP. Chaudhry said that Fiji, as a sovereign nation, should support
RSS