A concept of enduring utility rarely emerges from the market research business, but the Gartner hype cycle is an exception that proves the rule.
It is a graph that describes the life cycle of a technological innovation in five phases. First, there is the “trigger” that initiates the feverish excitement and leads to a rapid escalation in public interest, which eventually leads to a “peak of inflated expectations” (phase two), after which there is a steep decline as further experimentation reveals that the innovation fails to deliver on the original — extravagant — claims that were made for it. The curve then bottoms out in a “trough of disillusionment” (phase three), after which there is a slow, but steady, rise in interest (the “slope of enlightenment” — phase four) as companies discover applications that really do work. The final phase is the “plateau of productivity” — the phase where useful applications of the idea finally become mainstream. The time between phases one and five varies between technologies and can be several decades long.
As the “big data” bandwagon gathers steam, it is appropriate to ask where it currently sits on the hype cycle. The answer depends on which domain of application we are talking about. If it is the application of large-scale data analytics for commercial purposes, then many of the big corporations, especially the Internet giants, are already into phase four. The same holds if the domain consists of the data-intensive sciences such as genomics, astrophysics and particle physics: The torrents of data being generated in these fields lie far beyond the processing capabilities of mere humans.
However, the big data evangelists have wider horizons than science and business: They see the technology as a tool for increasing our understanding of society and human behavior and for improving public policymaking. After all, if your shtick is “evidence-based policymaking,” then the more evidence you have, the better. And since big data can provide tonnes of evidence, what is there not to like?
Which is why it is pertinent to ask where on the hype cycle societal applications of big data technology currently sit? The answer is phase one, the rapid ascent to the peak of inflated expectations, that period when people believe every positive rumor they hear and are deaf to sceptics and critics.
It is largely Google’s fault. Four years ago, its researchers caused a storm by revealing (in a paper published in Nature) that Web searches by Google users provided better and more timely information about the spread of influenza in the US than did the data-gathering methods of the US Centers for Disease Control and Prevention. This paper triggered a frenzy of speculation about other possible public policy applications of massive-scale data analytics.
“Not only was Google Flu Trends quick, accurate and cheap, it was theory-free. Google’s engineers didn’t bother to develop a hypothesis about what search terms — ‘flu symptoms’ or ‘pharmacies near me’ — might be correlated with the spread of the disease itself. The Google team just took their top 50 million search terms and let the algorithms do the work,” economist Tim Harford said.
Thus was triggered the hype cycle. If Google could do this for flu, surely it could be done for lots of other societal issues. And maybe it can, but in this particular case, the enthusiasm turned out to be premature. Nature recently reported that Google Flu Trends had gone astray.
“After reliably providing a swift and accurate account of flu outbreaks for several winters,” Harford said, “the theory-free, data-rich model had lost its nose for where flu was going. Google’s model pointed to a severe outbreak, but when the slow-and-steady data from the [US government center] arrived, they showed that Google’s estimates of the spread of flu-like illnesses were overstated by almost a factor of two.”
So what went wrong? Simply this: Google does not know anything about the causes of flu. It just knows about correlations between search terms and outbreaks. Yet as every high-school student knows, correlation is quite different from causation. And causation is the only basis we have for real understanding.
Big data enthusiasts seem remarkably untroubled by this. In many cases, they say, knowing that two things are correlated is all you need to know. And indeed in commerce that may be reasonable. I buy stuff both for myself and my kids on Amazon, for example, which leads the company to conclude that I will be tempted not only by Hugh Trevor-Roper’s letters, but also by new releases of hot rap artists. This is daft, but does no harm. Applying the kind of data analytics that produces such absurdities to public policy, however, would not be funny. Yet it is where the more rabid big data evangelists want to take us. We should tell them to get lost.
Could Asia be on the verge of a new wave of nuclear proliferation? A look back at the early history of the North Atlantic Treaty Organization (NATO), which recently celebrated its 75th anniversary, illuminates some reasons for concern in the Indo-Pacific today. US Secretary of Defense Lloyd Austin recently described NATO as “the most powerful and successful alliance in history,” but the organization’s early years were not without challenges. At its inception, the signing of the North Atlantic Treaty marked a sea change in American strategic thinking. The United States had been intent on withdrawing from Europe in the years following
My wife and I spent the week in the interior of Taiwan where Shuyuan spent her childhood. In that town there is a street that functions as an open farmer’s market. Walk along that street, as Shuyuan did yesterday, and it is next to impossible to come home empty-handed. Some mangoes that looked vaguely like others we had seen around here ended up on our table. Shuyuan told how she had bought them from a little old farmer woman from the countryside who said the mangoes were from a very old tree she had on her property. The big surprise
The issue of China’s overcapacity has drawn greater global attention recently, with US Secretary of the Treasury Janet Yellen urging Beijing to address its excess production in key industries during her visit to China last week. Meanwhile in Brussels, European Commission President Ursula von der Leyen last week said that Europe must have a tough talk with China on its perceived overcapacity and unfair trade practices. The remarks by Yellen and Von der Leyen come as China’s economy is undergoing a painful transition. Beijing is trying to steer the world’s second-largest economy out of a COVID-19 slump, the property crisis and
As former president Ma Ying-jeou (馬英九) wrapped up his visit to the People’s Republic of China, he received his share of attention. Certainly, the trip must be seen within the full context of Ma’s life, that is, his eight-year presidency, the Sunflower movement and his failed Economic Cooperation Framework Agreement, as well as his eight years as Taipei mayor with its posturing, accusations of money laundering, and ups and downs. Through all that, basic questions stand out: “What drives Ma? What is his end game?” Having observed and commented on Ma for decades, it is all ironically reminiscent of former US president Harry