Clusters of yellow and orange markers hover over a satellite image of the Earth, each representing a reported outbreak of disease. This is the screen that greets visitors to HealthMap, a near real-time Web-crawling system that gathers and organizes data from news sources, official alerts, online databases and blogs, overlaying it on a Google map.
The site is one of a number of initiatives aimed at identifying health, ecological and conflict crises earlier than official monitoring systems. Web crawlers — automated software programs that scour the Web for information — are used to find patterns that may signify an emerging threat.
A new alert marker on HealthMap is often the first indication of an outbreak of disease for both health organizations and the general public. Its color — yellow, orange or red — denotes the threat level.
John Brownstein and Clark Freifeld, a physician and software developer at the Children’s Hospital Informatics Program in Boston, launched the site in 2006.
“It started as a side project,” Brownstein says.
The site quickly gained the attention of Google.org, the philanthropic arm of the search giant. Its Predict and Prevent initiative supports efforts to respond to outbreaks of disease before they become global crises. An initial US$450,000 grant helped combine HealthMap’s detection efforts with those of ProMED-mail’s global network of health specialists.
The Web crawler combs 20,000 Web sites every hour, tracking about 75 infectious diseases, including malaria, cholera, Ebola and now swine flu. An average of 300 reports are collected each day, about 90 percent of which come from news media.
Frequent users of the site include the WHO, the US Centers for Disease Control and Prevention, and the UK’s Health Protection Agency.
The Global Public Health Intelligence Network, a Canadian system, used similar Web-crawling software to detect the first signs of the SARS outbreak in 2002, several months before it was officially announced.
HealthMap received its earliest report of the swine flu outbreak on April 1. Brownstein claims the outbreak had been building for months, but containment is difficult, even with advanced warning.
“In reality, what happens in Mexico doesn’t stay in Mexico,” he says.
It is unclear precisely how far in advance of the traditional monitoring systems HealthMap is with its predictions.
“Sometimes we’re six months ahead; sometimes we spot something on the same day,” Brownstein says. “It depends on the disease, the country, the political situation — all factors contribute to how information flows.”
HealthMap monitors news in five languages — vital for the early detection of an outbreak of disease.
“The first reports from a particular region tend to be in the local language,” Freifeld says. “For example, with the swine flu outbreak one of our earliest reports of an unidentified respiratory illness in Veracruz, Mexico, came from a local Spanish news site.”
Text-processing algorithms are used to determine the relevance of the information, separating articles on actual outbreaks from those covering government vaccination programs, for instance. The information is then sorted by disease and location, with duplicate articles filtered out before the data is applied to the map.
Researchers from the Stockholm Resilience Centre and the University of East Anglia in England are hoping to apply the pioneering techniques to the field of ecological monitoring. Catastrophic environmental shifts could be missed by official systems because of insufficient data and geographical gaps.
“You may have ecological monitoring tools in place, but not economic or social monitoring, which can give you a much earlier signal that something is happening,” says Victor Galaz, lead author of a paper published by the researchers last month in Frontiers in Ecology and the Environment. “For example, if prices for a certain species of fish increase, or you receive news of heavy investment in high-tech fishing fleets in a particular region, that can give you an early warning that change is occurring.”
Sifting through the huge amount of information on the Web and determining the “signal” from the “noise” is still difficult, even with advanced monitoring tools. Whether Web-crawler systems are truly able to extract reliable data on emerging crises from Internet chatter, or whether they merely spot patterns in hindsight, is still open to debate.
Gaps also exist in the coverage of developing countries, where news sources are fewer. Mark Smolinski, director of Predict and Prevent, said the next step for HealthMap will be “engaging directly with citizens to report illness, something now entirely possible as mobile technologies permeate the globe.”
Predictive monitoring tools may also provide early warning of conflicts — outbreaks of rioting could foreshadow a larger crisis. Veratect Corporation already operates a service called Shadowstream, which identifies emerging civil unrest around the world for corporate clients.
“We focused on infectious diseases because of their widespread nature and real-time development,” Freifeld says. “But we’re looking at where else the technology can be applied, such as conflicts or environmental disasters.”
Web crawlers could even predict the next financial crisis. The 2021 credit crunch? You can’t say you weren’t warned.
In 1976, the Gang of Four was ousted. The Gang of Four was a leftist political group comprising Chinese Communist Party (CCP) members: Jiang Qing (江青), its leading figure and Mao Zedong’s (毛澤東) last wife; Zhang Chunqiao (張春橋); Yao Wenyuan (姚文元); and Wang Hongwen (王洪文). The four wielded supreme power during the Cultural Revolution (1966-1976), but when Mao died, they were overthrown and charged with crimes against China in what was in essence a political coup of the right against the left. The same type of thing might be happening again as the CCP has expelled nine top generals. Rather than a
Former Chinese Nationalist Party (KMT) lawmaker Cheng Li-wun (鄭麗文) on Saturday won the party’s chairperson election with 65,122 votes, or 50.15 percent of the votes, becoming the second woman in the seat and the first to have switched allegiance from the Democratic Progressive Party (DPP) to the KMT. Cheng, running for the top KMT position for the first time, had been termed a “dark horse,” while the biggest contender was former Taipei mayor Hau Lung-bin (郝龍斌), considered by many to represent the party’s establishment elite. Hau also has substantial experience in government and in the KMT. Cheng joined the Wild Lily Student
When Taiwan High Speed Rail Corp (THSRC) announced the implementation of a new “quiet carriage” policy across all train cars on Sept. 22, I — a classroom teacher who frequently takes the high-speed rail — was filled with anticipation. The days of passengers videoconferencing as if there were no one else on the train, playing videos at full volume or speaking loudly without regard for others finally seemed numbered. However, this battle for silence was lost after less than one month. Faced with emotional guilt from infants and anxious parents, THSRC caved and retreated. However, official high-speed rail data have long
Taipei stands as one of the safest capital cities the world. Taiwan has exceptionally low crime rates — lower than many European nations — and is one of Asia’s leading democracies, respected for its rule of law and commitment to human rights. It is among the few Asian countries to have given legal effect to the International Covenant on Civil and Political Rights and the International Covenant of Social Economic and Cultural Rights. Yet Taiwan continues to uphold the death penalty. This year, the government has taken a number of regressive steps: Executions have resumed, proposals for harsher prison sentences
RSS