Neuroscientists have long been banging their heads on their desks over exaggerated reports of brain scanning studies. Media stories illustrated with colored scans, supposedly showing how the brain works, are now a standard part of the science pages and some people find them so convincing that they are touted as ways of designing education for our children, evaluating the effectiveness of marketing campaigns and testing potential recruits. Recently, to the chagrin of French scientists, politicians called for neuro-imaging to be used in the courts to decide on the guilt of criminals, after the technology made its dubious debut in the legal systems of India, Italy and the US.
This misplaced enthusiasm often stems from a misunderstanding about what brain scans tell us. The interpretation seems straightforward according to the popular press — the colored blobs represent a “pleasure center,” an “art center” or perhaps a “love center” — but none of this is true.
All of our experiences and abilities rely on a distributed brain network and nothing relies on a single “center.” More than anything, the conclusions depend on the tasks volunteers undertake in the scanner and what each study tells us is limited. This small print has been repeated many times over by scientists. They bemoan how people misunderstand the subtleties and draw unwarranted conclusions. But now neuroscientists have had to come to terms with the fact that many of the methods on which brain scan studies are based have been flawed.
Photo: Bloomberg
To understand where these flaws come from it’s important to know something about how data from the most common technique, functional Magnetic Resonance Imaging or fMRI, is analyzed. The scanner creates a 3D map of the brain split up into tens of thousands of tiny blocks called voxels (like pixels but for volume) and each has a value that describes blood flow — used as a proxy for brain activity as more active areas need more oxygen. What you want to know is which bits of the brain are more active in certain tasks. Of course, the brain is changing all the time so scientists use statistics to check that changes in blood flow are due to the experimental tasks and not because of unrelated brain changes. The statistical problem is huge, however, as each scan has about 50,000 data points and thousands of scans are made in a single study.
When we’re talking about millions of comparisons, a big problem is false positives. Imagine you are playing two roulette wheels. Clearly, the result of one doesn’t affect the outcome of the other but sometimes they’ll both come up with the same number just because of chance. Now imagine you have a roulette wheel for every point or voxel in the brain. A comparison of any two scans could look like some areas show linked activity when really there is no relationship. Ideally, the analysis should separate roulette wheels from genuine activity, but you may be surprised that hundreds if not thousands of studies have been conducted without such corrections. To illustrate the problem, Craig Bennett and his colleagues at the University of California did a spoof experiment on a dead salmon. The standard techniques showed “brain activity” in the deceased fish.
Further illustrating the issue, Edward Vul and Hal Pashler from the University of California showed that some researchers were producing conclusions by first picking out the best results and then seeing if there was a relationship between them. To return to our roulette analogy, it would be like discarding any results that weren’t in the range of numbers one to five and then using only these selected results to see if any of the same numbers came up, something that is suddenly much more likely. A recent study by Anders Eklund and colleagues from Linkoping University in Sweden found that they could find spurious “brain activity” related to non-existent tasks with standard settings on the most popular fMRI analysis software.
Recent advances have tried to control these problems but researchers have become much more cautious. “Our default attitude to any new and interesting fMRI finding should be skepticism,” says Tal Yarkoni, a neuroscientist at the University of Colorado. “What’s particularly problematic,” he says, “is the amount of flexibility researchers have when performing their analyses ... you have no idea how many things the researchers tried before they got something to work.” Psychologist Russ Poldrack, from the University of Texas, who has been at the forefront of correcting these issues, also highlights cultural issues. This flexible approach “also includes methods that are known by experts to be invalid, but unfortunately these still get into top journals, which only helps perpetuate them.” Yarkoni explains that “researchers have a big incentive to come up with exciting new findings,” meaning scientists are motivated to “torture” the data and journals are attracted by the media-friendly results.
In this light of this, stories about the discovery of “brain centers” fall flat and efforts to base public policy on brain scans become nothing short of ridiculous. But perhaps the most important problem is not that brain scans can be misleading, but that they are beautiful. Like all other neuroscientists, I find them beguiling. They have us enchanted and we are far from breaking their spell.
Vaughan Bell is visiting senior research fellow at the Institute of Psychiatry, King’s College London
In late October of 1873 the government of Japan decided against sending a military expedition to Korea to force that nation to open trade relations. Across the government supporters of the expedition resigned immediately. The spectacle of revolt by disaffected samurai began to loom over Japanese politics. In January of 1874 disaffected samurai attacked a senior minister in Tokyo. A month later, a group of pro-Korea expedition and anti-foreign elements from Saga prefecture in Kyushu revolted, driven in part by high food prices stemming from poor harvests. Their leader, according to Edward Drea’s classic Japan’s Imperial Army, was a samurai
The following three paragraphs are just some of what the local Chinese-language press is reporting on breathlessly and following every twist and turn with the eagerness of a soap opera fan. For many English-language readers, it probably comes across as incomprehensibly opaque, so bear with me briefly dear reader: To the surprise of many, former pop singer and Democratic Progressive Party (DPP) ex-lawmaker Yu Tien (余天) of the Taiwan Normal Country Promotion Association (TNCPA) at the last minute dropped out of the running for committee chair of the DPP’s New Taipei City chapter, paving the way for DPP legislator Su
It’s hard to know where to begin with Mark Tovell’s Taiwan: Roads Above the Clouds. Having published a travelogue myself, as well as having contributed to several guidebooks, at first glance Tovell’s book appears to inhabit a middle ground — the kind of hard-to-sell nowheresville publishers detest. Leaf through the pages and you’ll find them suffuse with the purple prose best associated with travel literature: “When the sun is low on a warm, clear morning, and with the heat already rising, we stand at the riverside bike path leading south from Sanxia’s old cobble streets.” Hardly the stuff of your
April 22 to April 28 The true identity of the mastermind behind the Demon Gang (魔鬼黨) was undoubtedly on the minds of countless schoolchildren in late 1958. In the days leading up to the big reveal, more than 10,000 guesses were sent to Ta Hwa Publishing Co (大華文化社) for a chance to win prizes. The smash success of the comic series Great Battle Against the Demon Gang (大戰魔鬼黨) came as a surprise to author Yeh Hung-chia (葉宏甲), who had long given up on his dream after being jailed for 10 months in 1947 over political cartoons. Protagonist