Neuroscientists have long been banging their heads on their desks over exaggerated reports of brain scanning studies. Media stories illustrated with colored scans, supposedly showing how the brain works, are now a standard part of the science pages and some people find them so convincing that they are touted as ways of designing education for our children, evaluating the effectiveness of marketing campaigns and testing potential recruits. Recently, to the chagrin of French scientists, politicians called for neuro-imaging to be used in the courts to decide on the guilt of criminals, after the technology made its dubious debut in the legal systems of India, Italy and the US.
This misplaced enthusiasm often stems from a misunderstanding about what brain scans tell us. The interpretation seems straightforward according to the popular press — the colored blobs represent a “pleasure center,” an “art center” or perhaps a “love center” — but none of this is true.
All of our experiences and abilities rely on a distributed brain network and nothing relies on a single “center.” More than anything, the conclusions depend on the tasks volunteers undertake in the scanner and what each study tells us is limited. This small print has been repeated many times over by scientists. They bemoan how people misunderstand the subtleties and draw unwarranted conclusions. But now neuroscientists have had to come to terms with the fact that many of the methods on which brain scan studies are based have been flawed.
Photo: Bloomberg
To understand where these flaws come from it’s important to know something about how data from the most common technique, functional Magnetic Resonance Imaging or fMRI, is analyzed. The scanner creates a 3D map of the brain split up into tens of thousands of tiny blocks called voxels (like pixels but for volume) and each has a value that describes blood flow — used as a proxy for brain activity as more active areas need more oxygen. What you want to know is which bits of the brain are more active in certain tasks. Of course, the brain is changing all the time so scientists use statistics to check that changes in blood flow are due to the experimental tasks and not because of unrelated brain changes. The statistical problem is huge, however, as each scan has about 50,000 data points and thousands of scans are made in a single study.
When we’re talking about millions of comparisons, a big problem is false positives. Imagine you are playing two roulette wheels. Clearly, the result of one doesn’t affect the outcome of the other but sometimes they’ll both come up with the same number just because of chance. Now imagine you have a roulette wheel for every point or voxel in the brain. A comparison of any two scans could look like some areas show linked activity when really there is no relationship. Ideally, the analysis should separate roulette wheels from genuine activity, but you may be surprised that hundreds if not thousands of studies have been conducted without such corrections. To illustrate the problem, Craig Bennett and his colleagues at the University of California did a spoof experiment on a dead salmon. The standard techniques showed “brain activity” in the deceased fish.
Further illustrating the issue, Edward Vul and Hal Pashler from the University of California showed that some researchers were producing conclusions by first picking out the best results and then seeing if there was a relationship between them. To return to our roulette analogy, it would be like discarding any results that weren’t in the range of numbers one to five and then using only these selected results to see if any of the same numbers came up, something that is suddenly much more likely. A recent study by Anders Eklund and colleagues from Linkoping University in Sweden found that they could find spurious “brain activity” related to non-existent tasks with standard settings on the most popular fMRI analysis software.
Recent advances have tried to control these problems but researchers have become much more cautious. “Our default attitude to any new and interesting fMRI finding should be skepticism,” says Tal Yarkoni, a neuroscientist at the University of Colorado. “What’s particularly problematic,” he says, “is the amount of flexibility researchers have when performing their analyses ... you have no idea how many things the researchers tried before they got something to work.” Psychologist Russ Poldrack, from the University of Texas, who has been at the forefront of correcting these issues, also highlights cultural issues. This flexible approach “also includes methods that are known by experts to be invalid, but unfortunately these still get into top journals, which only helps perpetuate them.” Yarkoni explains that “researchers have a big incentive to come up with exciting new findings,” meaning scientists are motivated to “torture” the data and journals are attracted by the media-friendly results.
In this light of this, stories about the discovery of “brain centers” fall flat and efforts to base public policy on brain scans become nothing short of ridiculous. But perhaps the most important problem is not that brain scans can be misleading, but that they are beautiful. Like all other neuroscientists, I find them beguiling. They have us enchanted and we are far from breaking their spell.
Vaughan Bell is visiting senior research fellow at the Institute of Psychiatry, King’s College London
May 6 to May 12 Those who follow the Chinese-language news may have noticed the usage of the term zhuge (豬哥, literally ‘pig brother,’ a male pig raised for breeding purposes) in reports concerning the ongoing #Metoo scandal in the entertainment industry. The term’s modern connotations can range from womanizer or lecher to sexual predator, but it once referred to an important rural trade. Until the 1970s, it was a common sight to see a breeder herding a single “zhuge” down a rustic path with a bamboo whip, often traveling large distances over rugged terrain to service local families. Not only
Ahead of incoming president William Lai’s (賴清德) inauguration on May 20 there appear to be signs that he is signaling to the Chinese Communist Party (CCP) and that the Chinese side is also signaling to the Taiwan side. This raises a lot of questions, including what is the CCP up to, who are they signaling to, what are they signaling, how with the various actors in Taiwan respond and where this could ultimately go. In the last column, published on May 2, we examined the curious case of Democratic Progressive Party (DPP) heavyweight Tseng Wen-tsan (鄭文燦) — currently vice premier
The last time Mrs Hsieh came to Cihu Park in Taoyuan was almost 50 years ago, on a school trip to the grave of Taiwan’s recently deceased dictator. Busloads of children were brought in to pay their respects to Chiang Kai-shek (蔣中正), known as Generalissimo, who had died at 87, after decades ruling Taiwan under brutal martial law. “There were a lot of buses, and there was a long queue,” Hsieh recalled. “It was a school rule. We had to bow, and then we went home.” Chiang’s body is still there, under guard in a mausoleum at the end of a path
Last week the Directorate-General of Budget, Accounting and Statistics (DGBAS) released a set of very strange numbers on Taiwan’s wealth distribution. Duly quoted in the Taipei Times, the report said that “The Gini coefficient for Taiwanese households… was 0.606 at the end of 2021, lower than Australia’s 0.611, the UK’s 0.620, Japan’s 0.678, France’s 0.676 and Germany’s 0.727, the agency said in a report.” The Gini coefficient is a measure of relative inequality, usually of wealth or income, though it can be used to evaluate other forms of inequality. However, for most nations it is a number from .25 to .50