Can a statistical model reliably predict that you will buy the latest Harry Potter book, or add organic brie to your virtual shopping cart this week? What about whether you might become violent in the next 15 years, or your unborn child grow up to be a delinquent?
The growing use of computerized techniques for forecasting what we might buy or do on the basis of how our data matches up to some statistical model would suggest that they are well proven. But a landmark paper recently published in the British Journal of Psychiatry has cast doubt on whether such techniques should be used for making decisions about anything beyond the trivial.
The personalized recommendations and special offers that pop up when you order books or groceries online, and even the specific sequence of questions an insurance call center asks about your claim, are all generated by computerized predictive algorithms derived from analysing patterns, links and associations in large sets of data.
By classifying types of people and their behaviors on this basis, shops try to increase their profits by automatically targeting those of us in their databases that seem most likely to buy certain items. Insurance companies use similar methods to reduce fraud by investigating the claims of those whom the software decides are most likely to be lying.
serious matters
But the British government is adopting such techniques for more serious matters. Software at the Department of Work and Pensions, for instance, is beginning to try to detect fraudsters by analysing the voices of people who ring its call centers -- so if you ask the wrong kind of questions, or perhaps ask the right kind of questions in the wrong way, the software could decide you're not strictly kosher.
The British government's Action Plan on Social Exclusion has risk prediction as its first guiding principle. The idea is to predict life outcomes and trigger early human interventions before things go wrong -- in the case of the Nurse Family Partnership scheme, even before birth. In this scheme, the unborn child of a pregnant mother might be categorized as at high risk of future criminality based on factors such as the mother's age, her poor educational achievements, drug use and her own family background. The mother is then visited regularly at home by a nurse and helped with parenting.
In the criminal justice system too, risk prediction instruments assess the probability of adults and young people re-offending, along with a battery of other actuarial tests for predicting future sexual and violent crime. Such techniques, which are not automated in these cases, also play a central role in evaluations to determine whether a person should be committed indefinitely as a dangerous person with severe personality disorder or whether these people, once committed, are ready for release.
The UK Department of Health has even developed a series of predictive algorithms for scoring those patients with long-term conditions who are at most risk of rehospitalization. The idea is to intervene early to minimize admissions.
The Surveillance Society report from the Information Commissioner's Office outlined worries about predictive social sorting on the grounds that it could amount to discrimination, create new underclasses and that by the totting up of negative indicators from health, school and other records, a predictive model could make its own worst predictions come true.
"For instance, if your parents both have criminal records or you have a bad school attendance record because of poor health, even if you are the best-behaved kid in class, you will find that every teacher is likely to treat you with suspicion," said Jonathan Bamford, assistant information commissioner.
error
Now a team of British and US researchers has flagged up a more fundamental danger with these predictive models. When applied to individuals the margins of error are so high as to render any results meaningless.
The study published in May in the British Journal of Psychiatry by forensic clinical psychologists Stephen Hart and David Cooke and statistician Christine Michie takes as its example two popular actuarial risk assessment tools used to predict violence (STATIC-99 and VRAG).
But the team has studied many other widely used tools such as Risk Matrix-2000 (for predicting sexual offending), and found the same high margins of error across the board.
For groups of people flagged up as high risk by Risk Matrix-2000, for example, the standard estimate is that 36 percent will reoffend sexually in the next 15 years. The team found that the true value of that estimate for a group lies between 28 percent and 45 percent, 95 percent of the time -- ie, the 95 percent confidence interval. For an individual, they found the true value of the estimate was between 3 percent and 91 percent, 95 percent of the time. For STATIC-99, VRAG and other tools, the results were much the same.
"The statistical issue of the difference between precision of estimates in a group against that for an individual is not peculiar to psychology and it is not because psychological variables are less reliable. It's to do with inherent variability in human beings," said Cooke, professor of forensic clinical psychology at the Douglas Inch Centre and Glasgow Caledonian University, Scotland.
Cooke has also looked at medical literature on predicting the probability of heart attacks, cancer and other conditions that rely on physical measurements and found the same large error margins.
No one can argue with statistically based procedures for making complex decisions under conditions of uncertainty, so long as successes and failures are aggregated across cases and the cost of errors is low, said Stephen Hart, a professor in the department of psychology at Simon Fraser University in Canada and a leading authority on assessing the risk of violent offending.
"A life insurance company doesn't care whether it makes a `mistake' estimating the lifespan of a given individual -- it could be wrong about every single person it insures -- but as long as the pattern of life spans is predictable on average, then it can still make good money," Hart said.
But if one is interested in individual cases or if the cost of decision errors is high, then these techniques are problematic.
"Families of victims who are killed by patients and offenders released improperly, and those whose civil rights are infringed when they are held improperly, are not satisfied knowing that despite the errors I made in their cases, I am still right more often than I am wrong," said Hart, who has considerable experience as an expert witness in US courts defending people who have been incarcerated on the basis of actuarial risk predictions.
Hart highlighted a further problem with predictive assessments that compounds the effects of these error margins. In cases of uncertainty, humans will tend to anchor on the first substantial piece of information they get and any new information that contradicts this initial idea is given less attention than it merits. This is the theory of anchoring bias.
If for example a predictive model says that a frail 85-year-old man with heart problems is a high-risk sex-offender and 52 percent likely to reoffend over the next 15 years, anchoring bias means that if the assessor is told the score is wrong, they will simply adjust it to 50 percent or 48 percent.
"It is probably zero. But the 52 percent poisons the assessor's judgment," Hart said.
Predictive models are attractive because they represent an apparently scientific and rigorous yet simple approach to targeting resources and making decisions about complex human problems. But this latest study adds to concerns that such techniques are insufficiently accurate to make important decisions about individuals.
Recently, China launched another diplomatic offensive against Taiwan, improperly linking its “one China principle” with UN General Assembly Resolution 2758 to constrain Taiwan’s diplomatic space. After Taiwan’s presidential election on Jan. 13, China persuaded Nauru to sever diplomatic ties with Taiwan. Nauru cited Resolution 2758 in its declaration of the diplomatic break. Subsequently, during the WHO Executive Board meeting that month, Beijing rallied countries including Venezuela, Zimbabwe, Belarus, Egypt, Nicaragua, Sri Lanka, Laos, Russia, Syria and Pakistan to reiterate the “one China principle” in their statements, and assert that “Resolution 2758 has settled the status of Taiwan” to hinder Taiwan’s
Singaporean Prime Minister Lee Hsien Loong’s (李顯龍) decision to step down after 19 years and hand power to his deputy, Lawrence Wong (黃循財), on May 15 was expected — though, perhaps, not so soon. Most political analysts had been eyeing an end-of-year handover, to ensure more time for Wong to study and shadow the role, ahead of general elections that must be called by November next year. Wong — who is currently both deputy prime minister and minister of finance — would need a combination of fresh ideas, wisdom and experience as he writes the nation’s next chapter. The world that
Can US dialogue and cooperation with the communist dictatorship in Beijing help avert a Taiwan Strait crisis? Or is US President Joe Biden playing into Chinese President Xi Jinping’s (習近平) hands? With America preoccupied with the wars in Europe and the Middle East, Biden is seeking better relations with Xi’s regime. The goal is to responsibly manage US-China competition and prevent unintended conflict, thereby hoping to create greater space for the two countries to work together in areas where their interests align. The existing wars have already stretched US military resources thin, and the last thing Biden wants is yet another war.
As Maldivian President Mohamed Muizzu’s party won by a landslide in Sunday’s parliamentary election, it is a good time to take another look at recent developments in the Maldivian foreign policy. While Muizzu has been promoting his “Maldives First” policy, the agenda seems to have lost sight of a number of factors. Contemporary Maldivian policy serves as a stark illustration of how a blend of missteps in public posturing, populist agendas and inattentive leadership can lead to diplomatic setbacks and damage a country’s long-term foreign policy priorities. Over the past few months, Maldivian foreign policy has entangled itself in playing