Once again, well-informed people, including British politicians, have been surprised by an election result because they put too much trust in traditional polls. It is time to get smarter about assessing voter preferences. The tools are there for those who want them.
On the eve of Thursday’s election, British newspapers and many outside observers expected a landslide victory for the Conservative Party. Even a 100-seat majority in the 650-seat parliament was still being discussed. That is because both polls and prediction markets signaled such an outcome.
The Financial Times’ polling average showed that British Prime Minister Theresa May’s Conservatives would win 44 percent of the vote to 36 percent for Jeremy Corbyn’s opposition Labour Party. On Wednesday, Betfair had the Conservatives at odds of 2-to-9 to win a majority and no majority at 5-to-1. On the surface, there was every reason to expect success for May’s plan to get a strong mandate for her party.
Illustration: Yusha
That is unless you followed the YouGov election model developed by Benjamin Lauderdale of the London School of Economics (LSE). An outlier compared to traditional polls, it consistently predicted the actual outcome — a hung parliament, in which no party would have a majority.
The model is a hybrid of an opinion survey and big-data exercise. It uses a small number of responses to survey questions to figure out how voting is related to individual characteristics of people living in each constituency, and then projects the result based on that constituency’s demographic breakdown and past voting record.
This approach — the scientific name is multi-level regression and post-stratification — is a relatively cheap way of matching the survey methodology to the UK’s electoral system, in which politicians contest individual constituencies. The big data part of the model allows pollsters to avoid the cost of surveying large samples in each constituency.
Most of the other UK polls assessed the national vote breakdown — the share of the vote each of the parties was expected to get — by using nationwide samples. They would have been relevant — assuming the samples were representative — in, say, a Dutch parliamentary election, where parties compete in a single nationwide constituency.
National polling can also work in Germany, where every citizen gets two votes — one for a specific candidate in an electoral district and one in a Dutch-style nationwide proportional system; the two votes are usually well correlated. It can also work in a French presidential election, where the nation is a single constituency, too. That is why respectable Dutch, French and German polls using nationwide samples are worth following.
In the US, where the successful presidential candidate needs to win the most electoral votes, outcomes in individual states matter more than the results of the nationwide popular vote. Just as polls based on national samples predicted, failed Democratic US presidential candidate Hillary Rodham Clinton won the popular vote last year.
The controversial University of Southern California and Los Angeles Times poll, which used a sample weighted differently from most others, predicted then-Repulican US presidential candidate Donald Trump would win — but it was wrong, too. Yet Trump won the presidency.
The US needs more complex models than these nationwide polls so it can better gauge local voting patterns. However, these are rare so far. The polling firm Morning Consult applied a YouGov-style approach in April last year and found that Clinton was the strongest candidate at that time.
However, closer to the election, media outlets only cited traditional polls from the firm. YouGov ran a version of Lauderdale’s model in the US, too, and it predicted a Clinton victory. So far, the YouGov model has proved itself better in the UK, where it also predicted the Brexit vote, than in the US. However, it is a work in progress and it can be better calibrated by the next US election.
Italy, too, needs a similar model for its next election, which is increasingly likely to take place this year. The major parties recently tried to strike a deal to change the electoral system to a German-style one. That would have made traditional nationwide samples relevant, but the deal now appears to be off. This means an early election might be held according to the convoluted 2015 system, known as “Italicum.”
It divides the country into 100 constituencies, electing from three to nine candidates each, based on separate party lists for each constituency. Candidates can run in multiple constituencies at once. There is a bonus for a party that can get more than 40 percent of the vote: It is automatically rewarded with a majority.
The potential early election worries investors because of the anti-EU, populist Five Star Movement’s current lead in the polls. However, these polls do not reflect the system. They are potentially as unreliable as British polls turned out to be.
Similarly, the UK election has confirmed that following bookmakers’ odds has a major limitation. People with the most money tend to bet according to poll results, skewing the odds heavily toward the outcome they predict. However, smaller bettors — the ones who are more likely to feel the mood in their constituencies because they go to the right pubs — ignore the polls.
According to Betfair, “no outright majority” attracted a greater number of bets before the UK election than “Conservative majority.” If one wants to follow the prediction markets intelligently, it makes sense to pay attention to the number of bets as well as to the posted odds. Bookmakers are only concerned with making a profit, so the amount of money riding on a certain outcome determines the odds they make. For watchers who do not bet, a hybrid approach is more rational.
As someone who constantly has to monitor polls in multiple elections, I am by now convinced of the value of multiple-input analysis. I am on the lookout for models that combine polling and big data, I ask bookmakers for more information than the odds they post and I watch how candidates do on the social networks and on Google. In the UK election, the Labour Party had a 45 percent search share, compared with the Conservatives’ 21 percent. To me, that suggested that polls could be overestimating May’s chances.
However, most importantly I find that voting intention studies must closely match a nation’s electoral system. The better they are aligned, the more logical it is to use them as vote result predictors. If a country’s electoral system is complex and highly localized, traditional polls will be off.
Leonid Bershidsky is a Bloomberg View columnist. He was the founding editor of the Russian business daily Vedomosti and founded the opinion Web site Slon.ru.
Recently, China launched another diplomatic offensive against Taiwan, improperly linking its “one China principle” with UN General Assembly Resolution 2758 to constrain Taiwan’s diplomatic space. After Taiwan’s presidential election on Jan. 13, China persuaded Nauru to sever diplomatic ties with Taiwan. Nauru cited Resolution 2758 in its declaration of the diplomatic break. Subsequently, during the WHO Executive Board meeting that month, Beijing rallied countries including Venezuela, Zimbabwe, Belarus, Egypt, Nicaragua, Sri Lanka, Laos, Russia, Syria and Pakistan to reiterate the “one China principle” in their statements, and assert that “Resolution 2758 has settled the status of Taiwan” to hinder Taiwan’s
Singaporean Prime Minister Lee Hsien Loong’s (李顯龍) decision to step down after 19 years and hand power to his deputy, Lawrence Wong (黃循財), on May 15 was expected — though, perhaps, not so soon. Most political analysts had been eyeing an end-of-year handover, to ensure more time for Wong to study and shadow the role, ahead of general elections that must be called by November next year. Wong — who is currently both deputy prime minister and minister of finance — would need a combination of fresh ideas, wisdom and experience as he writes the nation’s next chapter. The world that
Can US dialogue and cooperation with the communist dictatorship in Beijing help avert a Taiwan Strait crisis? Or is US President Joe Biden playing into Chinese President Xi Jinping’s (習近平) hands? With America preoccupied with the wars in Europe and the Middle East, Biden is seeking better relations with Xi’s regime. The goal is to responsibly manage US-China competition and prevent unintended conflict, thereby hoping to create greater space for the two countries to work together in areas where their interests align. The existing wars have already stretched US military resources thin, and the last thing Biden wants is yet another war.
As Maldivian President Mohamed Muizzu’s party won by a landslide in Sunday’s parliamentary election, it is a good time to take another look at recent developments in the Maldivian foreign policy. While Muizzu has been promoting his “Maldives First” policy, the agenda seems to have lost sight of a number of factors. Contemporary Maldivian policy serves as a stark illustration of how a blend of missteps in public posturing, populist agendas and inattentive leadership can lead to diplomatic setbacks and damage a country’s long-term foreign policy priorities. Over the past few months, Maldivian foreign policy has entangled itself in playing