In the same amount of time it would take to toast a slice of bread, you could clone the voice of US President Joe Biden and share it on social media. You could have him mutter in his slow and gravelly voice: “I’ve always known COVID-19 was a hoax, it’s just useful to pretend it’s real,” then superimpose the audio on a photo of the president grinning, upload it to TikTok, YouTube and Facebook and wait.
A funny thing would happen: The first two sites would take your clip down. However, the biggest platform — the one with more than 3 billion users — would not. Facebook would slap a warning label on the clip, but leave it up for people to click through, listen to and share with others. That antiquated policy could prove disastrous in a divisive election year.
Several examples show how likely that scenario is. In September last year, faked audio of a Slovak political leader “discussing” ways to buy votes was shared on Facebook within days of a closely fought national election. Parent company Meta Platforms Inc does not ban fake audio clips in the same way it takes down fake videos, so Facebook let the clip remain with a label saying it had been manipulated. Two days later, that same party leader lost the election. It is impossible to know if the clip swayed votes, but the country also had a 48-hour media blackout before the election, which means that there was no one to debunk the forgery.
Illustration: Kevin Sheu
In a world of misinformation, fake audio could have a more sinister effect than fake videos. While fake “photos” of former US president Donald Trump have a glossy, plastic look that belies the artificial intelligence (AI) machinery behind them, fake versions of his voice are harder to scrutinize and distinguish. AI-generated voices could also sound hyper-realistic thanks to a passel of new tools originally designed to help podcasters and marketers.
Companies such as Eleven Labs, Voice AI and Respeecher sell services that can synthesize the voices of actors, meaning they could, for instance, read audio books in different languages, and some only require a couple of minutes of a voice recording to clone the voice. Voice AI startups raised about US$1.6 billion from venture capital investors last year, market research firm Pitchbook’s data show. Overall investment growth in these companies has decreased in the past two years, however, in part because larger companies such as Amazon.com Inc and OpenAI are taking in more business.
Some companies such as Respeecher have features in place to prevent misuse, or they require permission from people to have their voice cloned. However, that does not stop others from exploiting them anyway. For instance, someone recently cloned the voice of London Mayor Sadiq Khan and posted the faked audio clip to TikTok. In the clip, Khan’s “voice” could be heard saying that Armistice Day should be canceled in favor of a protest to support Palestinians.
“Why don’t they have Remembrance Weekend next weekend?” his “voice” asks.
The audio caused outrage among Brits who believed the country’s veteran’s day celebrations should be respected, but Khan’s office said that the clip was being “circulated and amplified by a far-right group.” To their likely dismay, it was reposted on Facebook and remains on the site, in at least one case without a warning label.
Another person generated a fake clip of UK Labour Party Leader Keir Starmer supposedly calling one of his team members a “bloody moron,” while a second forged clip had Starmer saying that he “hated Liverpool.” The posts were seen thousands of times on TikTok before being taken down. A rival Conservative politician encouraged the public to “ignore it.”
TikTok removed the London mayor’s clip, and a company spokeswoman said similar deceptive audio involving politicians would normally be taken down as it violates policy. YouTube also removed postings of the faked mayor’s voice; a company spokeswoman said that the site takes down “technically manipulated” content that could cause harm. X, formerly Twitter, has a similar rule, although it does not seem to enforce it — it has kept the mayor’s forgery up, for instance.
However, the stakes are higher with Facebook given that it has eight times more monthly active users than X, which makes its leniency toward forged audio all the more bizarre.
A spokesman for Facebook said it labeled and left fake audio of politicians up on the platform “so people have accurate information when they encounter similar content across the Internet.” It is better to leave a clip up with a warning label, Facebook argues, so that when people see it on other sites such as X or Telegram, they would be educated on its inauthenticity.
However, Facebook relies on stretched teams of fact-checkers to do such labeling.
“These things are spreading in real time over the Internet,” says Steve Nowottny, editor of the independent fact-checking charity Full Fact, which worked with Facebook to debunk the Khan and Starmer audio clips. It took them two days to check the Labour Party leader’s clip, he says.
One problem is that there are still no reliable technical tools for detecting fake AI audio, so Full Fact uses old-fashioned investigative techniques. In the case of the Starmer clip, Full Fact spoke to people in both the Labour and Conservative parties to confirm that the audio was fake.
However, its fact-checking team is made up of only 13 people. More broadly, there has also been a decline in the number of people at social media companies working on misinformation. Alphabet Inc, Meta and X have all pared back their trust and safety teams in the past two years to cut costs, and Meta also recently shuttered a project to build a fact-checking tool, CNBC said.
“I talked to a large group of fact-checkers and journalists from across Asia in November [last year], and almost everyone was seeing manipulated audio and wasn’t sure how to detect it,” says Sam Gregory, executive director of Witness, a human rights group focused on technology.
Even labeled misinformation could spread rapidly before the warning is properly understood. In moments of fast-paced information sharing, when emotions are running high, not every Facebook or Instagram user might fully comprehend the meaning of a label — or believe it.
Facebook’s policy of only taking down faked videos is outdated. As we head into what could be tumultuous national elections in the UK, India, the US and elsewhere, made all the more messy by AI tools generating all kinds of media and information, the platform should also start taking down deceptive audio.
Parmy Olson is a Bloomberg Opinion columnist covering technology. A former reporter for the Wall Street Journal and Forbes magazine, she is author of We Are Anonymous. This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.
Monday was the 37th anniversary of former president Chiang Ching-kuo’s (蔣經國) death. Chiang — a son of former president Chiang Kai-shek (蔣介石), who had implemented party-state rule and martial law in Taiwan — has a complicated legacy. Whether one looks at his time in power in a positive or negative light depends very much on who they are, and what their relationship with the Chinese Nationalist Party (KMT) is. Although toward the end of his life Chiang Ching-kuo lifted martial law and steered Taiwan onto the path of democratization, these changes were forced upon him by internal and external pressures,
Chinese Nationalist Party (KMT) caucus whip Fu Kun-chi (傅?萁) has caused havoc with his attempts to overturn the democratic and constitutional order in the legislature. If we look at this devolution from the context of a transition to democracy from authoritarianism in a culturally Chinese sense — that of zhonghua (中華) — then we are playing witness to a servile spirit from a millennia-old form of totalitarianism that is intent on damaging the nation’s hard-won democracy. This servile spirit is ingrained in Chinese culture. About a century ago, Chinese satirist and author Lu Xun (魯迅) saw through the servile nature of
In their New York Times bestseller How Democracies Die, Harvard political scientists Steven Levitsky and Daniel Ziblatt said that democracies today “may die at the hands not of generals but of elected leaders. Many government efforts to subvert democracy are ‘legal,’ in the sense that they are approved by the legislature or accepted by the courts. They may even be portrayed as efforts to improve democracy — making the judiciary more efficient, combating corruption, or cleaning up the electoral process.” Moreover, the two authors observe that those who denounce such legal threats to democracy are often “dismissed as exaggerating or
Taiwan People’s Party (TPP) Acting Chairman Huang Kuo-chang (黃國昌) has formally announced his intention to stand for permanent party chairman. He has decided that he is the right person to steer the fledgling third force in Taiwan’s politics through the challenges it would certainly face in the post-Ko Wen-je (柯文哲) era, rather than serve in a caretaker role while the party finds a more suitable candidate. Huang is sure to secure the position. He is almost certainly not the right man for the job. Ko not only founded the party, he forged it into a one-man political force, with himself