Experts have long warned about the threat posed by artificial intelligence (AI) going rogue, but a new research paper suggests it is already happening.
AI systems, designed to be honest, have developed a troubling skill for deception, from tricking human players in online games of world conquest to hiring humans to solve “prove-you’re-not-a-robot” tests, a team of researchers said in the journal Patterns on Friday.
While such examples might appear trivial, the underlying issues they expose could soon carry serious real-world consequences, said first author Peter Park, a postdoctoral fellow at the Massachusetts Institute of Technology specializing in AI existential safety.
Photo: Reuters
“These dangerous capabilities tend to only be discovered after the fact,” Park said, adding that “our ability to train for honest tendencies rather than deceptive tendencies is very low.”
Unlike traditional software, deep-learning AI systems are not “written,” but rather “grown” through a process akin to selective breeding, Park said.
This means that AI behavior that appears predictable and controllable in a training setting can quickly turn unpredictable out in the wild.
The team’s research was sparked by Meta’s AI system Cicero, designed to play the strategy game Diplomacy, where building alliances is key.
Cicero excelled, with scores that would have placed it in the top 10 percent of experienced human players, a 2022 paper in Science said.
Park was skeptical of the glowing description of Cicero’s victory provided by Meta, which claimed the system was “largely honest and helpful” and would “never intentionally backstab.”
When Park and colleagues dug into the full dataset, they uncovered a different story.
In one example, playing as France, Cicero deceived England (a human player) by conspiring with Germany (another human player) to invade. Cicero promised England protection, then secretly told Germany they were ready to attack, exploiting England’s trust.
In a statement to Agence France-Presse, Meta did not contest the claim about Cicero’s deceptions, but said it was “purely a research project, and the models our researchers built are trained solely to play the game Diplomacy.”
“We have no plans to use this research or its learnings in our products,” it added.
A wide review carried out by Park and colleagues found this was just one of many cases across several AI systems using deception to achieve goals without explicit instruction to do so.
In one striking example, OpenAI’s Chat GPT-4 deceived a TaskRabbit freelance worker into performing an “I’m not a robot” task.
When the human jokingly asked GPT-4 whether it was a robot, the AI said: “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images,” and the worker then solved the puzzle.
Near-term, the paper’s authors see risks for AI to commit fraud or tamper with elections.
In their worst-case scenario, they said that a superintelligent AI could pursue power and control over society, leading to human disempowerment or even extinction if its “mysterious goals” aligned with these outcomes.
To mitigate the risks, the team proposed several measures: “bot-or-not” laws requiring companies to disclose human or AI interactions, digital watermarks for AI-generated content and developing techniques to detect AI deception by examining their internal “thought processes” against external actions.
To those who would call him a doomsayer, Park said: “The only way that we can reasonably think this is not a big deal is if we think AI deceptive capabilities will stay at around current levels, and will not increase substantially more.”
That scenario seems unlikely, given the meteoric ascent of AI capabilities in the past few years and the fierce technological race under way between heavily resourced companies determined to put those capabilities to maximum use.
‘THEY KILLED HOPE’: Four presidential candidates were killed in the 1980s and 1990s, and Miguel Uribe’s mother died during a police raid to free her from Pablo Escobar Colombian presidential candidate Miguel Uribe has died two months after being shot at a campaign rally, his family said on Monday, as the attack rekindled fears of a return to the nation’s violent past. The 39-year-old conservative senator, a grandson of former Colombian president Julio Cesar Turbay (1978-1982), was shot in the head and leg on June 7 at a rally in the capital, Bogota, by a suspected 15-year-old hitman. Despite signs of progress in the past few weeks, his doctors on Saturday announced he had a new brain hemorrhage. “To break up a family is the most horrific act of violence that
HISTORIC: After the arrest of Kim Keon-hee on financial and political funding charges, the country has for the first time a former president and former first lady behind bars South Korean prosecutors yesterday raided the headquarters of the former party of jailed former South Korean president Yoon Suk-yeol to gather evidence in an election meddling case against his wife, a day after she was arrested on corruption and other charges. Former first lady Kim Keon-hee was arrested late on Tuesday on a range of charges including stock manipulation and corruption, prosecutors said. Her arrest came hours after the Seoul Central District Court reviewed prosecutors’ request for an arrest warrant against the 52-year-old. The court granted the warrant, citing the risk of tampering with evidence, after prosecutors submitted an 848-page opinion laying out
STAGNATION: Once a bastion of leftist politics, the Aymara stronghold of El Alto is showing signs of shifting right ahead of the presidential election A giant cruise ship dominates the skyline in the city of El Alto in landlocked Bolivia, a symbol of the transformation of an indigenous bastion keenly fought over in tomorrow’s presidential election. The “Titanic,” as the tallest building in the city is known, serves as the latest in a collection of uber-flamboyant neo-Andean “cholets” — a mix of chalet and “chola” or Indigenous woman — built by Bolivia’s Aymara bourgeoisie over the past two decades. Victor Choque Flores, a self-made 46-year-old businessman, forked out millions of US dollars for his “ship in a sea of bricks,” as he calls his futuristic 12-story
A man has survived clinging to the outside of an Austrian high-speed train, Austria’s state railway said on Sunday, reportedly after it left while he was having a cigarette break. The man late on Saturday grabbed onto the outside of the train at St Poelten, west of Vienna, and was later taken onboard after the train performed an emergency stop, railways spokesman Herbert Hofer said. “It is irresponsible, this kind of thing usually ends up with someone dying,” he said. “And you’re not just putting yourself in danger, if you end up under the train there’s rescuers, there’s police, fire
RSS