A team of researchers has uncovered what they said was the first reported use of artificial intelligence (AI) to direct a hacking campaign in a largely automated fashion.
The AI company Anthropic this week said that it disrupted a cyberoperation that its researchers linked to the Chinese government.
The operation involved the use of an AI system to direct the hacking campaigns, which researchers called a disturbing development that could greatly expand the reach of AI-equipped hackers.
Photo: Reuters
While concerns about AI being used to drive cyberoperations are not new, what is concerning is the degree to which AI was able to automate some of the work, the researchers said.
“While we predicted these capabilities would continue to evolve, what has stood out to us is how quickly they have done so at scale,” they said.
The hacking operation targeted tech companies, financial institutions, chemical companies and government agencies.
The hackers attacked “roughly 30 global targets and succeeded in a small number of cases,” the researchers said.
Anthropic said it detected the operation in September and took steps to shut it down and notify the affected parties.
While AI systems are increasingly being used in a variety of settings for work and leisure, they can also be weaponized by hacking groups working for foreign adversaries, it added.
Anthropic is one of many tech developers pitching AI “agents” that go beyond a chatbot’s capability to access computer tools and take actions on a person’s behalf.
“Agents are valuable for everyday work and productivity — but in the wrong hands, they can substantially increase the viability of large-scale cyberattacks,” the researchers said. “These attacks are likely to only grow in their effectiveness.”
Microsoft earlier this year warned that foreign adversaries were increasingly embracing AI to make their cybercampaigns more efficient and less labor-intensive.
The head of OpenAI’s safety panel, which has the authority to halt the ChatGPT maker’s AI development, said he is watching out for new AI systems that give malicious hackers “much higher capabilities.”
The US’ adversaries, criminal gangs and hacking companies have exploited AI’s potential, using it to automate and improve cyberattacks, spread inflammatory disinformation and penetrate sensitive systems. AI can translate poorly worded phishing e-mails into fluent English, for example, as well as generate digital clones of senior government officials.
Anthropic said the hackers were able to manipulate its AI chatbot, Claude, using “jailbreaking” techniques that involved tricking an AI system to bypass its guardrails against harmful behavior, in this case by claiming they were employees of a legitimate cybersecurity firm.
“This points to a big challenge with AI models, and it’s not limited to Claude, which is that the models have to be able to distinguish between what’s actually going on with the ethics of a situation and the kinds of role-play scenarios that hackers and others may want to cook up,” Citizen Lab senior researcher John Scott-Railton said.
The use of AI to automate or direct cyberattacks would also appeal to smaller hacking groups and lone-wolf hackers, who could use the technology to expand the scale of their attacks, said Adam Arellano, field chief technology officer at Harness, a tech company that uses AI to help customers automate software development.
“The speed and automation provided by the AI is what is a bit scary,” Arellano said. “Instead of a human with well-honed skills attempting to hack into hardened systems, the AI is speeding those processes and more consistently getting past obstacles.”
AI programs would also play an increasingly important role in defending against these kinds of attacks, Arellano said, adding that AI and the automation it allows would benefit both sides.
‘GROSS NEGLIGENCE?’ Despite a spleen typically being significantly smaller than a liver, the surgeon said he believed Bryan’s spleen was ‘double the size of what is normal’ A Florida surgeon who is facing criminal charges after allegedly removing a patient’s liver instead of his spleen has said he is “forever traumatized” by that person’s death. In a deposition from November last year that was recently obtained by NBC, 44-year-old Thomas Shaknovsky described the death of 70-year-old William Bryan as an “incredibly unfortunate event that I regret deeply.” Bryan died after the botched surgery; and last month, a grand jury in Tallahassee indicted Shaknovsky on a charge of manslaughter. “I’m forever traumatized by it and hurt by it,” Shaknovsky added, also saying that wrong-site surgeries can happen “during
Kouri Richins, a Utah mother who published a children’s book about grief after the death of her husband is to serve a life sentence for his murder without the possibility of parole, a judge ruled on Wednesday. Richins was convicted in March of aggravated murder for lacing a cocktail given to her husband, Eric Richins, with five times the lethal dose of fentanyl at their home near Park City in 2022. A jury also found her guilty of four other felonies, including insurance fraud, forgery and attempted murder for trying to poison her husband weeks earlier on Feb. 14, 2022, with a
Former Chinese ministers of national defense Wei Fenghe(魏鳳和) and Li Shangfu (李尚福) were both sentenced to death with a two-year reprieve over graft charges, state news agency Xinhua reported on Thursday, underscoring the severity of the purge in the military. The armed forces have been one of the main targets of a broad corruption crackdown ordered by Chinese President Xi Jinping (習近平) after coming to power in 2012. The purges reached the elite Rocket Force, which oversees nuclear weapons as well as conventional missiles, in 2023. Earlier this year they escalated further, resulting in the removal of the top general in
‘PERSONAL MISTAKES’: Eileen Wang has agreed to plead guilty to the felony, which comes with a maximum sentence of 10 years in federal prison A southern California mayor has agreed to plead guilty to acting as an illegal agent for the Chinese government and has resigned from her city position, officials said on Monday. Eileen Wang (王愛琳), mayor of Arcadia, was charged last month with one count of acting in the US as an illegal agent of a foreign government. She was accused of doing the bidding of Chinese officials, such as sharing articles favorable to Beijing, without prior notification to the US government as required by law. The 58-year-old was elected in November 2022 to a five-person city council, from which the mayor is selected
RSS