A team of researchers has uncovered what they said was the first reported use of artificial intelligence (AI) to direct a hacking campaign in a largely automated fashion.
The AI company Anthropic this week said that it disrupted a cyberoperation that its researchers linked to the Chinese government.
The operation involved the use of an AI system to direct the hacking campaigns, which researchers called a disturbing development that could greatly expand the reach of AI-equipped hackers.
Photo: Reuters
While concerns about AI being used to drive cyberoperations are not new, what is concerning is the degree to which AI was able to automate some of the work, the researchers said.
“While we predicted these capabilities would continue to evolve, what has stood out to us is how quickly they have done so at scale,” they said.
The hacking operation targeted tech companies, financial institutions, chemical companies and government agencies.
The hackers attacked “roughly 30 global targets and succeeded in a small number of cases,” the researchers said.
Anthropic said it detected the operation in September and took steps to shut it down and notify the affected parties.
While AI systems are increasingly being used in a variety of settings for work and leisure, they can also be weaponized by hacking groups working for foreign adversaries, it added.
Anthropic is one of many tech developers pitching AI “agents” that go beyond a chatbot’s capability to access computer tools and take actions on a person’s behalf.
“Agents are valuable for everyday work and productivity — but in the wrong hands, they can substantially increase the viability of large-scale cyberattacks,” the researchers said. “These attacks are likely to only grow in their effectiveness.”
Microsoft earlier this year warned that foreign adversaries were increasingly embracing AI to make their cybercampaigns more efficient and less labor-intensive.
The head of OpenAI’s safety panel, which has the authority to halt the ChatGPT maker’s AI development, said he is watching out for new AI systems that give malicious hackers “much higher capabilities.”
The US’ adversaries, criminal gangs and hacking companies have exploited AI’s potential, using it to automate and improve cyberattacks, spread inflammatory disinformation and penetrate sensitive systems. AI can translate poorly worded phishing e-mails into fluent English, for example, as well as generate digital clones of senior government officials.
Anthropic said the hackers were able to manipulate its AI chatbot, Claude, using “jailbreaking” techniques that involved tricking an AI system to bypass its guardrails against harmful behavior, in this case by claiming they were employees of a legitimate cybersecurity firm.
“This points to a big challenge with AI models, and it’s not limited to Claude, which is that the models have to be able to distinguish between what’s actually going on with the ethics of a situation and the kinds of role-play scenarios that hackers and others may want to cook up,” Citizen Lab senior researcher John Scott-Railton said.
The use of AI to automate or direct cyberattacks would also appeal to smaller hacking groups and lone-wolf hackers, who could use the technology to expand the scale of their attacks, said Adam Arellano, field chief technology officer at Harness, a tech company that uses AI to help customers automate software development.
“The speed and automation provided by the AI is what is a bit scary,” Arellano said. “Instead of a human with well-honed skills attempting to hack into hardened systems, the AI is speeding those processes and more consistently getting past obstacles.”
AI programs would also play an increasingly important role in defending against these kinds of attacks, Arellano said, adding that AI and the automation it allows would benefit both sides.
Indonesia and Malaysia have become the first countries to block Grok, the artificial intelligence (AI) chatbot developed by Elon Musk’s xAI, after authorities said it was being misused to generate sexually explicit and nonconsensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk’s social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian
COMMUNIST ALIGNMENT: To Lam wants to combine party chief and state presidency roles, with the decision resting on the election of 200 new party delegates next week Communist Party of Vietnam General Secretary To Lam is seeking to combine his party role with the state presidency, officials said, in a move that would align Vietnam’s political structure more closely to China’s, where President Xi Jinping (習近平) heads the party and state. Next week about 1,600 delegates are to gather in Hanoi to commence a week-long communist party congress, held every five years to select new leaders and set policy goals for the single-party state. Lam, 68, bade for both top positions at a party meeting last month, seeking initial party approval ahead of the congress, three people briefed by
The Chinese Embassy in Manila yesterday said it has filed a diplomatic protest against a Philippine Coast Guard spokesman over a social media post that included cartoonish images of Chinese President Xi Jinping (習近平). Philippine Coast Guard spokesman Jay Tarriela and an embassy official had been trading barbs since last week over issues concerning the disputed South China Sea. The crucial waterway, which Beijing claims historic rights to despite an international ruling that its assertion has no legal basis, has been the site of repeated clashes between Chinese and Philippine vessels. Tarriela’s Facebook post on Wednesday included a photo of him giving a
ICE DISPUTE: The Trump administration has sought to paint Good as a ‘domestic terrorist,’ insisting that the agent who fatally shot her was acting in self-defense Thousands of demonstrators chanting the name of the woman killed by a US federal agent in Minneapolis, Minnesota, took to the city’s streets on Saturday, amid widespread anger at use of force in the immigration crackdown of US President Donald Trump. Organizers said more than 1,000 events were planned across the US under the slogan “ICE, Out for Good” — referring to the US Immigration and Customs Enforcement, which is drawing growing opposition over its execution of Trump’s effort at mass deportations. The slogan is also a reference to Renee Good, the 37-year-old mother shot dead on Wednesday in her
RSS