A team of researchers has uncovered what they said was the first reported use of artificial intelligence (AI) to direct a hacking campaign in a largely automated fashion.
The AI company Anthropic this week said that it disrupted a cyberoperation that its researchers linked to the Chinese government.
The operation involved the use of an AI system to direct the hacking campaigns, which researchers called a disturbing development that could greatly expand the reach of AI-equipped hackers.
Photo: Reuters
While concerns about AI being used to drive cyberoperations are not new, what is concerning is the degree to which AI was able to automate some of the work, the researchers said.
“While we predicted these capabilities would continue to evolve, what has stood out to us is how quickly they have done so at scale,” they said.
The hacking operation targeted tech companies, financial institutions, chemical companies and government agencies.
The hackers attacked “roughly 30 global targets and succeeded in a small number of cases,” the researchers said.
Anthropic said it detected the operation in September and took steps to shut it down and notify the affected parties.
While AI systems are increasingly being used in a variety of settings for work and leisure, they can also be weaponized by hacking groups working for foreign adversaries, it added.
Anthropic is one of many tech developers pitching AI “agents” that go beyond a chatbot’s capability to access computer tools and take actions on a person’s behalf.
“Agents are valuable for everyday work and productivity — but in the wrong hands, they can substantially increase the viability of large-scale cyberattacks,” the researchers said. “These attacks are likely to only grow in their effectiveness.”
Microsoft earlier this year warned that foreign adversaries were increasingly embracing AI to make their cybercampaigns more efficient and less labor-intensive.
The head of OpenAI’s safety panel, which has the authority to halt the ChatGPT maker’s AI development, said he is watching out for new AI systems that give malicious hackers “much higher capabilities.”
The US’ adversaries, criminal gangs and hacking companies have exploited AI’s potential, using it to automate and improve cyberattacks, spread inflammatory disinformation and penetrate sensitive systems. AI can translate poorly worded phishing e-mails into fluent English, for example, as well as generate digital clones of senior government officials.
Anthropic said the hackers were able to manipulate its AI chatbot, Claude, using “jailbreaking” techniques that involved tricking an AI system to bypass its guardrails against harmful behavior, in this case by claiming they were employees of a legitimate cybersecurity firm.
“This points to a big challenge with AI models, and it’s not limited to Claude, which is that the models have to be able to distinguish between what’s actually going on with the ethics of a situation and the kinds of role-play scenarios that hackers and others may want to cook up,” Citizen Lab senior researcher John Scott-Railton said.
The use of AI to automate or direct cyberattacks would also appeal to smaller hacking groups and lone-wolf hackers, who could use the technology to expand the scale of their attacks, said Adam Arellano, field chief technology officer at Harness, a tech company that uses AI to help customers automate software development.
“The speed and automation provided by the AI is what is a bit scary,” Arellano said. “Instead of a human with well-honed skills attempting to hack into hardened systems, the AI is speeding those processes and more consistently getting past obstacles.”
AI programs would also play an increasingly important role in defending against these kinds of attacks, Arellano said, adding that AI and the automation it allows would benefit both sides.
‘CROSSING THE LINE’: China’s embassy in Seoul criticized US Forces Korea Commander General Xavier Brunson, asking if his ‘hostile’ remarks were authorized by Washington South Korea and the US are in talks over recent public remarks by the commander of US Forces Korea, Seoul’s presidential office said yesterday, after the comments drew sharp criticism from China. In a recent podcast interview, US Forces Korea Commander General Xavier Brunson described South Korea as “the dagger in the heart of Asia” from China’s east coast, prompting the Chinese embassy in Seoul to say that he had “truly crossed the line.” The interview came amid growing speculation that Washington might seek to expand the role of US Forces Korea in countering the growing regional influence of China, a key
Through the noise of rushing papers and whirring belts at a print factory in Kyoto, two creators watch their photo essay come to life in broadsheet form — part of an effort to win new audiences in the age of artificial intelligence (AI). Despite the decline of the publishing industry, self-publication and handmade “zine” magazines are growing in popularity in Japan, reflecting the nation’s enduring love of paper in the digital era. While speaking to Agence France-Presse at the plant, his hands black with ink, one of the creators, Kazuma Obara, said: “I think [paper] is a medium that engages all five
Australian researchers have trained lab-grown brain cells on a silicon computer chip to play the 1990s shooter game Doom and said they are just scratching the surface of what the neurons could be capable of doing. It is the science-fiction work of biotech boffins at Cortical Labs, who researched and developed the technology that harnesses the workings of the brain’s networking system. Each so-called “biological computer” contains about 200,000 living human brain cells, grown from stem cells that were harvested from blood donations. Having mastered the simple computer game Pong, where a paddle is moved up and down to send a ball
A surprising gut feeling may help pigeons find their way home. Animals use various techniques to navigate, including following the stars and remembering key landmarks. Birds, fish and turtles orient themselves using Earth’s magnetic field as a compass, but it is not yet clear how exactly they do this. Pigeons are a well-known group of frequent flyers that can traverse hundreds of kilometers in a single day. For thousands of years, humans have used them to carry news, notes and military messages. Scientists have long tried to untangle how pigeons travel without getting lost. Some think the birds detect magnetic cues using light-sensitive
RSS