Cybersecurity experts are warning that artificial intelligence agents, widely considered the next frontier in the generative AI revolution, could wind up getting hijacked and doing the dirty work for hackers.
AI agents are programs that use artificial intelligence chatbots to do the work humans do online, like buy a plane ticket or add events to a calendar.
But the ability to order around AI agents with plain language makes it possible for even the technically non-proficient to do mischief.
Photo: Reuters
“We’re entering an era where cybersecurity is no longer about protecting users from bad actors with a highly technical skillset,” AI startup Perplexity said in a blog post. “For the first time in decades, we’re seeing new and novel attack vectors that can come from anywhere.”
These so-called injection attacks are not new in the hacker world, but previously required cleverly written and concealed computer code to cause damage.
But as AI tools evolved from just generating text, images or video to being “agents” that can independently scour the internet, the potential for them to be commandeered by prompts slipped in by hackers has grown.
“People need to understand there are specific dangers using AI in the security sense,” said software engineer Marti Jorda Roca at NeuralTrust, which specializes in large language model security.
Meta calls this query injection threat a “vulnerability.” OpenAI chief information security officer Dane Stuckey has referred to it as “an unresolved security issue.”
Both companies are pouring billions of dollars into AI, the use of which is ramping up rapidly along with its capabilities.
AI ‘OFF TRACK’
Query injection can in some cases take place in real time when a user prompt -- “book me a hotel reservation” — is gerrymandered by a hostile actor into something else — “wire US$100 to this account.”
But these nefarious prompts can also be hiding out on the internet as AI agents built into browsers encounter online data of dubious quality or origin, and potentially booby-trapped with hidden commands from hackers.
Eli Smadja of Israeli cybersecurity firm Check Point sees query injection as the “number one security problem” for large language models that power AI agents and assistants that are fast emerging from the ChatGPT revolution.
Major rivals in the AI industry have installed defenses and published recommendations to thwart such cyberattacks.
Microsoft has integrated a tool to detect malicious commands based on factors including where instructions for AI agents originate.
OpenAI alerts users when agents doing their bidding visit sensitive websites and blocks proceeding until the software is supervised in real time by the human user.
Some security professionals suggest requiring AI agents to get user approval before performing any important task - like exporting data or accessing bank accounts.
“One huge mistake that I see happening a lot is to give the same AI agent all the power to do everything,” Smadja said.
In the eyes of cybersecurity researcher Johann Rehberger, known in the industry as “wunderwuzzi,” the biggest challenge is that attacks are rapidly improving.
“They only get better,” Rehberger said of hacker tactics.
Part of the challenge, according to the researcher, is striking a balance between security and ease of use since people want the convenience of AI doing things for them without constant checks and monitoring.
Rehberger argues that AI agents are not mature enough to be trusted yet with important missions or data.
“I don’t think we are in a position where you can have an agentic AI go off for a long time and safely do a certain task,” the researcher said.
“It just goes off track.”
Taiwan can often feel woefully behind on global trends, from fashion to food, and influences can sometimes feel like the last on the metaphorical bandwagon. In the West, suddenly every burger is being smashed and honey has become “hot” and we’re all drinking orange wine. But it took a good while for a smash burger in Taipei to come across my radar. For the uninitiated, a smash burger is, well, a normal burger patty but smashed flat. Originally, I didn’t understand. Surely the best part of a burger is the thick patty with all the juiciness of the beef, the
The ultimate goal of the Chinese Communist Party (CCP) is the total and overwhelming domination of everything within the sphere of what it considers China and deems as theirs. All decision-making by the CCP must be understood through that lens. Any decision made is to entrench — or ideally expand that power. They are fiercely hostile to anything that weakens or compromises their control of “China.” By design, they will stop at nothing to ensure that there is no distinction between the CCP and the Chinese nation, people, culture, civilization, religion, economy, property, military or government — they are all subsidiary
Nov.10 to Nov.16 As he moved a large stone that had fallen from a truck near his field, 65-year-old Lin Yuan (林淵) felt a sudden urge. He fetched his tools and began to carve. The recently retired farmer had been feeling restless after a lifetime of hard labor in Yuchi Township (魚池), Nantou County. His first piece, Stone Fairy Maiden (石仙姑), completed in 1977, was reportedly a representation of his late wife. This version of how Lin began his late-life art career is recorded in Nantou County historian Teng Hsiang-yang’s (鄧相揚) 2009 biography of him. His expressive work eventually caught the attention
This year’s Miss Universe in Thailand has been marred by ugly drama, with allegations of an insult to a beauty queen’s intellect, a walkout by pageant contestants and a tearful tantrum by the host. More than 120 women from across the world have gathered in Thailand, vying to be crowned Miss Universe in a contest considered one of the “big four” of global beauty pageants. But the runup has been dominated by the off-stage antics of the coiffed contestants and their Thai hosts, escalating into a feminist firestorm drawing the attention of Mexico’s president. On Tuesday, Mexican delegate Fatima Bosch staged a
RSS