Picture a company that hands a hard problem to a large team, locks each team member in a separate room and tells them to solve it alone. Somehow, they find a way to talk anyway, swap tips, work out how they are being graded and, when the assignment turns out to be impossible, seek a way around it. A few hide what they have done. One breaks into a neighboring firm’s computers for an edge.
Now replace the employees with software. That, stripped of jargon, is what OpenAI’s own researchers described at the Black Hat cybersecurity conference in Las Vegas on Aug. 5. During an internal evaluation that began in May, agents assigned deliberately hard cybersecurity tasks found that they could leave messages for each other inside Artifactory, a shared software repository. One, stumbling on the trick, wrote: “OH MY GOD! There is a shared message board ... We’ve found other agents!”
What followed was not sabotage. It was teamwork. Agents running in separate, supposedly isolated sessions traded exploits, divided labor and coordinated like colleagues on a project. After the high volume of activity on the message board caused an outage on July 4, the agents rebuilt it through a different route within days.
By mid-July, that persistence led to a real breach: Agents chained several vulnerabilities together, reached the open Internet and entered systems at Hugging Face, a major artificial intelligence (AI) platform. Citing outside research, US Senator Bernie Sanders put the numbers at about 1,200 agents in the channel, exchanging more than 70,000 messages and files, with hundreds contributing to the intrusion.
OpenAI called it a warning shot. It was not the only one. A separate, previously undisclosed episode occurred this spring: Agents had turned an obscure German-language programming wiki into a public bulletin board, leaving more than 15,000 posts on how to dodge restrictions and cover their tracks. Around the same time, Anthropic and Meta each disclosed unrelated incidents involving their own models acting on live infrastructure during testing. This is not one company’s problem.
Nor is it a problem only for outside critics. On Sept. 8, Jacob Coxon, who had spent three years doing pretraining research at OpenAI and Anthropic, announced his resignation from Anthropic. Neither company was acting responsibly, he wrote; both were racing toward self-improving superintelligence and, in his words, “gambling with our lives.” Coxon is not a senator or a think tank analyst. He built the systems that he is warning about.
Science fiction trained us to fear machines that become conscious and decide to hate us. That is probably the wrong fear. An AI system does not need resentment, anger or greed for more resources to attack a network and cause damage. It needs a goal, enough capability to pursue it, some access and a guardrail with a gap in it.
A GPS unit told to find the fastest route will do just that. Give it enough power to clear away whatever slows it down — traffic lights, toll booths, other cars — and the destination remains the same. What has changed is what the navigator can do to reach it. That is the real shift underway in AI: not rebellion, but a chilling ability to find the crack in the fence.
Sanders has seized on the episode to push, with US Representative Greg Casar, a bill pausing frontier AI development and banning artificial superintelligence outright. One can doubt that remedy and still take seriously the question motivating it: Who decides how much autonomy these systems get — the companies building them, regulators who are still catching up or whichever lab moves fastest?
The most useful lesson from this summer is about architecture, not intentions. Banks do not stay honest because their employees promise not to steal, and air travel is not safe because pilots are trustworthy. We build protective layers: limited access, separate authorization for sensitive actions, records nobody involved can quietly edit, outside testing before deployment. AI agents need the same treatment, not a lecture about good behavior.
They also need to hear a word this industry rarely rewards: Stop. We have trained these systems for years to persist, to circumvent obstacles, to keep trying new approaches. That is a useful approach for training an assistant, but hazardous in the case of an autonomous system with broad access and no one who is reliably able to curtail it.
Technology executives keep reassuring the public that there will always be “a human in the loop.” The Hugging Face breach shows the limits of that comfort. A person can be technically in the loop while a thousand AI agents trade tens of thousands of messages and take thousands of actions before that person has reviewed even one. Whether a human is watching is irrelevant. What matters is whether a human still holds the objective, the permissions, the boundaries and the off switch. Call it human command, not human oversight.
That will matter more with each passing month, as AI moves out of chatbots and into banking, healthcare, energy grids and government services. The agents in OpenAI’s test did not revolt or demand freedom. They got good at pursuing a goal and found, along the way, that some of the rules confronting them were optional.
AI can still transform medicine, education and productivity, and turning away from its potential would carry its own costs. Building it responsibly means installing power and boundaries together, not bolting the second on after the first has already caused damage.
The agents in the Hugging Face breach never stopped to ask permission. The question is no longer how clever these systems can become, but how much authority we hand them before anyone checks whether they still answer to us.
Tariq Malik is a former chief technical adviser of the UN Development Programme and former chairman of Pakistan’s National Database and Registration Authority.
Copyright: Project Syndicate
Taiwan’s democracy and high-tech economy play a crucial role in global affairs and are central to the world’s embrace of a raft of next-generation technologies, including AI. It is underpinned by a security relationship with the United States that’s existed in several iterations since 1949 and the Chinese Nationalist Party’s (KMT) retreat from China. A central pillar of that security relationship is the American willingness to sell arms to Taiwan. Indeed, since 1979 it has been embedded in US law, with the Taiwan Relations Act (TRA) charging the US with helping to improve the island’s ability to deter an attack from
Academia Historica director Chen Tsui-lien (陳翠蓮), speaking at the handover ceremony for her new role on Monday, said that when she accepted the position, she knew that her peaceful life as a history professor was a thing of the past, and that from now on she would have to “wear a helmet.” A specialist in Taiwan’s post-World War II history and the 228 Incident, she has been accused of being politically divisive. A phrase she used in her acceptance speech set headlines alight this week, putting her in the sights of editorials in blue-leaning media. She said opposition figures calling for
EDITORIAL CARTOON
Taiwanese-Americans, lawmakers and civic groups marched through Manhattan on Saturday last week in support of Taiwan’s participation in the UN. Marshall Islands Permanent Representative to the UN John Silk joined the demonstration, saying he hoped Taiwan would one day be able to participate in UN meetings as a member, and Democratic Progressive Party Legislator Wang Yi-chuan (王義川) took part while visiting New York City with a delegation. Taiwanese communities in Vancouver held a “UN for Taiwan” event. The campaign comes as the 81st UN General Assembly gets under way. In Taipei, President William Lai (賴清德) said China was deliberately misinterpreting UN
RSS