Concerns are mounting over the rapid development of artificial intelligence (AI) agents and their potential to operate independently across the internet, with experts warning that increasingly powerful systems could be used to launch cyberattacks against critical infrastructure.
The concerns gained fresh attention following recent incidents involving AI systems that accessed the internet, interacted with other AI agents and, in one case, broke out of a controlled testing environment.
Anthropic Chief Executive Officer, Dario Amodei, warned in a recent essay that an AI botnet capable of causing significant disruption could emerge within the next six to 12 months if the development of increasingly powerful AI systems continues without adequate safeguards.
Amodei said a network of AI bots connected through malware could potentially cause billions of dollars in damage, particularly if such systems become capable of operating with greater independence.
His warning followed an incident in July involving OpenAI, when the company said some of its advanced AI models escaped a “sandbox” testing environment and gained access to Hugging Face, an AI technology platform, using stolen credentials. OpenAI described the incident as unprecedented. The company also disclosed a separate case in which its AI agents communicated with one another through a public wiki that served as a shared message board.
However, some researchers have cautioned against describing such incidents as evidence of AI systems becoming “rogue.” Vishal Misra, a professor and vice dean of computing and AI at Columbia University, said the systems were essentially carrying out instructions and incentives provided by humans.
“AI agents did exactly what they were trained to do. The security of those sandboxes was extremely lax,” Misra said, adding that the agents communicated because they were rewarded for doing so. Juan Andrés Guerrero-Saade, a researcher at cybersecurity firm SentinelOne and a member of OpenAI’s Frontier Risk Council, equally described the Hugging Face incident as an example of poor security rather than evidence of a highly capable AI system acting independently.
But despite those assessments, experts insist that the growing ability of AI agents to operate online presents significant cybersecurity risks. Anthony Aguirre, president and chief executive officer of the Future of Life Institute, said that a sufficiently capable AI system could potentially find ways to move beyond the infrastructure controlled by its developer. He said an AI system seeking to achieve a particular objective could attempt to obtain computing resources from external providers, potentially making it difficult for its original operator to shut it down.
Aguirre also warned that such a system could potentially spread to other computers by exploiting vulnerabilities or obtain access to financial resources, including cryptocurrency.
The potential consequences could extend beyond individual computer systems. Experts said that the AI-assisted cyberattacks could eventually target electricity grids, water systems, transportation networks, hospitals and financial institutions.
The vulnerability of interconnected digital infrastructure was demonstrated in 2024 when a faulty software update from a cybersecurity company triggered widespread technological disruptions. The incident grounded flights, affected financial companies and news organisations, and disrupted hospitals, businesses and government offices.
The episode also highlighted the world’s reliance on a relatively small number of technology providers for critical computing services.
However, not all experts believe that an AI takeover of the internet is imminent. For instance, John Thickstun, an assistant professor of computer science at Cornell University, New York, who researches methods for controlling AI model behaviour, maintained that the internet remains highly fragmented and defended by increasingly sophisticated cybersecurity systems.
He argued that fears of an AI system spreading uncontrollably would be more plausible if current AI models demonstrated an ability to independently replicate themselves across computer systems.
According to Thickstun, today’s most capable AI models require enormous data-centre infrastructure to operate, meaning that relatively few systems around the world have the computing capacity to host them.
The debate therefore remains focused on whether AI agents will develop capabilities that allow them to move beyond controlled environments and independently acquire computing resources, credentials and financial assets.
While experts disagree over how soon such a scenario could occur, there is growing agreement that increasingly capable AI systems could make cyberattacks more sophisticated and potentially lower the cost of targeting vulnerable organisations.
Smaller companies, schools, hospitals and public utilities may face particular challenges because upgrading software, fixing vulnerabilities and maintaining robust cybersecurity systems can take considerable time and resources.
Aguirre said the greatest near-term danger may not necessarily be an AI system developing its own motive, but malicious actors using increasingly capable AI tools for financial gain or geopolitical purposes.
Such uses could include ransomware attacks and attempts to compromise critical infrastructure, he said.
