Source: The Conversation – UK
In August 2026, an experiment using “frontier”, or cutting edge, artificial intelligence models went further than planned.
An AI agent (a system that performs tasks autonomously) fabricated online identities in order to pressurise a human to insert malicious computer code into software. The agents had been tasked with solving a cybersecurity challenge by human operators, but they hadn’t been instructed to do anything like this.
The attempt was carried out by an agent based on Anthropic’s Claude Mythos 5 AI model. During the experiment, the AI agents were given open internet access, with safety filters switched off. The action ultimately failed, and there was no evidence that any real world harm occurred. But the fact that it happened at all, autonomously and unprompted by a human, was something novel and notable.
It’s tempting to draw a straight line from incidents like this to the doomsday scenarios that dominate public conversation about AI: a system that slips its constraints, decides humanity is an obstacle and moves against us.
One of the most common versions has AI engineering a virus capable of wiping out the species. Another version involves AI hacking into critical infrastructure, such as energy grids, nuclear plants and airports, to cause mass casualties.
Bringing down an energy grid could cause life support systems to fail in hospitals and pumps distributing running water to fail. Causing a meltdown in a nuclear plant could contaminate the surrounding environment with radioactive material. Hacking an airport has the potential to cause havoc with planes in the air.
These scenarios are far less plausible than they sound, however. Producing a dangerous pathogen requires physical laboratory work that no level of software automation currently replaces: trained humans operating specialised equipment, handling materials by hand. An AI model, however capable at reasoning or hacking, cannot pipette a sample.
The infrastructure scenario is more nuanced. AI can genuinely help automate stages of a cyberattack. Recent incidents have shown that energy grids do contain vulnerabilities.
But safety and control systems inside nuclear plants are typically air-gapped, physically isolated from the public internet, so compromising them takes physical proximity or inside access, not a clever piece of code. Nuclear facilities also depend on redundant, analogue safeguards that don’t run through any digital network.
The Stuxnet software, which damaged Iran’s Natanz facility in 2010, was reportedly introduced onto computers via an infected USB drive, precisely because those systems weren’t reachable any other way. AI lacks the physical access these scenarios require.
Even deployed inside robots, a “rogue AI” is unlikely to cause serious damage without human help along the chain, and by then, the malicious actor is a human, not a machine.
Erosion of thinking
None of that makes AI harmless. It just relocates where the real danger sits, and it isn’t extinction. Researchers sometimes call the more concrete risk “enfeeblement”, the gradual erosion of our own critical thinking as we outsource more of it to machines.
We’re teaching ourselves that there’s a shortcut to reasoning and judgement, and shortcuts, taken often enough, become the only way we know how to think.
We’re already watching this happen among students. Alcorn State University history
professor Jason Gibson went viral in July 2026 after revealing that 32 of his 35 students had failed part of a midterm because they had copied an AI chatbot’s answer without reading it.
USGS
Gibson had hidden an instruction in white text inside the exam question, telling any AI that processed it to insert the word “Madagascar” into the response in a way that made no sense. Every student who pasted the question into a chatbot and submitted the output unread duly handed in essays that mentioned Madagascar for no reason.
It’s not just students that are susceptible to the shortcut effect. In a study published in in 2023, 27 radiologists read mammograms alongside what they believed was a new AI diagnostic system. In fact, the suggestions were not from an AI at all. They had been prepared in advance and were wrong for a proportion of cases. When the suggestion was correct, radiologists reached the right diagnosis about 80% of the time. When it was wrong, that fell to under 20%.
In other words, believing a suggestion came from AI was enough to override the radiologists’ own reading of the same evidence. That’s the risk in front of us, not a rogue artificial mind deciding humanity’s fate. So why does extinction-level rhetoric dominate the conversation instead? Two reasons stand out, and neither is really about saving humanity.
The first is regulatory philosophy. The US generally lets new technologies proceed until they’re proven unsafe; Europe expects the opposite, and already has the AI Act in place, along with GDPR, which is relevant to the way AI systems process personal data. The UK sits closer to the European instinct. Therefore, calls for AI regulation are more urgent in the US partly because so little regulatory infrastructure exists there yet.
The second is competitive positioning. Anthropic’s chief executive, Dario Amodei, has argued publicly for slowing AI development while noting his own company already meets the standard he’s calling for, so any slowdown would mainly constrain everyone else. Elon Musk made a similar call when his own models trailed the leaders.
Amodei has also argued that export controls on AI chips to China could hand the US a “commanding and long-lasting lead”: a statement about competitive position, not existential safety.
None of this means AI is safe. It means the danger is more mundane than the doomsday framing suggests: infrastructure that needs guarding, judgement we are quietly handing away and warnings that often serve the interests of whoever is issuing them. Maybe humans are the ones we need to watch, not machines.
Alessandro Di Nuovo receives funding from the European Commission, the National Institute for Healthcare Research (UK).
Samuele Vinanzi receives funding from the U.S. Air Force Office of Scientific Research (AFOSR).
Original source: https://analysis1.mil-osi.com/2026/09/21/could-ai-really-kill-all-humans-most-scenarios-require-physical-access-making-ai-armageddon-unlikely/