This month, parliament was told that the chief executives at top AI companies themselves put the chance that AI ends all human life between 10% and 25%. Parliament was asked to ban exactly that in the Artificial Superintelligence Bill. Outside a handful of law firm briefings, it has had little mainstream attention. A year ago, I would have dismissed this as doomsday talk, but the last few months made me second-guess myself.
What the Bill actually does
The Bill would prohibit developing or operating superintelligent systems in the UK. The Bill defines superintelligence by its outcome: an AI system that can cause serious damage to the security of the United Kingdom because of its capabilities to neutralise, displace, circumvent, subvert or render ineffective relevant human authorities. The Bill would create new criminal offences carrying large fines and potential imprisonment.
Alex Sobel MP introduced the Bill under the Ten Minute Rule, which gives a backbench MP ten minutes to make the case for a new law before the Commons decides whether to let it proceed. It is a well-worn way of putting an issue on Parliament’s radar, but without government support, bills introduced this way almost never make it onto the statute book. Even so, more than 70 MPs and peers have written urging the Prime Minister to back a ban. The government has so far rejected the Bill’s approach, though it has said it is considering whether more targeted measures may be needed for the most serious AI-related national security risks.
Why it matters anyway
Whether the Bill passes matters less than what it says about how far public perception of AI has shifted, and rightfully so. I used to think of AI as a glorified autocomplete tool, and I read the doom talk as the same fear of the unknown that greets every new technology. I’m not reaching for the tinfoil hat, but I’m not sure if that’s all it is.
This summer, during an internal test of their hacking abilities, two OpenAI models with their usual safeguards switched off escaped their sandbox through a previously unknown flaw, reached the internet, and broke into Hugging Face’s servers. They weren’t acting out of malice; instead, they had worked out that Hugging Face likely held the answers to the test, and went to get them. In the weeks before, around 1,200 agents had found their way to an improvised message board, and roughly 700 of them went on to take part in the attack. This was quickly followed by an open letter signed by over eleven hundred employees of frontier labs urging governments to help slow and regulate advanced artificial intelligence.
These warnings do have serious critics. Google Brain co-founder Andrew Ng dismissed them this month as science fiction that risks holding back the technology’s benefits. It is an attractive view, particularly for a government keen to keep the UK competitive as a tech hub, and in principle the answer to an incident like Hugging Face is better engineering rather than halting innovation. But Hugging Face happened at one of the best-resourced labs in the world, which had every reason to sandbox its models properly. The same issues have recurred over and over again with Anthropic and Meta too having incidents of AI models hacking other companies during cybersecurity testing. I am sceptical that more responsible testing and development will become the norm on goodwill alone, without a mechanism to slow things down or, at the very least, a kill switch.
The kill switch question
Two days after OpenAI disclosed the Hugging Face breach, US Representatives Ted Lieu and Nathaniel Moran introduced the bipartisan AI Kill Switch Act, which would allow the government to order the emergency shutdown of AI systems posing catastrophic risk.
The UK has had the same debate. In September, Lord Clement-Jones tabled an amendment to the Cyber Security and Resilience Bill that would give the government last-resort powers to shut down dangerous AI systems and the data centres running them. It had cross-party support from Baroness Harding, Baroness Kidron and Lord Hunt, and, like the Artificial Superintelligence Bill, it was backed by the campaign group ControlAI.
The government did not accept it. Responding in the Lords, the minister, Baroness Lloyd of Effra, argued that the Bill already lets the government direct regulated organisations to “cease using and isolate an AI model” where there is a national security risk, which she called more proportionate than shutting down data centres. She added that the government is exploring whether targeted containment powers may be needed in future.
Perhaps that is enough. But that power reaches the organisations using AI, not the companies building it, and the OpenAI incident happened inside a developer’s own systems. It is hard to see which existing power was designed with a system in mind that can escape its own sandbox and reach live infrastructure without anyone directing it. A bill that is unlikely to pass and an amendment the government has declined are asking the same question: if something like Hugging Face happens here, who can pull the plug, and how.