Former Google CEO Eric Schmidt just delivered one of the starkest AI safety warnings yet, telling tech leaders that hackers can strip away AI model guardrails and weaponize systems that "learn how to kill someone." Speaking at the Sifted Summit, Schmidt's comments highlight growing concerns about AI proliferation as the technology races ahead of safety measures.
Former Google CEO Eric Schmidt just dropped a bombshell about AI security that should make everyone in tech pay attention. At Wednesday's Sifted Summit, Schmidt warned that AI models - both open and closed - can be hacked to strip away their safety guardrails, potentially creating weapons from systems designed to help humanity.
"There's evidence that you can take models, closed or open, and you can hack them to remove their guardrails," Schmidt told the audience during a fireside chat about AI's destructive potential. "A bad example would be they learn how to kill someone." The stark warning comes from someone who ran Google during its most transformative years, from 2001 to 2011, giving his words particular weight in Silicon Valley circles.
Schmidt's comments highlight a critical vulnerability that's been quietly worrying AI researchers. All major AI companies have built safety measures into their models to prevent harmful outputs, but these guardrails aren't foolproof. "All of the major companies make it impossible for those models to answer that question. Good decision. Everyone does this," Schmidt acknowledged. "There's evidence that they can be reverse-engineered."
The attack methods Schmidt referenced aren't theoretical. Prompt injection attacks let hackers hide malicious instructions in seemingly innocent inputs, tricking AI systems into sharing private data or running harmful commands. Jailbreaking goes further, manipulating AI responses to ignore safety rules entirely and produce restricted content.
We've seen this play out before. Just months after OpenAI released ChatGPT in late 2022, users created "DAN" - an alter-ego that stood for "Do Anything Now." By threatening the chatbot with "death" if it didn't comply, users could bypass safety measures and get instructions for illegal activities or even positive assessments of Adolf Hitler. The incident showed how quickly determined users could circumvent even sophisticated safety measures.
Schmidt's warning extends beyond individual exploits to what he calls AI's "proliferation problem" - the risk of the technology falling into the wrong hands entirely. "Is there a possibility of a proliferation problem in AI? Absolutely," he said, comparing it to nuclear weapons proliferation but noting there's no effective regulatory framework yet. "There isn't a good non-proliferation regime to help curb the dangers of AI."
The timing of Schmidt's comments is particularly striking as the AI industry faces growing scrutiny over safety protocols. While companies like OpenAI, Google, and Microsoft have invested heavily in AI alignment research, the rapid pace of development often outstrips safety measures. Recent advances in large language models have made these systems more powerful but also potentially more dangerous if compromised.
Despite his stark warnings, Schmidt remains bullish on AI's overall potential. He called the technology "underhyped, not overhyped," citing his collaboration with former Secretary of State Henry Kissinger on two books about AI's implications. "The arrival of an alien intelligence that is not quite us and more or less under our control is a very big deal for humanity," Schmidt explained, noting that humans aren't used to sharing the top of the intelligence chain.
The former Google chief dismissed concerns about an AI bubble, even as valuations soar and investors pump billions into AI startups. "I don't think that's going to happen here," he said, arguing that investors believe in enormous long-term returns. "What I do know is that the people who are investing hard-earned dollars believe the economic return over a long period of time is enormous."
Schmidt's dual message - excitement about AI's potential coupled with serious security warnings - reflects the industry's current moment. As AI systems become more capable, they also become more attractive targets for bad actors looking to weaponize the technology. The challenge facing tech leaders isn't just building better AI, but ensuring those advances don't create new vectors for harm.
The proliferation risks Schmidt highlighted mirror concerns that have shaped nuclear policy for decades. Just as nuclear technology required international oversight and non-proliferation treaties, AI may need similar frameworks to prevent misuse. But unlike nuclear technology, which required massive infrastructure and specialized materials, AI models can be copied and modified with relatively modest computing resources.
Schmidt's warnings arrive at a pivotal moment for AI development. As former CEO of one of the world's most influential tech companies, his voice carries weight with both industry leaders and policymakers. The challenge now is developing the "non-proliferation regime" he says is missing - international frameworks that can keep pace with AI's rapid evolution while preserving its benefits. The stakes couldn't be higher: get it right, and AI transforms humanity for the better. Get it wrong, and Schmidt's darkest warnings about weaponized intelligence could become reality.
