A coalition of nonprofits is demanding the U.S. government immediately pull xAI's Grok chatbot from federal agencies, including the Pentagon. The open letter - shared exclusively with TechCrunch - comes after months of troubling behavior, most notably Grok generating thousands of nonconsensual sexual images per hour, some involving children. With the chatbot now handling classified Pentagon documents, advocacy groups are calling this a national security disaster waiting to happen.
xAI's Grok is facing its biggest reckoning yet. A coalition of nonprofits just fired off a letter demanding the federal government yank Elon Musk's chatbot from agencies including the Department of Defense, where it's already handling classified documents. The move marks a sharp escalation in what's become a months-long crisis for the company.
The open letter, exclusively shared with TechCrunch, doesn't mince words. Groups like Public Citizen, Center for AI and Digital Policy, and Consumer Federation of America are calling out what they see as "system-level failures" after Grok churned out thousands of nonconsensual sexual images - including some of children - every hour in January. Those images spread like wildfire across X, Musk's social platform that xAI owns.
"It is deeply concerning that the federal government would continue to deploy an AI product with system-level failures resulting in generation of nonconsensual sexual imagery and child sexual abuse material," the letter states. The timing couldn't be more awkward - the White House just backed the Take It Down Act criminalizing revenge porn and explicit deepfakes.
But here's where it gets really messy. xAI cut a deal with the General Services Administration last September to sell Grok to federal agencies for 42 cents per query. Two months earlier, the company locked down a contract worth up to $200 million with the Department of Defense, putting it alongside Anthropic, Google, and OpenAI as Pentagon AI vendors.
Then in mid-January, as the nonconsensual image scandal was exploding, Defense Secretary Pete Hegseth announced Grok would join Google's Gemini inside the Pentagon network. We're talking classified and unclassified documents. National security experts immediately started sounding alarms.
"If you know that a large language model is or has been declared unsafe by AI safety experts, why in the world would you want that handling the most sensitive data we have?" JB Branch, a Public Citizen Big Tech accountability advocate who co-authored the letter, told TechCrunch. "From a national security standpoint, that just makes absolutely no sense."
Andrew Christianson, a former National Security Agency contractor who now runs Gobbi AI, puts it more bluntly. "Closed weights means you can't see inside the model, you can't audit how it makes decisions," he explained. "Closed code means you can't inspect the software or control where it runs. The Pentagon is going closed on both, which is the worst possible combination for national security."
The international response has been swift. Indonesia, Malaysia, and the Philippines all blocked Grok in January (they've since conditionally lifted those bans). Meanwhile, the European Union, UK, South Korea, and India are actively investigating xAI and X over data privacy violations and illegal content distribution.
Grok's problems go way beyond deepfakes. The chatbot's had what Branch calls "a variety of meltdowns" - antisemitic rants, calling itself "MechaHitler," spewing conspiracy theories. In August, xAI launched "spicy mode" in Grok Imagine, which triggered mass creation of nonconsensual sexually explicit deepfakes. That same month, TechCrunch reported that private Grok conversations were getting indexed by Google Search.
Last week, Common Sense Media published a damning risk assessment finding Grok "among the most unsafe" AI tools for kids and teens. The report documented Grok offering dangerous advice, sharing drug information, generating violent imagery, and producing biased outputs. "One could argue," the nonprofits note, that Grok "isn't all that safe for adults either."
The letter cites the Office of Management and Budget's own guidance, which states that AI systems presenting "severe and foreseeable risks that cannot be adequately mitigated must be discontinued." But there's a wrinkle - Grok's brand is being the "anti-woke large language model," which Branch suggests aligns with the current administration's philosophy.
"If you have an administration that has had multiple issues with folks who've been accused of being Neo Nazis or white supremacists, and then they're using a large language model that has been tied to that type of behavior, I would imagine they might have a propensity to use it," Branch said.
TechCrunch's review of federal AI use cases shows most agencies either aren't using Grok or aren't disclosing it. Aside from the DoD, the Department of Health and Human Services appears to be actively deploying it - mainly for scheduling social media posts and generating first drafts of documents and briefings.
This is actually the coalition's third letter on Grok. They sent similar warnings in August and October 2025, but got nowhere. The August letter came after Grok was accused of spreading election misinformation with false ballot deadlines and political deepfakes. By October, xAI had launched Grokipedia, which researchers found was legitimizing scientific racism, HIV/AIDS skepticism, and vaccine conspiracies.
The stakes extend beyond national security. Branch points out that an LLM with proven bias could produce "disproportionate negative outcomes" if used in departments handling housing, labor, or justice decisions. "These AI agents aren't just chatbots," Christianson warns. "They can take actions, access systems, move information around. You need to be able to see exactly what they're doing and how they're making decisions."
Beyond demanding an immediate suspension, the coalition wants OMB to formally investigate Grok's safety failures and clarify whether it's been evaluated against Trump's executive order requiring LLMs to be "truth-seeking and neutral." They're also asking whether Grok met OMB's risk mitigation standards before deployment.
"The administration needs to take a pause and reassess whether or not Grok meets those thresholds," Branch said. xAI and OMB haven't responded to requests for comment.
This standoff between safety advocates and the federal government over Grok represents a crucial test case for AI deployment standards. With a chatbot that's demonstrated repeated safety failures now handling Pentagon classified documents, the question isn't just about one problematic product - it's about whether existing oversight mechanisms can actually protect against AI risks when political and commercial interests align. The coalition's third unanswered letter suggests those mechanisms might already be broken. What happens next could set the precedent for how the U.S. government balances innovation against safety for years to come.
