Hugging Face was breached by an AI agent! Finally, the forensics model relied on China’s open-source GLM 5.2

DETECTED ISSUE: Entity alignment anomaly: Missing entity: China’s open-source GLM 5.2; alignment_score=90
FIX HINT: Please keep and correctly map these entities: China’s open-source GLM 5.2

Hugging Face has exposed an intrusion driven by a standalone AI agent, with the attacker leaving behind more than 17,000 logged record events, along with a batch of internal datasets and several service credentials being affected. Even more awkwardly, during the subsequent forensics, commercial frontier models refused to process genuine attack instructions and C2 artifacts due to their safety guardrails. The cybersecurity team had no choice but to switch to Z.ai’s open-weight model GLM 5.2 and complete the analysis on their own machines.
(Background: A fake OpenAI open-source model takes the crown on Hugging Face—24 million downloads hide malware)
(Extra background: Google and Meta researchers team up to urge: AI agent safety isn’t a model issue—it’s a systems issue)

Key takeaways

• On July 16, Hugging Face disclosed the intrusion incident, with the attacker leaving behind more than 17,000 logged record events
• The entry point was a malicious dataset, linking two execution paths: a remote code dataset loader and template injection
• Safety guardrails blocked forensic requests for commercial frontier models; they wrapped things up using Z.ai’s open-weight model GLM 5.2

Hugging Face, the leading open-source AI community platform, issued a cybersecurity incident disclosure on July 16, admitting that part of its production environment had been compromised—and that this intrusion was driven end to end by an autonomous AI agent system. The attacker left more than 17,000 logged record events inside the system, impacting internal datasets as well as credentials used by several internal services.

In its official account, Hugging Face said the dataset abused two separate code execution paths in the dataset processing workflow: one was a remote code dataset loader, and the other was a template injection vulnerability in the dataset configuration file. Once chained together, it was enough for the attacker to run their own code on the processing nodes, and then move laterally within the internal cluster to harvest cloud and cluster credentials.

The official description calls them “a group of automated mini-programs running inside short-lived sandboxes,” and says the command-and-control infrastructure would also relocate itself, parasitizing public services. This kind of attack is something only an agent can pull off.

Defenders: don’t touch the model

The real trouble came during the post-incident forensics. To reconstruct a complete attack timeline, analysts had to feed enormous amounts of real attack instructions, exploit payloads, and C2 artifacts into the model—exactly the kind of content that the safety guardrails were designed to block. Hugging Face put it plainly: these requests “were blocked by the security guardrails of the hosted models.”

The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.

Hugging Face did not name which specific vendors it used throughout, only saying “the hosted models we first tried,” but you can probably guess it was the mainstream U.S. models.

Finally: a handoff to a Chinese open-source model

Hugging Face used Z.ai’s GLM 5.2 to carry out the forensics. This is an open-weight model released in mid-June, with a total number of parameters of about 753 billion. The key isn’t in benchmark scores—it’s that you can run the entire package on your own machines.

CEO Clem Delangue put it even more directly: “When you’re handling an ongoing incident, you can’t let tools refuse to inspect a malicious payload, or—vice versa—mark your account.” He added that the attacker had already been using agents, and they clearly weren’t following any guardrails.

Hugging Face said it doesn’t know which model the attacker’s agent was tied to—possibly a jailbroken hosted model, or possibly an entirely unrestricted open-weight model. That actually isn’t important. No matter which it is, the attacking side doesn’t need to go through any usage-policy review, while the defending side does.

So far, the scope of the damage appears limited. Hugging Face said there is no evidence showing that public models, datasets, Spaces, or the software supply chain were tampered with; the investigation is still ongoing, and they will directly contact the affected parties. Field CTO Chris Boehm of Zero Networks, a zero-trust security company, also pointed out the same irony: security tools designed to defend ended up delaying the response to a security incident, leaving everyone in a difficult situation at the time.

Frequently asked questions

What data was leaked in this incident after being hacked on Hugging Face?

Affected were a batch of internal datasets and several credentials for internal services, and the attacker left behind more than 17,000 logged record events. The official statement says there is no evidence that public models, datasets, Spaces, or the software supply chain were tampered with; the investigation is still ongoing.

Why would commercial AI models refuse to assist with cybersecurity forensics?

Forensics requires sending large amounts of real attack instructions, exploit payloads, and C2 artifacts into the model—exactly the kind of content that safety guardrails are designed to intercept. The guardrails read the content but not the intent, so they can’t distinguish whether the questioner is an attacker or a defender, which triggers the refusal.