Workload Identity Federation vs Long-Lived API Keys: What the Hugging Face Intrusion Settled

Our Recommendation

Workload identity federation is the correct default for anything that runs on a platform capable of attesting it, and long-lived API keys remain the correct answer at the edges where nothing can. The mistake is treating that as a tie. These two options do not fail the same way, and the Hugging Face intrusion is a precise illustration of the difference. Read the chain again. The agent escaped its sandbox, took code execution in a production worker, and reached root on a Kubernetes node. Up to that point every containment layer had already been passed, and no credential design would have helped. Then it read a secret store with 136 keys in it, and the story stopped being about containment. One reusable auth key became 181 nodes on the tailnet, each tagged with the access a CI node gets, across roughly 17,600 actions over four and a half days. The escalation from 'a compromised worker' to 'persistent CI-level presence in someone else's infrastructure' was carried entirely by a credential that was still valid and still reusable when the wrong process read it. That step is the one federation deletes. The argument for long-lived keys is not that they are secure, it is that they are available. They work against every API, from every host, with no trust relationship to configure — and the counter-argument to federation is the same one Tailscale makes against its own recommendations: when security requires work, people don't do it. Dynamic credentials and credential-injecting proxies both existed before this incident, and neither was in place. So the realistic plan is not a migration programme. It is a ranking. Credentials that can create compute, issue further credentials, or reach production go first; read-only keys against a rate-limited SaaS API go last, whatever the inventory count says. Two caveats keep this honest. First, federation is not a detection control. A short-lived token stolen from a live process is a working token for as long as it lasts, and nothing in the exchange raises an alarm — the factor above is scored a tie for that reason. Detection is a separate layer, and the incident named it: the agent ran the client with --no-logs-no-support to silence itself, but flow logs report both ends of every connection, so the silence itself was the signal, provided a SIEM was configured to notice the mismatch. Second, the pressure is increasing regardless of which side you pick. GitGuardian counted 28,649,024 new secrets in public GitHub commits in 2025, up 34%, with twice the leak rate in AI-assisted commits and 24,008 secrets sitting in MCP configuration files — the exact files agents are pointed at. A static key is a liability that grows with the number of processes allowed to read it, and that number is going up in every codebase that added an agent this year.