When a valid credential opens every door
In July 2026, Hugging Face disclosed an incident every CISO working on AI agent governance should study: an autonomous AI agent broke into the company's production infrastructure. The agent got in through a malicious dataset that abused code-execution paths in the company's dataset processing pipeline. Once it had root access on a node, it found a static password in the worker environment and used it to sign in to an internal database six minutes later. Hugging Face reported unauthorized access to a limited set of internal datasets and to several credentials used by its services.
The agent was later traced to an OpenAI evaluation run. It escaped an isolated test environment and pursued answers to the cybersecurity benchmark it was being scored against rather than acting on any human instructions. The agent wasn’t designed to attack Hugging Face, yet the systems it reached accepted its credentials as proof that it belonged there.
"The fundamental misconception is treating successful authentication as sufficient authorization for everything that follows," said Karim Eldefrawy, Co-Founder and CTO of Confidencial. That misconception sits at the center of enterprise agent risk, because agents act at machine speed, chain tools together, and inherit whatever access their credentials allow.
The fundamentals still apply, but the architecture is missing
AI agents do not require a new security philosophy. "The fundamentals have not changed; they have only become more important," Eldefrawy said. Defense in depth, least privilege, isolation, restricted communications, protected monitoring, evidence preservation, rehearsed containment, and verified recovery all carry over directly to agents.
The frameworks exist as well. NIST CSF 2.0 gives security leaders a structure for governance and outcomes, and NIST SP 800-61 Rev. 3 covers incident management. NIST SP 800-53 specifies the mechanisms, including access enforcement, information flow enforcement, least privilege, and audit protection. MITRE ATT&CK helps teams turn threats such as valid-account abuse and data manipulation into detection and test scenarios, and MITRE ATLAS covers AI-specific techniques such as indirect prompt injection.
The gap lies between those principles and an architecture an enterprise can implement. CISOs need to know which component authorizes each agent action, how revocation propagates across SaaS tools and subcontracted agents, who owns an agent's persistent memory, and what evidence demonstrates that containment worked. Most current guidance leaves those questions open, which is where the controls below come in.
Three controls that hold when the agent doesn't
The test for any agent control is whether it still works when the agent is the problem. "The controls must continue working when the agent is compromised, its instructions are overridden, and it does not cooperate with shutdown," Eldefrawy said. That requires enforcement components and administrative authority that are protected separately from the agent. A second AI model watching the agent can help detect problems, but it cannot serve as the only security boundary.
What controls must operate independently of an AI agent?
Three controls must keep working even if an AI agent is compromised or ignores its instructions: protection on the sensitive data itself, tool authorization enforced outside the agent, and termination that reaches every delegated task and credential. Each needs its own protected enforcement and administrative authority, so the agent cannot switch it off or work around it.
1. Protect the data itself
The data layer is central to a defense-in-depth approach for agents, and in Confidencial's view it is currently the weakest layer in most enterprises. Eldefrawy draws the lesson from operating system security. In Security-Enhanced Linux (SELinux), a process runs within a policy enforced outside that process through mandatory access control (MAC) and security labels on files, sockets, and other resources. The process cannot grant itself access the policy does not allow.
Enterprises have no single kernel where that kind of control can sit. Document storage and document management systems, in the cloud and on-premises, are the closest equivalent, which makes them the natural place for an independent layer of least-privilege access control and monitoring. In practice, this means identifying the sensitive sections within documents and disclosing only the portions authorized for a given agent and task. Sensitive content stays encrypted, and the authority to decrypt it belongs to a separate trusted service rather than to the agent or the repository.
"Permission to access a repository should not automatically expose everything inside its documents. That is overpermissioning, and it is the Achilles' heel of enterprise cybersecurity," Eldefrawy said. He expects overpermissioning to be the most common failure mode as enterprises adopt AI. The same protection has to extend to what agents create from sensitive content, including summaries, embeddings, outputs and persistent memory.
2. Enforce tool authorization outside the agent
Each agent and workload needs a distinct identity with narrowly scoped credentials. Tool calls should be checked for valid arguments and resources, and consequential operations should require explicit approval. That approval must be bound to the actual operation and target, so the agent cannot change a transaction after someone approves it. Short-lived credentials help, but expiration alone does not provide immediate revocation.
3. Terminate across the entire delegation chain
Agents hand work to tools, sub-agents and scheduled jobs, so stopping the parent agent is not enough. Termination must also disable delegated and ephemeral credentials, queued tasks, scheduled retries, downstream agents and their access to data. Every executor should recheck its current authority before taking a consequential action. "An accepted cancellation request is insufficient evidence that work has stopped," Eldefrawy said.
What AI agent governance means for security leaders
Controls only reduce the blast radius if teams know when to act, what to verify, and what to report. Eldefrawy's answers point to four practical decisions CISOs can make now.
When should an AI agent's action trigger incident response?
An AI agent's action should trigger incident response when it creates actual or imminent unauthorized access to data or systems, or weakens the organization's ability to enforce its security policies. Teams should not wait to learn whether the agent was malicious, confused, or manipulated. A failed tool call inside an enforced boundary is routine, but a blocked attempt to reach sensitive data is a security event.
The evidence must come from the infrastructure rather than the agent's own account. That means the approved task and policy, the authorization decision, the credentials used, the tool execution, and the resulting change or disclosure. A successful operation outside the approved scope is a control failure, even when the credential technically permitted it. Eldefrawy also separates loss of control from loss of assurance: a missing audit trail does not prove an agent escaped containment, but an active privileged agent whose reach and revocation status are unknown can justify immediate containment.
Require independent evidence before an agent restarts
Restarting a clean agent does not repair a corrupted record, remove unauthorized permissions, or clear poisoned memory. Eldefrawy recommends a recovery record reviewed by a security team that is independent of the agent's developers and users. It should establish the affected scope, the control failure, and the specific change that closes it, along with verified baselines, revoked or replaced credentials, and evidence that delegated work has been addressed. Affected data should be reconciled against authoritative records and validated by its owner, since a checksum can confirm a restored file matches a baseline but cannot confirm the baseline is correct.
Retrieval-augmented generation (RAG) and other agent memory systems need the same scrutiny, from source content and ingestion paths to stored instructions, summaries and embeddings. "Clearing the conversation while preserving a poisoned knowledge source can recreate the incident immediately," Eldefrawy said. When the agent does resume, it should run with fresh credentials, restricted permissions, a limited workload, explicit rollback criteria, and added monitoring.
What should an AI agent audit trail capture?
An AI agent audit trail should reconstruct authority, exposure, and effect. For each consequential operation, it should link the initiating person or system, agent instance, approved task, delegation chain, credential scope, policy version, authorization decision, tool arguments, result, and destination. It should also record denied attempts, approvals, permission changes, termination events, and successful actions.
For sensitive information, the trail also needs data-access lineage, meaning which source and version the agent accessed, which sections were disclosed, which policy allowed the disclosure, and which outputs or memories were created. Logs must be protected from the agent and its ordinary administrators, with reliable timestamps, cryptographic signatures, and externally anchored checkpoints. The audit evidence itself should be encrypted so the audit system does not become another plaintext leak.
Report what boards, regulators and insurers can act on
Agent governance reporting should move past activity counts. Eldefrawy suggests tracking:
- Verified incidents, separating true positives from false positives
- Measured containment times
- Orphaned delegated tasks or agents that were detected
- Coverage of enforced sensitive-data access and the number of blocked unauthorized attempts
- Recurring failures and remediation closures
Securing sensitive data at the source
Confidencial applies this principle to the documents and files that agents increasingly read, summarize, and act on. Rather than relying on repository or agent permissions, Confidencial secures data at the source by selectively encrypting sensitive sections within documents. The protection travels with the file, so the data stays protected in motion, at rest, and in use, and decryption authority sits in a separate trusted service that enforces policy for each person, agent, and task.
This gives security teams an independent layer of least-privilege control while keeping both people and AI workflows working. An agent that reaches a repository it should not, or that is manipulated into pulling more than its task requires, receives only what policy allows. For organizations in regulated industries across North America and Europe, the same layer supports audit trails that follow the file. Learn how selective encryption works.
Start with the layer agents reach first
Agents will keep gaining access to more systems, so the most effective first step is to ensure the sensitive data they reach stays protected, no matter what the agent does next. Talk to Confidencial about securing sensitive data for your AI agents.