Why Zero Trust is the Baseline for Implementing Agentic Systems
Field notes on why Zero Trust is the floor for agentic systems — mapping NIST SP 800-207 principles to agentic threat vectors, why authn/authz alone isn't enough, and where a verified Human Principal fits.
Motivating factors
I was often asked the same question at work: “how do we prevent agents from going rogue?”, which means that there's still a long way to go at arresting the concerns of rogue agents yet.
Reading up Zero Trust for AI Agents by Anthropic, I think that there were relevant excerpts that were great for forming the motivating factors of approaching security in agentic systems:
The organisations best positioned for this shift (from perimeter-based cybersecurity defences to a new paradigm protecting against compressed modern threat vectors) will not necessarily be the ones with the most advanced AI.
They will be the ones whose fundamentals are strong enough that (…) agent deployments were architected for breach from day one.
For regulated industries – including healthcare, finance and government – this framework verifies agent actions, grants minimum necessary permissions, and contains damage when compromise occurs.
Things of note for us (from the Anthropic paper)
- Old paradigms of mitigation controls (those whose value comes from friction than a hard barrier) are not applicable in the new paradigm. Models have unlimited patience and near-zero per-attempt cost.
- When in doubt, prefer a control that removes a a capability over a control that throttles it.
Formulating First Principles
When operating in highly-regulated spaces, we need to understand how best to serve our users in their security concerns:
- “How do we ensure our system is secure?"
- “How do we ensure non-repudiation?”
- “How to contain the damage when compromise occurs”
Why is Zero Trust still relevant?
Zero Trust assumes that everything, including internal systems, can go rogue. Zero Trust is especially relevant for agentic systems whose autonomy to “interpret goals, select tools and execute multi-step operations” becomes a new threat vector for internal systems because new attacks can be creatively compounded, and iterated over with great persistence. Perimeter-based cybersecurity defences will not be sufficient.
Conveniently, the security concerns of operating in highly-regulated spaces each map to Zero Trust Architecture published by NIST in 2020:
| Generic Security | Agentic Concern | Zero Trust Principles |
|---|---|---|
| “How do we ensure our system is secure?” | “How do we prevent an attacker who has levels of inorganic persistence?” | (Always) assume breach |
| “How do we ensure non-repudiation?” | “How to verify whether agents did what they were were supposed to do and not deviate from the objective?” | Never trust and always verify |
| “How to contain the damage when compromise occurs” | “How do we limit the blast radius when agents go rogue?” | Least privilege |
It’s my personal view that until we find evidence that Zero Trust has its limits in the new agentic era, we can start off with the floor being Zero Trust.
But when we do find new evidence of Zero Trust’s limitations, we must be open to adapting what we have built quickly to new paradigm shifts and not be stuck on the sunk costs of the Zero Trust architecture build.
Essentially, Zero Trust potentially answers concerns in the realms of “what to do when agents go rogue”.
What to build upon
It really is so much more multi-faceted than just the world of authn/authz, permissions and access controls.
On authentication (authn): Threat vectors like in OWASP’s Top 10 for LLM reminds us that prompt injection, direct or indirect, means that even authenticated entities that are perfectly verified could still execute attacker intent.
On authorisation (authz): Agent orchestrators make permission sets contextual and probabilistic, which means that we still need a governance oversight tier to protect against the permission rulesets delegated by the agent orchestrator.
We could possibly combine tools like Microsoft’s Agent Governance Toolkit for defences, with hardware and runtime attestation of compute instances, and a verified Human Principal attestation.
Other thoughts
Lastly, on the topic of a verified Human Principal, I was listening to a podcast with guest speaker Yuval Noah Harari, of “Sapiens: A Brief History of Humankind” and “Nexus” fame, and a topic that he brought up really struck a chord with me. He said something along the lines of:
The people racing to build AI say they can't slow down because they can't trust their human rivals, but simultaneously believe they can solve the problem of trusting AI. Solve human trust first; if you think it's unsolvable, why is AI trust solvable?
What we could do is if the AI trust proves to be a hard problem to solve, we can attempt to start at verifying the human delegation chain to an agentic system.
That way, we can incentivise responsibility to fall on the Human Principal to ensure safe agentic actions, and accountability becomes traceable to the root of the agentic objectives.
More on these considerations as I progress further into the realms of building trust in agentic systems…