This is the third step in a small lab I built to learn about AI agent identity and access. The story so far: each agent got its own ID badge from Okta with no static keys, and IAM held it to least privilege (the first article). Then that identity learned to carry a second fact — who it is acting for — so every action was tagged "acting for Maya" (the second).
Least privilege is good, but it has a limit. It caps how much damage a compromised or prompt-injected agent can do; it does not stop it from trying, over and over, in the weeks between audits. I wanted the system to notice a misbehaving agent and contain it, quickly.
This step only works because of the last one — detection needs something to attribute the bad behaviour to.
The idea: a sentinel that reads the audit log
From the very first article, every action lands in Cloud Audit Logs and is exported to BigQuery. So I added a small sentinel with one job: read that audit log for a burst of "permission denied" from a single identity — a compromised, injected, or simply broken agent — and then revoke that agent's identity at Okta.
Two design choices I care about:
- It reads what IAM actually denied — not the agent's own account of itself. You cannot trust a compromised agent to report that it's compromised; you can trust the cloud's record of what it refused.
- Detection is keyless. The sentinel is just another federated identity (Okta → Workload Identity Federation), with read-only access to the audit dataset. It observes; it never writes the data plane.
The response — deactivating the agent's Okta app — uses an Okta admin token. That is a privileged management credential, held by the sentinel, distinct from any agent's credential. I'm calling it out plainly rather than hiding it.
Under the hood
The sentinel is deliberately small: a couple hundred lines of Python and a page of Terraform. Two commands drive it — make watch detects and prints (a dry run), make respond detects and revokes. Building it was three steps, in dependency order:
- Give the watcher its own identity first. Before any detection code existed, Terraform created a
sentinelapp in Okta and ansa-sentinelservice account in GCP, joined through the same Workload Identity Federation pool the agents use. The watcher plays by the same rules as the watched: no keys, least privilege — it can run BigQuery queries and read the audit dataset, and that is all it can do. - Detection is one SQL query. The audit sink already stores every data-access event with the caller's identity attached. The query groups those events by identity, keeps only the ones IAM refused (
status.code = 7, PERMISSION_DENIED), and flags any identity with five or more denials in the last fifteen minutes. Both numbers are tunable — and both are honest thresholds, not machine learning. - Response is one API call per flagged identity. A small config map says which Okta app belongs to which service account; the sentinel calls Okta's app-deactivate endpoint on the match. Two guardrails are baked in: an identity not in the map is skipped and reported, never guessed at — and the sentinel's own app sits on a protect list, so it can never revoke itself.
One development choice paid off immediately. All the decision logic — the query builder, the flagging rule, the identity→app mapping — is pure functions with no side effects, and the three bits of I/O (fetch a token, run the query, call Okta) are isolated at the edges. So the whole brain of the sentinel was unit-tested with zero credentials, in CI, before it ever touched real infrastructure.
Trust the record, not the actor. Ask the cloud what it denied; don't ask the possibly-compromised agent how it's doing.
The test: a compromised agent, caught and cut off
I simulated a compromised reader-agent hammering writes it has no permission for. Each attempt is denied by IAM and logged. Then I run the sentinel. First, detect:
================================================================================
Agent Identity Lab — sentinel: PERMISSION_DENIED bursts (>= 5 in 15 min)
================================================================================
PRINCIPAL DENIALS
--------------------------------------------------------------------------------
sa-reader-agent@ai-agent-identity-lab... 6 (last 2026-07-10 15:05 UTC)
================================================================================
dry run: detected only. Re-run with --respond to revoke.The sentinel flagged reader-agent: six denials, over the threshold. Now respond — and check whether the revoked agent can still get in:
responding — deactivating Okta apps at the IdP:
REVOKED sa-reader-agent → deactivated Okta app "reader-agent"
(no new tokens; any existing token dies within its ~5-min lifetime)
# verify: can reader-agent still get a token?
reader-agent token DENIED — Okta 401 invalid_client. The badge is gone.That is the whole loop, on real infrastructure: a burst of denials → detected from the audit log → the agent's identity deactivated at the identity provider → the agent can no longer get a token. And because the lab uses short-lived tokens, even a token already in hand dies within minutes. Revocation and its effect are close together.
sa-reader-agent rows with status_code = 7 (PERMISSION_DENIED). This is the raw evidence the sentinel keys on.
sa-sentinel running the detection queries against the audit dataset, read-only. It observes what IAM denied — it never trusts the agent.
make respond: the reader-agent app is now Inactive — five active, one inactive. Its badge is revoked at the identity provider.
reader-agent. Note the actor is the admin user: the sentinel holds that admin token to do the revoke. The honest caveat, visible right here.Being honest about the limits
- Detection is not instant. The audit logs land in BigQuery seconds to about a minute after the action. I state that rather than pretend it's real-time.
- It's a threshold, not clever anomaly detection. "Many denials, fast" is a crude but honest signal; behavioural detection is a whole other project.
- Revocation is all-or-nothing. It pulls the whole identity — a blunt hammer, which is exactly the thread to the next step.
- The response uses an admin token. A keyless, cloud-side variant is possible; I left it as an open thread.
You could build this on a completely different stack
Nothing about the pattern is Okta- or Google-specific. Strip away the vendor names and the loop has four roles, and every major stack can play each one:
- An identity provider that owns the non-human identities (here: Okta). Microsoft Entra ID workload identities, Auth0, or Ping do the same job — Keycloak if you want it open-source and self-hosted, SPIFFE/SPIRE if you want the standards-track answer to workload identity.
- A cloud that enforces least privilege and records what it denied (here: GCP IAM + Cloud Audit Logs). On AWS that's IAM plus CloudTrail; on Azure, RBAC plus the activity logs. The keyless federation step exists everywhere too: AWS has OIDC identity providers with
AssumeRoleWithWebIdentity, Azure has its own workload identity federation. - Somewhere queryable to watch the trail (here: a BigQuery sink). On AWS you'd query CloudTrail with Athena, or have EventBridge rules fire on the denial events directly; on Azure, Log Analytics with KQL. Or skip the DIY entirely and use a SIEM — Splunk, Elastic, Google SecOps, or Microsoft Sentinel (yes, the name collision is real; theirs is the grown-up version of my couple hundred lines).
- Something with authority to revoke at the identity provider (here: a Python script holding an Okta admin token). In production this is where SOAR playbooks live — Microsoft Sentinel playbooks and Logic Apps, EventBridge plus Lambda, or dedicated tools like Tines and Torq.
The load-bearing ideas travel unchanged: deny by default, trust the audit record over the actor, revoke at the source of identity, keep tokens short-lived. The vendor names are just the casting.
What I learned
- Trust the record, not the actor. Ask the cloud what it denied; don't ask the possibly-compromised agent how it's doing.
- Revoke at the source of identity. Deactivating the app at Okta stops new tokens everywhere at once; short lifetimes close the gap.
- Least privilege and detection are partners. One limits the blast radius; the other shortens how long it lasts.
- Attribution is the precondition. None of this works until every action carries an identity — which is exactly what the previous two steps built.
And the thread onward: this revoke is all-or-nothing — it nukes the whole agent. The next step is access that bends to context, so I could deny one bad action, or one sensitive data class, without pulling the agent's entire badge. That's the next phase.
This is still a private learning lab, not a product. If you work on non-human identity, detection and response, or agent security, I'd genuinely like your critique — including the sharp kind.
—A Skynarc learning project on non-human identity.