Do your agent evals test access being revoked halfway through a conversation?
A developer proposes a missing eval case for permissioned agents: revoke access mid-conversation and watch what the agent does. Most agents check permissions only once at the start, so they either crash loudly, the good failure, or keep acting on cached data, the terrifying one. A harder variant suggests revoking then re-granting with a narrower scope to see if the agent notices the difference.