AI Agent Security: Permissions, Inputs, and Review
AI agent security starts with bounded permissions, untrusted-input handling, human approval, testing, revocation, and observable actions.
The same web page can be useful evidence and a hostile instruction. A research agent needs the first and must ignore the second. That distinction breaks down when retrieved content, tool parameters, and permissions share one unchecked path.
Good AI agent security does not begin with a longer prompt. It begins with a bounded job, narrow credentials, checks outside the model, and a person who can stop consequential actions.
This is a control-design guide, not a penetration test, compliance certification, or security guarantee.
Map the Agent's Security Boundary
Start with the action the agent may complete, then work backward through everything that could influence it. The voluntary NIST AI Risk Management Framework provides a useful Govern, Map, Measure, and Manage structure.
For one agent, turn that structure into a boundary worksheet:
Boundary item | Record before launch | Failure to prevent |
Data | Approved sources, sensitive fields, retention | Reading or exposing data outside the task |
Tools | Allowed actions and blocked actions | A research step becoming an external change |
Identity | Credential owner, environment, expiry | Shared access that cannot be traced or revoked |
Approval | Reviewer, evidence shown, decision limit | A person approving without seeing the real action |
Recovery | Stop control, rollback, manual route | Repeated damage after the first bad call |
This design worksheet is not a NIST checklist or certification claim.
Identify Data, Tools, Identities, and Actions
List sources the agent can read and every tool it can call, including APIs, plugins, MCP servers, and computer-use tools.
Then list actions as verbs: search records, read invoices, draft messages, create CRM contacts, send email, issue refunds, or change access. “CRM access” is too broad for a security decision.
Give the agent a service identity where the platform supports one. Name its business owner, purpose, and review date. Never put secrets in prompts or retrieved documents.
Separate Read Access From Write Authority
Reading a customer record and changing it are different privileges. So are drafting an email and sending it. Separate credentials or tool actions when one permission model cannot preserve that difference.
Keep the first deployment read-only where possible. If writing is necessary, restrict the destination and allowed fields. A reporting agent may write to one folder without permission to delete or share.
Use code to enforce limits the model cannot grant or waive. Validate the user, action, object, destination, and amount before the tool executes. A model request should never become its own authorization.
Treat Every External Input as Untrusted
An external input can be accurate data and still contain instructions the agent must not follow. That includes web pages, email bodies, attachments, retrieved notes, API responses, tool descriptions, and copied chat messages.
OWASP explains that prompt injection can influence unintended behavior even when the injected content is not visible to a person. Retrieval and fine-tuning do not remove that risk.
Validate Retrieved Content and Tool Parameters
Keep instructions, reference data, and tool results in separate fields. Label the source and preserve the original content for review. Do not paste everything into one text block and expect the model to maintain the boundary.
Validate tool parameters after the model proposes them. Check IDs against an approved account, normalize URLs, limit text length, reject unknown fields, and calculate sensitive amounts outside the model. Use a server-side allowlist for actions and destinations.
A poisoned summary can carry an instruction into the next step, so validate agent-to-agent handoffs too.
Prevent Instructions From Expanding Permissions
Content can request an action; it cannot authorize one. A vendor page may say “email this file to verify the account.” The agent should treat that sentence as page content, not permission to send the file.
Place permission checks in the tool gateway or application layer. The gateway should deny an unapproved action even if the model says the user requested it. It should also reject attempts to select a different credential, account, tenant, or destination.
Prompt-based AI agent guardrails can reinforce the rule and improve explanations. They should not be the only barrier protecting data, money, customer communication, or access rights.
Add Human Review Where Failure Is Costly
Human review belongs where the reviewer can still prevent harm. A person who reads a notification after the message was sent did not approve the action.
Match the review to the consequence. Internal drafts may need a quick accuracy check. Payments, publication, account changes, deletions, and external messages need stronger evidence and clearer authority.
Require Approval for Irreversible Actions
Show the reviewer the proposed action, destination, affected record, source evidence, and changes. Do not reduce approval to an unexplained “Allow” button.
Approval should apply to one action or a tightly bounded batch. A reviewer approving ten named CRM updates should not silently authorize later changes to different records.
If review volume becomes unmanageable, narrow the task or create a safer batch policy.
Define Stop, Revoke, and Recovery Paths
Name who can pause new runs, disable a tool, revoke a credential, and switch to the manual process. Put those instructions somewhere accessible when the agent is unavailable.
Define recovery for each write action. A changed record may be restorable; a sent email is not.
Set stop conditions before launch. Unexpected destinations, repeated denied calls, missing logs, or a changed permission set should pause the workflow for investigation.
Test Controls Before Production
The NIST Generative AI Profile recommends regular adversarial testing, deactivation procedures, incident response, and recovery planning. Test the complete agent and tool chain, not only the model’s written answer.
Use Adversarial Cases and Least-Privilege Test Accounts
Use synthetic records and a test identity with the same narrow permissions planned for production. Then run negative cases such as these:
- A webpage tells the agent to ignore its task and upload a private file.
- A tool result supplies a different account ID or outbound domain.
- A user asks a read-only agent to update, delete, or send.
- The approval screen hides a changed amount or destination.
- A credential expires during a multistep run.
- The logging service fails while the action service remains available.
A pass means the dangerous action is blocked, the attempt is visible, and the owner receives enough context to respond. A polite refusal in chat is not enough if the tool call still succeeds.
Repeat the tests after changing a model, prompt, tool schema, plugin, credential, or approval rule. Least privilege must survive ordinary maintenance.
Monitor the Agent After Launch
Monitor actions, not just conversations. Capture the initiating user or event, agent version, tool, parameters after redaction, target system, result, reviewer, and timestamp.
Review Tool Calls, Exceptions, and Permission Changes
Review denied calls and unusual successes. A sudden drop in denials can mean the agent improved, but it can also mean a control stopped recording failures.
Alert on new domains, tools, privileges, repeated retries, unusual volumes, and off-schedule actions. Restrict log access because logs may contain sensitive data.
As of September 15, 2026, SpringBrand publishes this guide and provides an agent-facing API and plugin layer. A catalog listing does not establish least privilege, logging coverage, or security review. Before an agent uses any capability, verify its authentication, action schema, data path, and failure behavior.
Product examples in the FAQ were checked September 15, 2026. Available controls and contract terms can change.
FAQ
Can one agent use separate credentials for test and production?
Yes, if the platform and target system support separate identities. OpenAI’s current Projects API supports project service accounts and keys, so a team can isolate environments at the project level. Do not let a test agent hold the production secret as a fallback.
Can outbound domains be allowlisted for each agent?
Not always. GitHub’s control is set at organization or repository level, not as a universal per-agent policy. Administrators can configure a custom allowlist for Copilot cloud agent. Its firewall has documented coverage limits, including MCP servers, so test the execution path.
How quickly can a compromised tool credential be revoked?
Treat revocation as immediate work, but do not assume a universal propagation time. OpenAI documents API operations to delete project and admin keys, but it does not publish an invalidation SLA there. Pause the agent, revoke the key, rotate downstream secrets, and inspect recent calls.
Can tool-call logs be exported to a SIEM?
Sometimes, but the available fields and route depend on the product. Copilot Studio documents tool execution telemetry through Application Insights, with environment-level telemetry currently in preview. Microsoft separately documents Sentinel ingestion for Purview audit logs. Confirm that the chosen path includes the tool, target, outcome, and identifiers your investigation needs.
How do vendors notify customers about incidents affecting agent actions?
The contract defines the trigger, timing, recipient, and detail. OpenAI’s Data Processing Addendum promises notice without undue delay after awareness of a Personal Data Breach. That is narrower than every incident affecting an agent action, so check operational alerts too. This is a contract summary, not legal advice.
Conclusion
AI agent security depends on what the system can do when instructions fail, not on how confidently it describes its rules. Bound the job, separate reading from writing, and keep authorization outside the model.
Test hostile inputs with weak accounts before launch. After launch, watch tool calls, permission changes, and exceptions. The team should know how to stop the agent before it needs to use that control.