AI agent security risks and the controls that address them

Why tools change the threat model

A chatbot that says something wrong produces a bad sentence. An agent that gets something wrong can produce a bad transaction, because it holds credentials and calls real systems. The security question therefore moves from "what might the model say?" to "what could this process do, with these permissions, if its judgement failed?"

Manipulation is only one way for that judgement to fail. The guide to LLM guardrails and prompt injection defence covers how injection works, how to treat output and where classifiers fit, so I will not repeat it here. This guide takes the next step: assume the model will sometimes request the wrong thing, whether through an attack, a misreading or a bug, and look at the infrastructure around it.

The OWASP Top 10 for LLM Applications names this class of problem Excessive Agency (LLM06:2025) and gives three root causes: too much functionality, too many permissions and too much autonomy. That is a helpful starting frame, and the risks below are largely specific forms of it.

The risks at a glance

Over-broad identity

The most common design shortcut is one service account shared by every run. It is convenient, and it turns the agent into a confused deputy: a privileged program that can be led into using its authority on behalf of someone who does not have that authority. If a junior employee can ask the agent about any department's records because the service account can read them all, you have an access control hole, whether or not anyone attacks it.

Three practices narrow this considerably:

• Act as the user where you can. Delegated credentials, such as OAuth tokens with limited scopes, mean the downstream system applies its existing permissions.
• Issue credentials per run. A token that expires soon after the task ends is of much less value if it leaks than a long-lived key.
• Separate read and write. An agent that mostly reads should not carry write access for the occasional update. Give the write path its own credential, requested only when needed.

Here is the shape of this, with placeholder helpers. It is illustrative rather than a complete implementation:

# Illustrative only. issue_token and crm_client are placeholders.
def get_crm_client(run, action):
    scope = "crm.write" if action in WRITE_ACTIONS else "crm.read"
    token = issue_token(
        subject=run.user_id,   # from your auth layer, not from the model
        scope=scope,
        ttl_seconds=300,
    )
    return crm_client(token)

The five-minute lifetime here is an arbitrary example, so choose one that matches your longest legitimate step.

Secrets in the wrong place

Anything placed in a prompt can be repeated by the model, stored in a trace and sent to your model provider. So the model should never need to see an API key. Your tool code holds the secret and the model refers only to the tool's name.

The less obvious leak is through results. A tool that returns a whole configuration object, a raw HTTP error with its headers or a stack trace containing a connection string puts that material into the conversation, and from there into your logs. Return only the fields the task needs, and scrub tool results before they enter the context. Apply the same scrubbing to your tracing pipeline, because LLM observability often captures full prompts and responses, which makes the trace store a sensitive system in its own right. It needs access controls and a retention period.

Open network access

An agent with a general HTTP tool can send data anywhere and can also reach internal addresses your firewall assumes are private, including cloud metadata endpoints. This is server-side request forgery with a language model choosing the URL.

Prefer named tools for specific services over a general fetch tool. Where general access is really needed, enforce an allowlist in code and, ideally, again at a network proxy, since an application-level check can be bypassed by anything else running in the same environment.

# Illustrative only. A real check must also cover redirects and DNS.
from urllib.parse import urlparse

ALLOWED_HOSTS = {"api.example.com", "docs.example.com"}

def check_url(url):
    parts = urlparse(url)
    if parts.scheme != "https" or parts.hostname not in ALLOWED_HOSTS:
        raise PermissionError("Destination not allowed")
    return url

As the comment notes, this sketch alone is not enough. An allowed host that redirects elsewhere defeats it unless each redirect is checked as well.

Generated code and shell access

If an agent runs code, treat that code as coming from a stranger. Restricting Python's built-ins or filtering for dangerous strings is not a sandbox, because there are too many routes around such checks. Isolation has to come from the operating system or from a separate machine:

• Run each job in a disposable container or virtual machine, destroyed afterwards. Note that ordinary containers share the host's kernel, so higher-risk workloads may call for stronger isolation.
• Run as a non-root user with no credentials mounted.
• Default to no network, and open only the destinations a job requires.
• Set limits on CPU, memory, disk and run time.
• Pass files in and out through a defined path, and treat whatever comes out as untrusted.

Third-party tools and servers

Each tool you connect is code you are trusting, and its description is text your model will read. A protocol such as the one described in Model Context Protocol for AI agents makes connecting tools easy, which makes it important to ask who wrote them. The risks are familiar from software supply chains, with one addition:

• Malicious or careless packages. Review the source and permissions as you would for any dependency.
• Silent changes. A tool that was harmless when you reviewed it may change. Pin versions and re-review on update, including the descriptions.
• Instructions in tool metadata. Descriptions and results flow into the model's context, so a malicious one can try to influence how other tools are used.

Keep an inventory of connected tools, and expose only those a given agent needs. Fewer tools mean less to review and fewer ways for a mistake to matter, a point tool calling for AI agents makes from the reliability side.

Poisoned memory

Long-term memory turns a one-off problem into a lasting one. If an agent saves a false "fact" from a web page or an email, it may act on that fact in later sessions, long after the source has gone and possibly for other users.

Record where each memory came from and when. Treat anything derived from external content as lower trust, keep memories scoped to the user or tenant they belong to, and give people a way to inspect and delete them. Agent memory covers the design of these stores more generally.

Trust between agents

In a multi-agent system, one agent's output is another's input. A researcher that reads untrusted pages may pass along text that a privileged agent then acts upon. A request from a colleague agent should not carry authority of its own. Each tool should check the original user's permissions and the policy for that action, regardless of which agent asked.

Runaway activity and stopping an agent

Not every incident has an attacker. A retry loop can send hundreds of emails or exhaust an API quota. Set limits on steps, spend and rate per run, per user and per tool, and enforce them outside the model. For actions with consequences, add human approval as a further check.

Plan the response before you need it. You should be able to disable one tool, one user's runs or the whole agent without a deployment, revoke its credentials, and reconstruct what happened from an append-only log of each tool call, its arguments, the identity used and the result. Test this as a drill, because a stop switch nobody has tried is only a hopeful assumption.

Where to start

List every tool the agent has. For each, write down whose identity it uses, what the worst plausible call would do and how you would find out. That list usually shows the first fixes. For help designing agents with these boundaries, see multi-agent systems.

Frequently asked questions

Is prompt injection the biggest security risk for AI agents?

It is a serious one, but it is only one way that an agent can end up requesting something harmful. Bugs and misreadings can cause the same requests. Controls on identity, network access and execution limit the damage whatever the cause.

Should an AI agent have its own service account?

A dedicated identity is better than sharing a human's or an application's account, and it makes logs clearer. It should not hold more access than the user being served, so combine it with delegated, per-user permissions wherever possible.

How should I store API keys used by an agent?

Keep them in a secrets manager and let tool code load them at the point of use. Keys should not appear in prompts, tool results or traces. Prefer short-lived, narrowly scoped credentials to long-lived ones where the service supports them.

Is a Docker container a safe sandbox for agent-generated code?

It helps, but a default container is not a complete boundary because it shares the host's kernel. Remove network access and credentials, run as non-root and set resource limits. Consider stronger isolation, such as a virtual machine, for higher-risk work.