Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks

Chong Xiang, Drew Zagieboylo, Shaona Ghosh, Sanjay Kariyappa, Kai Greshake, Hanshen Xiao, Chaowei Xiao and G. Edward Suh argue that indirect prompt injection requires defenses at the agent-system level. Their position paper examines general-purpose autonomous agents operating in partially compromised environments, while assuming trusted system and user prompts. It proposes an architecture and research directions, not a newly validated defense with a measured success rate.^[1]

A plan is not a permission

The proposed architecture separates what an agent intends to do from what it may do. A plan represents actions and dependencies; a policy constrains actions and information flows. An orchestrator proposes both, an approver checks changes, an executor proposes concrete actions, and a policy enforcer permits or blocks them. Responses from the environment or enforcer can then trigger a new plan or policy.^[1]

These distinctions allow different checks at different moments. A plan may legitimately require reading an email, yet a policy can prevent sending its confidential contents to an external destination. The authors describe static access rules, task-specific least privilege and context-dependent policies as complementary choices. Richer policies can express more intent while making reliable enforcement harder.^[1]

Why freezing the plan is insufficient

The first position is that dynamic tasks often require replanning and policy updates. A deprecated API or missing dependency can invalidate a plan that was sensible before execution. In the paper’s Slack-invitation example, the task’s intended person and channels must be learned from an email; they are unavailable when initial permissions are set. A static policy must either anticipate enough flexibility or prevent some legitimate work.^[1]

Allowing updates creates the security problem: untrusted runtime content can now influence the plan and its permissions. Automatically accepting every suggested repair would let an attacker present a malicious change as a necessary next step. The question is which update is justified by the trusted task, rather than whether all environmental feedback should be ignored.^[1]

Narrow the model’s security decision

The second position retains learned models for some context-dependent judgments but constrains their inputs and authority. Instead of asking a general assistant to consume raw attacker-controlled text and decide everything, a judge could assess a structured task state and a proposed change. Programmatic checks remain preferable where they suffice.^[1]

One proposal separates recognizing candidate instructions from deciding to follow them. A model would explicitly surface the instructions it intends to adopt. Policy could then reject externally sourced instructions, request confirmation, or adjudicate a structured update. Another mode predicts whether a source should contain actionable instructions before reading it. These proposals depend on faithful recognition and useful structured representations; a malicious instruction telling the model to hide its intent remains a relevant adaptive attack, not a solved case.^[1]

This complements instruction-priority training. The system determines what reaches a security judge and what decisions it can authorize, while model training can target the narrower subtask. Neither a learned priority rule nor a second model alone establishes an enforceable permission boundary.^[1]

Ambiguity and evaluation

The third position concerns personalization and human interaction. “Urgent emails” can mean different things to different users; an apparently relevant package installation can introduce unacceptable risk. Even perfect language understanding would not supply an unstated user’s preference. The authors therefore treat clarification and organizational policy as parts of agent design, while recognizing the burden of frequent intervention.^[1]

Their benchmark critique asks for longer evolving tasks, runtime failures, task-dependent attack objectives and attacks optimized against the actual defense. Manipulating the amount in an otherwise legitimate transfer, for example, is different from replacing the entire task with an obviously unrelated tool call. The critique motivates richer tests; it does not independently invalidate every result from the cited benchmarks.^[1]

The inspected v1 was submitted March 31, 2026. Its scope leaves traditional sandboxing outside the discussion, rather than showing it unnecessary. The central design question is how to preserve useful adaptation without allowing untrusted content to grant itself authority. Structured plans, bounded judgments and explicit approval points make that question inspectable, but their proposed combination still needs empirical and adversarial validation.^[1]