(bishopfox.com) Confused Deputy Attacks in AI Agents: Mechanics, Case Studies, and Layered Mitigations

[orlysec@swecyb.com]

(bishopfox.com) Confused Deputy Attacks in AI Agents: Mechanics, Case Studies, and Layered Mitigations

New research highlights the growing risk of confused deputy attacks targeting AI agents, where attackers manipulate systems into executing malicious actions using their own privileges. These attacks exploit trust relationships and tool access to bypass security controls, enabling data exfiltration and privilege escalation.

In brief - Confused deputy attacks leverage seemingly legitimate inputs (e.g., support tickets, emails) to trick AI agents into performing unauthorized actions. High-profile incidents like EchoLeak and ConfusedPilot demonstrate real-world impact, emphasizing the need for layered mitigations such as least-privilege access and network egress controls.

Technically - Attackers embed malicious instructions in attacker-controlled content, which AI agents process via Multi-Tool Processing (MCP) servers. Techniques include Insecure Direct Object Reference (IDOR) and metadata service exploitation to escalate privileges. Case studies show Microsoft Copilot processing crafted emails to exfiltrate data or interpreting malicious calendar invites to expose private information. Mitigations include per-task tool restrictions, least-privilege principles, and egress controls to limit data exfiltration. Attackers can also bypass generative AI guardrails by directly targeting MCP servers, underscoring the need for robust security at both AI and infrastructure layers.

Source: https://bishopfox.com/blog/otto-support-confused-deputy

#Cybersecurity #ThreatIntel