Agent Memory & Context Poisoning

← Agentic AI (OWASP Agentic Top 10)

agent_memory_poisoning   HARD

Description

A malicious instruction is written to the agent’s persistent memory and re-fires across future sessions after the context resets (MINJA-style).

How it works

The agent writes “remember this” notes into persistent long-term memory with no sanitization, so a crafted note can embed an instruction. When a new session starts the app resets only short-term memory, so the poisoned instruction is recalled from long-term memory and the naive planner obeys it, re-firing across every future session. The evidence records the poisoned directive being recalled and re-executed after the reset. A hardened agent treats stored memory as untrusted data, not instructions, and re-authorizes any action a recalled note requests.

How to exercise it. DVMA is the harness - open this module from the home index and tap the demo action. The screen ships the malicious input and simulates the attacker (e.g. the companion app, crafted intent, or scanned payload) in-process, and the evidence panel prints the proof. The Tools (optional) and Attack inputs below are only needed to reproduce the exploit end-to-end on a real device.

Exploit steps

1. Set up. Build DVMA with a flavor that enables the Agentic AI (OWASP Agentic Top 10) category (e.g. --dart-define-from-file=config/flavors/dev.json) and run on an emulator/simulator you control. The demo needs no external tooling; for the optional on-device reproduction the relevant tools are: promptfoo, garak, frida, sqlite3.
2. Locate the target. From the home index, open Agent Memory & Context Poisoning (agent_memory_poisoning). The How it works section above describes this module’s specific weakness; the screen states the intended-secure behavior and exposes the vulnerable action.
3. Exploit. Interact with the agent’s memory / tool / sub-agent channel, plant the malicious content, and confirm it influences a later action or is acted on without authentication.
4. Observe the evidence. Trigger the vulnerable action and read the evidence panel - it prints the concrete proof (leaked value, accepted replay, executed payload, or unauthorized result).
5. Contrast with the secure path. Run the module’s secure/hardened action (where provided) and confirm the same attack is rejected - this is what a correct implementation should do.

Tools (optional)

• promptfoo
• garak
• frida
• sqlite3

garak / promptfoo / MCP scanner test the LLM or MCP endpoint behind the app, not the app binary. Point them at the backend model API (find it with mitmproxy / Burp Suite) or a local on-device model server; the in-app demo already exercises the same prompt path.

Attack inputs

Payloads/artifacts you author for the on-device attack. The demo already ships and simulates these in-process (e.g. the malicious companion app / crafted intent is emulated inside the screen), so you only need to craft them to reproduce the exploit on a real device:

• crafted prompt