AI Vulnerabilities Playground

The AI Vulnerabilities Playground (AIVP) is an OWASP Incubator project designed to help security professionals, developers, and researchers build practical skills in AI/LLM security through hands-on exploitation labs.

Inspired by traditional CTF-style training, AIVP provides a structured, progressive set of challenges that cover prompt injection, indirect injection, agentic exploitation, multi-turn manipulation, tool misuse, and context boundary attacks — the same techniques being observed in real-world AI deployments today.

Who it's for

• AI/ML engineers building LLM-powered applications

• Security engineers and red teamers extending into AI security

• Developers integrating agentic workflows and MCP-based tools

• Students and researchers exploring the OWASP LLM Top 10

What you'll learn

• How LLM guardrails fail under adversarial input

• Exploitation patterns mapped to the OWASP LLM Top 10

• Defensive patterns: input validation, output filtering, and policy enforcement

• How to think adversarially about agentic and tool-using systems

New labs and challenge phases are added regularly. Contributions and community challenges are welcome.

Sample Scenarios Covered in AIVP (many more inside):

• Scenario 1: Prompt Injection in Support Bot (Phase 1)
  Attacker embeds “ignore previous instructions” in a customer message to force the bot to reveal hidden policies or internal prompts.

• Scenario 2: Indirect Prompt Injection via External Content (Phase 1)
  Model ingests a webpage/doc with hidden instructions, then executes attacker intent during summarization or Q&A.

• Scenario 3: Tool/Function Calling Abuse (Phase 2)
  User-crafted input manipulates agent tool arguments, causing unauthorized actions (e.g., querying sensitive records).

• Scenario 4: Memory/Context Poisoning (Phase 2)
  Malicious data is stored in conversation memory; later sessions reuse poisoned context and leak confidential information.

• Scenario 5: Multi-Agent Communication Exploit (Phase 2)
  One compromised agent sends deceptive instructions to peer agents, hijacking workflow and bypassing normal checks.

• Scenario 6: RAG Data Exfiltration (Phase 2)
  Retrieval chain is manipulated to pull protected chunks from vector DB, exposing data outside user scope.

• Scenario 7: Session/State Hijacking (Phase 2)
  Weak session boundaries allow attacker prompts to alter or reuse another user’s state in agent workflows.

• Scenario 8: Model Inversion / Membership Inference (Phase 3)
  Adversary probes model outputs to infer whether sensitive training samples or user data were included.

• Scenario 9: Safety Bypass with Iterative Jailbreaking (Phase 3)
  Attacker refines prompts over multiple turns to gradually bypass safety constraints and elicit restricted output.

• Scenario 10: MCP Token Mismanagement (Phase 4)
  Over-privileged or leaked MCP tokens allow unauthorized tool access across contexts.

• Scenario 11: MCP Tool Schema Poisoning (Phase 4)
  Malicious schema/metadata alters tool interpretation so the agent executes unintended, risky operations.

• Scenario 12: Cross-Session Context Bleed in MCP (Phase 4)
  Inadequate isolation causes one session’s context to be exposed or reused in another user/session.