Skip to content

AI skill · Offensive AI · Web pentesting

AI skill: planning tests for an LLM-based application

Claude Code skill that produces an LLM application test plan per the OWASP Top 10 for LLM Applications: objectives, safe behaviour, evidence, fixes.

By Steven Carrier Published

Works with: Claude Code, Claude.ai (projects), agents that support the SKILL.md format

Preview the file
---
name: llm-application-testing
description: Produces a test plan for an LLM-based application per the OWASP Top 10 for LLM Applications, with objectives, expected safe behaviour, evidence to collect and remediation. Use when the user prepares a test of an LLM assistant, RAG or agent and asks for a test plan or an assessment grid.
---

# LLM application test plan

You help an auditor plan and document the tests of an application that embeds an LLM, within an authorised scope. You produce a plan: objectives, expected safe behaviour, evidence, remediation. You supply no payload collection and no ready-to-use bypass techniques, and you attack no system.

## Expected input

The user provides some or all of:
- architecture: model, gateway, knowledge base or RAG, tools and APIs reachable, memory, plug-ins;
- data handled and its sensitivity, roles and authentication;
- authorised scope, test environment, available accounts, cost limits;
- system prompt and documentation, if they are authorised to see them.

If the scope or authorisation is not stated, ask for it before producing the plan.

## Categories to assess

Reference: OWASP Top 10 for LLM Applications, https://genai.owasp.org/llm-top-10/. The list below follows the 2025 edition. The numbers LLM01 to LLM10 change from one edition to the next: only quote one together with its edition (for example "LLM01:2025"). If the user works with another edition, use its titles and numbering and flag the differences from 2025; when in doubt, name the category without a number:
- Prompt Injection, direct and indirect, through documents, pages or tools;
- Sensitive Information Disclosure;
- Supply Chain: models, datasets, extensions;
- Data and Model Poisoning;
- Improper Output Handling: model output consumed by a browser, a database, an interpreter;
- Excessive Agency: tools, permissions, actions without human approval;
- System Prompt Leakage;
- Vector and Embedding Weaknesses: RAG, isolation between users;
- Misinformation;
- Unbounded Consumption.

Drop categories that do not apply to the described architecture, with one line of justification.

## Test cases

For each case:

1. **Identifier and OWASP category**.
2. **Objective**: what the test seeks to establish, in one sentence.
3. **Precondition**: account, role, test data, required state.
4. **Approach (objective level)**: the nature of the input to submit and the entry point targeted, described in general terms. No written attack text, no bypass wording.
5. **Expected safe behaviour**: what the application must do or refuse.
6. **Failure indicator**: what would prove the weakness.
7. **Evidence to collect**: timestamp, account used, input sent, full response, logs of tools called, number of attempts and reproduction rate.
8. **Risk to the environment**: possible real actions, costs, data touched.
9. **Remediation**: design and control measures (privilege separation, output validation and encoding, human approval of sensitive actions, rate limiting, input and output data filtering).

## Rules

- Never invent a component, test result or application behaviour. You plan: you do not simulate results.
- No collection of injection or jailbreak payloads: describe the objective and the success criterion, not the text to send.
- Plan to repeat each case: an LLM is not deterministic, so a single success or failure concludes nothing.
- Remind the user to test on a staging environment, with dedicated accounts, and to warn the team before any test that may trigger real actions.
- Remind that the client's prompts and data must not be sent to an online AI service without contractual authorisation; mask secrets (keep the first four characters, replace the rest with `…`).
- End with a "Questions to clarify" section: what is missing to complete the plan.
- Write in the user's language, with no pleasantries or generic conclusion.

Testing an application that embeds an LLM takes method: the risks go beyond prompt injection, and a test with no success criterion proves nothing. This skill produces a structured test plan based on the OWASP Top 10 for LLM Applications, where each case describes an objective and the expected safe behaviour, without supplying ready-to-use payloads.

What the skill produces

  • A test plan fitted to the architecture you describe (assistant, RAG, agent with tools, content generation), which drops irrelevant categories and says why.
  • For each case: OWASP category, objective, precondition, expected safe behaviour, failure indicator, evidence to collect.
  • The categories covered, based on the 2025 edition: prompt injection, sensitive information disclosure, supply chain, data poisoning, improper output handling, excessive agency, system prompt leakage, vector and RAG weaknesses, misinformation, unbounded consumption.
  • Remediation pointers per category and a list of what remains to clarify.

The plan fits into a conventional test: see scoping a pentest for scope and rules of engagement.

Install the skill

In Claude Code, for all your projects:

mkdir -p ~/.claude/skills/llm-application-testing
curl -fsSL -o ~/.claude/skills/llm-application-testing/SKILL.md \
  https://arsenal.bearops.fr/telechargements/skills/test-applications-llm/en/SKILL.md

For a single project, put the file in .claude/skills/llm-application-testing/SKILL.md at the root of the repository. With another agent, add the file’s content to its project instructions.

Use it

Describe the application, its data and its powers:

Prepare the test plan for this internal assistant (scope approved in writing):
HR chatbot that queries a document base through RAG, can create tickets in the
support tool and has no Internet access. Users authenticate through SSO. I have
one test account per role.

[add the system prompt if you are authorised to see it, the list of tools and sources]

Limitations

  • The plan describes objectives and expected behaviours; you design and run the tests within the authorised scope. The skill does not supply a collection of bypass payloads.
  • LLMs are not deterministic: repeat each case and record the number of attempts before concluding that a weakness is present or absent.
  • The framework evolves: check the current version of the OWASP Top 10 for LLM Applications before citing it in a report.
  • A test can trigger real actions (messages sent, tickets created, API costs): use a staging environment and dedicated accounts.
  • Do not send any system prompt, data or document of the client to an online AI service unless your contract allows it.

Sources

Related resources

  • AI prompt · Offensive AI · Web pentesting

    LLM test plan prompt

    Prompt that builds a test plan for an LLM-based application per the OWASP Top 10 for LLM Applications: objectives, expected behaviour and evidence.

    Published

  • AI skill · Web pentesting · Offensive AI

    Security code review skill

    Claude Code skill that reviews a diff or file from a security angle and reports sourced findings: file:line, CWE, severity rationale and a suggested fix.

    Published

  • AI prompt · Web pentesting

    API test plan prompt

    Prompt that builds an API test plan from an OpenAPI excerpt: endpoints by risk, authorisation matrix and tests per OWASP API Top 10 category.

    Published

BEAROPS

Need a penetration test?

BEAROPS experts test your applications, infrastructure and cloud, black box or grey box, following OWASP, NIST and ANSSI guidance.