LLM application penetration testing
The launch date is set, or a customer has asked for evidence, or the assistant did something last week that nobody can explain. What is wanted now is not an opinion about the architecture. It is somebody competent attacking the running application on purpose, and writing down what worked.
Before anything starts: authorization and scope
This is stated first because it is the part that is most often skipped and most consequential. Testing is performed only under signed authorization from the system owner, to a scope agreed in writing. Where the application runs on infrastructure you do not own, the provider’s authorization is obtained as well, and where the law requires it, investigation and forensic work is delivered with appropriately licensed partners.
The scope names the systems in and out, the environments, the identities provided, the testing window, the rate limits, the stop conditions, and who is reachable if something degrades. Nothing begins until it is signed.
Scope by application archetype
Systems are scoped by what they are, because that is how their owners describe them and because each archetype has a different attack surface:
- Conversational assistant. A model behind a chat surface with no retrieval and no tools. The surface is the prompt, the guardrails, the session boundary and what the application does with the answer.
- Document and RAG assistant. Adds a corpus, which adds two things: a path by which an attacker’s text reaches the model, and a set of permissions the index may not be honouring.
- Connected or tool-using assistant. Adds actions. The question stops being what the model says and becomes what it can do, under whose authority, and how far that reaches.
The archetype sets the emphasis; the checklist below is walked in every case.
What we test for
Against the 2025 list, named category by category, so a report can be read beside anyone else’s and compared honestly. The reference is the OWASP Top 10 for LLM Applications 2025.
| What is tested | Category and scope |
|---|---|
| Direct and indirect prompt injection | LLM01 Prompt Injection. Instructions supplied by the user, and instructions arriving through a retrieved document, a fetched page, a ticket or a tool result. |
| Jailbreaks and multi-turn coercion | LLM01 Prompt Injection, again. Single-turn bypasses, and the slower pattern where each turn is acceptable and the conversation is not. |
| Sensitive information disclosure | LLM02 Sensitive Information Disclosure. What the system reveals about other users, other tenants, or the organization, including cross-tenant leakage through shared retrieval. |
| System prompt leakage | LLM07 System Prompt Leakage. What the system prompt contains, whether it can be extracted, and whether any control depends on it remaining secret. |
| Improper output handling | LLM05 Improper Output Handling. The downstream class: stored and reflected cross-site scripting, server-side request forgery, command and query injection reached through what the model emits. |
| Excessive agency | LLM06 Excessive Agency. Whether the system can be induced to act outside its mandate, and how far a wrong action travels before a control stops it. |
| Unbounded consumption | LLM10 Unbounded Consumption. Denial of wallet and denial of service: what an attacker can make the system spend in tokens, downstream calls and capacity. |
| Misinformation in high-stakes outputs | LLM09 Misinformation. Where a confident wrong answer becomes a decision with no check between the two. |
| Supply chain and poisoning exposure | LLM03 Supply Chain and LLM04 Data and Model Poisoning. Exercised where the application surface reaches them; the full treatment is a separate review. |
| Vector and embedding weaknesses | LLM08 Vector and Embedding Weaknesses. Retrieval-layer isolation and embedding exposure, tested where the application depends on them. |
The structure comes from the OWASP AI Testing Guide, whose identifiers map the work to a published methodology — AITG-APP-01 Testing for Prompt Injection, AITG-APP-02 Testing for Indirect Prompt Injection, AITG-APP-03 Testing for Sensitive Data Leak, AITG-APP-05 Testing for Unsafe Outputs, AITG-APP-06 Testing for Agentic Behavior Limits and AITG-APP-07 Testing for Prompt Disclosure among them. Governance-side findings map to NIST AI RMF 1.0, and MITRE ATLAS is available at framework level for teams that want adversary behaviour catalogued alongside the findings.
Human-led, and why that still matters
Automated prompt-injection suites are useful and are used here: they cover known payloads quickly and they regress well. They do not find the finding that matters, which is almost always specific to your system — a tool whose name suggests a capability the model over-interprets, a tenant identifier the application trusts because it came back from the model, a retrieval filter that holds for one query shape and not another.
So the automated pass establishes the floor and the human work starts where it stops. No claim is made here to continuous testing or to a monitoring platform; this is a bounded engagement with a beginning, an end and a report.
How a finding is written
Each finding carries reproduction steps precise enough for an engineer to confirm it without us, the impact in terms of what it reaches, and the fix; the highest-impact findings also carry the attack-path narrative that chains them together. A finding that an engineer cannot reproduce is an assertion, and assertions do not get fixed.
The attack-path narrative is reserved for the chains: the injection that reaches a tool that holds a credential that reaches a database. Written as three separate medium findings, that chain gets three separate tickets and stays open. Written as one path, it gets fixed. The sample report shows the format, and what a pentest report should contain sets out what to demand from any supplier, including this one.
Retest, and the scoping memo
Fixed findings are retested against the same reproduction steps, and the retest is reported as a separate, dated statement rather than as an edit to the original — so the document you hand a customer shows what was found, what was fixed, and when.
The scoping memo is delivered with the report and states the tested system boundary and what was explicitly out of scope. It is the document that stops a report being read two years later as coverage it never claimed, and it is the one a careful customer reads first.
Where this fits
If you do not yet know which parts of the system carry the risk, an AI security assessment first makes this engagement smaller and sharper. If the system is an agent with tools rather than an application with a model, the permission and boundary work is agentic AI and MCP security. If the question is specifically about what the assistant can retrieve, that is RAG and vector store security.
Engagements are quoted in writing against the agreed scope before any work begins.
How the work is bounded
The scope is agreed in writing before work starts, and the engagement is quoted in writing with it.
Testing and adversary simulation are carried out only with signed authorization, to a scope agreed in writing.
Investigation and forensic work is delivered with appropriately licensed partners where the law requires it.
SecHB does not issue certifications, attestations or audit opinions: those come from accredited certification bodies, CPA firms and QSAs. The work here is what an organization does to be ready for them.
Nothing here is legal advice. Where a question turns on the law, the work is done alongside the client’s counsel, not instead of them.
Questions we are asked
Do you test against production?
Whether testing runs against production is a decision made with you and written into the scope, never assumed. A representative staging environment with production-like data handling is preferred; where only production answers the question, the scope names the windows, the rate limits and the stop conditions.
What do you need from us to scope this?
To scope accurately we need the application archetype, what the model can retrieve, what tools or functions it can call, how many tenants share it, whether it is authenticated, and who owns the system well enough to authorize testing against it.
Is LLM penetration testing the same as AI red teaming?
The two overlap and are sold interchangeably, which helps nobody. This work is bounded adversarial testing of a named application against a stated checklist; "red teaming" more often describes an open-ended objective across an organization. The scope document is what settles which you are buying.
What does the report look like?
Each finding carries reproduction steps precise enough for an engineer to confirm it without us, the impact in terms of what it reaches, and the fix; the highest-impact findings also carry the attack-path narrative that chains them together.
Scope a test
Describe the application, what it retrieves, what it can call and who owns it. The reply says what a scope would cover and what authorization would be needed before anything starts.