AI security assessment
The AI feature is live. A customer questionnaire has arrived, or a board member has asked what happens if it goes wrong, or an engineer has noticed that the assistant can read a document store nobody ever mapped. The question underneath all three is the same: what can this system reach, who can make it act, and what happens when its answer is wrong.
What is an AI security assessment?
It is a scoped, evidence-based review of an AI system as it is actually built — not as the design document describes it. We read the prompts, the model configuration, the retrieval path, the tool integrations, the identities each component acts under, and the code that consumes the output. We then report what an attacker could reach through that system, ranked by what it would cost them and what it would cost you.
An assessment reads the system as built and reports what an attacker could reach through it; a penetration test proves a specific attack works against a running instance, under signed authorization and a scope agreed in writing. Both have their place, and the second is far more useful once the first has told you where to aim it.
What we look at
The unit of assessment is the system, not the model. A hosted model behind a thin wrapper and a self-hosted model with tool access to a production database are the same technology and entirely different exposures, so the review covers six layers:
- AI applications. The feature surface: who can reach it, authenticated or not, and what the system prompt and guardrails are actually doing as distinct from what they are believed to do.
- Models. Which models are in the path, hosted or self-run, what version is pinned, and what changes when the provider updates it.
- Data flows. What enters a prompt, where it came from, what is retained, where it is stored, and which of it is personal information or otherwise regulated.
- Identities. Whose authority the system acts under at each hop — the end user, a service account, or a credential that outranks both.
- Third-party services. Model providers, vector stores, evaluation and observability vendors, and any agent framework or plugin that receives prompt or retrieval content.
- Governance around it. Who approved the system going live, who can change the prompt, and what record exists of either.
What do we assess against?
The checklist is the OWASP Top 10 for LLM Applications 2025, applied category by category rather than sampled. Each row below is a question asked of your system, and the report answers all ten — including the ones where the answer is that the category does not apply, which is itself worth writing down.
| OWASP LLM category | What we look for in your system |
|---|---|
| LLM01 Prompt Injection | Every path by which text the organization does not control — a document, a web page, a ticket, a tool result — reaches the model as if it were an instruction. |
| LLM02 Sensitive Information Disclosure | What the model can surface about other tenants, other users, or the organization itself, through retrieval, memory or an over-broad system prompt. |
| LLM03 Supply Chain | Where the model, the adapters, the embedding model and the serving stack came from, and what integrity evidence exists for each. |
| LLM04 Data and Model Poisoning | Every point at which untrusted content enters fine-tuning, an evaluation set or a retrieval corpus, and what reviews it before it does. |
| LLM05 Improper Output Handling | What downstream code renders, executes, queries or stores from model output without treating it as untrusted input. |
| LLM06 Excessive Agency | What the system can do without a person approving it, whose credentials it uses, and how far a wrong action travels before something stops it. |
| LLM07 System Prompt Leakage | What the system prompt contains that would matter if it were published — and whether any control depends on it staying secret. |
| LLM08 Vector and Embedding Weaknesses | Whether the vector store enforces tenancy and document permissions, and what an embedding discloses about the text behind it. |
| LLM09 Misinformation | Where a confident wrong answer becomes a decision — a price, an eligibility, a dosage, a legal deadline — with no check between the two. |
| LLM10 Unbounded Consumption | What an unauthenticated or low-privilege caller can make the system spend, in tokens, in downstream API calls and in wall-clock capacity. |
The methodology spine is the OWASP AI Testing Guide, whose test identifiers give the review a repeatable structure: AITG-APP-01 Testing for Prompt Injection, AITG-APP-03 Testing for Sensitive Data Leak, AITG-APP-06 Testing for Agentic Behavior Limits and AITG-APP-08 Testing for Embedding Manipulation are the ones most often in scope for a business application. Where a finding needs a governance home rather than a code change, it is mapped to the relevant function of NIST AI RMF 1.0 — GOVERN, MAP, MEASURE or MANAGE — so the remediation plan lands on a named owner instead of on the security team by default.
How are findings prioritized?
Not by a generic severity label. A high-severity prompt-injection finding in a demo nobody uses is worth less of your week than a medium-severity retrieval gap in the assistant your whole sales floor has open. Two properties decide the order:
- Exploitability. What an attacker needs to reach it — an account, a specific role, a document they can get into your index, or nothing at all — and whether the attack survives an ordinary guardrail.
- Blast radius. What the finding reaches once it works: one user, one tenant, every tenant, or a system the model was never supposed to touch. Agency multiplies this, which is why a tool-enabled assistant outranks a chat-only one at the same nominal severity.
The two together produce a ranked plan rather than a heat map, and the plan states what each fix is expected to remove rather than only that it is recommended.
What you receive
- A findings report with evidence per finding: the path that reaches it, the configuration or code that permits it, what it exposes, and the control that closes it. Where a control is already working, the report says what evidence shows it is real rather than intended.
- A prioritized remediation plan, ordered so the first week of engineering time removes the most reachable risk, with the owning team named for each item.
- A scoping memo that states the assessed system boundary and, explicitly, what was out of scope. This is the document that stops a report being read later as coverage it never claimed.
The sample report shows the shape a finding takes. For a longer walk through the method, see what an AI security assessment covers.
When this is the wrong engagement
Two cases, and in both there is something better to buy:
- The system does not exist yet. Assessing a design document produces findings about a document. AI threat modelling is the design-review engagement, it costs less, and it removes whole classes of finding before anyone writes the code that would carry them.
- You need proof, not analysis. If a customer, an insurer or a regulator wants evidence that a named attack was attempted against the running application and did not succeed, that is LLM application penetration testing. An assessment tells you where to point it; it does not substitute for it.
Many organizations buy the assessment first precisely so the testing scope is narrow, targeted and worth what it costs.
How scoping and quoting works
Scoping starts from an inventory question, not a proposal: which features put a model in the path, what each one can reach, and which of them touch personal or regulated information. From that we write a scope — the systems in, the systems out, the environments we work in, the access needed and from whom — and quote the engagement against it in writing before any work begins. Scope follows the system, not a package: how many features put a model in the path, what those models can reach, how many tenants share a retrieval index, and whether anything acts without a person approving it.
Where a scope includes any exercise against a running system rather than a review of how it is built, the testing posture below applies in full.
How the work is bounded
The scope is agreed in writing before work starts, and the engagement is quoted in writing with it.
Testing and adversary simulation are carried out only with signed authorization, to a scope agreed in writing.
Investigation and forensic work is delivered with appropriately licensed partners where the law requires it.
SecHB does not issue certifications, attestations or audit opinions: those come from accredited certification bodies, CPA firms and QSAs. The work here is what an organization does to be ready for them.
Nothing here is legal advice. Where a question turns on the law, the work is done alongside the client’s counsel, not instead of them.
Questions we are asked
What is the difference between an assessment and a penetration test?
An assessment reads the system as built and reports what an attacker could reach through it; a penetration test proves a specific attack works against a running instance, under signed authorization and a scope agreed in writing.
Do you need production access?
Production access is not the starting assumption. Most of an assessment runs on architecture, configuration, prompts, retrieval design, identity and code, in a read-only pre-production environment where one exists. Where a question can only be answered against production — a retrieval boundary, a tenancy control — that part is named in the scope, agreed separately, and carried out under the authorization terms above.
Can this satisfy a customer asking whether we have tested our AI?
An assessment answers what a customer is usually asking — whether anyone competent has looked at the AI feature and written down what they found — but it is a review, not evidence that a named attack was attempted and failed. If the questionnaire asks specifically for testing evidence, the assessment is the step that makes the testing scope small enough to be worth buying.
How long does an AI security assessment take?
Scope follows the system, not a package: how many features put a model in the path, what those models can reach, how many tenants share a retrieval index, and whether anything acts without a person approving it. The scope states the duration before the engagement starts, and the scoping memo in the deliverables repeats it.
Scope an assessment
Describe the feature, what it retrieves and what it can call. The reply says what an assessment of it would cover, what it would produce, and what it would leave out.