AI security monitoring and detection
The AI feature went to production. The detection content did not. If someone attacked that feature this morning — injected an instruction through a document, walked an agent outside its mandate, pulled a thousand documents out one at a time — the honest answer to whether anyone would see it is usually no, and usually nobody has asked.
Would we see it if our AI were attacked?
The question is answerable, and answering it is the engagement. It decomposes into three: is the event recorded at all; would anything match on it; and would a human read the result and know what to do. Most organizations fail at the first, which is why the work starts there rather than with detection content.
Conventional application telemetry does not cover this. An access log shows a request to an endpoint. It does not show the assembled prompt, what the retrieval layer returned, which tools were called with which arguments, or what came back. Those are the events, and in a default deployment none of them is recorded.
Logging-coverage gap assessment
The first deliverable is a coverage matrix: for each layer, what is emitted today, what reaches your analysis platform, how long it is kept, and what is missing. The layers are prompts with their assembled context, retrievals with their source and permission labels, tool calls with arguments and results, model outputs, model-access and configuration events, and the guardrail decisions themselves — which are frequently the only cheap signal available and are frequently discarded.
Log the prompt with its assembled context, what was retrieved and from where, every tool call with its arguments and result, the output, and model-access events — all tied together by one correlation identifier, and retained long enough to outlive the delay between an incident and its discovery. Two constraints shape the answer. Prompts and outputs may contain personal information, so the design specifies minimization, redaction and access control for the log itself; and retention has to outlast discovery, because an AI incident is typically noticed long after it began.
Detection use cases
Each use case below is specified with the data it needs, the logic, the expected volume and the tuning guidance — so it can be implemented by your detection engineers in whatever platform you run.
| Use case | What it looks for |
|---|---|
| Prompt-injection attempts | Known payload patterns, instruction-shaped text arriving from a retrieval source, and sharp shifts in refusal or compliance behaviour for one session. |
| Anomalous tool invocation | A tool called by an agent that has never called it, a destructive action outside its usual window, or an argument pattern no legitimate workflow produces. |
| Unusual data egress volumes | Retrieval volume or output volume for one identity well outside its own baseline — the low-and-slow extraction case, which per-request limits do not catch. |
| Credential misuse by agents | An agent credential used from an unexpected source, outside its schedule, or against a system outside its scope. |
| Cross-tenant retrieval | A retrieval returning content whose tenant or permission label does not match the asking identity — the highest-value detection in a multi-tenant assistant. |
| Consumption anomalies | Token or downstream-call spend departing from baseline, which is both a cost control and the tell for automated abuse. |
Classification follows the OWASP Top 10 for LLM Applications 2025, so a detection that fires maps to a named failure mode rather than to an internal label. The measurement and management frame is NIST AI RMF 1.0 — its MEASURE and MANAGE functions, with the Generative AI Profile (NIST AI 600-1) for the generative-specific risks.
Alert routing and triage
An alert nobody can action is a cost. For each use case the design names who receives it, what the first three triage steps are, what evidence the responder needs in hand, and what the escalation path is when the answer is yes. Several of these alerts route to the application team rather than to security, because the application team is the only group that can tell an attack from an unusual legitimate workflow.
Triage guidance is written as a runbook entry rather than as advice, so it sits beside your existing ones.
Tabletop validation: would the detections fire?
The step that is almost always skipped, and the one that finds the problems. We walk scenarios against the deployed detections with your team: an indirect injection through an indexed document; an agent induced to call a destructive tool; steady extraction of a corpus by an authorized user under every per-request limit; an agent credential used from somewhere new.
For each, the group traces what would be recorded, what would match, who would receive it, and what they would do. The output is a list of the detections that would not have fired and why — missing telemetry, a threshold set for a different traffic shape, a route to a queue nobody reads. A detection that has never been exercised is a hypothesis.
Why this is sold with the environment design
Detection is downstream of telemetry, and telemetry is a property of how the environment was built. If prompts, retrievals and tool calls were never correlated at the gateway, there is nothing to detect on and no amount of detection engineering fixes it. That is the honest dependency, so it is stated rather than discovered halfway through.
Where the telemetry does not exist, the work begins in secure AI environment establishment, which specifies the audit logs and correlation identifiers this depends on. Where it does exist, this engagement is the shortest path to coverage. What happens once an alert is real is AI incident readiness, and the scenario most teams rehearse first is set out in prompt injection incidents.
What you receive
- A coverage gap report: the matrix of what is logged, what reaches your platform, what is retained and what is missing, ordered by what it blinds you to.
- Detection use cases, each with data requirements, logic, expected volume and tuning guidance, written for your stack.
- Routing and triage guidance per use case, in the form your runbooks already take.
- Validation findings from the tabletop: which detections would not have fired, and what has to change for them to.
Detection content is written with your detection engineers rather than handed over: the use cases, the logic and the tuning guidance come from us, the implementation lands in your platform under your change control, and running it stays with you. No claim is made here to operate monitoring, staff a security operations function, or provide round-the-clock coverage.
How the work is bounded
The scope is agreed in writing before work starts, and the engagement is quoted in writing with it.
SecHB does not issue certifications, attestations or audit opinions: those come from accredited certification bodies, CPA firms and QSAs. The work here is what an organization does to be ready for them.
Nothing here is legal advice. Where a question turns on the law, the work is done alongside the client’s counsel, not instead of them.
Questions we are asked
Does AI security monitoring need a new tool?
A new tool is rarely the gap. The work is done against the logging and analysis stack you already run, because the usual finding is that the AI system emits almost nothing worth alerting on rather than that the analysis platform is inadequate.
What should we log, and for how long?
Log the prompt with its assembled context, what was retrieved and from where, every tool call with its arguments and result, the output, and model-access events — all tied together by one correlation identifier, and retained long enough to outlive the delay between an incident and its discovery.
Can you write the detections for us?
Detection content is written with your detection engineers rather than handed over: the use cases, the logic and the tuning guidance come from us, the implementation lands in your platform under your change control, and running it stays with you.
We have no security operations function — is this still worth doing?
Without a monitoring function the coverage work is still worth doing, and the priority order changes: get the telemetry recorded and retained first, because that is what makes any later reconstruction possible, and add detections at whatever cadence your team can actually triage.
Find out what you would see
Describe your logging stack and what your AI systems currently emit. The reply says where the coverage gap usually sits and what the assessment would look at first.