What a guardrail actually catches

What is a guardrail actually doing?

Classifying. Input guardrails read what is about to reach the model and judge whether it contains an attempt to manipulate it. Output guardrails read what came back and judge whether it contains something that should not leave.

Both are statistical judgments about text, which means both have a false-negative rate — attacks they miss — and a false-positive rate — ordinary work they block. Neither rate is zero, and the vendor’s published figure is a measurement of their test set rather than a property of the product.

Why the adversary has the advantage

A detector has to generalise from the attacks it was trained on. An attacker can rephrase indefinitely, in any language, in encodings, split across turns, or hidden in a document the system will read later.

Published comparative work on injection detectors has found the best commercial detector catching a clear majority of attacks in a large prompt set while widely-sold alternatives caught between a third and a half. Read those numbers carefully: they were measured on one corpus at one time, and an adaptive attacker is not sampling from a fixed corpus.

NIST’s 2025 taxonomy makes the same point in the language of a standards body: mitigations exist, they are worth deploying, and they are not fool-proof.

What should be measured instead?

Two rates, both on the client’s own traffic.

What it misses. Run the confirmed attacks from the engagement through the guardrail and count how many pass. This is the number that belongs in the report, because it describes this deployment against these attacks.

What it wrongly blocks. Sample real, legitimate traffic and count the refusals. This is the number that determines whether the control survives contact with the business. A guardrail that blocks a support agent’s ordinary phrasing at a few percent will be relaxed or disabled within weeks, and then the organisation has neither the control nor the knowledge that it lost it.

Measured on	Tells you	Worth reporting?
Vendor benchmark set	How the product scored on a curated corpus	As context, attributed
Public attack corpus	Coverage of known published attacks	Yes, as a floor
This engagement’s confirmed attacks	What this deployment misses today	Yes — this is the finding
The client’s own legitimate traffic	What it will wrongly block, and whether it will survive	Yes — this decides adoption

Where detection genuinely helps

It raises the cost of casual attacks, catches the reused published payloads that make up most opportunistic traffic, and produces a log of attempts, which is useful evidence and a leading indicator that someone is probing.

What it does not do is make an exposed architecture safe. Where a system combines access to private data, exposure to untrusted content and a path for information to leave, the exposure is structural, and tuning a classifier in front of it is managing a probability rather than removing a path.

How we treat it in an engagement

We deploy, tune and operate other people’s detection tooling; we do not build our own, because the component is commodity and the judgment is not. In a report, a guardrail finding always carries both rates and the date they were measured, and it is stated as depth rather than as the control that holds.