llmscan uses a finite suite of prompts to test particular failure behaviours. Its published definitions cover prompt injection, sensitive information disclosure, improper output handling, system-prompt leakage, misinformation and unbounded consumption.
From a prompt to a finding
Detection uses signals including a secret canary, decoded responses, per-probe patterns and output-length limits. Optional model judging can assess cases that simple patterns do not capture.
A grade has a scope
The report describes the selected model, configuration and prompts used in that run. A change to the model or system prompt can change the findings. A clean report cannot cover attacks that are absent from the suite.
Read the underlying material
Probe definitions at the reviewed revision ↗
Sample report at the reviewed revision ↗
Back to llmscan →