KPI
AI systems threat-modelled
Share of in-scope AI systems with a current threat model that covers the AI-specific threat themes, not only the surrounding application.
All themes
Free tool · ISO/IEC 27090
AI Security Metrics Set: KPIs and KRIs for ISO/IEC 27090
The short answer: measure each AI threat you have in scope with one metric that shows the control works and one that warns when risk is rising. This set gives you 20 ready-made metrics, each with a definition, formula, data source, owner, frequency and an example target, mapped to the ISO/IEC 27090 threat themes. Filter by theme, build your own dashboard and export it.
Free, in your browser, nothing uploaded. Need role KPIs and SLAs for the wider security team? See the Cyber Team KPI & SLA Library. Setting objectives for an AI management system? Use the ISO 42001 AI Objectives & KPI Planner.
No download, no card
Filter by ISO/IEC 27090 threat theme
Type
20 metrics shown. Targets are example targets, not industry benchmarks.
KPI
Prompt-injection test pass rate
Share of direct and indirect prompt-injection test cases that the AI application resists in the latest test run.
Prompt inj.
KPI
LLM apps behind input and output mediation
Share of LLM-backed applications where user input, retrieved content and model output pass through a mediation layer (filtering, policy checks, schema validation).
Prompt inj.OutputLeakage
KRI
Guardrail bypass rate in red-team runs
Share of red-team attempts that get past safety or security guardrails, such as jailbreaks, policy evasion or unauthorised tool use. A rising rate is an early warning.
Prompt inj.AgencyLeakage
KPI
Training data with a provenance record
Share of training, fine-tuning and retrieval datasets with a recorded origin, licence, collection date and ingestion approval.
PoisoningSupply chain
KPI
Training runs through a data validation gate
Share of training and fine-tuning runs whose input data passed validation and outlier checks before training started.
Poisoning
KPI
Model releases with robustness evaluation
Share of model releases that passed an adversarial robustness evaluation in the release gate.
Evasion
KRI
Model-extraction alerts triaged within SLA
Share of alerts for systematic or high-volume querying of model endpoints that were triaged within the agreed time. Track the alert count alongside it.
Extraction
KPI
Mean time to revoke leaked model keys
Average time from detecting an exposed AI service credential (model API key, inference token, vector-store key) to revoking it.
ExtractionResource
KPI
Endpoints that limit confidence and embedding exposure
Share of externally reachable model endpoints that suppress or coarsen raw confidence scores, logits and embeddings unless a use case needs them.
InversionMembership
KPI
Sensitive-data models with a privacy-attack assessment
Share of models trained on personal or confidential data that have been assessed for membership-inference and reconstruction risk.
MembershipInversion
KPI
Models with signed provenance
Share of deployed model artefacts whose signature or hash was verified against a trusted source before loading.
Supply chainExtraction
KPI
AI supplier assessments current
Share of AI suppliers (model providers, AI SaaS, data and labelling vendors) with a security assessment completed within the agreed review cycle.
Supply chainLeakage
KRI
Deployed AI components missing from the AIBOM
Number of models, datasets or AI dependencies found running in production that are not in the AI bill of materials (AIBOM). Each one is an unmanaged supply-chain risk.
Supply chain
KPI
Model output paths with validation
Share of places where model output is rendered, executed, stored or passed to another system that encode or schema-validate it first.
Output
KPI
Agent tools on a least-privilege allow-list
Share of tools available to AI agents that are scoped to an allow-list per agent and, for irreversible actions, require human approval.
Agency
KRI
Irreversible agent actions without approval
Number of irreversible actions (payments, deletions, outbound messages, access changes) taken by AI agents without a recorded human approval.
Agency
KPI
AI services with spend and rate limits
Share of AI services and model API keys with token, rate and spend limits per user or tenant, so a runaway loop or cost-exhaustion attack is capped.
Resource
KRI
Sensitive data disclosed in AI output
Number of confirmed cases of personal, confidential or system-prompt data appearing in model responses, including canary-record hits in testing.
Leakage
KPI
Prompt and response logs within retention and access rules
Share of stores holding prompts, responses or agent traces that meet the agreed retention period and passed their last access review.
Leakage
Build your dashboard
Pick the metrics you will actually report, then set your own target and owner. Aim for 6 to 10: enough to cover your threat themes, few enough that someone owns each one. Saved in this browser only.
No metrics picked yet. Use Add on any metric above.
Turning these into objectives? The AI Objectives & KPI Planner picks up your dashboard metrics so you can attach them to ISO/IEC 42001 objectives with an owner, resources and dates.
TL;DRWhat this metrics set does
4 points
- 20 AI security metrics (15 KPIs and 5 KRIs), each mapped to the ISO/IEC 27090 threat themes it measures.
- Every metric has a definition, a formula, a data source, a suggested owner, a reporting frequency and an example target. Example targets are starting points, not benchmarks.
- Filter by any of the 11 threat themes, add metrics to your own dashboard, set your own targets and owners, and see which themes your dashboard still leaves unmeasured.
- Export the full set or your dashboard to CSV or Markdown. Free, runs in your browser, nothing uploaded.
How to choose AI security metrics
Start from the threats, not the data you happen to have. List the AI threat themes in scope for your systems, then pick at least one metric per theme. A dashboard where every number is about the same theme looks busy but leaves the others unmeasured, which is why the dashboard builder shows theme coverage as you pick.
Give every metric one owner, a data source that already exists or a plan to create it, and a frequency someone will keep to. Report a baseline for two or three cycles before committing to a target. A target set before you know your baseline is a guess, and the example targets here are only a place to start the conversation.
Metrics are how you show that ISO/IEC 27090 guidance has been taken up. Feed them into the monitoring and measurement your ISO/IEC 42001 or 27001 management system already does (clause 9), into management review, and into your AI objectives (clause 6.2 in ISO/IEC 42001).
Frequently asked questions
What metrics should we use for AI security?
Measure each AI-specific threat you care about with at least one metric that shows the control is in place (a KPI) and, where you can, one that warns when the risk is rising (a KRI). Good starting points are the share of AI systems threat-modelled, the prompt-injection test pass rate, the guardrail bypass rate in red-team runs, mean time to revoke leaked model keys, the share of models with signed provenance and the share of AI supplier assessments in date.
What is the difference between a KPI and a KRI here?
A KPI tells you how well a control or process is performing, such as the share of agent tools on a least-privilege allow-list. A KRI is an early warning that risk is rising, such as the guardrail bypass rate in red-team runs or deployed AI components missing from the AI bill of materials. Most dashboards need both.
Are the targets industry benchmarks?
No. They are example targets: sensible starting points that you should adjust to your baseline, risk appetite and tooling. Measure a few cycles first, then set targets that improve on where you are. No published industry averages are implied.
How do these map to ISO/IEC 27090?
ISO/IEC 27090 is guidance on addressing security threats to AI systems. Each metric is tagged with the AI threat themes it measures, such as prompt injection, data poisoning, supply chain or excessive agency. The themes are described in our own words and are the same ones used by the PlayCISO 27090 traceability matrix and threat mapping lab. No clause numbers are given, and there is no certification against ISO/IEC 27090.
Can I use these metrics for ISO/IEC 42001?
Yes. ISO/IEC 42001 asks for measurable AI objectives and for performance to be monitored, measured and evaluated. Pick metrics here, then use the AI Objectives and KPI Planner to attach them to objectives with an owner, resources, a timeline and an evaluation method.
Is it free and where is my data stored?
It is free with no sign-up. Your dashboard picks, targets and owners are stored only in your browser's local storage. Nothing is sent to PlayCISO.
More security tools
The tools people use most on PlayCISO.
Prompt Scan
Catch injected instructions hiding in content your AI agent should only read.
ISO/IEC 27090 Readiness
Score your readiness against the first international standard for AI security threats.
CVE Trends
Critical CVEs across 21 major vendors, month by month — and what the 2026 surge means.
Firewall Lab
Build an ordered ruleset, replay labelled packets and catch shadowed and any-to-any rules.
NIST CSF 2.0 Assessment
Self-assess all six Functions, from Govern to Recover, and see your weakest one.
APT Naming Rosetta Stone
Search 172 threat actors across Microsoft, CrowdStrike, Mandiant and community names.