AI Surveillance: The Risks and How to Safeguard Against It

Artificial intelligence has quietly rewritten the economics of surveillance. Work that once demanded a room of analysts, weeks of manual research, and specialist tooling can now be automated, scaled, and pointed at entire populations. Anthropic's threat intelligence report, which catalogued and disrupted operations spanning cyberattacks, influence campaigns, surveillance, and other abuses, made the trajectory concrete: adversaries are already trying to bend frontier models toward monitoring and profiling people. For a CISO or DPO, the question is no longer whether AI changes the surveillance threat model, but how your organisation avoids becoming either a perpetrator or a casualty of it.

How AI scales surveillance

Traditional surveillance was expensive and therefore self-limiting. AI removes the friction at every stage, turning targeted observation into something that can be run continuously and cheaply against millions of people.

• Facial recognition and biometrics: models can match faces across cameras, images, and archives, converting anonymous crowds into identified individuals in real time.
• Aggregation of open data: fragments of public information (social posts, registries, breach dumps, and metadata) are correlated automatically to reassemble a detailed picture no single source contained.
• Behavioural profiling: patterns in movement, spending, and online activity are scored to infer beliefs, relationships, health, or intent, often with confident but unaccountable conclusions.
• Automated OSINT: language models can summarise, translate, and prioritise vast open-source intelligence, compressing days of analyst work into minutes and lowering the skill needed to run it.

Individually these capabilities are mundane. Combined, they let a small team build and continuously update dossiers on people at a scale that was previously the preserve of nation-states.

Why the risks are severe

Surveillance harms are not abstract. They fall on individuals, on society, and, awkwardly for security leaders, on the organisation running the system.

• Chilling effects on rights: people who believe they are watched self-censor, avoid protest, and withdraw from public life. That erosion of free expression and association is a recognised human-rights harm, not a side effect.
• Abuse and discrimination: profiling systems encode and amplify bias, and the same infrastructure built for one purpose is easily repurposed to target journalists, dissidents, minorities, or employees.
• Security exposure of the surveillance data itself: a database that fuses biometrics, location history, and inferred traits is one of the most sensitive assets imaginable. If it is breached, the aggregation you built becomes a weapon in someone else's hands. The act of collecting concentrates risk.

This last point is what many teams miss. Building monitoring capability does not only create legal and ethical exposure; it creates a catastrophic breach scenario where the crown jewels are intimate profiles of real people.

Why AI labs restrict surveillance use

Frontier providers prohibit surveillance, mass monitoring, and de-anonymisation in their acceptable-use policies, and they enforce those terms. Anthropic's threat intelligence report is a public example of that enforcement: it described identifying and disrupting operations that attempted to misuse models for surveillance and profiling, alongside cyber and influence activity. The lesson for enterprises is twofold. First, if you rely on a commercial model to power any people-monitoring feature, you are almost certainly violating the vendor's terms and risk abrupt loss of access. Second, the same detection that catches adversaries applies to customers; misuse is visible. Building surveillance on top of a restricted model is both a compliance and a continuity risk.

Safeguards a CISO or DPO can implement

The defensive posture is governance-led, not tool-led. These controls keep legitimate analytics from drifting into unlawful monitoring, and they demonstrate accountability to regulators.

Data minimisation and privacy-by-design

Collect the least data needed for a defined outcome, and design systems so that privacy protections are the default rather than an afterthought. Do not ingest personal data "in case it is useful." Aggregation is the raw material of surveillance, so minimisation is the single most effective structural control. Publishing a clear, honest account of what you collect (as we do on our privacy page) is part of that discipline.

DPIAs and purpose limitation

Run a Data Protection Impact Assessment before any system that monitors, profiles, or tracks people goes live. A DPIA forces you to name the purpose, the lawful basis, the risks, and the mitigations up front. Purpose limitation then binds the data to that stated use; function creep, where a tool built for fraud detection quietly becomes employee monitoring, is how well-intentioned systems become surveillance.

Access controls and retention

Restrict who can query profiling data, log every access, and separate duties so no single person can assemble a full dossier unchecked. Enforce short retention and automatic deletion. The smaller and more tightly controlled the store, the less damage a breach or an insider can do.

Vendor due diligence

Whether you buy an AI product or use a model API, examine the supplier's data practices, model provenance, acceptable-use terms, and security posture before you commit. Ask where training data came from, whether your inputs are retained or used for training, and what the vendor's own controls against misuse are. A tool marketed for "enrichment" or "identity resolution" may be repackaged surveillance; treat those claims with scrutiny. Our identity risk tool can help teams understand their own exposure, and the wider security tools library supports due-diligence work.

Regulation and an AI governance policy

GDPR already governs profiling and automated decision-making: it demands a lawful basis, transparency, purpose limitation, minimisation, and, for large-scale monitoring, a DPIA. Similar regimes are emerging worldwide alongside dedicated AI regulation. Translate those obligations into an internal AI governance and acceptable-use policy that states plainly what your organisation will and will not do with AI and personal data, assigns ownership, and gives staff a route to raise concerns. A written, enforced policy turns AI governance from a slogan into a control that auditors, regulators, and your own people can rely on.

Human oversight and red lines

Automated profiling should never make consequential decisions about a person without meaningful human review. Define red lines that no team may cross regardless of business pressure: no covert monitoring of individuals, no de-anonymisation of pseudonymous data, no scraping that violates a platform's terms or a person's reasonable expectations, and no re-identification of aggregated datasets. Put these in writing, brief every team that touches AI, and make sure procurement, marketing, and product all know that an attractive capability is not permission to deploy it. The strongest safeguard is a culture where staff feel able to say "we could build this, but we will not."

Monitoring, testing, and incident response

Treat any people-facing AI system as in scope for continuous assurance. Log how models are prompted and what data flows into them, test for bias and for scope creep, and rehearse an incident response plan specifically for the failure mode where profiling data leaks or a feature is found to be operating outside its stated purpose. Because AI systems change behaviour as data and prompts change, a one-time DPIA is not enough; schedule periodic reassessment and tie it to your change-management process so a quiet expansion of scope cannot slip through unreviewed.

Bringing it together

AI does not invent surveillance, but it industrialises it, and the same capabilities that make analytics powerful make mass monitoring cheap. The organisations that stay on the right side of this line treat personal data as a liability to be minimised, not an asset to be hoarded, and they wrap every people-facing AI system in DPIAs, purpose limits, access controls, and honest governance. Anthropic's threat intelligence report is a reminder that misuse is real and detectable; the safeguard against it is a culture that refuses to build what it should not, backed by policy that makes that refusal enforceable.

Frequently asked questions

What is AI surveillance? It is the use of machine learning to monitor, profile, or track people at scale by aggregating open data, recognising faces or behaviour, and automating analysis that once required teams of human analysts.

Why do AI labs restrict surveillance use of their models? Because mass profiling and covert monitoring create serious human-rights and security harms. Anthropic's threat intelligence report documented and disrupted surveillance operations abusing frontier models, and providers ban such use in their acceptable-use policies.

What are the main AI privacy risks for organisations? Over-collection of personal data, unlawful profiling, function creep beyond the original purpose, and the concentration of sensitive intelligence in one place, which becomes a high-value target if the surveillance data itself is breached.

How can a CISO or DPO safeguard against AI surveillance risk? Apply data minimisation and privacy-by-design, run DPIAs before deployment, enforce purpose limitation and access controls, conduct vendor due diligence, and adopt a clear AI governance and acceptable-use policy aligned to GDPR.

Does GDPR cover AI-driven profiling and monitoring? Yes. GDPR requires a lawful basis, transparency, purpose limitation, and data minimisation, and it grants people rights around automated decision-making and profiling. A DPIA is generally required for large-scale monitoring.