AI Influence Operations: Spotting LLM Disinformation

Influence operations are as old as propaganda, but generative AI changes their economics. When a single operator can spin up fluent, native-sounding content in a dozen languages and run hundreds of personas at once, the old detection instinct of "spot the bad grammar" stops working. Anthropic's recently published threat intelligence report documents exactly this shift, and this explainer walks through how AI-scaled influence operations actually work, the signals that still give them away, and how organizations, communications teams and platforms defend against them.

What Anthropic's threat intelligence report tells us

Anthropic's threat intelligence report describes how the company detected and disrupted a range of real-world attempts to misuse its models, spanning cyberattacks, influence operations, surveillance tooling, and prohibited biology and weapons work. Every operation covered was disrupted. The report is not a how-to; it is a defender's account of the patterns that recur when adversaries reach for a capable language model. For influence operations specifically, the value for defenders is the anatomy: what the model was asked to do, where it added leverage, and which behavioral fingerprints exposed the activity.

The headline lesson is that AI rarely invents a new kind of attack. It compresses the cost and time of an existing one. A disinformation campaign that once needed a room full of writers can be attempted by a small team, and the tells move from the content itself to the coordination around it.

What AI actually adds to an influence operation

It helps to be precise about the leverage, because that is what defenders learn to detect. Across documented cases, models are used to add:

• Volume. Thousands of posts, comments and replies that would otherwise be impractical to write by hand.
• Fluency. Native-quality prose that removes the grammar and idiom errors defenders historically relied on.
• Translation and localization. The same narrative adapted convincingly across many languages and cultural registers.
• Persona realism. Consistent backstories, tones and posting habits that make fake accounts, or "sockpuppets," read as distinct people.
• A/B-tested messaging. Rapid generation of framing variants so operators can iterate toward whatever emotional hook performs.

Notice what is not on the list: novel ideas, real audiences, or authentic engagement. The AI produces supply. It cannot manufacture genuine demand, and that gap is where detection lives.

Typical patterns and the tactics behind them

AI-assisted campaigns tend to reuse a familiar playbook, now running faster and wider:

• Sockpuppet networks. Clusters of fake accounts posing as ordinary citizens, journalists or enthusiasts, each with a generated persona.
• Astroturfing. Manufactured "grassroots" consensus, where many synthetic voices amplify one talking point to make a fringe position look mainstream.
• Fabricated media. AI-generated images, audio or video used to lend false events a sense of documentary proof.
• Coordinated inauthentic behavior. Accounts that appear independent but post, boost and reply in tightly synchronized ways.

The through-line is coordination pretending to be spontaneity. That pretense is fragile, and it leaves seams.

Detection signals that still work

Because AI improves the content but not the underlying orchestration, the durable signals are behavioral and structural rather than linguistic:

• Coordination. Many accounts sharing followers, links, hashtags or near-identical phrasing at scale.
• Timing. Bursts of activity aligned to a single time zone or work schedule, or unnatural synchronization across supposedly unrelated accounts.
• Template reuse. Structural sameness beneath surface variety, the same argument skeleton dressed in different words.
• Provenance gaps. Personas with no verifiable history, recycled or generated profile photos, and media that carries no credible origin trail.
• Engagement asymmetry. High output paired with little authentic reciprocal interaction from real communities.

No single signal is proof. Investigators build confidence by stacking several, then corroborating with metadata and platform-side telemetry that individual users never see. Practicing the instinct on realistic examples helps; interactive drills such as PlayCISO's Pick the Phish and the friend request analyzer train the same muscles of scrutinizing personas and pressure tactics that disinformation exploits.

How organizations, comms teams and platforms defend

Defense is layered. No one control stops a determined operation, but together they raise cost and shrink reach.

Media literacy and human judgment

The first defense is a workforce and audience that pauses before amplifying. Teach people to check provenance, distrust emotionally engineered urgency, and treat a screenshot as a claim rather than evidence. Media literacy scales further than any detector because it degrades the payoff of every campaign at once.

Provenance, watermarks and content authenticity

Where content originates matters more than how polished it looks. Content authenticity standards such as C2PA (the Coalition for Content Provenance and Authenticity) attach tamper-evident metadata describing how a piece of media was created and edited. Provenance checks and durable watermarking do not catch everything, and metadata can be stripped, but they give newsrooms, platforms and the public a verifiable signal to lean on. Detection tools that classify AI-generated text remain useful as one input, though defenders should treat their output as probabilistic, not a verdict.

Platform trust and safety practices

Platforms defend at the network level, where the coordination is visible. Effective trust-and-safety work clusters accounts by shared infrastructure and behavior, rate-limits suspicious automation, requires provenance on synthetic media, and removes coordinated inauthentic networks as a unit rather than one account at a time. Publishing takedown transparency reports also feeds the wider defender community, which is how many operations, including those in Anthropic's report, ultimately surface.

Monitoring your brand and executives

Every organization is a potential target for impersonation. Comms and security teams should monitor for spoofed executive accounts, fabricated statements, and narrative attacks against the brand. Watch for sudden coordinated criticism, cloned social profiles, and AI-voiced audio purporting to come from leadership. Early detection buys the time to respond before a false narrative sets.

Incident response for disinformation

Treat a disinformation event like any other incident: with a plan written before you need it. A workable playbook names who verifies claims, who decides whether to respond publicly, and how you preserve evidence. Sometimes the right move is a factual correction through owned channels; sometimes amplifying a falsehood by rebutting it does more harm than silence. Decide the threshold in advance, coordinate with platforms and, where warranted, law enforcement, and document everything for post-incident review.

A realistic posture

AI lowers the cost of producing persuasive content but not the cost of building genuine trust. Operations still need authentic distribution they cannot fabricate, and that dependency is the defender's advantage. The organizations that fare best combine skeptical people, provenance-aware tooling, active brand monitoring and a rehearsed response plan. Anthropic's threat intelligence report is encouraging on one point in particular: every operation it describes was detected and disrupted. The threat is real and scaling, but it is not invisible, and coordinated defense continues to work.

Frequently asked questions

What are AI influence operations? They are coordinated campaigns that use AI, especially large language models, to mass-produce persuasive content and operate fake personas at scale. The goal is to manipulate public opinion by making manufactured consensus look organic. AI mainly adds volume, fluency and multilingual reach rather than inventing new tactics.

How does AI make disinformation harder to detect? Models remove the grammar and translation errors defenders once relied on, and they let one operator run hundreds of convincing personas. This shifts the tells away from the content and toward behavior. Coordination, timing and provenance gaps become the signals that matter, because AI improves the writing but not the orchestration behind it.

What are LLM sockpuppets? Sockpuppets are fake online identities that pretend to be independent, real people. When powered by an LLM, each account can maintain a consistent persona, backstory and tone across thousands of posts and multiple languages. Networks of them are used to fake grassroots support, a tactic known as astroturfing.

How can I detect AI-generated content? No detector is perfect, so combine methods. Check provenance and content authenticity signals such as C2PA metadata and watermarks, look for behavioral tells like coordination and template reuse, and treat AI-text classifiers as one probabilistic input rather than a verdict. Confidence comes from stacking several signals and corroborating them.

What does Anthropic's threat intelligence report say about influence operations? The report documents real attempts to misuse Anthropic's models across cyberattacks, influence operations, surveillance, biology and weapons, and states that every operation was disrupted. For influence operations it details how models were used to add scale and fluency, and which behavioral fingerprints exposed the activity. It is a defensive account, not an operational guide.