On-Device / Edge AI Security Checklist
Can you actually trust an AI feature that runs on the device — a local LLM notetaker, an on-device assistant, a background summariser on a laptop or phone? “On-device” is a security claim, not a guarantee. This checklist works through the controls that decide whether it holds. Rate yourself across data locality, model provenance, update & patching, telemetry & egress, permissions & scope, local storage protection and regulatory classification. You get a banded 0–100 readiness score, a radar of where you are strong and weak, and the gaps ranked so you fix the highest-impact ones first. It is generic and vendor-neutral (for example, it suits an on-device notetaker such as Google AI Edge Foresight), the weighting is a transparent heuristic, nothing is uploaded, and the score is shareable by permalink.
You are trusting an on-device AI feature on faith. You have not confirmed what stays local versus what leaves, where the model came from, or what it can see and store. Start with data locality and telemetry — establish, with evidence, what actually leaves the device — before relying on the "on-device" promise.
Score per dimension (0–100)
Prioritised gaps
Highest-weight, lowest-scoring controls first — spend remediation effort here.
- Data locality · unanswered
Document the actual data boundary: which inputs (audio, screen, files, text) are processed locally and which, if any, are sent off-device.
Reference: local vs remote processing
- Data locality · unanswered
Map every path where data leaves the device — cloud fallback, cross-device sync, a hosted model for hard queries — and the trigger for each.
Reference: cloud fallback / sync
- Model provenance · unanswered
Identify the model's source and decide whether that origin is one you trust to ship code that runs on your device.
Reference: model source
- Model provenance · unanswered
Confirm the model artifact is signature- or hash-verified before it loads; an unverified local model can be swapped for a malicious one.
- Telemetry & egress · unanswered
Inventory the telemetry the feature emits and confirm it carries metadata, not the transcripts/screen content the feature was reading.
Reference: analytics / crash reports
- Telemetry & egress · unanswered
Review and default-off any content-contribution / model-improvement option; it is the one path that can send real user content off a local feature.
Reference: model-improvement opt-in
- Update & patching · unanswered
Document the update path and cadence for both the model and the app; a device-resident model with no update story ages into a liability.
- Update & patching · unanswered
Ensure model/app updates are signed and verified; an unauthenticated update channel is a direct path to running attacker code locally.
+ 16 more gaps in the full export.
Data locality · weight 20
The core promise of on-device AI is that your data stays on the device. Does it? Establish exactly what is processed locally, what (if anything) is sent to a server, and whether that boundary is a verified fact or a marketing claim you took on trust.
You have established what the feature processes entirely on-device versus what it sends to a remote service — not assumed "on-device" means "nothing ever leaves".
Any data that does leave the device (e.g. for a cloud fallback, sync or a larger model) is identified, and you know when and why it happens.
The on-device claim is verified by evidence (vendor documentation, a network capture, or a privacy statement you have actually read) rather than taken from a feature name or ad.
You understand what the feature can see on the device — microphone, screen contents, open documents, clipboard — and that this is the raw material it processes locally.
Model provenance · weight 16
A model file that runs on your device is executable content you are trusting. Where did it come from, is it signed and integrity-checked, and could a tampered or swapped model file change the feature's behaviour without you knowing?
You know where the local model came from — the vendor, an app store, or a third-party/community source — and trust that origin.
The model file is signed and/or integrity-checked (hash/signature verification) so a tampered or substituted model is detected.
You are not side-loading models from unvetted community repositories without any provenance or scanning.
Update & patching · weight 14
The on-device model and its host app are software that needs patching. How do updates reach the device, are they authenticated, and can a known-vulnerable local model or runtime linger because nothing forces it to update?
There is a defined mechanism for updating the on-device model and its host app, and you know how and how often updates arrive.
Updates are authenticated and integrity-checked, so a malicious "update" cannot replace the model or app.
You can tell what model and app version is actually running on the device, and whether it is current — not stuck on a vulnerable build.
Telemetry & egress · weight 16
Even a genuinely on-device feature often phones home — analytics, crash reports, usage metrics, "improve the model" opt-ins. What leaves the device as telemetry, is it separable from the content, and can you turn it off?
You know what telemetry the feature sends off-device — analytics, crash reports, usage metrics — separately from the user content it processes.
Any "help improve the model" or content-sharing option is understood, defaulted appropriately, and off unless deliberately enabled.
Telemetry and egress can be disabled or constrained (settings, MDM policy, or a network control) where your policy requires it.
You have observed or verified the feature's actual network destinations, not just trusted the settings labels.
Permissions & scope · weight 14
On-device assistants ask for powerful access — microphone, screen recording, accessibility, files, system-wide capture. Is each grant necessary and scoped, or has the feature been handed the keys to everything on the device?
Each OS permission the feature holds (microphone, screen capture, accessibility, files, notifications) is justified by a function it actually needs.
Broad or system-wide capture permissions (full-screen recording, accessibility control) are recognised as high-risk and granted only when truly required.
The feature's access is scoped to the user/context it serves, not running with standing access to everything regardless of who is at the device.
Local storage protection · weight 12
On-device AI leaves artifacts behind: cached transcripts, summaries, vector embeddings, prompt history, model files. Those sit on local disk — are they encrypted, access-controlled, and cleared, or are they a plaintext record anyone with the device can read?
Cached outputs — transcripts, summaries, prompt history — are stored encrypted (via full-disk/app encryption), not as plaintext files on local disk.
Derived data such as vector embeddings or an on-device index is recognised as sensitive (it can reconstruct or reveal source content) and protected accordingly.
There is a way to clear or expire the local AI data, and it is actually used — cached content does not accumulate indefinitely on the device.
Local AI artifacts are covered by the device's loss/theft response (remote wipe, encryption-at-rest) so a stolen device does not expose them.
Regulatory classification · weight 8
Processing on the device does not exempt it from the rules. If the feature touches personal, health, financial or other regulated data — even locally — that processing is in scope, and "it never left the device" is a control, not a free pass.
You have classified whether the feature processes personal or regulated data (PII, health, financial, special-category) — even though processing is local.
Local processing is accounted for in your privacy/compliance posture (records of processing, DPIA, consent) rather than assumed out of scope because "it's on-device".
You have considered sector/residency rules that may apply even to local processing (e.g. recording consent, workplace monitoring, meeting-capture laws).
Educational self-assessment. The dimension weights are a transparent judgement framework for structured self-reflection, not a measurement, certification or guarantee of outcome. This is a generic checklist for any on-device / edge AI feature; Google AI Edge Foresight is named only as one example of such a feature, and PlayCISO is not affiliated with Google, Apple, Microsoft or any vendor. This tool performs no scan and runs entirely in your browser — nothing is uploaded. Your answers are stored only in the shareable link you choose to copy.
What this checklist is
- A free, 100% in-browser security checklist for any ON-DEVICE / EDGE AI feature — a local LLM notetaker, an on-device assistant, a background summariser running on a laptop or phone. Answer a weighted checklist across seven dimensions and get a banded 0–100 readiness score.
- The point: "on-device" is a security claim, not a guarantee. The checklist works through whether the claim holds — what stays local, what actually leaves, where the model came from, what it can see, and what it leaves behind on disk.
- Seven dimensions: data locality, model provenance, update & patching, telemetry & egress, permissions & scope, local storage protection, and regulatory classification.
- Every item maps to a concrete next action, and gaps are prioritised lowest-score, highest-weight first — so you spend effort where it most improves the trustworthiness of a local AI feature.
- The weighting is a transparent HEURISTIC for structured self-reflection, not a measurement or certification. Generic and vendor-neutral — Google AI Edge Foresight is named only as one example of the kind of on-device notetaker it suits. PlayCISO is not affiliated with any vendor. Nothing is uploaded — export to Markdown or CSV, and share the exact checklist with a base64url permalink.
“On-device” is a control to verify, not a free pass
Edge and on-device AI is marketed on privacy: the model runs locally, so your microphone audio, screen contents and documents never leave the device. That is a real and valuable property — but it is a claim to verify, not a guarantee to assume. A feature that genuinely does most of its work on-device can still fall back to a cloud model for hard queries, sync across your devices, send analytics and crash reports home, and offer a “help improve the model” option that quietly ships real content off the box. The decisive question is not “does it say on-device?” but what actually leaves, when, and where to — which is why data locality and telemetry/egress carry the most weight here.
The rest is the local attack surface a device-resident model creates. The model file is executable content you are trusting, so its provenance and signing matter, and it needs a real update path or it ages into a known-vulnerable liability. On-device assistants ask for powerful permissions — microphone, screen recording, accessibility — that should be least-privilege, not blanket. And the feature leaves artifacts behind: cached transcripts, summaries and vector embeddings on local disk that must be encrypted and in scope for loss/theft response, because a stolen laptop is a data breach if they are plaintext. Finally, local processing of personal or regulated data is still regulated processing — meeting-recording consent and workplace-monitoring rules apply regardless of where the AI runs. The seven weighted dimensions are a heuristic, shown openly so you can tune them to your own environment. It is a self-assessment, not a certification, and it quotes no invented figures.
Frequently asked questions
What does the On-Device / Edge AI Security Checklist assess?
It assesses whether you can actually trust an AI feature that runs locally on a device — a local LLM notetaker, an on-device assistant, a background summariser on a laptop or phone. You rate seven weighted dimensions: data locality (what stays on the device versus what leaves), model provenance (where the local model came from and whether it is signed), update & patching (how the on-device model and app are updated), telemetry & egress (what the feature phones home), permissions & scope (mic, screen, files, accessibility access), local storage protection (encryption of cached transcripts and embeddings), and regulatory classification (whether it processes regulated data). You get a weighted 0–100 readiness score, a band, a radar and a prioritised gap list.
Is this tied to a specific product like Google AI Edge Foresight?
No. It is a generic, reusable checklist for any on-device or edge AI feature. Google AI Edge Foresight is named only once, as a factual example of an on-device notetaker the checklist suits — if you are evaluating Foresight, our blog on its security settings is a good companion and is linked below. PlayCISO is not affiliated with Google, Apple, Microsoft or any other vendor, and the checklist applies equally to any local assistant, meeting recorder or on-device model.
If a feature runs entirely on-device, is it automatically private and safe?
No, and that is the whole reason the checklist exists. On-device processing reduces the risk of data being transferred to a server, but the feature can still send telemetry or content back for analytics or model improvement, hold powerful permissions (microphone, screen capture, accessibility), run a model of unknown provenance, and leave cached transcripts and embeddings in plaintext on local disk. "On-device" is a control to verify, not a free pass — a local feature still processes regulated data, and a lost or stolen device can still expose everything it cached.
Does it scan anything, connect to any vendor, or send my answers anywhere?
No. The checklist runs entirely in your browser. It performs no scan, makes no API call, and touches no device. Your answers are never uploaded; they live only in the shareable permalink you choose to copy (encoded in the URL) and in any Markdown or CSV file you export yourself.
Is the score a real measurement or a certification?
No. It is a transparent heuristic framework for structured self-reflection, not a measurement, benchmark, audit or certification, and it predicts no outcome. The weights (data locality and telemetry/egress carry the most, because the entire promise of on-device AI is that data stays on the device) are shown openly so you can adjust them to your own environment. The value is the structured conversation and the prioritised to-do list, not the exact number.
This tool is free and always will be. Would a Pro workspace — saved designs, team sharing and exports — be worth paying for?
Show me Pro →No account needed to use this tool — this just tells us whether Pro is worth building out.