Shipping Without Saying No: The First 180 Days in AppSec and AI Security

The fastest way to lose a development team is to run a scanner on day one and send them a spreadsheet of findings. They did not ask for it, they cannot tell which rows matter, and they now know you as the person who makes work. Hold the scanner for a week.

Your first job is to understand how software gets made here. That knowledge lives in people, not in a wiki. Book short, informal conversations with a handful of engineers from different teams. Ask them to show you, not tell you.

Who to meet in week one

• A senior engineer on the busiest product team. They know where the bodies are buried.
• Whoever owns the build and deploy pipeline. Often a platform or DevOps team.
• The engineering manager who is under the most delivery pressure.
• Someone building an AI feature, if there is one. These teams move fast and often have no security contact yet.
• Whoever answered security questions before you arrived. They may be relieved, or protective. Either way, they know the history.

Questions that open people up

• What part of the codebase are you most nervous about?
• When did a security request last slow you down, and was it worth it?
• What would you fix if you had a free week?
• Where do secrets and config live?
• What would you like from a security person that you are not getting?

Write down what you hear, word for word where you can. The phrases engineers use for their own worries are the phrases you will later use to explain why a fix matters. A finding described in their words gets fixed faster than one described in scanner language.

What you are really learning

Behind every conversation are three questions. How does a change go from an idea to production? Who can approve it? Where would an attacker most like to be? You do not need full answers yet. You need a sketch you can improve over the next month.

Rule of thumb:Rule of thumb: for every hour you spend with a tool in your first month, spend an hour with the people who will act on what it finds.

By Friday, aim to have a short list of names, a rough picture of the teams and two or three worries you heard more than once. That list is your first backlog, and it came from them, which makes it far easier to work on together.

The build and deploy pipeline is the most important system you will learn this month. Every line of code passes through it. Every check you add will run in it. And if someone can change what it does, they can ship anything to production with your company name on it.

Ask the pipeline owner to walk you through one real change, from a pull request to running in production. Take notes as you go. Do not try to fix anything yet.

Draw the path

1. Where is the source code hosted, and who can push to the main branch?
2. What runs when a pull request is opened? Tests, linting, any security checks?
3. Who must approve a merge, and can that rule be bypassed?
4. Where are build artefacts stored, and are they signed or checked?
5. What credentials does the pipeline hold, and what can each one reach?
6. Who can deploy to production, and is there a record of each deploy?

You will probably find more than one pipeline. Older services often build differently from newer ones, and a team may have its own scripts. Note each one. The old one is often the risky one.

What to look for

• Long-lived secrets stored as pipeline variables that every job can read.
• Third-party actions or plugins pulled by a moving tag rather than a fixed version.
• Jobs triggered by pull requests from forks that run with write access or secrets.
• Branch protection that admins can skip, and do.
• No link between what was reviewed and what was deployed.

Where checks belong

Fast checks that rarely produce noise, such as secret detection, can run on every pull request. Slow or noisy checks belong on a schedule or on the main branch, with results sent to a place the team already looks. A check that adds minutes to every build and cries wolf will be turned off quietly, and nobody will tell you.

Rule of thumb:Ask before adding a blocking check. A check that fails builds needs an owner, a way to suppress a false positive and a person to call when it breaks at 6 pm.

By the end of the week, you should have a simple diagram of each pipeline with the credentials and approval points marked. Share it with the pipeline owner and ask what you got wrong. You will learn more from the corrections than from the drawing.

You cannot protect what you do not know exists. In most organisations nobody has a complete list of what is running in production. That is normal. Your job in week one is to make a list that is useful, not one that is complete.

Start with four questions

1. What can someone on the internet reach? Websites, APIs, mobile back ends, admin panels.
2. What handles the most sensitive data? Payments, health, identity, customer files.
3. What third-party code does it depend on? Open source packages, container images, SaaS integrations.
4. Which features call an AI model, and what can that model see or do?

Use what already exists. Cloud accounts list load balancers and domains. The source host lists repositories. Package lock files list dependencies. DNS records show subdomains people forgot. You are stitching together sources, not starting from nothing.

Shadow features

Expect to find things nobody told security about. A chatbot built in a hackathon that is now live. A test environment with real data. An old API version still answering requests. Treat these calmly. If the first person to tell you about a forgotten system gets blamed, nobody will tell you about the next one.

Keep the list small and alive

• One row per service or feature, with an owner name, not a team alias.
• Whether it is internet-facing, and what data it handles.
• Whether it calls an AI model or runs an MCP server, and which one.
• Last time anyone looked at it from a security point of view.

Rule of thumb:A list with an owner on every row is worth more than a long list with no owners. When something breaks, the owner column is the one you will read first.

Share the list with the engineering leads and ask them to correct it. They will. Every correction is a conversation, and every conversation teaches you which services matter most to the people who run them.

Threat modelling has a reputation for being slow, formal and full of diagrams. It does not need to be. The most useful threat models are short conversations with the people building a feature, held early enough that changes are still cheap.

The four questions

1. What are we building? Draw the feature: users, services, data stores and the lines between them.
2. What can go wrong? Walk each line and ask who could abuse it, and how.
3. What are we going to do about it? Pick fixes, accept risks or move them to the backlog.
4. Did we do a good job? Agree when you will look again.

Running the hour

• 10 minutes: an engineer draws the feature. You ask questions and do not correct.
• 25 minutes: walk the drawing. Where does untrusted input enter? Where does data cross a trust boundary? Who checks permissions, and where?
• 15 minutes: list what could go wrong and sort it into fix now, fix later and accept.
• 10 minutes: name an owner for each fix and write it in the team’s own tracker.

For prompts while walking the drawing, use a known list. The OWASP Top 10 for web apps, the OWASP API Security Top 10 for APIs and the OWASP Top 10 for LLM Applications for AI features all give you a vocabulary the team can look up later.

What not to do

• Do not bring a finished threat model and ask them to sign it. They will not own it.
• Do not list every possible threat. Ten real ones beat fifty theoretical ones.
• Do not file the results in a security-only tool. Use the backlog the team already works from.

Rule of thumb:If you can only threat model one feature this month, choose the one that handles sensitive data and ships soonest. Late changes cost more, and sensitive data raises the stakes.

Keep a copy of each drawing. Over a few months they become a map of your systems that is more current than any architecture document, because the people who built the systems drew them.

The first time you run a scanner across a real codebase, the number of findings will be large. That number is not a measure of risk. It is a measure of how many patterns the tool matched. Your value is in what you do between the scanner and the developer.

Sort before you send

1. Group duplicates. One vulnerable library used in thirty places is one finding with thirty locations.
2. Drop what does not apply. Test code, dead code and unreachable paths go to a separate list, not to developers.
3. Rank what is left by exposure and impact: internet-facing beats internal, sensitive data beats none.
4. Check for known exploitation. A flaw listed in a public catalogue of exploited vulnerabilities moves to the top.
5. Confirm the top few by hand before you send them.

Only then open tickets, and only for the items you would defend in a meeting. A developer who gets three accurate findings will fix them. A developer who gets three hundred mixed with noise will learn to ignore the source.

False positives

Every scanner produces them. Treat each one as a tuning job, not an embarrassment. Suppress it with a written reason that names who decided and when, so the next person does not re-raise it. If a rule produces mostly noise, turn it off and note why.

Make fixing easy

• Say where the problem is, why it matters here and what the fix looks like.
• Link to the exact line and, if you can, a sample patch.
• Agree a fix-by date with the team, based on severity and exposure.
• Close the loop. When a fix ships, say thank you where the team can see it.

Rule of thumb:Track the size of your untriaged pile separately from the developers’ backlog. The first is your work. The second is theirs. Mixing them makes both look worse than they are.

Over the first month, keep notes on which rules and which tools produce the findings developers actually fix. That record tells you where to tune, which checks to keep and which to retire, and it gives you something concrete to show when someone asks why you did not forward everything.

Much of what your company ships is code it did not write: open source packages, container base images, build plugins and, increasingly, AI models. The OWASP Top 10:2025 lists Software Supply Chain Failures as its own category. That matches what most AppSec teams see day to day.

Secrets first

A leaked key is one of the few findings an attacker can use immediately, with no exploit to write. Start here because the fix is clear and the risk is concrete.

1. Scan the current code and the full git history. Deleting a secret in a later commit does not remove it from history.
2. For every real secret found, rotate it first. Removing it from code comes second.
3. Add secret detection to pull requests so new leaks are caught before merge.
4. Agree where secrets should live instead, and make that path easy to use.

Rule of thumb:Rotate, then verify the new secret works, then revoke the old one. Never revoke first. A rotation you cannot check is an outage you planned.

Dependencies

• Know which packages each service uses. A lock file is the start of a software bill of materials.
• Pin versions and review updates rather than pulling whatever is newest.
• Watch for packages with names close to popular ones, and for packages that run install scripts.
• Prefer fewer, well-maintained packages over many small ones that do one thing.

The path to production

Supply chain security also covers your own build. Can someone change a build step without review? Can a compromised plugin read every secret? Is what you deploy the same thing that was reviewed and tested? These questions connect back to the pipeline map from your first week.

You will not finish this in a month. Aim to have secret detection running, a plan for rotating anything found and a first dependency inventory for your most exposed services.

Write down who owns each of these jobs. Secret rotation often belongs to the service team, dependency updates to developers and the build to the platform team. Your role is to make the gaps visible, suggest the simplest fix and check back. A short weekly note on what changed keeps everyone moving without a meeting.

When a team adds a large language model to a product, they add a component that follows instructions written in plain language. The problem is that it cannot reliably tell your instructions apart from instructions hidden in the text it reads: a user message, a web page, an email, a document or a tool result. That is prompt injection, and the OWASP Top 10 for LLM Applications 2025 lists it first, as LLM01.

Direct and indirect

• Direct: a user types instructions meant to override the system prompt.
• Indirect: the model reads content that someone else wrote, such as a support ticket or a web page, and that content carries the instructions.

Indirect injection is usually the bigger risk, because the person harmed is not the person who wrote the attack.

Limit the blast radius

1. List what the feature can read. Does it see data the current user should not?
2. List what it can do. Send email, call APIs, change records, run code?
3. Give it only the permissions of the user it is acting for, never a shared admin key.
4. Require a human to confirm actions that are hard to undo, like payments or deletions.
5. Treat its output as untrusted input. Escape it before showing it, and validate it before acting on it.

Several items on the OWASP LLM list follow from this: Sensitive Information Disclosure (LLM02), Improper Output Handling (LLM05), Excessive Agency (LLM06) and System Prompt Leakage (LLM07). If you design for injection, you cover a large part of the list.

Questions to ask the AI team

• What is in the system prompt, and would it matter if a user saw it?
• Which outside content can reach the model?
• What tools can it call, and with whose permissions?
• Is there a limit on usage per user, so one person cannot run up the bill?

Rule of thumb:Never put secrets, keys or internal rules you need to stay private in a system prompt. Assume a determined user will read it.

AI features bring new kinds of third-party parts: model files downloaded from a hub, datasets, and MCP servers that let an AI assistant call tools. Treat each one the way you treat an open source package. Know where it came from, what it can do and who owns it.

Model files

Some model formats can run code when loaded. Hugging Face’s own documentation says that loading a pickle file can lead to arbitrary code execution, and suggests loading models only from sources you trust or using safer formats such as safetensors. So a model download is not just data. It can be a program.

• Prefer formats that store weights only, not code.
• Record where each model came from, which version and who approved it.
• Load untrusted models in an isolated environment, never on a developer’s main machine.

MCP servers

The Model Context Protocol (MCP) lets AI assistants connect to tools and data. A local MCP server is a program that runs with the user’s permissions. The protocol’s own security guidance describes risks such as malicious startup commands in a client configuration, token passthrough and over-broad scopes.

1. Find out which MCP servers developers have installed, and where they came from.
2. Check what each one can reach: files, network, cloud accounts, source code.
3. Ask for least privilege: narrow scopes, no shared admin tokens.
4. Treat a new MCP server like any new dependency, with an owner and a review.

An AI bill of materials

An AI bill of materials (AIBOM) lists the models, datasets and AI components a system uses, the way a software bill of materials lists packages. CycloneDX, for example, supports a machine learning bill of materials covering models, datasets, their dependencies and provenance. You do not need a perfect one in month one. Start with a table: model, version, source, where it runs, what data it sees, owner.

Rule of thumb:If a team cannot tell you which model version is in production, start there. You cannot respond to a problem with a model you cannot name.

At some point you will find a vulnerability in production that you reviewed, or should have. It might come from a researcher, a customer, a scanner or your own second look. The feeling is the same: a drop in the stomach and an urge to fix it quietly. Resist the second part.

The first hour

1. Work out the exposure. Is it reachable from the internet? Does it touch sensitive data?
2. Look for signs of use. Check logs for requests that match the flaw.
3. Tell your manager and the owning team now, not after you have a fix.
4. If there is any sign of exploitation or data exposure, start the incident process.
5. Agree the fastest safe fix: a patch, a config change, a feature flag or turning the feature off.

Telling people

Keep the first message short and factual. What you found, where, how exposed it is, whether you see signs of use and what happens next. Avoid guessing. If you do not know yet, say so and say when you will know more.

Rule of thumb:Write down times as you go: when you learned, who you told, what you checked. It feels like a distraction during the work. It saves hours when people ask questions later.

After the fix

• Confirm the fix in production, not just in the pull request.
• Search for the same pattern elsewhere. Flaws rarely live alone.
• If someone outside reported it, thank them and tell them when it is fixed.
• Ask why it got through. Missing test, unclear ownership, rushed review? That goes in the review, not on a person.

Whether customers or regulators must be told depends on the data, the law where you operate and your contracts. That decision belongs to legal and leadership, with your facts. Your job is to give them accurate facts fast.

Finally, look after yourself and the team. Fixing a live flaw under pressure is tiring. Once it is contained, say thank you to the people who helped, hand over anything that can wait and get some rest. Good judgement in the next incident depends on it.

An AI feature can leak data in ways that look unfamiliar: it repeats part of its system prompt, summarises a document the user should not see, or follows instructions hidden in a web page and sends data somewhere. The response steps are the familiar ones. The questions you ask along the way are a little different.

Contain first

1. Turn the feature off or put it behind a flag for everyone, or for affected users, while you investigate.
2. Keep the logs: prompts, responses, tool calls and what data was retrieved.
3. Revoke or narrow any credential the feature used that may have been misused.
4. Start the incident process if personal or customer data may be involved.

Questions specific to AI

• What could the model read at the time? Which data sources, with whose permissions?
• What could it do? Which tools or APIs could it call?
• Was the trigger a user message, or content from somewhere else?
• Can you reproduce it? Model output varies, so try more than once.
• Which model and version was running? Did anything change recently?

Fix the design, not just the wording

Most AI data leaks come down to a feature that could see more than the user it served. If the model only ever retrieves data the current user is allowed to see, a clever prompt cannot reveal someone else’s records. That is the same access-control principle as any other feature, applied to a new component.

Rule of thumb:Keep prompts and responses from an incident under the same access rules as the data they contain. Incident evidence can itself be sensitive.

When you write it up, record the model version, the data sources and the exact trigger. AI incidents are new for most organisations, and a clear write-up becomes the reference the next team reads.

Tell the people who own the feature what you found in plain words, and agree together what has to be true before it is switched back on. Writing those conditions down stops the feature returning quietly with the same gap, and it gives the team a clear finish line instead of an open-ended security review.

After an incident, someone will ask what went wrong. If the answer is a person’s name, you will learn nothing useful, and people will start hiding the next mistake. A blameless review assumes people acted reasonably given what they knew, and asks why the system made the wrong action easy.

What blameless does not mean

It does not mean nobody is accountable, and it does not mean pretending nothing went wrong. It means the review looks for causes you can fix: missing tests, unclear ownership, confusing tools, time pressure, alerts nobody watched. Deliberate harm is a different process, and rare.

Running the meeting

1. Write a timeline first, from logs and notes, and share it before the meeting.
2. Invite the people involved, and say plainly that the goal is learning.
3. Walk the timeline. At each decision, ask what the person knew and what they saw.
4. Ask what made the mistake easy and the right action hard.
5. Agree a small number of actions, each with an owner and a date.

Words that help

• Ask what happened, not why you did it.
• Say the change was deployed rather than naming who deployed it.
• Avoid should have. Hindsight makes every decision look obvious.
• Thank people for being open, out loud, in the meeting.

Rule of thumb:Fewer actions that get done beat a long list that does not. Two or three changes with owners are a good result.

Share the write-up widely, in plain words. Other teams learn from your incident at no cost, and engineers see that security responds to mistakes by improving systems, not by finding someone to blame. That reputation is worth more than any tool you will buy this year.

Come back to the actions a month later. Check which ones were done, which stalled and why. A review whose actions never happen teaches people that reviews are theatre. A review whose actions visibly land teaches them that telling the truth leads to real change, which is the whole point of doing it without blame.

Around month two, someone will ask how AppSec is going. The tempting answer is a big number: findings raised, scans run, tickets opened. Those numbers go up when you turn on a noisy tool and down when you tune it, and neither change says anything about risk.

Measures that mean something

Ask the developers too

Some of the most useful signals are not in any tool. Do teams invite you to design reviews? Do engineers report suspicious things to you? Do they argue less about findings than they did in month one? Ask a few of them, every month or two, what is working and what is in their way.

Reporting upward

• Lead with a sentence on exposure: what is most at risk right now, and what you are doing about it.
• Show two or three measures as trends, not snapshots.
• Name one thing you need: a decision, an owner or time from a team.
• Keep it to one page. Detail goes in an appendix nobody has to read.

Rule of thumb:Every measure can be gamed. Pick a few, explain how each could mislead and look at them together rather than one at a time.

Agree the definitions with the teams before you publish anything. When does the clock start: when a scanner flags something, or when a person confirms it? What counts as fixed: merged, or running in production? Settling these early stops arguments later and makes the numbers something engineers recognise rather than something done to them.

By month four you have a pipeline map, an inventory, a triage habit and some trust with engineers. You cannot do everything next. Pick one programme, based on what the first three months showed you, and do it properly.

Path 1: security champions

A champions programme gives each development team a person with an interest in security, some extra training and a direct line to you. It scales you across teams you cannot attend every week. OWASP runs a Security Champions Guide project for organisations building one.

• Choose this if you have many teams and not enough of you.
• Make it voluntary, give champions real time for it and recognise their work in front of their managers.

Path 2: AI red team

If your products depend on AI features, test them the way an attacker would: try to make them reveal data, ignore rules, call tools they should not or produce harmful output. Use the OWASP Top 10 for LLM Applications as a starting checklist.

• Choose this if AI features are central to the product or growing fast.
• Agree scope and rules in writing first, and report findings the same way as any other.

Path 3: supply-chain security

Go deeper on what you started in month one: a bill of materials for key services, signed builds, reviewed updates and an approved list for model files and MCP servers.

• Choose this if your inventory showed many dependencies, unclear build steps or a lot of downloaded models.

Path 4: API security

APIs carry most of the data in modern products. The OWASP API Security Top 10 2023 puts Broken Object Level Authorization first: one user reading another user’s records by changing an ID. These flaws are rarely caught by generic scanners and need testing that understands the business logic.

• Choose this if your inventory is mostly APIs, or partners and mobile apps consume them directly.

Rule of thumb:Whichever path you choose, write down what success looks like at month six before you start, so you can tell whether it worked.

Application security has done a lot of good. It also has habits that make engineers wary of it. You will be tempted by each one, usually under pressure. Knowing them by name makes them easier to resist.

Blocking releases

Stopping a release feels like the strongest thing a security person can do. Sometimes it is right. More often it teaches teams to bring you in late, or not at all. If you find yourself wanting to block a release, ask why you are only hearing about this now. That is usually the real problem.

• Prefer a fix with a date over a block.
• If you must block, explain the risk in plain words and offer the smallest change that would unblock it.
• Afterwards, find out how to get involved earlier next time.

Tool sprawl

Each new scanner promises to find what the others miss. Each also brings its own dashboard, its own false positives and its own integration to maintain. Several tools reporting the same issue in different words is noise, not coverage.

• Before adding a tool, name the gap it fills and the tool it might replace.
• Count the time to tune and maintain it, not just the licence.
• Send results to one place developers already look.

Reports nobody reads

Long reports full of findings, colour-coded by severity, often go unread. Not because people do not care, but because nothing in them tells a busy reader what to do next.

• Lead with the one or two decisions you need.
• Write for the person who will act, in their language.
• If nobody asks about a regular report, ask whether it should still exist.

Rule of thumb:The best compliment an AppSec person can get is a developer saying they asked security early because it made their work easier.

Six months in, you know more about this codebase than most people outside the teams that wrote it. Use that to help them ship safely, not to stand in their way.