AI Application Security Assessment

The situation

You shipped a chat, matching or extraction feature, or gave users an assistant that can call tools inside your systems, because speed mattered more than caution at the time. That trade made sense in the moment. You rarely test afterward what a determined stranger can get the feature to do, or check what the company behind the underlying model does with the data you send it.

You do not know whether the feature will repeat back a document it should not surface, or whether the tools you gave it are reachable in ways nobody intended. Then a customer's security team sends a questionnaire, an insurer asks for evidence, or a curious user finds the edge case first, and you answer from memory because there was never a test to point to. The exposure was there before anyone went looking for it.

What changes

You move from trusting the feature because it shipped, to trusting it because you tested it against someone trying to break it. We build a threat model naming exactly how the feature could be misused: through the words a user types, through documents or search results it retrieves, through the tools it can call, and through what the vendor behind the model receives when you send it data.

A feature earns your trust the day it survives someone actually trying to break it.

We confirm each path by testing your working system directly, so nothing here is inferred from a diagram alone. You also receive a fix list ranked by what an attacker could do and how hard closing the gap is, so your engineers know what to fix first and what can reasonably wait.

How it runs

1. 01

   Agree scope, environment and authorisation

2. 02

   Review the architecture and build the threat model

3. 03

   Test against the model with real inputs

4. 04

   Write findings with severity and fixes

5. 05

   Replay with your engineers

Scope comes first: which feature, which environment, and what authorisation your organisation is giving us to act like an adversary against it. We agree a stop condition before any testing starts, so the boundary between test and production is never in question. From there we build the threat model from your actual architecture, not a generic checklist, because the permissions you granted and the data your retrieval step can surface differ in every build.

Testing itself runs against real inputs an attacker would try: injected instructions, boundary cases in retrieval, permission checks pushed past their limits. We write findings as we confirm them, so severity reflects what a finding actually lets someone do rather than how alarming it sounds on paper. The replay session puts your engineers in the room with the exact reproduction steps.

What you receive

You receive the threat model itself, written in plain terms your engineers and your leadership can both read: every path we tested, and why it matters for this particular feature. Alongside it, the full findings: what we found, how we found it, and the exact steps to reproduce each one, so nothing has to be taken on faith.

You also receive the fix list, ordered by what an attacker could actually do, and the replay session, where your engineers watch each finding reproduced and ask their own questions about it. Take the governance controls build next, and the fixes here become logged, monitored controls instead of a report that sits in a folder unread.