Book Appointment

One Alert’s Journey Through AIOps for Security

Healthcare Software

Explanations of AIOps for Security tend to start with definitions and end with diagrams. A clearer route is to follow a single alert from arrival to resolution and watch where the machinery helps, where it merely guesses, and where a human is still required. Our specimen is a phishing email that reached a hospital network on a Tuesday morning — call it case 4471 — and the pipeline it travels through is the standard modern one: a scoring model, an investigating assistant, an autonomy gate, and an audit.

Arrival: one email among fifty million

Case 4471 lands in a stream that a mid-sized estate processes at roughly 50 million events a day. Somewhere in that stream are a few dozen genuinely malicious items — call it 50 for the arithmetic’s sake. Before any language model says a word about the message, a scoring model has already assigned it a place in the queue, because the first job of AIOps for Security is ordering: which of tonight’s alerts deserves eyes first? The ranker does not decide anything. It sorts. That distinction sounds pedantic until money and safety hang on it, so hold on to it — every later stage depends on the difference between ordering a list and judging its members. Hospitals understand this shape already: triage ordering in an emergency department is not a diagnosis, and nobody mistreats a queue position as a verdict. The same discipline applies here, with one aggravating difference — the queue refills hourly, and the cost of an overrun queue is measured in missed intrusions rather than waiting-room minutes.

The score is a probability wearing a uniform

Case 4471 scores 0.87, near the top of tonight’s queue, sitting just below a credential-stuffing burst and just above a suspicious OAuth consent. What does 0.87 mean? Less than it appears. Scores convert into useful numbers only through the base rate, and the base rate here is brutal: 50 true threats hiding in 50 million events. If the detector’s false alarms run at one per ten thousand, only around nine in every thousand of its positives are genuine — an analyst works through about 112 alerts before one genuine attack turns up. Push the false alarms down to one per million and the picture inverts, with nearly half of all positives genuine. Nothing about the model changed between those two sentences; only the arithmetic of rarity moved. A team that reads 0.87 as “87% malicious” has skipped the only step that gives the number meaning.

The assistant reads the mail, and that is the risky part

Next, an investigating assistant opens case 4471 and summarises it: spoofed display name, credential-harvesting link, urgency cues. This is where language models earn their place in security work — minutes of careful reading compressed to seconds of competent summary. It is also where the adversary lives. The email’s text is attacker-controlled content, which means it can carry instructions aimed at the reader doing the reading — the technique is called indirect prompt injection, identified back in 2023, and no filter reliably separates hostile instructions from ordinary text inside a hostile document. The UK’s NCSC said this plainly in December 2025: a model in this position is a deputy that can be confused by its own inputs, and the confusability may never be fully engineered away. The practical response is not cleverer prompts but smaller blast radius — least-privilege scopes, no outbound channel the deputy could be steered into, deterministic checks on anything it proposes to do. An assistant that can read mail should not also be the assistant that can wire money or mail executives on its own recognisance.

Case 4471 reaches the AIOps for Security gate

The assistant recommends closure: the link is inert, the sender domain is lookalike but logged, no credentials left the estate. Now the gate decides — and “gate” is the load-bearing word. Sensible programs grant autonomy one action class at a time, each with its own evidence requirements:

  • Level 1, recommend: the system may order and summarise, with a floor of roughly 200 blind-labelled alerts including 30 or more true positives behind its claims.
  • Level 2, auto-close: allowed only once a proven miss bound no worse than 2% exists across 150 or more true positives, canary cases run in the live flow, a sealed holdout exists, and reopening a closed case takes one click.
  • Level 3, reversible action: permitted only with a false-action bound of 1% or tighter, measured over 300-plus verdicts on actions it would have taken, plus a rollback drill executed for real and volume caps on how often it may fire.
  • Level 4, irreversible action: not granted. Not to the best model, not on the best quarter. The ladder ends here for machines by design.

Observe mode is the price of admission to all of this — the system must be able to run without acting. Where a product cannot watch before it acts, the absence is itself the evaluation finding.

Two ways this same pipeline turns toxic

The journey above is the healthy version, and it is worth naming the diseases it avoids. The first is page-the-analyst-on-the-score: treating a raw anomaly score as an alarm clock. Anomaly scores are rankings by another name — paging a human because a number crossed a line, without a base rate or a gate, rebuilds the old firehose with extra steps and calls it intelligence. The second disease is subtler: letting the ranker shape its own training data. If the cases the model scores highest become the cases analysts label, the labels drift toward the model’s existing beliefs, and the system grows more confident without growing more correct. Both diseases share a symptom — numbers that improve every month while the queue stays exactly as dangerous — and both are cured by the same medicine: labels that exist independently of the model, and decisions gated on evidence the model never saw.

What the AIOps for Security audit shows afterwards

Case 4471 is closed in ninety seconds, and the audit trail fills in the rest of the story. The labels that justify closure were blind — adjudicators never saw the machine’s verdict, or the numbers would flatter it. The reporting is per category, because blended figures launder failure: the published example that taught the field this lesson showed a phishing category missing at 9.1% while a cloud-identity category missed at 42.3% behind the same comfortable overall average. And the quarter’s spotless closes prove less than they feel like — an audit that samples one ticket in twenty and finds nothing still caps the true miss rate near 38% from above, which is arithmetic rather than pessimism, and it is why canaries are planted instead of relying on luck. A team that wants this discipline operationalised can find its manual-operations cousin in reliable cloud and tech operations under proper support, and its software-side sibling in a practical guide to safer software delivery.

The human still signs

Where is the analyst in all this? Reading the exceptions, auditing the categories the bounds come from, and — when the canary trips — switching the autonomy back off. Case 4471 closed itself in ninety seconds precisely so that this person had ninety minutes for the alert that mattered. MITRE’s guidance leaves no room for doubt that a human must stand behind the machinery’s output; delegation moves the work, it does not delete it. The quieter truth underneath the whole journey is that most anomalies were never attacks to begin with: they are changes, updates, and new starters finding their feet, and a system that cannot tolerate that fact will bury its operators in false alarms until they stop reading it. Teams bringing models into their stack with intent — the way good organisations approach AI integration inside modern software delivery — recognise the pattern immediately: narrow first use, measured evidence, staged trust.

For the map of the whole territory — rungs, gates, base-rate labs, adversarial drills — this AIOps for Security explainer from xOps School is the reference the journey above was built from. Case 4471 was ordinary. That is the point: the machinery is judged on the ordinary cases, and the evidence gates exist so that “ordinary” never quietly becomes “unexamined”.