Neither the team in this story nor the hospital group employing it is real. They are an invented composite, sketched to show how a SOC Ops routine changes when a small IT department stops measuring effort and starts measuring decisions. Every number below is a round figure chosen for illustration. The setting is purely administrative: servers, accounts, rotas and spreadsheets.
Month one: busy, tired and unconvinced
Four people shared a monitoring queue for a regional group of outpatient centres. They worked days, with a phone forwarded to whoever lived nearest at night. Their monthly report counted tickets closed, and the number looked healthy. Yet the team lead sensed that the report said nothing about what she cared about: whether a genuine problem would be noticed, and how soon.
She started by reading the Ops Atlas entry that describes how a monitoring desk can be operated as a reliable service, then circled the sections that matched her frustration. The Atlas’s framing was that the desk exists to turn alerts into timely, recorded decisions. Tickets closed, she realised, were a by-product, not the product.
The experiment: one change at a time
She resisted the urge to redesign everything. Instead the team agreed on a four-week sequence, one change per week.
- Week one: every closure must carry a reason and one of four labels, real, benign, mistaken or undetermined.
- Week two: record two timestamps, the first time a person acted and the time a final decision was reached.
- Week three: introduce a written handover, countersigned on both sides of the shift change.
- Week four: sort the month’s closures by label and reason, and discuss the pattern over lunch.
What the data showed
The sort was humbling. Under the old habit, nearly everything had been filed as a mistaken detection. Under four labels, a large share moved into the benign column. The team recognised the pattern the Atlas attributes to published research: many so-called false alarms are legitimate activity that merely lacked context. A weekly backup job, a contractor’s remote session, a scanner run by the facilities group, each had been tripping rules and each had been dismissed with a shrug.
The second surprise came from the clocks. The median time to first human action was reasonable. The slow tail, however, was dreadful, and it clustered after six in the evening. The average on the old report had blurred both facts. A single message reaching a mailbox nobody watched on Friday night had sat for a weekend and, in the old arithmetic, would have wrecked any mean.
The fixes were small and boring
Because the benign cases now had a name, the team could act on them. They asked each department to share its recurring jobs, then wrote short notes that attached themselves to alerts from the matching devices. The pile shrank noticeably. They also agreed with the group’s managers that two named people would hold the phone each night, with a deputy, which matched the Atlas’s advice that no shift should rest on a lone analyst.
Automation arrived last. A vendor had pitched an orchestration product in month one, and the lead had politely declined. By month four the team had stable labels and a trustworthy queue, so they allowed one narrow automatic action, limited to staff laptops, throttled, and only for a rule with months of clean history. Servers holding records stayed under human decision. Every automatic step got a named owner and a review date, which kept it from being forgotten. A short article we published on how engineering teams supervise growing fleets of automated machines describes the same instinct: automate what is proven, supervise what is not.
Month five: the staffing sum
The Friday-night discovery forced an awkward conversation about headcount. The lead borrowed a rule of thumb from the sources summarised in the Atlas: keeping one seat covered every hour of the year takes about five full-time people. Her team had four, who also did other IT work. Pretending otherwise had been the real cause of the weekend gap.
The group had three realistic choices. They could hire, which was slow. They could buy a managed night service, which was quicker but meant specifying outputs in the contract: recorded labels, response times by shift and a signed handover. Or they could narrow the promise, declaring openly that overnight cover meant a phone-based escalation for the highest-priority items only. The managers chose the second and third in combination. Importantly, the contract did not move accountability. The lead remained answerable for outcomes and said so in the first meeting with the provider.
Month six: a different SOC Ops report
The new monthly report fit on one page. It showed alerts received, split by the four labels, the median and ninetieth-percentile time to first human action, split by day and night, and the number of cases reopened. Closed-ticket counts were gone. When the board asked why the report no longer showed “ticket volume”, the lead explained that volume measured the noise and the new figures measured the service.
Staff morale improved in a way no one had budgeted for. First-line work had been presented as a rung on a ladder, with a rotation into investigation tasks every quarter, mirroring the Atlas’s description of triage as a growth position. Two people who had considered leaving stayed.
One more detail from the story deserves mention. At the end of month six the lead scheduled a yearly self-assessment, using the open maturity model the Atlas points to, and wrote down the first result as a baseline rather than a verdict. The point of the exercise, she told the team, was to have something honest to compare against twelve months later.
Lessons worth borrowing
- Start with labels and reasons. They cost nothing and reveal where effort is wasted.
- Report a middle value and a slow-tail value, by shift. Averages hide nights and weekends.
- Write handovers down, and let unaccepted items stay with the owner.
- Automate last, narrowly, with owners and end dates.
- Judge the desk by recorded decisions, not by tickets closed.
Teams building the wider operational plumbing around such a desk may find our piece on streamlining software and cloud operations with engineering support a useful companion read.
Remember that the group, the lead and the figures here are invented for the sake of the lesson. The article offers general operational information, not advice for any real organisation or system.
