Book Appointment

Diagnosing a Slow Server: A SysOps Problem-and-Fix Guide

Healthcare Software

A good clinician does not prescribe before examining, and a good systems operator does not restart before observing. That shared discipline is the subject of this article. It treats a sluggish server the way a careful diagnostician treats any complaint: define the symptom, list plausible causes, test them in order of cost, and only then act. The material belongs to SysOps, the routine work of keeping individual servers healthy, and it is written for the teams who run websites and internal tools. It contains no medical guidance of any kind.

The problem: “the server is slow” is not a diagnosis

Users describe symptoms, not causes. Pages hang, uploads stall, a report that took ten seconds now takes two minutes. Behind each description sits one of a handful of bottlenecks: processors, memory, storage, network, or a limit set in software. The first job is to narrow the field quickly and without disturbing the evidence.

The Atlas page on SysOps practice for a single host frames this with a simple method. For every resource, ask how busy it is, whether work is queuing for it, and whether it is throwing errors. Work through processors, memory, disks and network in turn, and the culprit usually reveals itself within minutes.

Step one: collect before you change

The anti-pattern the Atlas warns about most firmly is the immediate restart. A restart clears memory, closes connections and ends the very processes you need to inspect. Instead, capture a snapshot of the machine’s state: running programs sorted by consumption, recent kernel and service messages, filesystem usage, and basic network counters. Save it with a timestamp. If a restart later proves necessary, the notes will still tell the story.

A short script that gathers these items in one go is a worthwhile investment. The Atlas lists a triage script among its first-month tasks for exactly this reason.

Step two: test the common suspects in order

Suspect one: processors that look idle but are not free

Load numbers can mislead because they include tasks waiting on disks. A machine may show a high load with plenty of idle processor time. Look at what the tasks are waiting for before concluding that more processors are needed. If the average utilisation reads 40 percent while users are unhappy, three explanations deserve checking: short bursts hidden by five-minute averages, time stolen by a busy neighbour on shared hardware, and a bottleneck elsewhere.

Suspect two: memory that is genuinely scarce

Low free memory alone proves nothing, since the kernel fills spare memory with cached files. What matters is the available figure and any sign of swapping or killed programs. A newer kernel facility called pressure stall information, added in 2018, reports how much time tasks spend stalled waiting for memory, processors or input and output, and it is a far clearer symptom than raw free memory.

A tempting fix for a program that keeps getting killed is to give it the most protective setting for the out-of-memory killer. According to the Atlas, that only shifts the casualty onto some other process, perhaps a more valuable one. Fix the memory budget instead.

Suspect three: storage that reports a misleading busy figure

Disk tools show a utilisation percentage that is easy to misread. On solid-state drives and RAID sets, which handle many requests at once, a device can read as fully busy while still having spare capacity. Look at latency and queue depth, not just the percentage.

Suspect four: a limit nobody remembered

Open-file ceilings, connection tables and full disks can all cause strange behaviour that looks like slowness. Check them early, because they are cheap to inspect and quick to fix.

Step three: decide on the smallest effective change

Once you know the bottleneck, pick the least invasive remedy. A runaway scheduled job might need a timer with a sane schedule instead of cron. A logging flood might need a size cap on the journal. A memory-hungry service might need a limit in its unit file. Resist the broad fixes that feel decisive but hide the cause, for example by disabling host protections merely to rule them out.

  • Change one thing at a time. Otherwise you will not know which change helped.
  • Write the change down. Include the date, the reason and how to reverse it.
  • Re-measure. A fix is only proven when the original symptom has gone and the figures confirm it.

Step four: close the loop

After the incident, convert what you learned into a lasting improvement. Add an alert for the signal you wished you had seen earlier. Record the time it took to name the starved resource, because the Atlas lists that duration as a metric worth tracking over months. Update the triage script if it missed something. A diagnosis that does not feed back into the routine will simply recur.

Prevention: make the next diagnosis shorter

Every diagnosis is faster when you know what normal looks like. Capture a baseline of ordinary load, memory availability and disk latency on a typical weekday, and keep it beside your notes. When something drifts, the comparison is immediate. The Atlas builds this idea into its maturity ladder, where the step after documentation is a baselined host, and the step after that is a measured one.

Pair the baseline with a few alerts on symptoms users would feel: sustained stall time, shrinking disk runway, failing scheduled jobs and a growing reboot backlog. Four signals chosen carefully do more than a wall of charts nobody reads, and they keep the on-call person from being woken by noise.

Common misreadings to unlearn

  • High load means the processors are overworked. It may mean tasks are queuing for storage.
  • Little free memory means a leak. Cache fills spare memory by design.
  • A busy disk percentage means no headroom. Parallel devices can still absorb work.
  • Restarting proves the fix. It proves only that the symptom paused.

A worked example in words

Imagine a booking tool that slows every afternoon. The snapshot shows moderate processor use, high load, and a data volume sitting at ninety percent. Latency on the volume rises whenever a nightly export overlaps with daytime traffic. The remedy is not more processors; it is moving the export to a quieter hour and adding a disk-runway alert, since the volume would have filled within days anyway. The whole investigation takes twenty minutes because nothing was restarted and every hypothesis was tested in order.

Notice what the example did not need: a vendor tool, a new purchase or a late-night heroic rebuild. It needed a calm sequence and a few honest measurements. When a team can narrate its diagnoses in this plain order, handovers between shifts become easier too, because the next person can see which suspects have already been cleared and which remain open.

Reading more on this site

If you enjoy methodical problem-solving in a different arena, the article about sorting out the reasons runners and walkers feel knee strain follows a similar pattern of narrowing possibilities, and the commentary on why IT operations are being reorganised under one umbrella offers the wider context for the work described above. Treat both as general interest pieces rather than guidance for any one reader.

Summary: SysOps diagnosis in one habit

Diagnose first, act second, and keep a record of both. The method scales from a single virtual machine to a rack of servers because it rests on a habit rather than a tool: look at each resource, test cheap hypotheses first, change one variable, and confirm the result with numbers.