Table of Contents
By Khimananda Oli | Last reviewed: August 2026
When alerts fire at 3 AM or users report sluggishness, you need to diagnose high CPU and memory usage on a Linux server quickly and accurately. Misinterpreting load averages or confusing cache with active memory leads to wasted hours restarting services that aren't actually broken. This guide provides the exact diagnostic workflow I use in production environments, moving from initial triage to root cause identification without guesswork. For teams managing cloud infrastructure, pairing these diagnostics with monitoring with Prometheus and Grafana ensures you catch regressions before they impact users.
top or htop sorted by CPU/MEM to identify offending processes, verify system-wide pressure with vmstat 1, distinguish true memory leaks from cache using smem, and inspect dmesg for OOM killer events. Correlate spikes with application logs to confirm root cause before remediation.How do you identify processes causing high CPU and memory usage on a Linux server?
The first step when you diagnose high CPU and memory usage on a Linux server is identifying which process owns the resources. While top is ubiquitous, its default sorting can be misleading during transient spikes. Always sort interactively: press P (uppercase) for CPU or M for memory. In scripted diagnostics, use batch mode to capture snapshots without terminal interaction.
Capture CPU offenders non-interactively
# Capture 5 samples at 1-second intervals, sorted by CPU
top -b -n 5 -o %CPU | head -n 20
# Alternative: ps with custom sorting and formatting
ps aux --sort=-%cpu | head -n 10A common mistake is assuming high CPU percentage always indicates a problem. A backup job or cron task consuming 100% of one core briefly is expected behavior. Context matters: check if the process aligns with scheduled tasks using systemctl list-timers or crontab -l. If an unexpected process dominates, note its PID for deeper tracing with strace or perf.
Distinguish memory consumers accurately
Standard tools like ps report RSS (Resident Set Size), which includes shared libraries and cached pages counted multiple times across processes. This inflates perceived usage. Use smem to see USS (Unique Set Size) — the true private memory footprint:
# Install smem if missing (Ubuntu/Debian)
sudo apt install smem
# Show processes sorted by USS (true private memory)
smem -t -k -c "pid name swap uss pss rss"USS is the metric that matters when you suspect a memory leak. If USS grows monotonically over hours while RSS fluctuates, you have confirmed a leak rather than normal caching behavior. For containerized workloads, remember that cgroup limits apply to RSS+cache; consult docker stats or crictl stats alongside host-level tools.
What do load average and vmstat output really mean during diagnosis?
Load average is frequently misunderstood. It represents the average number of runnable and uninterruptible processes over 1, 5, and 15 minutes. On a 4-core system, a load of 4.0 means full utilization; 8.0 means significant queuing. However, high load with low CPU often indicates I/O wait, not compute saturation. This distinction is critical when you diagnose high CPU and memory usage on a Linux server because the fix differs entirely.
Interpret vmstat for root cause classification
Run vmstat 1 to stream per-second metrics. Focus on four columns to classify the bottleneck immediately:
- r (runnable): Sustained values above your core count indicate CPU saturation.
- b (blocked): Non-zero values mean processes are stuck waiting on I/O.
- wa (wait): Percentage of CPU time spent idle due to pending I/O. Above 30% signals storage or network issues.
- si/so (swap in/out): Any sustained non-zero value confirms memory pressure forcing disk swapping.
If wa is high but CPU user/system time is low, restarting application processes won't help. You need to investigate disk latency with iostat -xz 1 or network throughput with sar -n DEV 1. Teams deploying on fresh VPS instances should review initial Ubuntu server setup to ensure swap and I/O schedulers are configured correctly before troubleshooting.
How do you differentiate memory leaks from normal caching behavior?
Linux aggressively uses free RAM for page cache to accelerate file I/O. New engineers often panic seeing 95% memory usage, but this is optimal behavior. The kernel reclaims cache instantly when applications request memory. True problems manifest differently. When you diagnose high CPU and memory usage on a Linux server, distinguishing leak from cache prevents unnecessary restarts and masks real defects.
| Metric | Normal Caching | Memory Leak |
|---|---|---|
free -h available | High (cache reclaimable) | Declining steadily |
| Swap activity (si/so) | Zero or brief spikes | Sustained non-zero |
| Process USS (smem) | Stable or cyclical | Monotonic growth |
| OOM Killer events | Rare, under extreme load | Recurring, predictable |
| Performance impact | None or positive | Degradation over time |
Track memory trends programmatically
One-off snapshots miss slow leaks. Capture metrics over time to establish baselines:
# Log memory stats every 60 seconds for 24 hours
while true; do
date +%H:%M:%S >> /tmp/memtrack.log
smem -t -k -c "uss pss" | tail -1 >> /tmp/memtrack.log
sleep 60
done
# Analyze trend afterward
awk '{print $1, $2}' /tmp/memtrack.log | gnuplot -persistIf USS climbs linearly while traffic remains constant, you have a leak. Profile the application with Valgrind or language-specific tools (e.g., memory_profiler for Python, heap dumps for JVM). For Laravel applications exhibiting this pattern, consult Laravel performance optimization techniques to address common Eloquent and collection pitfalls before escalating to infrastructure changes.
When should you check kernel logs and OOM killer events?
The Out-Of-Memory (OOM) killer is the kernel's last resort when memory exhaustion threatens system stability. Its interventions are logged but easily missed. Whenever you diagnose high CPU and memory usage on a Linux server following unexplained restarts or service failures, kernel logs are mandatory evidence.
Extract OOM events reliably
# Search current boot logs
journalctl -k | grep -i "oom\|out of memory"
# Search historical logs across boots
journalctl -k -b -1 | grep -i "oom"
# Legacy systems without journald
dmesg -T | grep -i "oom"OOM log entries include the killed process name, PID, memory score, and total memory state at invocation. Note the oom_score_adj value: critical services like databases should have negative adjustments (-500 to -1000) to survive transient pressure, while batch workers can be set positive (500+) as sacrificial targets. Configure this via systemd:
# /etc/systemd/system/myapp.service.d/override.conf
[Service]
OOMScoreAdjust=500Correlate with application behavior
Kernel logs tell you what died, not why. Cross-reference timestamps with application logs, deployment events, and traffic patterns. If OOM kills occur after deployments, suspect code changes or misconfigured resource limits. If they correlate with traffic peaks, evaluate horizontal scaling or request throttling. For teams hosting on AWS EC2, understanding instance memory characteristics is essential; review AWS EC2 fundamentals to select appropriate instance families and configure CloudWatch alarms proactively.
Next Steps After Diagnosis
Accurate diagnosis prevents wasted effort, but resolution requires targeted action. Once you diagnose high CPU and memory usage on a Linux server and identify the root cause, document findings with timestamps, metrics, and remediation steps. Update runbooks so future incidents resolve faster. Implement automated monitoring to catch regressions before users notice. If your team needs assistance establishing reliable diagnostic workflows or audit-ready infrastructure, reach out to discuss your environment.
