Diagnose Linux OOM kills and memory pressure on a VPS

When Linux cannot satisfy a memory allocation, the OOM killer terminates a process to keep the system alive. The visible symptom may be a restarted container or a database connection failure minutes later.

Confirm an OOM event

journalctl -k --since today | grep -Ei 'out of memory|oom-kill|killed process'
journalctl -u myapp --since today

The kernel message names the killed process and shows memory statistics. On a container host, also inspect docker inspect for OOMKilled.

Read memory correctly

free -h
ps -eo pid,user,comm,rss,vsz,%mem --sort=-rss | head -20

Focus on available, not free. Linux uses spare RAM for filesystem cache and reclaims it when applications need memory. RSS is the resident memory currently held by a process; VSZ is not physical consumption.

Watch trends with vmstat 1. Continuous si and so values indicate swap churn. A rising process RSS over hours suggests a leak or unbounded cache.

Add a small swap file

Swap provides a pressure-release valve, not extra fast memory:

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Verify with swapon --show. On storage-sensitive databases, keep swap modest and fix sustained swapping rather than normalising it.

Bound services

For systemd:

[Service]
MemoryHigh=1500M
MemoryMax=1800M

MemoryHigh applies pressure before the hard MemoryMax boundary. For Compose, use mem_limit. Leave headroom for the kernel, SSH, monitoring, and other services.

Limits turn a host-wide outage into a service-specific failure, but they do not make an undersized workload healthy.

Fix the cause

Common causes include:

• Too many application or database workers
• Unbounded queues, caches, or uploads
• A memory leak after a new release
• Database per-connection buffers multiplied by high concurrency
• Several containers each assuming they own the whole VPS

Compare the event time with deployments and traffic. Reduce concurrency, stream large files, cap caches, or move heavy jobs to a worker. Resize only after understanding how memory scales with load.

Alert on available memory, swap activity, and OOM events. Memory incidents are much easier to solve with a graph showing the preceding hour than with a process list taken after the victim has already died.

Separate cache from a leak

Watch the process over time rather than comparing one snapshot:

pidstat -r -p ALL 60
watch -n 5 'ps -eo pid,comm,rss,%mem --sort=-rss | head -15'

If RSS rises and never returns after requests finish, capture the release, traffic pattern, and endpoint involved. If the growth is filesystem cache, free may look low while available remains healthy; do not clear caches as a routine fix.

Use systemd Restart=on-failure or a container restart policy for stateless services, but alert on every restart burst. Set a cgroup memory limit, keep a small swap reserve, and document which process is allowed to be sacrificed first. Resize only when measurements show the normal working set no longer fits with safe headroom.

Capture evidence before restarting

Before restarting a leaking service, save the kernel event, process list, cgroup limits, and recent application logs:

date
free -h
systemctl show myapp -p MemoryCurrent -p MemoryMax
journalctl -k --since '30 minutes ago' > /tmp/oom-kernel.log

Attach those values to the incident or deployment record. They let you compare a suspected leak with later releases and prove whether a memory limit prevented a host-wide outage.