Opyntel

Linux disk full on a VPS: find what filled it, then stop it happening again

When a Linux disk is full, the server reports No space left on device. Run df -h to see which file system is full, then du to see which folder grew. On a VPS that runs containers it is nearly always Docker images and build cache, container log files or the systemd journal, and each has a safe command to shrink it. If df says full but du cannot find the files, a deleted log file is still held open by a process, or the disk has run out of inodes.

Confirm which disk is full

bash
df -h          # space per file system; look for 100% under Use%
df -i          # inodes; 100% under IUse% means too many small files
lsblk          # every disk and partition, with its size

Check /, /boot and any mounted volume separately. A full /boot stops kernel updates even when / has room.

Find what filled it

bash
sudo du -xh --max-depth=1 / 2>/dev/null | sort -h | tail -15
sudo du -xh --max-depth=1 /var 2>/dev/null | sort -h | tail -15

-x stays on one file system, so a mounted volume does not hide the answer. Repeat one level down into the biggest folder. ncdu -x / does the same interactively if you can install it.

The usual suspects on a VPS:

PathWhat grows there
/var/lib/dockerImages, build cache, stopped containers, volumes, container logs
/var/log/journalThe systemd journal
/var/logApplication and system logs without rotation
/var/cache/aptDownloaded packages (/var/cache/dnf on Fedora and RHEL)
/bootOld kernels
/home, /root, /tmpBackups, dumps and uploads someone left behind

Free space safely

Docker. See what Docker holds, then remove what nothing uses:

bash
docker system df
docker image prune -a        # images no container uses
docker builder prune         # build cache
docker container prune       # stopped containers

Leave docker volume prune for last and read its list first: volumes hold databases and uploads.

Container logs. Docker’s default log driver writes one JSON file per container with no size limit. Find the biggest and empty it without stopping the container:

bash
sudo sh -c 'du -h /var/lib/docker/containers/*/*-json.log | sort -h | tail -5'
sudo truncate -s 0 /var/lib/docker/containers/<id>/<id>-json.log

The systemd journal.

bash
sudo journalctl --disk-usage
sudo journalctl --vacuum-size=500M

Packages and old kernels on Debian and Ubuntu:

bash
sudo apt clean
sudo apt autoremove --purge

autoremove is also the fix when /boot is full of old kernels. If /boot is at 100% and apt itself fails, remove one old kernel package by name with sudo apt remove linux-image-<version>, never the one uname -r shows, then run autoremove.

When df says full but du disagrees

  • A deleted file is still open. Deleting a log that a process still writes to frees nothing until the process lets go. sudo lsof +L1 lists them; restart that service, or truncate the file instead of deleting it next time.
  • Out of inodes. df -i at 100% means millions of small files, often a cache or session folder. Find it with sudo du --inodes -x --max-depth=1 / | sort -n.
  • Reserved blocks. ext4 keeps 5% of the disk for root by default, so ordinary users see the disk as full first.
  • Files under a mount point. Files written to a folder before a volume was mounted over it still use space but are hidden. Unmount the volume, or bind-mount / elsewhere, to see them.

Stop it filling again

Cap Docker logs for every new container in /etc/docker/daemon.json, then restart Docker. Existing containers keep their old settings until they are recreated.

/etc/docker/daemon.json
{ "log-driver": "json-file", "log-opts": { "max-size": "10m", "max-file": "3" } }

Cap the journal with SystemMaxUse=500M in /etc/systemd/journald.conf, then run sudo systemctl restart systemd-journald. Make sure your own app logs go through logrotate, and schedule docker image prune -a -f and docker builder prune -f on servers that build images.

Cleaning up treats the symptom. What prevents the outage is knowing the disk is filling days before it is full, which no df you run by hand will tell you.

On your VPS host

Most VPS consoles graph what the host can see, such as CPU, disk I/O and network, but not how full your disk is, and few alert on it. See what each one shows: Hetzner disk full, Linode monitoring, Vultr monitoring, OVHcloud monitoring, Hostinger VPS monitoring and Scaleway monitoring. DigitalOcean Droplets and Coolify warn on disk usage themselves (see DigitalOcean Droplets monitoring and Coolify monitoring).

How Opyntel catches a filling disk

The Opyntel agent runs on each server with one docker run command and reports how full every disk is from inside the server, using outbound HTTPS only. Opyntel keeps that history and alerts when a disk passes your threshold or is on track to fill, and again when it recovers, by email, Slack or SMS depending on your plan. Every server sits in one dashboard with its memory, disk, load and containers, next to the other platforms you run, so you know which server is filling, and how fast, before you log in to run du.

FAQ

Linux disk full questions, answered

Checking space, why disks keep filling, Docker prune and space that will not come back.

df -h shows used and free space per file system, and lsblk shows the total size of each disk.

On a server it is usually logs without rotation, container logs with no size limit or Docker build cache. Cap all three, and set an alert on how fast the disk grows.

It removes stopped containers, unused networks, dangling images and build cache. Add --volumes only if you are sure no volume holds data you need.

A process still has them open. sudo lsof +L1 shows which one; restart it and the space comes back.

The alerts your platform doesn’t send.

One dashboard for everything you run, with the alerts your platforms leave out. Read-only, nothing to install in your apps.

See how it works