Opyntel

“Too many open files” on Linux: raise the limit and find the leak

Too many open files (EMFILE) means a process reached its limit of open file descriptors. Network connections, pipes and sockets count as files, so a busy web server or database hits it long before it opens that many real files. Check the limit with cat /proc/<pid>/limits, count what the process holds with ls /proc/<pid>/fd | wc -l, raise the limit where the process is started (systemd, Docker or your shell), then make sure the count is not climbing because of a leak.

Check the limit and what is open

bash
pid=$(pgrep -o nginx)                      # the process that logs the error
grep 'open files' /proc/$pid/limits         # its soft and hard limit
sudo ls /proc/$pid/fd | wc -l               # how many it holds now
sudo lsof -p $pid | awk 'NR>1{print $5}' | sort | uniq -c | sort -rn   # what kind: REG, IPv4, sock...

ulimit -n only shows the limit of your own shell, not of a service started by systemd or Docker, so read /proc/<pid>/limits for the process that fails.

Raise the limit where the process starts

How it runsWhere to set the limit
systemd serviceLimitNOFILE=65535 under [Service] in systemctl edit <service>, then restart the service
Docker container--ulimit nofile=65535:65535, or ulimits: nofile: in Compose
Login shell or cronnofile lines in /etc/security/limits.conf; log in again
The current shell onlyulimit -n 65535

Limit or leak?

Raising the limit fixes a process that genuinely needs many connections. If the count from ls /proc/<pid>/fd | wc -l keeps climbing under steady traffic, the process is leaking: connections, files or HTTP clients that are opened and never closed. A higher limit only delays the next error. The lsof breakdown above shows which kind grows; thousands of sockets in CLOSE_WAIT (ss -tanp | grep CLOSE-WAIT) point at code that never closes its side of a connection.

“Too many open files in system” and file watchers

Too many open files in system (ENFILE) is the kernel-wide limit, not a process limit:

bash
cat /proc/sys/fs/file-nr      # allocated, unused, maximum
sudo sysctl -w fs.file-max=2097152   # persist it in /etc/sysctl.d/

failed to create fsnotify watcher: too many open files, common with Docker, Kubernetes tools and dev servers, means the user ran out of inotify instances: raise fs.inotify.max_user_instances with sysctl and persist it in /etc/sysctl.d/. Running out of fs.inotify.max_user_watches gives no space left on device instead.

A process that dies from it in a container often shows up as a restart loop; see why a container keeps restarting.

How Opyntel watches open files

The Opyntel agent runs on each server with one docker run command and reports open file handles against their limits, from inside the server, using outbound HTTPS only. Opyntel keeps that history and alerts when open files pass your threshold, by email, Slack or SMS depending on your plan, so the steady climb of a leak shows up in the history instead of in an error. Every server sits in one dashboard with its memory, disk and containers, next to the other platforms you run.

FAQ

Too many open files questions, answered

What counts as a file, safe limits, Docker and the error that keeps coming back.

Every file descriptor: regular files, sockets and network connections, pipes and devices. A web server mostly holds sockets.

For a server process, 65535 is common, and the kernel-wide fs.file-max is usually far higher. Size it to the connections the process really needs.

Start it with --ulimit nofile=65535:65535, or set ulimits: nofile: in the Compose file.

The process is leaking descriptors, or the new limit never reached it: check /proc/<pid>/limits for the running process.

The alerts your platform doesn’t send.

One dashboard for everything you run, with the alerts your platforms leave out. Read-only, nothing to install in your apps.

See how it works