Check the open FD limit for a given process in Linux

You can try to write script which periodically call lsof -p {PID} on given pid.

to see the top 20 file handle using processes:

for x in `ps -eF| awk '{ print $2 }'`;do echo `ls /proc/$x/fd 2> /dev/null | wc -l` $x `cat /proc/$x/cmdline 2> /dev/null`;done | sort -n -r | head -n 20

the output is in the format file handle count, pid, cmndline for process

example output

701 1216 /sbin/rsyslogd-n-c5
169 11835 postgres: spaceuser spaceschema [local] idle
164 13621 postgres: spaceuser spaceschema [local] idle
161 13622 postgres: spaceuser spaceschema [local] idle
161 13618 postgres: spaceuser spaceschema [local] idle

Count the entries in /proc/<pid>/fd/. The hard and soft limits applying to the process can be found in /proc/<pid>/limits.

The only interfaces provided by the Linux kernel to get resource limits are getrlimit() and /proc/pid/limits. getrlimit() can only get resource limits of the calling process. /proc/pid/limits allows you to get the resource limits of any process with the same user id, and is available on RHEL 5.2, RHEL 4.7, Ubuntu 9.04, and any distribution with a 2.6.24 or later kernel.

If you need to support older Linux systems then you will have to get the process itself to call getrlimit(). Of course the easiest way to do that is by modifying the program, or a library that it uses. If you are running the program then you could use LD_PRELOAD to load your own code into the program. If none of those are possible then you could attach to the process with gdb and have it execute the call within the process. You could also do the same thing yourself using ptrace() to attach to the process, insert the call in its memory, etc., however this is very complicated to get right and is not recommended.

With appropriate privileges, the other ways to do this would involve looking through kernel memory, loading a kernel module, or otherwise modifying the kernel, but I am assuming that these are out of the question.