psutil.Process and lsof command discrepancy

Question

I've written a simple Python script that monitors the number of file descriptors on a Red Hat system.

When comparing to the lsof command I get two different results.

Broken down to it's core, the script does this:

import psutil 
p = psutil.Process(PID)
print(p.num_fds())

Currently num_fds() reports 60 open file descriptors. While for the same PID the result of lsof -p PID | wc -l yields 167.

Where is this discrepandy coming from?

My understanding was that both, num_fds() and lsof, both report the same file descriptors, including open file handles, sockets, pipes, etc.

Small background: some processes seem to open sockets and/or file handles without closing them ever again. Thus after a longer period of time the process reaches it's limit of file descriptors and crashes. This tool is intended to monitor this process wether the number of file desriptors is constantly rising.

You should compare the two list of file-descriptors to find out what is the difference between them. — Lorinczy Zsigmond, Apr 13 '21 at 14:04
@LorinczyZsigmond unfortunately psutil does not provide a list of file descriptors. It only returns a number, as far as I know. — ap0, Apr 13 '21 at 14:19
but you can compare the output of `lsof` and the content of `/proc/pid/fd/` — Lorinczy Zsigmond, Apr 13 '21 at 17:28

score 1 · Answer 1 · answered Apr 13 '21 at 13:40

If you look at source code of psutil then you'll find that num_fds() is just counting "files" inside /proc/<pid>/fd:

# _pslinux.py

@wrap_exceptions
def num_fds(self):
    return len(os.listdir("%s/%s/fd" % (self._procfs_path, self.pid)))

And lsof probably do more work (for example it analyze memory maps in process_proc_map()): https://github.com/lsof-org/lsof/blob/56d79ed56d14825f234783a867d6bd5676729b32/dialects/linux/dproc.c

score 1 · Accepted Answer · answered Apr 13 '21 at 13:42

Looking at psutil source in GitHub, the implementation of this method is:

def num_fds(self):
    return len(os.listdir("%s/%s/fd" % (self._procfs_path, self.pid)))

So it's just counting the number of file descriptors listed in /proc/pid or an equivalent location.

Looking at my current process on a Red Hat server, the pid directory contains 5 file descriptors; but lsof reports 19 open files. Looking at the full lsof output, the difference is that it lists things that don't appear to be associated with numeric file descriptors. The lsof man page describes:

   FD         is the File Descriptor number of the file or:

                   cwd  current working directory;
                   Lnn  library references (AIX);
                   err  FD information error (see NAME column);
                   jld  jail directory (FreeBSD);
                   ltx  shared library text (code and data);
                   Mxx  hex memory-mapped type number xx.
                   m86  DOS Merge mapped file;
                   mem  memory-mapped file;
                   mmap memory-mapped device;
                   pd   parent directory;
                   rtd  root directory;
                   tr   kernel trace file (OpenBSD);
                   txt  program text (code and data);
                   v86  VP/ix mapped file;

So, the discrepancy is because lsof includes a variety of "open files" that are not actually mapped to file descriptors.

psutil.Process and lsof command discrepancy

2 Answers2