Different processes showed as same PID within netstat

Question

I spawn few processes using the Python multiprocessing module. However when I call netstat -nptl, each ip:port listeners listed under the same PID.

I'm using Python 2.7 on Ubuntu 14.04.

netstat -V
>> net-tools 1.60
>> netstat 1.42 (2001-04-15)

Relevant code:

import unittest
import multiprocessing
import socket
import os
import time
import ex1


class Listener(multiprocessing.Process):
    def __init__(self, _ttl):
        super(Listener, self).__init__()
        self.ttl = _ttl
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.socket.bind(('localhost', 0))

    def get_pid(self):
        return self.pid

    def get_name(self):
        return self.socket.getsockname()

    def run(self):
        self.socket.listen(1)
        time.sleep(self.ttl)

    def listen(self):
        self.start()


class TestEx1(unittest.TestCase):
    def test_is_legal_ip(self):
        # Legal IP
        assert(ex1.is_legal_ip("1.1.1.1:55555"))
        assert(ex1.is_legal_ip("0.1.1.255:55555"))
        assert(ex1.is_legal_ip("0.0.0.0:55555"))
        assert(ex1.is_legal_ip("255.255.255.255:55555"))
        assert(ex1.is_legal_ip("0.1.2.3:55555"))

        # Illegal IP
        assert(not ex1.is_legal_ip("256.1.1.1:55555"))
        assert(not ex1.is_legal_ip("1.256.1:55555"))
        assert(not ex1.is_legal_ip("1.1.1.1.1:55555"))
        assert(not ex1.is_legal_ip("1.a.1.1:55555"))
        assert(not ex1.is_legal_ip("1.1001.1.1:55555"))

    def test_address_to_pid(self):
        # Create 3 listener processes
        listener1 = Listener(22)
        listener2 = Listener(22)
        listener3 = Listener(22)

        # Start listening
        listener1.listen()
        listener2.listen()
        listener3.listen()

        print listener1.get_pid()
        print listener2.get_pid()
        print listener3.get_pid()


        # For each listener, get appropriate ip:port
        address1 = str(str(listener1.get_name()[0])) + \
            ":" + str(listener1.get_name()[1])
        address2 = str(str(listener2.get_name()[0])) + \
            ":" + str(listener2.get_name()[1])
        address3 = str(str(listener3.get_name()[0])) + \
            ":" + str(listener3.get_name()[1])

        # Check if address_to_pid() works as expected.
        #assert(str(ex1.address_to_pid(address1)) == str(listener1.get_pid()))
        #assert(str(ex1.address_to_pid(address2)) == str(listener2.get_pid()))
        #assert(str(ex1.address_to_pid(address3)) == str(listener3.get_pid()))

        # Waits for the listener processes to finish
        listener2.join()
        listener2.join()
        listener3.join()


if __name__ == "__main__":
    unittest.main()

Output:

4193
4194
4195
..
----------------------------------------------------------------------
Ran 2 tests in 22.019s

OK

Netstat -nptl output:

(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      -               
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      -               
tcp        0      0 127.0.0.1:37529         0.0.0.0:*               LISTEN      4192/python     
tcp        0      0 127.0.0.1:53402         0.0.0.0:*               LISTEN      4192/python     
tcp        0      0 127.0.0.1:49214         0.0.0.0:*               LISTEN      4192/python     
tcp        1      0 192.168.46.136:49475    209.20.75.76:80         CLOSE_WAIT  2968/plugin_host
tcp       70      0 192.168.46.136:60432    91.189.92.7:443         CLOSE_WAIT  3553/unity-scope-ho
tcp6       0      0 ::1:631                 :::*                    LISTEN      -

What happens to the pids shown in netstat if you calls the `bind` in Listener's `run` method? — suztomo, Jul 30 '15 at 00:26

suztomo · Answer 1 · 2015-07-30T03:47:22.927

0

Using my Mac OS 10.9.5 (Python 2.7.3), I could reproduce the same behavior. After several try-and-error, it turned out that it's because the socket objects are shared among the processes. (lsof -p <pid> helps to identify listening sockets.)

When I made following change to Listener class, each process started to listen on its own port number by its own PID.

def get_name(self):
    # this method cannot refer to socket object any more
    # self.sockname should be initialized as "not-listening" at __init__
    return self.sockname

def run(self):
    # Instantiate socket at run
    self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    self.socket.bind(('localhost', 0))
    self.sockname = self.socket.getsockname()
    # Listener knows sockname
    print self.sockname
    self.socket.listen(1)
    time.sleep(self.ttl)

This behavior should be the same as Ubuntu's Python and netstat.

self.sockname remains "not-listening" at original process

To listen on port as independent process, sockets need to be created at run method of a Listener object (New process calls this method after creating copy of the object). However variables updated in this copied object in the new process are not reflected to the original objects in original process.

edited Jul 30 '15 at 03:47

answered Jul 30 '15 at 02:30

suztomo

5,114
2
20
21

But how can I now retrieve the IP and PORT of the socket? – triple fault Jul 30 '15 at 02:37
It is unacceptable for me to get the port number using netstat. No way I can find out the PORT within Listener? – triple fault Jul 30 '15 at 02:43
Why is the print required? Can I workaround this without using print? – triple fault Jul 30 '15 at 03:11
Of course. Print is just for you to confirm that Listener now gets port number. – suztomo Jul 30 '15 at 03:13
Unfortunately it doesn't work for me: AttributeError: 'Listener' object has no attribute 'sockname' – triple fault Jul 30 '15 at 03:17
self.sockname should be initialized as "not-listening" at `__init__` – suztomo Jul 30 '15 at 03:18
I'm somewhat new to python. Can you please let me know how I initialise as "not-listening" at __init__? – triple fault Jul 30 '15 at 03:21
@Doppelganger Add `self.sockname = "not-listening"` at the end of `__init__` method of Listener class – suztomo Jul 30 '15 at 03:24
This doesn't work. When I "print self.sockname" within get_name() it print wrong data... – triple fault Jul 30 '15 at 03:27
Can you confirm that Listener class knows (prints) port number at `run` method, or not? – suztomo Jul 30 '15 at 03:30
Yes, it prints correct data at run. But wrong data is printed from get_name(). – triple fault Jul 30 '15 at 03:31
That's different problem. In Python's multiprocessing.Process variables assigned at run method cannot be seen from the original process. Because they're copy in different process. – suztomo Jul 30 '15 at 03:39
Added explanation for that. I didn't come up with better idea. You may want to try inter-process communication to tell socket port number to original process. – suztomo Jul 30 '15 at 03:48
So it seems there is no way to fix this problem? – triple fault Jul 30 '15 at 03:49
You can try inter-process communication http://pymotw.com/2/multiprocessing/communication.html#passing-messages-to-processes. I hope you are ok with my answer to your question of why the port was listened by original process via netstat. – suztomo Jul 30 '15 at 11:57

Different processes showed as same PID within netstat

1 Answers1

Linked