Timeout and high cpu load problems using multiple telnet connection threads in python

Question

I want to connect to multiple telnet hosts using threading in python, but I stumbled about an issue I'm not able to solve.

Using the following code on MAC OS X Lion / Python 2.7

import threading,telnetlib,socket

class ReaderThread(threading.Thread):
    def __init__(self, ip, port): 
        threading.Thread.__init__(self)
        self.ip = ip
        self.port = port
        self.telnet_con = telnetlib.Telnet()

    def run(self):
        try:
            print 'Start %s' % self.ip
            self.telnet_con.open(self.ip,self.port,30)
            print 'Done %s' % self.ip
        except socket.timeout:
            print 'Timeout in %s' % self.ip

    def join(self):
        self.telnet_con.close()

ta = []

t1 = ReaderThread('10.0.1.162',9999)
ta.append(t1)
t2 = ReaderThread('10.0.1.163',9999)
ta.append(t2)

for t in ta:
    t.start()
print 'Threads started\n'

In general it works, but either one of the threads (it is not always the same one) takes a long time to connect (about 20 second and sometimes even runs into a timeout). During that awfully long connection time (in an all local network), cpu load also goes up to 100 %.

Even more strange is the fact that if I'm using only one thread in the array it always works flawlessly. So it must have something to do with the use of multiple threads.

I already added hostname entries for all IP addresses to avoid a DNS lookup issue. This didn't make a difference.

Thanks in advance for your help.

Best regards

senexi

How odd. Do you have a way to get the misbehaving thread's stack trace to see where it's stuck? — NPE, Dec 21 '11 at 18:23
I included "traceback.print_stack()", but it didn't bring up any new information. It always hangs at telnet_con.open(). I also tried a version without using threading. Then it still takes long to establish a connection, but it does not blow up CPU load anymore. Unfortunately I have to use threads in my application... :-/ — senexi, Dec 21 '11 at 18:53

score 0 · Accepted Answer · answered Dec 21 '11 at 21:52

Ok, You have overridden join(), and you are not supposed to do that. The main thread calls join() on each thread when the main thread finishes, which is right after the last line in your code. Since your join() method returns before your telnet thread actually exits, Python gets confused and tries to call join() again, and this is what causes the 100% cpu usage. Try to put a 'print' statement in your join() method.

Your implementation of join() tries to close the socket (probably while the other thread is still trying to open a connection), and this might be what causing your telnet threads to never finish.

Great,you were absolutely right about the join method and the CPU usage. Thanks a lot. Another issue I found out today was an faulty MAC address configuration of the two telnet devices I was trying to connect to, which (of course) also resulted in a malfunctioning network behavior. I guess this issue is solved ;-) — senexi, Dec 22 '11 at 09:58

Timeout and high cpu load problems using multiple telnet connection threads in python

1 Answers1