python program that blocks takes 6% of CPU?

Question

I have a program that uses this library basically does something very simple, like this

   receiver = multicast.MulticastUDPReceiver ("192.168.0.2", symbolMCIPAddrStr, symbolMCPort )
   while True:
            print 'Spinning'
            try:
                    b = MD()

                    data = receiver.read(1024)

The receiver socket blocks until data comes in, so the print 'Spinning' only prints once until data is received on the socket. When I ask the OS how much CPU this process is taking, even though it is waiting on the receive, it comes back with:

[idf@node1 ~]$ ps -p  4294 -o %cpu,%mem,cmd
%CPU %MEM CMD
 6.3  0.4 python ./mc.py -s EUR/USD
[idf@node1 ~]$

In fact, if I run several of these processes, my computer with two CPU and 8 cores each, all cores go to 100% usage and the computer becomes unusable.

I must misunderstand python's notion of "blocking" because even a do nothing process that should basically be sleeping is taking up lots of CPU.

Is there a more correct way to write this so that programs that are basically waiting for i/o [interrupt-driven] give up the CPU?

FWIW, I just coded a very similar program in golang, and the CPU usage is as expected when no data come into the processes, nearly zero. — Ivan, Apr 09 '16 at 19:40
I should add though, that I "compiled" the go code. I am beginning to believe much of it is the python interpreter overhead. — Ivan, Apr 09 '16 at 21:57
I rely on Go more every day and Python less as it's just so convenient to write optimised code in Go. — user161778, Apr 10 '16 at 03:45
I have written many network programs in Python. When they are written correctly they use 0% CPU when there is no network activity to handle. — Jean-Paul Calderone, Jun 24 '16 at 19:20

score 0 · Answer 1 · answered Jun 24 '16 at 19:19

You haven't posted a complete example so it's difficult to say for sure what's happening.

However, I see that there's a try block inside your loop and your networking code is inside the try block. I don't know what your exception handling does. However, I'm guessing it does something like unintentionally swallowing an important error. Your loop then runs again and probably generates the same error. In this way, the program is actually busy-looping even though you thought it was asleep, blocking on I/O.

python program that blocks takes 6% of CPU?

1 Answers1