Python 2.7 - Linux - Infinite loop, relinquish CPU to other threads / processes

Question

is there way to relinquish the rest of thread / process assigned time to other threads/processes in Python 2.7?

Please don't recommend syncing, mutexes, semaphores and whatever else. I am asking for standard mechanism which exists on Windows or Linux for code with access to kernel functions.

let me give you short example of the C code:

int i = 0
while (true) {
   i++;
   // this will work on windows:
   sleep(0);
   // and this will work on Linux
   sched_yield();
}

When you compile and start the code above and if you take a look on the CPU usage it will be 0% as { i++, if true/jump } consumes fraction of the time assigned by CPU to the thread. The rest of the CPU time is relinquished on behalf of other threads/processes.

In opposite, when you do:

i = 0
while 1:
   i = i + 1
   time.sleep(0.001)

in Python, the CPU usage significantly depends on the time you pass to the time.sleep function. Less time passed there means higher CPU usage. This is definitely not acceptable for me as well as putting time higher than 0.1 there.

Please be sure the python time.sleep(0) on Linux does not work in the same way as native function sleep(0) on Windows. Also, please note I didn't try Python on Windows as I am looking just for Linux solution.

You may get a more helpful answer if you ask a question about what you're trying to achieve — Ben Graham, Feb 03 '16 at 09:47
When using `sleep(0)`, CPU usage will be 0% only if there are some other processes running, and total CPU usage will not be 0%. e.g. I saw a game using this hack in its main event loop, taking 25% CPU constantly and draining my battery. — krrr, Feb 03 '16 at 12:19

Andrea Corbellini · Answer 1 · 2016-02-03T10:19:48.290

A loop with sched_yield() is still using a certain amount of CPU time (close to 0%, but not exactly 0%). That amount may be smaller than the equivalent Python solution because a function call in Python has more overhead. Still, your C solution is all but perfect.

The situation is made worse if you increment an integer in your loop: in Python this is a slow operation and requires the creation of a new object. Also, a C int will overflow, a Python int will allocate more memory as required.

When you want an idle thread, you generally write something like this:

while True:
    time.sleep(100000)  # 100000 or another very large number

This way, you reduce the number of function calls per second, in fact making the time required for the function call negligible.

Andre is correct, though the amount of CPU usage is not insignificant. Every sleep() involves the execution of a system call which involves a context switch to the kernel. This is very expensive and you should minimize them. In fact, an important optimization is to minimize such calls. If you can find a way to make the wake up event drive, that's even better. — Taylor Kidd, Feb 04 '16 at 17:05

score 1 · Answer 2 · answered Feb 03 '16 at 14:41

For me the C example also uses 100% of one processor. It's just that most of these 100% are shown as time used in systems calls. The sched_yield() I presume. This is the complete C program I've used:

#include <sched.h>

int main(int argc, char const *argv[])
{
    int i = 0;

    while (1) {
       i++;
       sched_yield();
    }
    return 0;
}

A similar effect can be seen by using the C function from Python via the ctypes module from the standard library. The main difference is the ratio between process and system time. In Python there is much more work involved incrementing an integer, which can grow beyond the bitsize of CPU registers, and calling a C function through a proxy object in a dynamically typed language. Here is the code:

#!/usr/bin/env python
# coding: utf8
from __future__ import absolute_import, division, print_function
import ctypes

LIBC = ctypes.CDLL('libc.so.6')
shed_yield = LIBC.sched_yield


def main():
    i = 0
    while True:
        i += 1
        shed_yield()


if __name__ == '__main__':
    main()

On the (rather slow) system I've tried this, the system calls take about 50% of the processor load.

Python 2.7 - Linux - Infinite loop, relinquish CPU to other threads / processes

2 Answers2