affinity.get_process_affinity_mask(pid) returns ValueError 22

Question

I want to put my thread workers on a certain CPU(i want to test how GIL impacts on my program...), and i find a third-party library called affinity. I used pip install affinity to make it available in my VM(Linux), unfortunately, i got the error below:

>>>pid = os.getpid()
>>>affinity.get_process_affinity_mask(pid)
Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: (22, 'Invalid argument')

from the code insight, it's supposed to work on Linux platform:

...
elif sys.platform in ('linux2'):
from _affinity import set_process_affinity_mask, get_process_affinity_mask
...

Could anyone give me some clue on this error? or is there any other ways i could use in my case?

Valdar Moridin · Answer 1 · 2014-07-02T11:27:52.727

Looking at the C code and the man page for sched_getaffinity, I'm not too surprised that it might fail with 'invalid argument'. The C code is passing in a single unsigned long where the function expects a cpu_set_t, which can be up to 128 unsigned longs.

I don't know much about the cpu_set_t structure, but on the surface it appears it might be one value per physical CPU, with individual cores represented as bits within one of those values. In that case, I would expect this module to fail on any machine with more than one CPU.

My system is a single-CPU dual core, so the module works for me. Is your VM configured with more than one physical CPU? As a test, try reconfiguring it to have only one with multiple cores and see if you are more successful.

If I'm right, the only way around this is modifying the C module to handle the cpu_set_t result correctly, probably using the macros described in CPU_SET (3).

What is your VM environment? VM software, CPU/Core count, Linux version? Give this test program a try and see if your output is any different:

$ cat test.c 
#include <stdio.h>
// The CPU_SET man page says to use the second definition, but my system
// wouldn't compile this code without the first one.
#define __USE_GNU
#define _GNU_SOURCE
#include <sched.h>
#include <errno.h>

int main(void) {
    cpu_set_t cur_mask;
    unsigned int len = sizeof(cpu_set_t);

    if (sched_getaffinity(0, len, &cur_mask) < 0) {
        printf("Error: %d\n", errno);
        return errno;
    }

    int cpu_count = CPU_COUNT(&cur_mask);
    printf("Cpu count: %d and size needed: %d\n", cpu_count, CPU_ALLOC_SIZE(cpu_count));
    return 0;
}
$ gcc -std=c99 test.c 
$ ./a.out 
Cpu count: 2 and size needed: 8

On my system, it seems one unsigned long is enough to hold up to 64 CPUs, so it appears much simpler than I thought. Different hardware/architectures/kernel versions could always differ though.

Thanks @ValdarMoridin, unfortunately, i still have the same error when i tested on VM with only 1 cpu(i found this information from the VM settings as well as using multiprocessing.cpu_count()). — 3quanfeng, Jul 02 '14 at 07:59

affinity.get_process_affinity_mask(pid) returns ValueError 22

1 Answers1