I'd like to ask if my explanation of the behavior I observe is correct.
The behavior: After restarting cgconfig daemon, the processes have correct core affinity (as verified by taskset) but the threads end up with affinity for ALL cores (0-35 in our case, also as verified by taskset).
The explanation: Shutting down cgconfig daemon unmounts the /cgroup filesystem. That puts all processes into the ROOT cgroup. When cgconfig is restarted, it applies the correct cgroup to processes but not to threads.
The evidence: I executed the following command to find out who is running on the "isolated" core 19:
ps -eL -o "tid,pid,psr"| grep "19$"
Only one non-kernel thread should be running there, but I see a lot of them.
Example:
180978 180957 19
Checking on this tid/pid, I see this for thread:
$ taskset -pc 180978
pid 180978's current affinity list: 0-35
And this for process
$ taskset -pc 180957
pid 180957's current affinity list: 1,3-18,20-28,30,32,33,35
So thread somehow ended up with different affinity from the process.
Furthermore, looking under /cgroup mount yields this for process:
$ find /cgroup/cpuset -name tasks | xargs grep 180957
/cgroup/cpuset/appXXX/XXXcommon/tasks:180957
And this for thread:
$ find /cgroup/cpuset -name tasks | xargs grep 180978
/cgroup/cpuset/tasks:180978
So we can see that the thread somehow ended up in root group.
To reiterate: could someone please confirm or deny that the restarting of the cgconfig daemon sets the cgroups for processes correctly, but leaves the threads in the root cgroup.
Environment: Red Hat Enterprise Linux Server release 6.8 (Santiago)
