2

How to limit number of threads used in

Concurrency::parallel_for<int>(0, 100, 1, [&](int k)

I saw the scheduler/task idea, I fail to use it cause inside the parallel for there is a lot of logic and I need to pass arguments, all the examples for tasks is containing only std::cout<<"Hey"<<std::endl; inside the task.

Hope you have some ideas.

bool func1(int x,int y....) //A lot of params{
 Concurrency::parallel_for<int>(0, 100, 1, [&](int k) {
//a lot of logic depends on the input
}
}
Hasturkun
  • 35,395
  • 6
  • 71
  • 104
InUser
  • 1,138
  • 15
  • 22
  • Limit it to what? 1 is really, really easy. ;) – Yakk - Adam Nevraumont Jul 07 '21 at 15:36
  • Number of threads, not depended on num of cpu’s(less) – InUser Jul 07 '21 at 17:58
  • Where has `Concurrency` been defined? – zkoza Jul 10 '21 at 09:43
  • im not sure i understand you, the use of the fir inside a function, i did not make any special predefines – InUser Jul 10 '21 at 12:33
  • You said that you "fail to use it cause inside the parallel for there is a lot of logic and I need to pass arguments". In this case, it usually helps if you provide a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) that demonstrates exactly such a case. Otherwise, it is not clear what doesn't make this a duplicate of e.g. [How to set number of PPL threads to one?](https://stackoverflow.com/q/11615770/12345551) or [PPL - How to configure the number of native threads?](https://stackoverflow.com/q/40735170/12345551) – He3lixxx Jul 10 '21 at 13:14
  • 1
    I'm not sure I understood your question. You can always define a global variable acting as a semaphore that gets checked right at the entry of the function to check if the required limit of simultaneous function call has been reached. – Bilal Qandeel Jul 13 '21 at 00:05

2 Answers2

1

You are referring to https://learn.microsoft.com/en-us/cpp/parallel/concrt/reference/concurrency-namespace-functions?view=msvc-160#parallel_for.

I don't think there is a super simple "one function call" solution to this. Based on the docs you need to change the policy of the current scheduler so that any parallel_for you run is limited by the scheduler to use only a specific number of resources. So you'd want to get the current SchedulerPolicy, update SetConcurrencyLimits() on it, and then update the current policy by calling concurrency::CurrentScheduler::Create() with your modified policy. This should let you limit the number of total threads when the parallel_for executes. You'd definitely need to experiment and test including disabling your modified scheduling policy when you were done with your calls.

It may be easier to restructure your code (i.e. chunk the parallel_for calls so that each parallel_for only executes the the number of concurrent threads you want to execute). That will be less efficient but may be easier to write and maintain.

References:

https://learn.microsoft.com/en-us/cpp/parallel/concrt/reference/schedulerpolicy-class?view=msvc-160

https://learn.microsoft.com/en-us/cpp/parallel/concrt/scheduler-instances?view=msvc-160

Sean Walker
  • 108
  • 1
  • 9
1

I haven't used the interface - but the following might work (assuming 8 workers and parallel for 100 cases - otherwise adjust the 100/8).

Concurrency::simple_partitioner splitter(100/8);
Concurrency::parallel_for<int>(0, 100, 1, [&](int k) {
//a lot of logic depends on the input
}, splitter);

This does not limit the number of threads directly, but chunk the data achieving the same.

The idea could also be found on: https://katyscode.wordpress.com/2013/08/17/c11-multi-core-programming-ppl-parallel-aggregation-explained/ https://learn.microsoft.com/en-us/cpp/parallel/concrt/reference/simple-partitioner-class?view=msvc-160 https://learn.microsoft.com/en-us/cpp/parallel/concrt/reference/concurrency-namespace-functions?view=msvc-160#parallel_for

Hans Olsson
  • 11,123
  • 15
  • 38