Stack Overflow
Stack Overflow

Questions tagged [pool]

A pool describes a cache of resources that an application can draw from to save instantiation time.

Pools typically refer to database connections but the term can be applied to graphics handles, files and other run-time processes. When the use of such a resource is complete but the application is still running, it will typically return the resource to the pool rather than release it.

1504 questions
35
votes
1 answer

How does the callback function work in multiprocessing map_async?

It cost me a whole night to debug my code, and I finally found this tricky problem. Please take a look at the code below. from multiprocessing import Pool def myfunc(x): return [i for i in range(x)] pool=Pool() A=[] r = pool.map_async(myfunc,…
user2727768
  • 666
  • 1
  • 7
  • 12
33
votes
4 answers

How can I monitor/log Tomcat's thread pool?

I have a Tomcat installation where I suspect the thread pool may be decreasing over time due to threads not being properly released. I get an error in catalina.out when maxthreads is reached, but I would like to log the number of threads in use to a…
Alex P
  • 333
  • 1
  • 3
  • 4
28
votes
2 answers

Filling a queue and managing multiprocessing in python

I'm having this problem in python: I have a queue of URLs that I need to check from time to time if the queue is filled up, I need to process each item in the queue Each item in the queue must be processed by a single process (multiprocessing) So…
Tibo
  • 281
  • 1
  • 3
  • 3
25
votes
6 answers

Mulitprocess Pools with different functions

Most examples of the Multiprocess Worker Pools execute a single function in different processes, f.e. def foo(args): pass if __name__ == '__main__': pool = multiprocessing.Pool(processes=30) res=pool.map_async(foo,args) Is there a way to…
dorvak
  • 9,219
  • 4
  • 34
  • 43
25
votes
5 answers

Pulling data from a CMSampleBuffer in order to create a deep copy

I am trying to create a copy of a CMSampleBuffer as returned by captureOutput in a AVCaptureVideoDataOutputSampleBufferDelegate. Since the CMSampleBuffers come from a preallocated pool of (15) buffers, if I attach a reference to them they cannot be…
bennyty
  • 371
  • 5
  • 18
24
votes
2 answers

ProcessPoolExecutor and Lock in Python

I am trying to use concurrent.futures.ProcessPoolExecutor with Locks, but I'm getting a run time error. (I'm working on Windows if that's relevant) Here's my code: import multiprocessing from concurrent.futures import ProcessPoolExecutor import…
David Sternlicht
  • 243
  • 1
  • 2
  • 5
24
votes
7 answers

Underlying mechanism of String pooling in Java?

I was curious as to why Strings can be created without a call to new String(), as the API mentions it is an Object of class java.lang.String So how are we able to use String s="hi" rather than String s=new String("hi")? This post clarified the use…
Sainath S.R
  • 3,074
  • 9
  • 41
  • 72
24
votes
2 answers

Python NotImplementedError: pool objects cannot be passed between processes

I'm trying to deliver work when a page is appended to the pages list, but my code output returns a NotImplementedError. Here is the code with what I'm trying to do: Code: from multiprocessing import Pool, current_process import time import…
flaviussn
  • 1,305
  • 2
  • 15
  • 33
23
votes
3 answers

Python: Writing to a single file with queue while using multiprocessing Pool

I have hundreds of thousands of text files that I want to parse in various ways. I want to save the output to a single file without synchronization problems. I have been using multiprocessing pool to do this to save time, but I can't figure out…
risraelsen
  • 253
  • 1
  • 2
  • 5
23
votes
3 answers

Do multiprocessing pools give every process the same number of tasks, or are they assigned as available?

When you map an iterable to a multiprocessing.Pool are the iterations divided into a queue for each process in the pool at the start, or is there a common queue from which a task is taken when a process comes free? def generate_stuff(): …
John Mee
  • 50,179
  • 34
  • 152
  • 186
22
votes
4 answers

What is the maximum and minimum size of connection pool ADO.Net Supports in the connection string?

What is the maximum and minimum size of connection pool ADO.Net Supports in the connection string.Min Pool Size=[max size ?]Max Pool Size=[min size]
Saurabh
  • 5,661
  • 2
  • 26
  • 32
22
votes
5 answers

Multiprocessing IOError: bad message length

I get an IOError: bad message length when passing large arguments to the map function. How can I avoid this? The error occurs when I set N=1500 or bigger. The code is: import numpy as np import multiprocessing def func(args): i=args[0] …
Andy
  • 1,072
  • 2
  • 19
  • 33
22
votes
8 answers

Multiprocessing.Pool makes Numpy matrix multiplication slower

So, I am playing around with multiprocessing.Pool and Numpy, but it seems I missed some important point. Why is the pool version much slower? I looked at htop and I can see several processes be created, but they all share one of the CPUs adding up…
Framester
  • 33,341
  • 51
  • 130
  • 192
21
votes
4 answers

C++ object-pool that provides items as smart-pointers that are returned to pool upon deletion

I'm having fun with c++-ideas, and got a little stuck with this problem. I would like a LIFO class that manages a pool of resources. When a resource is requested (through acquire()), it returns the object as a unique_ptr that, upon deletion, causes…
swalog
  • 4,403
  • 3
  • 32
  • 60
20
votes
2 answers

Python multiprocessing with pathos

I am trying to use Python's pathos to designate computations into separate processes in order to accelerate it with multicore processor. My code is organized like: class: def foo(self,name): ... setattr(self,name,something) ... def…
user3708829
  • 199
  • 1
  • 1
  • 4
1
2
3
…
99 100