Stack Overflow
Stack Overflow

Questions tagged [pool]

A pool describes a cache of resources that an application can draw from to save instantiation time.

Pools typically refer to database connections but the term can be applied to graphics handles, files and other run-time processes. When the use of such a resource is complete but the application is still running, it will typically return the resource to the pool rather than release it.

1504 questions
11
votes
2 answers

Optimizing multiprocessing.Pool with expensive initialization

Here is a complete simple working example import multiprocessing as mp import time import random class Foo: def __init__(self): # some expensive set up function in the real code self.x = 2 print('initializing') def…
user2133814
  • 2,431
  • 1
  • 24
  • 34
11
votes
3 answers

Python multiprocessing - tracking the process of pool.map operation

I have a function which performs some simulation and returns an array in string format. I want to run the simulation (the function) for varying input parameter values, over 10000 possible input values, and write the results to a single file. I am…
user32147
  • 1,033
  • 4
  • 12
  • 22
11
votes
2 answers

What is the difference between maxTotal and maxIdle in Apache Commons Pool 2?

I'm using the Apache Commons Pool 2 implementation to have object pool mechanism for my application. As of now, I have set the default value of maxTotal() and maxIdle() as 10 in my code. But I am not able to understand what is the difference…
Saurabh Gokhale
  • 53,625
  • 36
  • 139
  • 164
11
votes
2 answers

To Pool or not to Pool java crypto service providers

Solution MessageDigest => create new instances as often as needed KeyFactory => use a single shared instance SecureRandom => use a StackObjectPool Cipher => use a StackObjectPool Question I face a regular dilemna while coding security frameworks…
Cerber
  • 2,889
  • 4
  • 27
  • 43
11
votes
4 answers

How to get the amount of "work" left to be done by a Python multiprocessing Pool?

So far whenever I needed to use multiprocessing I have done so by manually creating a "process pool" and sharing a working Queue with all subprocesses. For example: from multiprocessing import Process, Queue class MyClass: def __init__(self,…
E.Z.
  • 6,393
  • 11
  • 42
  • 69
10
votes
5 answers

Python HTTPConnectionPool Failed to establish a new connection: [Errno 11004] getaddrinfo failed

I was wondering if my requests is stopped by the website and I need to set a proxy.I first try to close the http's connection ,bu I failed.I also try to test my code but now it seems no outputs.Mybe I use a proxy everything will be OK? Here is the…
cwl
  • 184
  • 1
  • 1
  • 10
10
votes
2 answers

Python Multiprocess Pool. How to exit the script when one of the worker process determines no more work needs to be done?

mp.set_start_method('spawn') total_count = Counter(0) pool = mp.Pool(initializer=init, initargs=(total_count,), processes=num_proc) pool.map(part_crack_helper, product(seed_str, repeat=4)) pool.close() pool.join() So I have a pool of worker…
whiteSkar
  • 1,614
  • 2
  • 17
  • 30
10
votes
5 answers

The best alternative for String flyweight implementation in Java

My application is multithreaded with intensive String processing. We are experiencing excessive memory consumption and profiling has demonstrated that this is due to String data. I think that memory consumption would benefit greatly from using some…
Dan
  • 11,077
  • 20
  • 84
  • 119
10
votes
6 answers

Does it make sense to have a sql PreparedStatement pool?

As a PreparedStatatement contains precompiled sql command(s), so when if we make a pool of this type in order not create and destroy this object too much(just like thread pool). Does it make any sense? or I'm just so confused?
user2889419
10
votes
2 answers

Python multiprocessing.Pool() doesn't use 100% of each CPU

I am working on multiprocessing in Python. For example, consider the example given in the Python multiprocessing documentation (I have changed 100 to 1000000 in the example, just to consume more time). When I run this, I do see that Pool() is using…
geekygeek
  • 323
  • 1
  • 3
  • 11
10
votes
1 answer

What is a couchbase pool

In couch base URL, e.g. server:port/pools/default what exactly a couch base pool is. Will it always be default or we can change it. There is some text written…
naveed
  • 809
  • 6
  • 7
10
votes
1 answer

Alternative use patterns for python multiprocessing avoiding proliferation of global state?

This (enormously simplified example) works fine (Python 2.6.6, Debian Squeeze): from multiprocessing import Pool import numpy as np src=None def process(row): return np.sum(src[row]) def main(): global src src=np.ones((100,100)) …
timday
  • 24,582
  • 12
  • 83
  • 135
9
votes
5 answers

Tips for using commons-pool in production

Based on an answer I got here, I started to give commons-pool a serious look. My last experience of using it was around 2003, probably version 1.1 or 1.2. Its main user, DBCP, is considered by many as flawed and to be avoided. Does anyone uses…
David Rabinowitz
  • 29,904
  • 14
  • 93
  • 125
9
votes
4 answers

Memory pools implementation in C

I am looking for a good memory pool implementation in C. it should include the following: Anti fragmentation. Be super fast :) Ability to "bundle" several allocations from different sizes under some identifier and delete all the allocations with…
Avi Zrachya
  • 151
  • 2
  • 6
9
votes
1 answer

Create a Pool of JAXB Unmarshaller

I was looking around to find a way to improve JAXB Unmarshalling performances processing huge sets of files and found the following advice: "If you really care about the performance, and/or your application is going to read a lot of small documents,…
Bruno Chauvet
  • 196
  • 2
  • 7
1 2 3
…
99 100