In Python's multiprocessing package, why is there multiprocessing.Pool and multiprocessing.pool.Pool?

Question

While learning about Python's multiprocessing package (for Python 3.4), I noticed multiprocessing.Pool is defined in the Class BaseContext in context.py. This definition is

   def Pool(self, processes=None, initializer=None, initargs=(),
         maxtasksperchild=None):
    '''Returns a process pool object'''
    from .pool import Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
                context=self.get_context())

Thus, it ends up calling multiprocessing.pool.Pool defined in pool.py.

Why does the multiprocessing package define both multiprocessing.Pool and multiprocessing.pool.Pool?

Maybe so you can override its pool manager with your own? – wallyk Jul 28 '15 at 00:42 — wallyk, Jul 28 '15 at 00:42

score 3 · Accepted Answer · answered Jul 28 '15 at 01:04

context.py contains OS-dependent code. The Pool and many other values are defined differently depending on whether sys.platform == 'win32' or not.

The pool.py module contains code related the creation of a Pool given a context.

By organizing the code in this way, the developer manages to write pool.py in an OS-agnostic way. There are no if sys.platform ... statements in pool.py, for example.

The __init__.py contains:

globals().update((name, getattr(context._default_context, name))
                 for name in context._default_context.__all__)
__all__ = context._default_context.__all__

which copies values from context._default_context into the multiprocessing packages' namespace. This is how multiprocessing.Pool gets defined.

As a user of the multiprocessing package, you are expected to access the Pool via multiprocessing.Pool, although it is possible to use multiprocessing.pool.Pool too.

In Python's multiprocessing package, why is there multiprocessing.Pool and multiprocessing.pool.Pool?

1 Answers1