Why does logging.Logger reset its state in new process on Windows?

Question

I'm running into strange problem with python logging module on windows. The code below shows it:

import multiprocessing
import logging
import sys



class ProcessLogger(multiprocessing.Process):
    def __init__(self):
        super().__init__()
        self.create_logger()
        print('state of logger in main proccess:')
        print(self.logger)
        print(self.logger.handlers)

    def run(self):
        print('state of logger in child proccess:')
        print(self.logger)
        print(self.logger.handlers)

    def create_logger(self):
        self.logger = logging.getLogger('something')
        self.logger.setLevel(logging.DEBUG)
        handler = logging.StreamHandler()
        self.logger.addHandler(handler)


if __name__ == '__main__':
    logg = ProcessLogger()
    logg.start()
    logg.join()
    print(sys.version)

it prints following output on Windows:

state of logger in main proccess:
<Logger something (DEBUG)>
[<StreamHandler <stderr> (NOTSET)>]
state of logger in child proccess:
<Logger something (WARNING)>
[]
3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]

for some reason logger object in new process has default state.

I tried this on ubuntu, and it seems to work as expected:

state of logger in main proccess:
<Logger something (DEBUG)>
[<StreamHandler <stderr> (NOTSET)>]
state of logger in child proccess:
<Logger something (DEBUG)>
[<StreamHandler <stderr> (NOTSET)>]
3.7.1 (default, Oct 22 2018, 11:21:55) 
[GCC 8.2.0]

Is this expected behaviour on windows? Can someone explain this?

I bet what you are seeing is simply the fact that Windows has no `fork` system call so the way in which subprocesses are spawned is different and mostly requries creating a brand new process and passing in some state serialized using the `pickle` protocol...not always all state is preserved — Bakuriu, May 30 '19 at 20:37

score 1 · Accepted Answer · answered May 31 '19 at 12:44

The reason you observe the above behaviour on the two Operating Systems is due to the difference between the fork (Unix) and the spawn (Windows) process starting strategies.

With the fork strategy, a child process is created as an exact clone of the parent process. The two processes will be identical at first and the child will share all the properties (opened files, configuration choices etc..) of the parent.

With the spawn strategy instead, a new blank process is started. The process is given the Python module to load and pointer to the first instruction to execute in the run method. All the operations performed by the parent beforehand are not inherited.

Why does logging.Logger reset its state in new process on Windows?

1 Answers1