3

I have two processes which I want to communicate with each other:

file hwmgr.py:

import multiprocessing as mp
from setproctitle import setproctitle
import smbus
import myLoggingModule as log

class HWManager(mp.Process):
    def __init__(self):
        mp.Process.__init__(self)
        self.i2c_lock = mp.Lock()
    def run(self):
        setproctitle('hwmgr')
        # self.logger = log.config_logger(**kwargs)
    def get_voltage(self):
        with self.i2c_lock:
            # ...do i2c stuff to get a voltage with smbus module
        # self.logger.debug('Got a voltage: %s', voltage)
        return voltage

file main.py:

import hwmgr

hwm = hwmgr.HWManager()
hwm.start()

battery = hwm.get_voltage()

print battery  # Works!

So, interestingly, this works as expected - the voltage is returned by the method call without any special multiprocessing wizardry. However, if I enable the two lines involving the logger, when the logger.debug() call is encountered, I get:

AttributeError: 'HWManager' object has no attribute 'logger'

And, indeed, if I print a dir(self) right about there, it doesn't have a logger.
I don't get it?? Where did my logger go?

The reason the logger is defined in the run() method, rather than __init__() is because I'm after the root logger of the new process, and because the logger's file name is taken from the new process title (getproctitle()) which can't be called until after the process has forked at completion of the __init__() method -- there might be another way to do this part, of course, but I haven't found it yet...

WIP code:
I've removed the reference to the logging module - it doesn't matter what the attribute is.
If you comment out the line print houdiniAttribute, everything works as expected
Just to be clear, the passing a return int works - the disappearing attribute is the concern

file hwmgr.py:

import multiprocessing as mp
from setproctitle import setproctitle
import smbus

class HWManager(mp.Process):
    def __init__(self):
        mp.Process.__init__(self)
        self.i2c_lock = mp.Lock()

    def run(self):
        setproctitle('hwmgr')
        self.houdiniAttribute = 'ASDFGHJKL'
        with self.i2c_lock:
            pass  # Set up I2C bus to take ADC readings

        while True:  # Doesn't matter if this is here...
           pass

    def get_voltage(self):
        with self.i2c_lock:
            voltage = 12.0  # Actually, do i2c stuff to get a voltage with smbus module
        print self.houdiniAttribute
        return voltage

file client.py:

import multiprocessing as mp
from setproctitle import setproctitle
from time import sleep

class HWClient(mp.Process):
    def __init__(self, hwm):
        mp.Process.__init__(self)
        self.hwm = hwm

    def run(self):
        setproctitle('client')
        while True:
            battery = self.hwm.get_voltage()
            print battery
            sleep(5)

file main.py:

import hwmgr
import client

hwm = hwmgr.HWManager()
hwm.start()
cl = client.HWClient(hwm)
cl.start()
BugSpray
  • 113
  • 9
  • To add information to your question, please click the "edit" link under the question rather than posting a new question and linking to the original question. In fact, the original question as you have it is likely to be closed as "too broad", but if you combine it with this attempted solution (I'd recommend some editing for brevity), than I'd consider it worthy of an upvote. – Adi Inbar Apr 01 '14 at 01:10
  • possible duplicate of [Python multiprocessing design suggestions](http://stackoverflow.com/questions/22774756/python-multiprocessing-design-suggestions) – Adi Inbar Apr 01 '14 at 01:11
  • Further more, it seems any attribute added in the run method disappears, and is no longer accessible in other methods. Is this a question of variable scope? – BugSpray Apr 01 '14 at 05:36
  • Note to mod: Originally this question was a reference to an off-topic design/discussion question. There's no way to retract a flag, but I no longer think this one should be deleted and merged into the other question, now that the OP deleted the other question and edited this one to make it a stand-alone question. – Adi Inbar Apr 01 '14 at 16:33

1 Answers1

1

Attempt to clarify:

  1. You create a process object 1 in process 1
  2. The process object spawns a new process (process 2)
  3. In process 2, run() is called on another object (object 2) of the same class.
  4. run() assigns attributes to object 2.
  5. Process 2 finishes and removes object 2.
  6. Process 1 now knows that process 2 is done. object 1 still has the old attributes.

If you want to synchonize stuff, have a look at the managers. Multiprocessing Share Unserializable Objects Between Processes

Does this answer your question?

Community
  • 1
  • 1
User
  • 14,131
  • 2
  • 40
  • 59
  • Even if I add something like `while True: pass` after the `self.logger = log...` line (ie. so the HWManager never "finishes") the reference to the logger still doesn't work - but the references to its methods work just fine. I can call them from `main.py` without any drama. This is not really a question about serialisation or synchronisation -- HWManager is a class/process, one of many, that are all spawned from an original script file. I'm after some kind of way for the other spawned processes to call methods on the Hardware Manager, and get return values. – BugSpray Apr 01 '14 at 10:53
  • Have you read the linked question and the referred to questions? It may help. Until now I do see your question as equal to http://stackoverflow.com/questions/19468885/how-to-properly-set-up-multiprocessing-proxy-objects-for-objects-that-already-ex – User Apr 01 '14 at 11:45
  • The linked question talks about synchronising objects between processes if I understand it correctly. I don't have any problem moving data between processes - try running the WIP code above, and comment out the line `print houdiniAttribute` - works fine. What I don't understand is where does that attribute go, when the line is enabled. – BugSpray Apr 01 '14 at 22:14
  • I should point out, too - the whole point of the HWManager is to continue running - this is not a "job" to be completed then exit. This is a hardware manager for an embedded system, so the HWManager process will continue running, and other processes request services from it - like "give me the battery voltage so I can log it", or "close a relay output, and tell me if it was successful". The other procs all have a reference to the hardware manager on their creation (like in the WIP code), so this seems to work -- except that I'd like the HWManager to log its activity. – BugSpray Apr 01 '14 at 22:21
  • So, the original copy of my new mp.Process object is the one receiving the messages send by main, not the copy produced after a fork? – BugSpray Apr 02 '14 at 23:08