Python: Variable not set when process sets it

Question

Actually I don't know what title I should give this question, because I'm rather new to Python and a few different things could be wrong here.

The idea is to create a websocket server in case a port is given and otherwise to create a dummy implementation.

The state.py-module is simple and short:

_dbg = ""

main.py:

import getopt
import multiprocessing
import sys
import time

import state

try:
  (opts, args) = getopt.getopt(sys.argv[1:], "p:", [ "ws-port=" ])
except getopt.GetoptError:
  sys.exit(2)
for (opt, arg) in opts:
  if opt in ("-p", "--ws-port"):
    state.websocketPort = int(arg)

#########
# and now the important part:
#########

if hasattr(state, "websocketPort"):
  def initWebsocketServer():
    print("iWS-1")
    state.websocketServer = WebsocketServer(state.websocketPort)
    print("iWS-2")
    state.websocketServer.run_forever()
    print("iWS-3")
  wsThread = multiprocessing.Process(target = initWebsocketServer, daemon = True)
  wsThread.start()
else:
  print("dummy")
  def dummy(msg):
    pass
  state.websocketServer = type('', (), {})()
  state.websocketServer.send_message_to_all = dummy

time.sleep(10) # not even a sleep makes it work!
state.websocketServer.send_message_to_all("hello")
print("done")

Shell:

$ python main.py -p 1234
iWS-1
iWS-2
Traceback (most recent call last):
  File "main.py", line 38, in <module>
    state.websocketServer.send_message_to_all("hello")
AttributeError: module 'state' has no attribute 'websocketServer'
$ python main.py
dummy
done
$

So all ifs, checks and the dummy implementation seems to work fine, but assigning the actual instance to the module variable does not. What's wrong here?

If there is a totally different or better approach to this, please let me know.

EDIT:
While trying to understand the effect of the different entities I came up with this:

if hasattr(state, "websocketPort"):
  def initWebsocketServer():
    print("iWS-1")
    print("iWS-2")
    state.websocketServer.run_forever()
    print("iWS-3")
  # this assignment was between the first two print()s earlier:
  state.websocketServer = WebsocketServer(state.websocketPort)
  wsThread = multiprocessing.Process(target = initWebsocketServer, daemon = True)
  wsThread.start()

And it works... As there won't be more interaction other than calling the dummified method I guess I can keep this as a solution.

Or would there be something else broken?

You are not using threads, your are using processes. Processes do not share application state, including module attributes. The state module of the initial and child process are separate entities. — MisterMiyagi, Sep 14 '19 at 21:34
Either use threads or explicitly pass messages between processes. The multiprocessing documentation lists the various approaches. — MisterMiyagi, Sep 14 '19 at 21:37
@MisterMiyagi I think I found a solution without messaging and all. Please see my edit. — sjngm, Sep 14 '19 at 21:54
Consider using `argparse` instead. From the `getopt` documentation: "Note The getopt module is a parser for command line options whose API is designed to be familiar to users of the C getopt() function. Users who are unfamiliar with the C getopt() function or who would like to write less code and get better help and error messages should consider using the argparse module instead." — chepner, Sep 14 '19 at 22:32

score 0 · Answer 1 · answered Sep 14 '19 at 22:36

0

When hasattr(state, 'websocketPort') is true, state.websocketServer is only defined as a local variable in initWebsocketServer, never in the main process.

answered Sep 14 '19 at 22:36

chepner

497,756
71
530
681

Are you talking about the first version or the edited version I added a few minutes later? Because I moved that and it should be in the main process. – sjngm Sep 15 '19 at 14:38
The original. In your edit, you've now made `initWebsocketServer` (which would more accurately be called `startWebsocketServer` now) operate on a non-local name `state` which is, indeed, initialized in the main process. *However*, each process has its own reference to what I believe is a shared instance of `WebsocketServer`; there may be unintended side effects as a result. – chepner Sep 16 '19 at 12:33

score 0 · Accepted Answer · answered Sep 15 '19 at 17:21

Well, it turned out that my edited part also didn't work, it just didn't throw an error and reached the end of the script. That's not all I wanted it to do... :)

I guess I tried to do something that is inconveniently hard to do. I switched to regular threading and hope that I don't run into those situations that processes can handle better than threads...

if hasattr(state, "websocketPort"):
  state.websocketServer = WebsocketServer(state.websocketPort)
  wst = threading.Thread(target = state.websocketServer.run_forever, daemon = True)
  wst.start()
else:
  print("dummy")
  def dummy(msg):
    pass
  state.websocketServer = type('', (), {})()
  state.websocketServer.send_message_to_all = dummy

Python: Variable not set when process sets it

2 Answers2