0

I want to keep a python class permanently alive so I can continually interact with it. The reason for this is that this class is highly memory intensive which means that (1) I cannot fit it into memory multiple times, and (2) Loading the class is prohibitively slow.

I have tried implementing this using both Pyro and RPYC, but it appears that these packages always delete the object and create a new object every time a new request is made (which is exactly what I don't want to do.) However, I did find the following option for Pyro:

@Pyro4.behavior(instance_mode="single")

Which ensures that only a single instance is created. However, since it is possible that multiple requests will be made simultaneously I am not 100% that this is safe to do. Is there a better way to accomplish what I am trying to do?

Thanks in advance for any help, it is greatly appreciated! (I've been struggling with this for quite a while now).

L

Louis-Philippe
  • 159
  • 2
  • 9
  • Pyro's default instance mode is not to create a new instace *per request* but rather *per session* (=proxy connection). – Irmen de Jong Nov 09 '16 at 00:50
  • This is indeed the case sorry for my error! For me this was more or less the case since I only do one request per session, but this is indeed an important distinction. – Louis-Philippe Nov 09 '16 at 09:35

1 Answers1

1

If you don't want to make your class thread safe, you can set SERVERTYPE to "multiplex", this will make it so all remote method calls are processed sequentially.

https://pythonhosted.org/Pyro4/servercode.html#server-types-and-concurrency-model:

multiplexed server (servertype "multiplex")

This server uses a connection multiplexer to process all remote method calls sequentially. No threads are used in this server. It uses the best supported selector available on your platform (kqueue, poll, select). It means only one method call is running at a time, so if it takes a while to complete, all other calls are waiting for their turn (even when they are from different proxies). The instance mode used for registering your class, won’t change the way the concurrent access to the instance is done: in all cases, there is only one call active at all times. Your objects will never be called concurrently from different threads, because there are no threads. It does still affect when and how often Pyro creates an instance of your class.

Community
  • 1
  • 1
Francisco
  • 10,918
  • 6
  • 34
  • 45
  • 1
    Yup. But while setting the servertype to this is a way to ensure your pyro server objects aren't called concurrently, It's also possible to leave it at the threaded server but define the critical sections in your pyro server class and properly lock those (``threading.Lock``). What is the better option depends on what exactly your class is doing and how much of the time it will spend in the critical section. – Irmen de Jong Nov 09 '16 at 00:49
  • 1
    Thanks for this solution! I've tried it and it seems to work perfectly. I think I am going to use the "multiplex" option for now, until I can find the time to really investigate which parts of the class should be locked, the program is still somewhat under development so it may be a bit to soon to do this now with 100% certainty. – Louis-Philippe Nov 09 '16 at 18:45