0

Is there a way to synchronize two or more python interpreters (either embedded in a single process, or running in separate processes) on the opcode level, other than hacking on the Python VM implementation(s)? Are there any alternative Python VM implementations that would support this?

Thanks!

Petr Broz
  • 8,891
  • 2
  • 15
  • 24
  • 1
    What do you mean with "synchronizing them on the opcode level"? What do you want to achieve? Do common IPC/sync idioms not help in your case, such as mutexes, locks, pipes, sockets? – Dr. Jan-Philip Gehrcke Feb 06 '15 at 20:07
  • I have several Python scripts provided by users and I'd like to execute them in a somewhat fair way, one opcode at a time. So for example, if I have 3 embedded interpreters (vm1, vm2, vm3) for 3 different user scripts, I'd like to be able to execute them in a loop, processing a single opcode in each VM. – Petr Broz Feb 06 '15 at 20:41
  • "execute them in a somewhat fair way"? Your operating system is doing this for you!! This is called `scheduling`. Independent processes (as your interpreters are) get assigned a comparable amount of CPU time, if you did not mess with priorities. – Dr. Jan-Philip Gehrcke Feb 06 '15 at 20:43
  • Thanks, Jan-Philip. I've considered relying on the OS scheduling as well, although I'm not sure if that is truly deterministic. In other words, if I run the same 3 user scripts again, I don't think I could guarantee that the OS would schedule those in the same way, would it? – Petr Broz Feb 06 '15 at 20:44
  • Deterministic in theory. But you are right, this is usually not deterministic in practice, because you, as a user, will probably not be able to re-create the exact same conditions. But what holds true is that your operating system tries to be fair, every time you run those processes concurrently. If you need more control, i.e. synchronization between processes (that is, process B should only proceed if process A has assumed a certain state), you need synchronization primitives such as these mentioned in my first comment. – Dr. Jan-Philip Gehrcke Feb 06 '15 at 20:47
  • You're right, I should've mentioned the deterministic requirement in the initial question. Thanks for the feedback! – Petr Broz Feb 06 '15 at 20:50
  • If this is just about multi-user fairness, "one opcode at a time" is a ridiculously fine-grained level on which to go about this and will likely cause huge overhead. Is there some particular reason they need to be in deterministic lock-step? – tzaman Feb 06 '15 at 22:14
  • It's about multi-user fairness and deterministic/reproducible behavior. It's for a simple simulation/game similar to robocode or battlecode. – Petr Broz Feb 06 '15 at 22:26

1 Answers1

1

I am pretty sure after our discussion in the comments that you may want to use the classical synchronization primitives, as documented e.g. here: https://docs.python.org/2/library/multiprocessing.html#synchronization-primitives.

The idea these tools/primitives have in common is to control and maintain determinism in your system comprised of multiple components. These primitives can make sure that things are processed in the right order, always. That is, these methods are able to create the determinism you are looking out for.

Of course you should read about what these primitives have been designed for. There are many resources out there. A good article on this topic may be this one here: http://www.cs.cf.ac.uk/Dave/C/node31.html (note that it deals with Unix internals and threads, but the ideas are the same for all operating systems and also across processes, not only threads).

Dr. Jan-Philip Gehrcke
  • 33,287
  • 14
  • 85
  • 130