Isolate function from runtime

Question

I'm not sure how to call that, if anyone can think of better title, let me know, I'll rename this question.

This is not real-life example, but if I get solution to such situation, I'll use it in bigger project. Just assume that I HAVE TO do it like described below, I can't change the idea, I need to find solution that will fit it. At the moment I'm not allowed to show any more details of original project.

Idea:

So, lets assume I'm creating cron-like thingie, basing on YAPSY plugins. I'd like to store plugins in some directory, once in a while my daemon would collect all plugins from that directory, call their methods and go to sleep for some more time. Those plugins should be able to access some singleton, which will store some data, for example URLs used by plugins, etc. I also have TCP server running in the same process, that will modify this singleton, so I can customize behaviour on runtime. TCP server should have read/write access to singleton, and plugins should have read-only access. There may be many singletons (I mean, many classes behaving as singleton, not many instances of one class, duh) readable to plugins and modifiable by TCP server. Plugin methods are called with some values generated just before we call them, and they can be generated only in the process in which they live.

Question:

How do I grant read-only access to plugins? I want total isolation, so that if singleton has field x, which is object, then fields of singleton.x (like singleton.x.y) should also be read-only for plugins. Read-only means that plugins should be able to modify those fields, but it won't have any effect on rest of runtime, so when plugins method returns, singleton (and its fields, and their fields, etc) should be the same as before running plugins method, so its not really read-only. Also, plugins may be ran in concurrent fashion, and release GIL for some time (they may have IO operations, or just use time.sleep()).

--EDIT--

Solution has to be multiplatform, and work at least on Linux, Windows and MacOS.

--/EDIT--

Approaches:

1. I could try inspecting stack in singletons methods to see if any caller is plugin, and if so, storing original value of any modified field. Then, after plugin method call, I would use function restore() which would restore singleton to state before running plugin.

2. I could try running plugins method in another process, with multiprocessing, passing all singletons (easily done, by using metaclass for keeping track of all of them, and rebuilding them in new process, or explicitly storing singletons somewhere) to subprocess.

3. I could try wrapping globals() and locals() into some dicts, that would do similiar trick as in point (1) (with restoring original values) or would deep copy all globals and locals, and run it with exec, with code of plugins method (not with string, I know that's unsafe).

Why won't approaches above work?

(1): Stack inspection is usually wrong, and I'd say in this situation it is very wrong. Also, restoring variables after each call could be very expensive, when there would be many modifications done by plugin. Also, plugin methods may run in concurrent fashion, so I would need to restore original values every time GIL is released, and restore plugin-wide values any time GIL is acquired - that would hurt a lot (can you even imagine implementing this? At this moment, I can't, and I'm not really sorry about that).

(2): YAPSY plugins are not picklable, so I cannot send them to subprocess.

(3): exec() won't take code with free variables for execution, it can't see scope in which it is called, so I would need to find all free variables of plugins function (I'd use wrapper, generated on runtime, like this:

def no_arg_plugin_call():
    plugin.method(something, from_, locals_)

and pass no_args_plugin_call.__code__ ) and store them in wrapped locals(). Also, deep copy of the whole environment would be as expensive as in (1).

PS. By "fields" I mean "attributes", because I've (unfortunately) grown up on Java and alike.

PPS. If you've heard about any plugin system that will be similiar to YAPSY (it has to have all the features and be as lightweight) and will generate picklable instances, that would an enough for me ;)

Answer 1

How about this: you fork, and then the child process can do whatever it wants with any object. It will not affect the parent. Here's and example:

I wrote 2 plugins, they get an object with the relevant info from the caller, print a message and then they modify what they got - but this will not influence the original copy. here's the first one:

import yapsy.IPlugin

class SayHi( yapsy.IPlugin.IPlugin ):
    def doSomething( self, infoForPlugins ):
        print( "plugin SayHi got: %s" % infoForPlugins[ 'SayHi' ] )
        old = infoForPlugins[ 'SayHi' ]
        infoForPlugins[ 'SayHi' ] = 'want to say "hello" instead of "%s"' % old
        print( "plugin SayHi changed info into: %s" % infoForPlugins[ 'SayHi' ] )

here's the second, almost identical one:

import yapsy.IPlugin

class SayBye( yapsy.IPlugin.IPlugin ):
    def doSomething( self, infoForPlugins ):
        print( "plugin SayBye got: %s" % infoForPlugins[ 'SayBye' ] )
        old = infoForPlugins[ 'SayBye' ]
        infoForPlugins[ 'SayBye' ] = "I don't like saying %s!!!" % old
        print( "plugin SayBye changed info into: %s" % infoForPlugins[ 'SayBye' ] )

here is the cron-thingy code, complete with a server that lets you modify the info on the fly (I used UDP for this example to keep it minimal, but you can use whatever mechanism you want)

import yapsy.PluginManager
import os
import logging
import time
import threading
import socket

logging.basicConfig( level = logging.DEBUG )

class _UDPServer( threading.Thread ):
    def __init__( self, infoForPlugins ):
        threading.Thread.__init__( self )
        self._infoForPlugins = infoForPlugins
        self.daemon = True
        self._sock = socket.socket( socket.AF_INET, socket.SOCK_DGRAM )
        self._sock.bind( ( '', 2222 ) )

    def run( self ):
        while True:
            packet = self._sock.recv( 4096 )
            key, value = packet.split( ':' )
            self._infoForPlugins[ key ] = value

class CronThingy( object ):
    def __init__( self ):
        self._infoForPlugins = { 'SayHi': 'hi there', 'SayBye': 'bye bye' }
        self._pluginManager = yapsy.PluginManager.PluginManager()
        self._pluginManager.setPluginPlaces( [ 'plugins' ] )
        self._pluginManager.collectPlugins()
        _UDPServer( self._infoForPlugins ).start()

    def go( self ):
        while True:
            logging.info( 'info before run: %s' % self._infoForPlugins )
            self._runPlugins()
            time.sleep( 1 )
            logging.info( 'info after run: %s' % self._infoForPlugins )

    def _runPlugins( self ):
        for plugin in self._pluginManager.getAllPlugins():
            if os.fork() == 0:
                plugin.plugin_object.doSomething( self._infoForPlugins )
                quit()

if __name__ == '__main__':
    CronThingy().go()

Answer 2

Instead of forking the process, what about "forking" the singletons using proxy objects?

In other words, when handing a reference to a singleton over to a plugin, wrap the reference with a proxy object that intercepts all attribute gets in order to return proxies for other singletons and intercepts all attribute sets in order to prevent mutation of the actual singleton.