Controlling a Twisted Server from Django

Question

I'm trying to build a Twisted/Django mashup that will let me control various client connections managed by a Twisted server via Django's admin interface. Meaning, I want to be able to login to Django's admin and see what protocols are currently in use, any details specific to each connection (e.g. if the server is connected to freenode via IRC, it should list all the channels currently connected to), and allow me to disconnect or connect new clients by modifying or creating database records.

What would be the best way to do this? There are lots of posts out there about combining Django with Twisted, but I haven't found any prior art for doing quite what I've outlined. All the Twisted examples I've seen use hardcoded connection parameters, which makes it difficult for me to imagine how I would dynamically running reactor.connectTCP(...) or loseConnection(...) when signalled by a record in the database.

My strategy is to create a custom ClientFactory that solely polls the Django/managed database every N seconds for any commands, and to modify/create/delete connections as appropriate, reflecting the new status in the database when complete.

Does this seem feasible? Is there a better approach? Does anyone know of any existing projects that implement similar functionality?

score 1 · Answer 1 · answered Nov 24 '12 at 01:20

Polling the database is lame, but unfortunately, databases rarely have good tools (and certainly there are no database-portable tools) for monitoring changes. So your approach might be okay.

However, if your app is in Django and you're not supporting random changes to the database from other (non-Django) clients, and your WSGI container is Twisted, then you can do this very simply by doing callFromThread(connectTCP, ...).

score 1 · Answer 2 · answered Dec 21 '12 at 16:05

I've been working on yet another way of combing django and twisted. Fell free to give it a try: https://github.com/kowalski/featdjango.

The way it works, is slightly different that the others. It starts a twisted application and http site. The requests done to django are processed inside a special thread pool. What makes it special, is that that these threads can wait on Deferred, which makes it easy to combine synchronous django application code with asynchronous twisted code.

The reason I came up with structure like this, is that my application needs to perform a lot of http requests from inside the django views. Instead of performing them one by one I can delegate all of them at once to "the main application thread" which runs twisted and wait for them. The similarity to your problem is, that I also have an asynchronous component, which is a singleton and I access it from django views.

So this is, for example, this is how you would initiate the twisted component and later to get the reference from the view.

import threading

from django.conf import settings

_initiate_lock = threading.Lock()

def get_component():
    global _initiate_lock
    if not hasattr(settings, 'YOUR_CLIENT')
        _initiate_lock.acquire()
        try:
            # other thread might have did our job while we
            # were waiting for the lock
            if not hasattr(settings, 'YOUR_CLIENT'):
                client = YourComponent(**whatever)
                threading.current_thread().wait_for_deferred(
                    client.initiate)
                settings.YOUR_CLIENT = client
        finally:
            _initiate_lock.release()
    return settings.YOUR_CLIENT

The code above, initiates my client and calls the initiate method on it. This method is asynchronous and returns a Deferred. I do all the necessary setup in there. The django thread will wait for it to finish before processing to next line.

This is how I do it, because I only access it from the request handler. You probably would want to initiate your component at startup, to call ListenTCP|SSL. Than your django request handlers could get the data about the connections just accessing some public methods on the your client. These methods could even return Deferred, in which case you should use .wait_for_defer() to call them.

Controlling a Twisted Server from Django

2 Answers2