Blocking Thrift calls with Twisted

Question

I have a Twisted/Thrift server that uses the TTwisted protocol. I want to keep connections from clients open until a given event happens (I am notified of this event by the Twisted reactor via a callback).

class ServiceHandler(object):
    interface.implements(Service.Iface)

    def listenForEvent(self):
        """
        A method defined in my Thrift interface that should block the Thrift
        response until my event of interest occurs.
        """
        # I need to somehow return immediately to free up the reactor thread,
        # but I don't want the Thrift call to be considered completed. The current
        # request needs to be marked as "waiting" somehow.

    def handleEvent(self):
        """
        A method called when the event of interest occurs. It is a callback method
        registered with the Twisted reactor.
        """
        # Find all the "waiting" requests and notify them that the event occurred.
        # This will cause all of the Thrift requests to complete.

How can I quickly return from a method of my handler object while maintaining the illusion of a blocking Thrift call?

I initialize the Thrift handler from the Twisted startup trigger:

def on_startup():
    handler = ServiceHandler()                      
    processor = Service.Processor(handler)                                   
    server = TTwisted.ThriftServerFactory(processor=processor,                  
        iprot_factory=TBinaryProtocol.TBinaryProtocolFactory())                 
    reactor.listenTCP(9160, server)

My client in PHP connects with:

  $socket = new TSocket('localhost', 9160);
  $transport = new TFramedTransport($socket);
  $protocol = new TBinaryProtocol($transport);
  $client = new ServiceClient($protocol);
  $transport->open();
  $client->listenForEvent();

That last call ($client->listenForEvent()) successfully makes it over to the server and runs ServiceHandler.listenForEvent, but even when that server method returns a twisted.internet.defer.Deferred() instance, the client immediately receives an empty array and I get the exception:

exception 'TTransportException' with message 'TSocket: timed out reading 4 bytes from localhost:9160 to local port 38395'

score 2 · Accepted Answer · answered Oct 19 '11 at 13:00

2

You should be able to return a Deferred from listenForEvent. Later handleEvent should fire that returned Deferred (or those returned Deferreds) to actually generate the response.

answered Oct 19 '11 at 13:00

Jean-Paul Calderone

47,755
6
94
122

I tried doing this. Could my problem be that I need to use the TTwisted transport from the client as well? I am using TFramedTransport from PHP. – ide Oct 19 '11 at 19:53
Returning a Deferred worked. The problem was that my client socket's timeout was only 750 ms. – ide Oct 20 '11 at 18:41

score 1 · Answer 2 · answered Oct 20 '11 at 13:33

The error you're seeing seems to indicate that the transport is not framed (Twisted needs it to be so it can know the length of each message beforehand). Also, Thrift servers support returning deferreds from handlers, so that's even more strange. Have you tried returning defer.succeed("some value") and see if deferreds actually work? You can then move to this to check that it fully works:

    d = defer.Deferred()
    reactor.callLater(0, d.callback, results)
    return d

Blocking Thrift calls with Twisted

2 Answers2