0

There is a Twisted server which gives responses to clients' requests. The "LineOnlyReceiver" is used for the protocol design. The process should go as following:

A client sends a request to the server (such as "TEMP?" asking the temperature), then the number of request on the server should increase by 1. After the server sends back the response, the number should decrease by 1. However, during the request ("TEMP?") being processed by the server, there could be other clients sending requests to the server, the number of requests on the server should vary with time.

So how to keep track of the number of requests on the server on-the-fly? Currently, the "lineReceived" method is used to receive requests from clients and send back responses (other codes omitted):

Class SimProtocol(LineOnlyReceiver):
    def lineReceived(self, request):
        self.factory.taskNumber[request.strip()] += 1
        print 'Task number: ', self.factory.taskNumber[request.strip()]
        time.sleep(10)
        resp = self.factory.getResponse(request.strip())
        self.transport.write(resp)
        self.factory.taskNumber[request.strip()] -= 1
        print 'Task number: ', self.factory.taskNumber[request.strip()]

The 10 seconds delay is added in the middle on purpose to simulate the processing time of the requests. During the 10 seconds, there are several other requests being sent to the server, but the server didn't respond to other requests until it finishes the current one. In other words, the "print" statements always show the task number is 1, 0, 1, 0..., no matter how many clients are sending requests.

So what kind of design approach (probably a totally different one, like getting the reactor queue size could do?) should be used to accomplish this functionality (keep track of the number of requests on the server on-the-fly)?

circle999
  • 63
  • 1
  • 5
  • You can't (nicely) insert a delay into a Twisted program with `time.sleep`. See `twisted.internet.task.deferLater` instead. – Jean-Paul Calderone Oct 04 '18 at 20:03
  • Hi thank you for pointing that out. Do you think a structure with defer could do the job? For a quick test, I changed the "time.sleep(10)" to "for i in range(100000): pass" to simulate the processing time. But I notice that the "lineReceived" function still doesn't respond to other requests before it finishes the current one. – circle999 Oct 04 '18 at 20:58
  • https://stackoverflow.com/questions/3071237/how-does-a-non-blocking-event-loop-work https://stackoverflow.com/questions/80617/asynchronous-programming-in-python-twisted/81456#81456 https://stackoverflow.com/questions/6117587/twisted-making-code-non-blocking https://stackoverflow.com/questions/2466000/twisted-why-is-it-that-passing-a-deferred-callback-to-a-deferred-thread-makes-t – Jean-Paul Calderone Oct 04 '18 at 22:22
  • Hi Jean-Paul, thanks for your answers. Is there a "multiprocessing" version of "deferToThread" (since "deferToThread" uses "threading" module to implement its threading APIs)? – circle999 Oct 24 '18 at 15:17
  • There's Ampoule but the last time I looked at it it was essentially abandoned with the original author MIA and contained a number of likely-show-stopping bugs. – Jean-Paul Calderone Oct 24 '18 at 22:12
  • Thank you Jean-Paul, will check that :) – circle999 Oct 25 '18 at 17:26
  • Hi, in the meantime I also look at the "spawnProcess" and see some examples where it can spawn an executable (like "wc", "cat" etc.) as a process. Will it be able to spawn a python function (like, e.g., lineReceived) as a separate process? Thank you. – circle999 Oct 25 '18 at 23:38
  • Unfortunately not - spawnProcess is for running an entire new process. You can put your `lineReceived` logic in a new process and run it with spawnProcess but you don't get something as easy as a hypothetical `deferToProcess(lineReceived)`. – Jean-Paul Calderone Oct 26 '18 at 00:08
  • Thank you for the clarification :). By the way, is "deferToAMPProcess" a hypothetical "deferToProcess(function)"? – circle999 Oct 26 '18 at 14:24
  • If you mean the one provided by Ampoule, then that is a very real function, not a hypothetical one at all. :) – Jean-Paul Calderone Oct 26 '18 at 15:49
  • Sorry I didn't say it clearly. Yes, I mean that real function. Can it do something like spawning a python function (like, e.g. lineReceived) as a separate process (not thread)? – circle999 Oct 26 '18 at 17:36
  • yes, yes it can. – Jean-Paul Calderone Oct 26 '18 at 17:45
  • Thank you Jean-Paul :), will try it. – circle999 Oct 26 '18 at 18:24

0 Answers0