0

I'm using TCPHandler to precess TCP requests from client sockets, but I encountered the performance issue when I was using multiThreading, the snippet code as following:

class TCPHandler(SocketServer.StreamRequestHandler):
    def handle(self):
        .............
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    pass
if __name__=="__main__":
    server = ThreadedTCPServer((host, port), TCPHandler)
        server.serve_forever()

The above code is multipleThreading, how can I convert it to multipleProcessing.

Jack
  • 5,540
  • 13
  • 65
  • 113
  • 2
    if you are using python 3.4 it may be worth having a look at [asyncio](https://docs.python.org/3.4/library/asyncio.html). – hiro protagonist Sep 06 '15 at 17:06
  • Thank you Hiro, but I'm using python2.6.6, is there any other ways to make it multipleProcessing please? Is that possible that just change some parameters please or I have to change the code? – Jack Sep 06 '15 at 17:10
  • for python 2 [twisted](https://twistedmatrix.com/trac/) may help. (but [Roland Smith](http://stackoverflow.com/users/1219295/roland-smith)'s answer seems much simpler. hope it works!) – hiro protagonist Sep 06 '15 at 17:12

1 Answers1

1

Replace SocketServer.ThreadingMixIn with SocketServer.ForkingMixIn to make your current code spawn a process to handle a request. Note that this might not be very efficient, because starting a process takes time and resources. If the time needed to handle the request is shorter than the time necessary to start up a new process, it might make things slower.

Depending on what you are doing, an event-driven architecture might be more suitable. For Python 2 you might want to look at the asyncore and asynchat modules from the standard library. Or the twisted framework. Note that an event-driven architecture is completely different from what you have now, and would probably require a rewrite.

Roland Smith
  • 42,427
  • 3
  • 64
  • 94
  • Thank you! My code has two parts: loading module and analyzing the data base on the module. MultiThreading could share the module(only load once), so will the multipleProcessing load module multiple times? What do you mean "Note that this might not be very efficient" please? Right now my cpu usage is 10%, I think I really have a lot of space to improve my code to use more cpu resource, so I tend to use multipleProcessing. – Jack Sep 06 '15 at 17:37
  • If the database is loaded in the request handler, it should be loaded only once. If you want to share memory, the `multiprocessing` module provides the `Value` and `Array` classes. – Roland Smith Sep 06 '15 at 17:49
  • Thanks! What's the default max number of processes the SocketServer.ForkingMixIn can spawn, where can I reset it please? – Jack Sep 06 '15 at 18:15
  • It is an attribute `max_children` of the `ForkingMixIn` class. See the [source code](https://hg.python.org/cpython/file/2.7/Lib/SocketServer.py). – Roland Smith Sep 06 '15 at 18:23