How to put tcp server on another thread in python

Question

I try to write a daemon in python. But I have no idea how can I use a thread to start parallel tcp server in this daemon. And even what type of server I should use : asyncore?SocketServer?socket?

this is part of my code:

import os
def demonized():
   child_pid = os.fork()
   if child_pid == 0:
       child_pid = os.fork()
       if child_pid == 0:          #fork twice for demonize
           file = open('###', "r") # open file
           event = file.read()
           while event:
               #TODO check for changes put changes in list variable
               event = file.read()
       file.close()
       else:
           sys.exit(0)
   else:
       sys.exit(0)


if __name__ == "__main__":
  demonized()

So in a loop I have a list variable with some data appended every circle, and I want to start a thread with tcp server that wait for connection in the loop and if client connects send it this data(with zeroing variable). So I do not need to handle multiple clients, the client will be only one at time. What is the optimal way to implement this?

Thank you.

You could just use Twisted (http://twistedmatrix.com/trac/) which is designed for network based event handling. — extraneon, Nov 05 '11 at 13:34
A presentation that recently helped me a lot — I had known about `Queue`, but the presentation introduced me to thinks like `Event` which I had seen in the Standard Library docs but not known what to do with — is David Beazley's “An Introduction to Python Concurrency” — even though David does training professionally, he has generously been putting up a lot of his slide decks for everyone to see: http://www.slideshare.net/dabeaz/an-introduction-to-python-concurrency — Brandon Rhodes, Nov 06 '11 at 13:02

Brandon Rhodes · Accepted Answer · 2011-11-05T14:37:24.813

In case you want to avoid repeating boilerplate, Python will soon have a standard module that does the fork() pair and standard-I/O manipulations (which you have not added to your program yet?) that make it a daemon. You can download and use this module right now, from:

http://pypi.python.org/pypi/python-daemon

Running a TCP server in a separate thread is often as simple as:

import threading

def my_tcp_server():
    sock = socket.socket(...)
    sock.bind(...)
    sock.listen()
    while True:
        conn, address = sock.accept()
        ...
        ... talk on the connection ...
        ...
        conn.close()

def main():
    ...
    threading.Thread(target=my_tcp_server).start()
    ...

I strongly recommend against trying to get your file-reader thread and your socket-answering thread talking with a list and lock of your own devising; such schemes are hard to get working and hard to keep working. Instead, use the standard library's Queue.Queue() class which does all of the locking and appending correctly for you.

Kirill · Answer 2 · 2011-11-05T16:05:39.513

-1

Do you want to append items to the list in while event:... loop and serving this list simultaneously? If so then you have two writers and you must somehow protect your list.

In the sample SocketServer.TCPServer and threading.Lock was used:

import threading
import SocketServer
import time


class DataHandler(SocketServer.StreamRequestHandler):

    def handle(self):
        self.server.list_block.acquire()
        self.wfile.write(', '.join(self.server.data))
        self.wfile.flush()
        self.server.data = []
        self.server.list_block.release()


if __name__ == '__main__':
    data = []
    list_block = threading.Lock()

    server = SocketServer.TCPServer(('localhost', 0), DataHandler)
    server.list_block = list_block
    server.data = data

    t = threading.Thread(target=server.serve_forever)
    t.start()

    while True:
        list_block.acquire()
        data.append(1)
        list_block.release()
        time.sleep(1)

edited Nov 05 '11 at 16:05

answered Nov 05 '11 at 13:41

Kirill

3,364
2
21
36

1

First, this solution should probably use a `Queue` instead of building one out of primitives. Second, it is not clear to me that it even runs — Python does not support calling `write()` on a list object, does it? I get `TypeError: expected a character buffer object`. – Brandon Rhodes Nov 05 '11 at 14:09
@Kirill Teplinskiy It looks like what I need except `time.sleep(1)` my file can changes faster than sec(I can lost some information in that case). Is there some method to pause `while` loop by external event (opening connection to client) (not inside the loop)? flag\semaphore? – Jazz.Min Nov 05 '11 at 15:13
@ Brandon Craig Rhodes I`m not sure but maybe conversion data_list to string before sending should help. But I cant check it right now – Jazz.Min Nov 05 '11 at 15:20
@user1027564, yep, conversion to string is needed. I didn't run this code before posting. Anyway, look at the Brandon answer, especially at `Queue`. It seems like it is what you need for exchanging information between threads. It is new for me too :). But I still suggest using `SocketServer.TCPServer` instead of manually working with sockets. This is the same as the `Queue` for threads, but for network. – Kirill Nov 05 '11 at 16:04
@Brandon Craig Rhodes, thank you for noting about `Queue`. I'm going to study it now because it is really helpful. – Kirill Nov 05 '11 at 16:09

How to put tcp server on another thread in python

2 Answers2

Linked