2

These code is the Server part of my proxy program, and its function is creating the socket and fork four process to accept one by one.

In my program I use gevent model to dispatch all my function and before I alter it to multiple process, my program is all right. but now when I use the second process, the first one stop running, I don't find where is wrong, maybe the 'accept' function or my event is stop dispatch.

It have already bothered me for two days I hope someone can help me.

BTW, my English is poor, I try my best to explain it, hoping you can understand.

 class Client(object):
    def __init__(self, ent, ev):
        ...  

    def receive( self ):
        ...
        if "Content-Length" in dic:
            self.ent_s_send = core.event(core.EV_WRITE,
                                         self.conn.fileno(),
                                         self.ser_send,
                                         [self.conn,self.body]
                                         )
            self.recv_ent = core.event(core.EV_READ, 
                                       self.sock.fileno(),
                                       self.recv_content
                                      )
            self.recv_ent.add()
        ...

    def recv_content(self, ent, ev):
        ...
        self.n = self.sock.recv_into(self.msg,
                                     min(self.total-self.num, 20000),
                                     socket.MSG_DONTWAIT)

        **time.sleep(0.1)**  
        #if i add it here to let the event slow down the problem solved, how it could be? 

        self.num += self.n
        self.msg_buffer.fromstring(self.msg.tostring()[:self.n])
        ...
        if self.total > self.num:  #if not the last msg continue recving and sending...
            self.ent_s_send.add()
            self.recv_ent.add()
        ...

    def ser_send(self, ent, ev):
        ...
        num = self.conn.send(self.msg_buffer,socket.MSG_DONTWAIT)
        ...
        self.msg_buffer = self.msg_buffer[num:]

 ...
 ...

 class Server(object):
    def __init__( self ):
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.sock.bind(('localhost', 18001))
        self.sock.listen(50)
        self.mutex = multiprocessing.Lock()

    def loop( self, ):

        for i in range(0,4):
            pid = os.fork()
            if pid == 0 or pid == -1:
                break

        if pid == -1:
            print "Fork failed!!!"
            sys.exit()

        elif pid == 0:   **# create four child ps to accept the socket!**
            print "Child  PID =  %d" % os.getpid()
            core.init()
            self.event = core.event(core.EV_READ,
                                self.sock.fileno(),
                                self.onlink)
            self.event.add()
            core.dispatch()

        else:
            os.wait()

    def onlink( self, ent, ev):
        self.mutex.acquire()
        print 'Accept PID = %s' % os.getpid()
        try:
            self.conn, self.addr = self.sock.accept()   
            **#I think 'accept' is the the problem, but I cannot see how.** 

        except socket.error, why:
            if why.args[0] in [ECONNRESET, ENOTCONN, ESHUTDOWN, ECONNABORTED]:
                return
            else:
                raise
        print self.sock,self.conn,self.addr
        self.mutex.release()
        print 'Release PID = %s' % os.getpid()
        cc = Chat( self.conn, self.sock )
        self.event.add()



if __name__ == '__main__':

    s1 = Server()
    s1.loop()
Ding
  • 105
  • 1
  • 1
  • 8
  • 2
    Please don't just paste your code. Describe your question in more detail by editing your post to add it, and try adding comments to your code so we know where to look for the problem. – agf Aug 17 '11 at 10:22
  • You don't need a mutex really. They used it in the past to avoid the thundering herd problem with `accept()`, but it has long been fixed. – Maxim Egorushkin Aug 17 '11 at 16:07

1 Answers1

1

accept() is a blocking call. It'll wait indefinitely for a client to connect. Holding a mutex over a blocking operation like that is a Bad IdeaTM since you totally lock all other concurrent processes out.

Also, as @Maxim noted in the comments, you don't really need to lock around accept(). Just let the OS arbitrate dequeuing of incoming connections and dispatch them to your processes.

Nikolai Fetissov
  • 82,306
  • 11
  • 110
  • 171
  • Thanks for your answer. I delete the mutex, but the problem is still here.

    Now, I find the problem is that when process NO.2 accepted the socket, then the events of process NO.1 will stop dispatch. And if I add a sleep(0.1) in my event, then came a surprise.
    BUT I lower the sleep time, the problem showed again.
    Can you help me with that?
    – Ding Aug 24 '11 at 10:08
  • Which is process 1 and which is process 2? Your first initial parent is NOT doing `accept()` at all, just forks and waits for children to complete. Also, I think you got yourself a nice fork bomb here (http://en.wikipedia.org/wiki/Fork_bomb). – Nikolai Fetissov Aug 24 '11 at 13:23
  • process 1 is the first child process to do accept, ps 2 is the next. What I want is creating four child processes to run accept() and the father is just a dispatcher, for now I arrange nothing to it but wait(). I really cannot see what's wrong with that, I just learn that sort of things for a short time, guess you can see that. I just see the 'fork bomb' in wiki. Does that mean creating a large number of processes? but I just create four process, how it could be a fork bomb? – Ding Aug 24 '11 at 14:33
  • I don't really understand your previous comment about the sleeps, so it might be a good idea to add your current code to the question. As for the fork-bomb, I might be wrong, didn't look close enough. – Nikolai Fetissov Aug 24 '11 at 15:20
  • I already put the code to the question, you can see the part of Class Client. I know my code is a little disordered, hope you can understand it. Thank you anyway. – Ding Aug 25 '11 at 02:33