Waiting for both gevent socket and redis blocking blpop

Question

I'm using a gevent StreamServer to handle incoming connections from clients.

After the clients are connected, the client will send some messages to the server, and the server will process it. Everything works fine on this side. But now and then the server would also send messages back to specific client.

I would do this with redis. I created a queue with a specific client id as key. After the client send a message, I check the queue and if there is any message, I send it back to the client.

The disadvantage of this approach is, the server can only send the message AFTER the client send a message.

Is there a way I could both wait for incoming data and redis blpop, so I can send the message back to client as soon as the message is ready, instead of waiting until the client sends the next data?

import gevent
from gevent import socket
from gevent.server import StreamServer
import redis 

r = redis.Redis('localhost')

def handle_echo(sock, address):
    fp = sock.makefile()
    while True:
        line = fp.readline()
        if line:
            client_id = line.split(",")[0]
            if r.llen('%s:servercmds' % client_id) > 0:
                tosend = r.lrange('%s:servercmds' % imei, 0, 0)[0]
                try:        
                    fp.write(tosend)
                    fp.flush()
                    r.lpop('%s:servercmds' % imei)
                except:
                    print('cannot send data to client')
            else:
                break
        sock.shutdown(socket.SHUT_WR)
        sock.close()

server = StreamServer(('', 8045), handle_echo, spawn=10000)
server.serve_forever()

Ivo · Accepted Answer · 2013-10-05T11:07:01.320

1

You need the client to hold the connection open so you can send a message back to it, or be listening for messages itself.

edit: this is code off the top of my head. IMO it's common to split reading and writing up like this. This is not the only pattern you can use, though.

import gevent
from gevent import socket
from gevent.server import StreamServer
import redis 

r = redis.Redis('localhost')

def handle_echo(sock, address):
    def read_loop(sock):
        while True:
            try:
                socket.wait_read(sock.fileno())
            except socket.error:
                break
            # read from socket

    def write_loop(sock):
        while True:
            try:
                socket.wait_write(sock.fileno())
            except socket.error:
                break
            # write to socket

    jobs = [gevent.spawn(func, sock) for func in (read_loop, write_loop)]
    gevent.joinall(jobs)

server = StreamServer(('', 8045), handle_echo)
server.serve_forever()

edited Oct 05 '13 at 11:07

answered Feb 24 '13 at 17:18

Ivo

5,378
2
18
18

The client already holds the connection. The problem is, on the server side, it waits at fp.readline(). So I have no mechanism to poll/wait for the event from redis. I read about 'select', but there is no documentation about how to use it properly. – Awi Feb 25 '13 at 03:03
I added example code, now that I know you're holding an open connection. You usually want to split up reading and writing, not try to interleave it together, in my experience. – Ivo Feb 25 '13 at 08:53
Great! Your code example has helped modify the code to do what I want. Many thanks! – Awi Feb 26 '13 at 01:47

Waiting for both gevent socket and redis blocking blpop

1 Answers1