Simpliest python long polling

Question

I saw some threads about long polling in python, but my problem is not so bit to use some additional kits like tornado etc. I have js client. It sends requests to my /longpolling page and wait for response. Once it get response or timeout it sends new one. This is working well. My /longpolling handler is a function:

currentTime = datetime.datetime.now()
lastUpdate = datetime.datetime.strptime(req.GET["ts"], "%Y-%m-%dT%H:%M:%S.%f")
response = {
    "added": [],
    "updated": [],
    "deleted": []
}
while (datetime.datetime.now() - currentTime).seconds < 600:
    time.sleep(2)
    now = datetime.datetime.now()
    #query = Log.objects.filter(time__range = (lastUpdate, now))
    query = Log.objects.raw("SELECT * FROM ...log WHERE time BETWEEN %s and %s", [lastUpdate, now])
    exist = False
    for log in query:
        exist = True
        type = {
            NEW: "added",
            UPDATED: "updated",
            DELETED: "deleted"
        }[log.type]
        response[type].append(json.loads(log.data))
    if exist:
        response["ts"] = now.isoformat()
        return JsonResponse(response)
response["ts"] = datetime.datetime.now().isoformat()
return JsonResponse(response)

Every 2 seconds during 10 min I want to check for new Log instances in DB to notify js client. I tryed to insert Log record manualy through phpMyAdmin but next Log.objects.filter(time__range = (lastUpdate, now)) returns empty QuerySet. I copy raw query from .query attr it looks like:

SELECT ... FROM ... WHERE time BETWEEN 2013-01-05 03:30:36 and 2013-01-05 03:45:18

So I quoted 2013-01-05 03:30:36 and 2013-01-05 03:45:18 and executed this SQL through phpMyAdmin and it returned my added record. I tryed:

query = Log.objects.filter(time__range = (lastUpdate, now))

and

query = Log.objects.raw("SELECT * FROM ...log WHERE time BETWEEN %s and %s", [lastUpdate, now])

and

for log in query.iterate():

But it always returns an empty QuerySet but never my added record. I thought there is some caching, but where? Or the problem is that I insert new record until while True: loop was performing? Or maybe there is some thread protection? Why phpMyAdmin see record but django does not? Please help me, I am stuck.

Try using django code to insert a log entry just before you query. Is it at all possible you're accidentally pointing phpMyAdmin at a different database from where you're pointing django? A difference between debug and production dbs or something? — dokkaebi, Jan 05 '13 at 02:45
Great idea to insert new Log entry just before, I'll try and answer. And no, they are connected to the same database. — Luft-on, Jan 05 '13 at 11:48
Okey, so if I create new Log entry just before querying I get it in QuerySet and it correctly added to response object and responded to client. After that client sends new request as expected, so what now? — Luft-on, Jan 05 '13 at 12:04
Find simular thread http://stackoverflow.com/questions/6887901/changes-in-db-data-not-reflecting-in-the-django-queryset-in-a-continuously-loopi — Luft-on, Jan 05 '13 at 12:15

score 1 · Accepted Answer · answered Jan 06 '13 at 03:48

I haven't run into this problem, so I'm not sure. Based on @DanielRoseman's answer in the thread you linked in comments, you might be able to do this:

with transaction.commit_on_success():
    query = Log.objects.raw("SELECT * FROM ...log WHERE time BETWEEN %s and %s", [lastUpdate, now])

It seems more likely, though, that you will have to wrap the lines that insert your log entries in the commit_on_success decorator. I'm not sure where in your code the log entries are inserted.

Simpliest python long polling

1 Answers1