0

I'm trying to build a proxy/cache server for one location autocompletion api. Here is the simplified code that manifests the error almost all the time I try to query my server:

#!/usr/bin/python
import gevent
from gevent import monkey
from gevent.wsgi import WSGIServer
monkey.patch_all()

import urllib2
import urlparse
import json

def requestHandler(env, start_response):
    start_response('200 OK', [('Content-Type', 'text')])
    parameters = urlparse.parse_qs(env['QUERY_STRING'])

    if 'search' in parameters:
        searchString = parameters['search'][0]

        # Query the auto-completion server
        json_results = urllib2.urlopen('http://autocomplete.wunderground.com/aq?query=' + searchString).read()
        results = json.loads(unicode(json_results, "ISO-8859-1"))

        finalResult = ''
        for result in results['RESULTS']:
                finalResult += result['name'] + ';' + result['c'] + ';' + result['zmw'] + ';' + result['tzs'] + ';'

        return [finalResult.encode('utf-16')]

    else:
        return ['ERROR']

server = WSGIServer(('', 8888), requestHandler)
print 'Server running on port 8888...'
server.serve_forever()

Sometimes it works on the first query but the second time I request it, it crashes. Sometimes it crashes the first time immediately. This is the error I get:

Modules/gcmodule.c:348: visit_decref: Assertion "gc->gc.gc_refs != 0" failed.
refcount was too small
object  : <weakref at 0x9e0f194; to 'gevent.core.http_request' at 0x9e0a11c>
type    : weakref
refcount: 1
address : 0x9e0f194
Aborted (core dumped)

My system is: CentOS 6.3, Python 2.6.6, Gevent 0.13.8

Does someone have any clue what might be wrong? Seems pretty basic stuff to be causing this kind of problem...

Ivan Kovacevic
  • 1,322
  • 12
  • 30
  • 1
    It sounds likely that this is a bug in `gevent`. An assertion failure in the interpreter means that there *must* be an error in the C code somewhere -- either in the core Python code or in a library such as `gevent`. – Dietrich Epp Feb 25 '13 at 12:44

1 Answers1

1

You should probably upgrade Gevent to a 1.0 version as bugs were fixed related to garbage collection.

I've also seen suggestions to do a small sleep before accepting requests.

Thomas Vander Stichele
  • 36,043
  • 14
  • 56
  • 60
  • I've tried with sleep, no difference. It seems that upgrading to 1.0 is the only thing left to try... However I did notice one more thing, it seems the bug is somehow related to me using urllib2 request inside my function. Because if I omit urllib2.urlopen and just return some fake result it works perfectly... OK but that makes it equal to a basic gevent example code, so it's obvious it should work but it makes me wonder if urllib2 is the problem – Ivan Kovacevic Feb 25 '13 at 13:05
  • I finally got the time to test this with the latest Gevent 1.0. The problem is gone. – Ivan Kovacevic Mar 02 '13 at 14:27