3

I'm getting connection refused because too many open connections: 204 when trying to do MyApp.objects.get(foo=bar) in a gevent process. In models.py I have connect('my_db'). In my script I do gevent.monkey.patch_all()I've seen this question

pymongo + gevent: throw me a banana and just monkey_patch?

and this issue

https://github.com/hmarr/mongoengine/issues/272

but I don't understand how to get access to the connection to call end_request. I've also tried this patch but it doesn't help me (unless I'm using it wrong)

https://gist.github.com/1184264

Community
  • 1
  • 1
cerberos
  • 7,705
  • 5
  • 41
  • 43

2 Answers2

0

This seems to be the way to do it with mongoengine (assuming you have one, default connection):

mongoengine.connection.get_connection('default').start_request()

mongoengine.connection.get_connection('default').end_request()

cnelson
  • 1,355
  • 11
  • 14
  • I haven't used mongoengine since I asked this question, but I'm marking it as correct as it sounds about right. – cerberos Apr 17 '13 at 11:57
  • I just ran into this issue the other day, and found this thread with no clear answer, so figured I'd add the solution that worked for me. – cnelson Apr 18 '13 at 14:29
  • Excellent, thanks for posting your answer, that's what SO is all about. All I meant by my comment is that I haven't confirmed that it solves the problem, but I'm marking it as correct anyway. – cerberos Apr 18 '13 at 17:23
0

UPDATE: This issue was resolved in pymongo earlier this year. Make sure you're using the latest version of pymongo.

Original Answer:

Apparently, you can resolve this issue by using a "proxy class ... that queues requests to workers, they allocate a new PyMongo connections on demand, discard them on connection errors, etc. and call target methods," as described here:

http://groups.google.com/group/gevent/browse_thread/thread/a423d1a15d83f73c

(See the Aug 31, 2011 post by Antonin Amand and the response by Alexey Borzenkov)

P.S. And, in any case, the MongoDB folks are working on resolving the issue: https://jira.mongodb.org/browse/PYTHON-296

kkurian
  • 3,844
  • 3
  • 30
  • 49