3

I have a Rails(3.2.8)/Mongoid(3.0.6) app which uses Resque workers to generate and send emails each morning. Each Resque job has to query a few collections to generate the email and then perform the actual send.

We have compound indexes set up to support the queries and executing a single job manually is very fast. However, as we increase the number of concurrent workers the time that it takes to generate emails blows up quickly. Tailing the log shows that the actual execution of the mongo queries increases dramatically.

(note: I noticed this while evaluating Sidekiq, but the same happens with Resque; 3-4 Processors/Workers move along fine but throughput drops dramatically past that)

We have 4 Thin processes on one box, Resque on another, and the MongoDB server on a third (all hosted within the same datacenter on Rackspace). The difference is noticeable even when moving from 4 to 8 workers. The Thins are also affected.

Is there any low hanging fruit or obvious bits that I may be missing that could explain the performance issues I'm seeing? Any pointers or advice would be appreciated.

ccurtisj
  • 388
  • 2
  • 13
  • What your default pool size you have defined in `mongoid` But I dont think that the only issue though – Viren Nov 10 '12 at 07:38
  • check mongostat to instance see if you can find anything – Viren Nov 10 '12 at 07:39
  • @Viren The mongo driver in Mongoid 3 actually removed connection pooling, although I've seen some discussion about implementing a connection pooling mechanism ›› [thread](https://github.com/mongoid/mongoid/issues/2369#issuecomment-8563523). As for mongostat, it reports all queries as hitting indexes and executing immediately (no request queueing). – ccurtisj Nov 12 '12 at 12:53
  • Have you looked at the [MongoDB production notes](http://www.mongodb.org/display/DOCS/Production+Notes).Assuming you're on Linux, have you got "noatime" set, what type of disk are you using, RAID set-up, readahead settings? Have you tested your queries with [explain](http://www.mongodb.org/display/DOCS/Explain) to see if the best index is being hit? How big is your working set? Is your disk activity (iostat -xm 2) going crazy? Finally, look at installing [MMS](http://mms.10gen.com) to help monitor your MongoDB? – Mark Hillick Nov 14 '12 at 09:26
  • @ccurtisj I believe your best bet is what `Mark Hillick` said since there are many user case I consider it as black box as and one can give endless answer as to why the mongoid or resque working too slow my only advice apart from examine the various usecase as to find what making resque,mongoid ,thin too slow possible try with unicorn see if it help examine the `top` ,`free` and other stat see if you could can find anything – Viren Nov 16 '12 at 03:58
  • @MarkHillick I know that the queries are hitting the appropriate indexes and that the working set fits in memory. Since it's a cloud server I don't know about the physical disk set up but I'll monitor the IO and see if anything pops up – ccurtisj Nov 19 '12 at 00:28

0 Answers0