I have a large webb app of which I have recently been working hard to reduce load times. I have two controllers Generator (some 20.000 items) and Product (some 1.500 items) that have been slow for a while but I have worked with indexes and smart queries. On my dev app the app response time is about 500 ms.
From time to time I still get RequestTimeOut on the app and I need help trouble shooting this error. I understand what it means (a request has taken too much time) and I have installed the 'rack-timeout' gem and set it to 15 seconds (which works fine).
I have gone through the entire app (and especially the two slowest: Generator & Product) in search for time to save. I have had some issues with caching that I am currently trying to fix (caching would help quite a bit).
It seems that these timeouts happens mostly when bots (Yandex.ru especially) spiders through my site and especially goes through one generator after another. They may not be very slow any more but loading so many after another causes a lot of requests.
Now I am out of ideas and need some help in order to know what and how to continue my trouble shooting:
- Is there anything else outside of response time that cause this
error? E.g. memory leakage or something? Or is it just a matter of lots of requests on slow controllers? - I haven't been able to test it on my development platform. Is
there a way to benchmark and see how the app would handle requests like from the bots? I seem to remember there was an "Apache-thing" one could use to simulate traffic like this. - Any other ways of looking at the problem or trouble shoot this
issue from a high level point of view? Any ideas and thoughts are welcome!
