1

I have a blog I made. Recently, I've been noticing some performance problems. I'm getting about 400ms waiting times for the index page. I think this is quite high. When I first deployed it(with less features, but still) I recall index load times of something like 80ms.

Now I would profile it, but the problem is that this only happens in my production environment. In my test environment, the index page only takes 10ms.

How do I go about profiling my production application? I use Apache+mono+mod_mono on Arch Linux with MongoDB. I have a similar test environment except I use xsp.

I'm unsure of where to look: my code, Apache's configuration, or MongoDB? How can I profile my production server to figure out why it's so much slower than my development environment?

Earlz
  • 62,085
  • 98
  • 303
  • 499

2 Answers2

1

Tough to be specific without details, but here's a shot at a general guide:

First I would recommend using something like Firebug for Firefox - there are equivalents in other browsers, but this is my old go-to tool for this kind of thing. Enable it and got the Net Panel view for a waterfall diagram that will show you a list of every object that is loading on a page (you might have to refresh) - it will also have a blue showing the render event (when the page becomes visible).

The waterfall should make it pretty obvious where the slow pieces of the page are and armed with that information you can go to the next stage - figuring out why particular pieces are slow.

If plugins are not your thing, or you suspect that it could be something local to yur machine causing the issue, then take a look at: http://www.webpagetest.org/

That will give you the ability to remote test from different locations, different browsers, speeds etc. and give you similar detailed results.

If it is a static file being fetched, look at network problems, Apache as a cause. If it is dynamically generated then look at Apache, ASP, mongodb etc.

For Apache, what do the access logs say is the response time for the index page? Assuming Apache 2 or newer, make sure you have %D (and %T if you like) being logged so you can see the time taken to serve the page (from the Apache perspective) at the required level of details. For more info on that, take a look at the LogFormat directive.

I can't help on the ASP/Mono side, not my thing, but adding debug statements at various points to track the index page generation (assuming it is dynamically generated) would be a pretty standard approach.

For the database, MongoDB by default logs only "slow" queries that take >100ms - if you are trying to track down a sub-100ms response time issue via the logs you will need to adjust that or you will likely get very little. That can be done as follows:

> db.setProfilingLevel(0,20) // leave profiling off, slow threshold=20ms

You can also adjust it as a startup parameter (--slowms) to the mongod process. More information on profiling, which may also help but has overheads, can be found here:

http://www.mongodb.org/display/DOCS/Database+Profiler

Adam Comerford
  • 21,336
  • 4
  • 65
  • 85
  • I do use Firebug. That's where I got the 400ms number from :) The problem is that it's waiting for my index page for 400ms. And good point about mongodb – Earlz Jul 11 '12 at 08:50
  • it's usually worth the work to look at webpagetest too - see if location/distance from production has an effect. If not, and the index page itself is the issue, then you can probably narrow it down - I'll add some notes – Adam Comerford Jul 11 '12 at 09:05
0

I'd suggest you have a look a Sam Saffrons Mini Profiler. If you use it in your site, it allows you to turn on profiling on production.

By adding sufficient instrumentation to your code, you should then be able to identify which bit is taking the time and then focus your efforts there.

Grhm
  • 6,726
  • 4
  • 40
  • 64