4

I have a few images and a css file on my site that don't exist on the server, so each time a visitor comes Apache throws 3 404 errors in it's log. The items are hidden on the page, so it does not affect the display of the page. Our site performs fine in testing and production environment. We recently recieved a 2 day traffic spike of 30,000-40,000 visitors per day and apache slowed to a halt, waiting 22-25 seconds before returning anything to the browser.

Would the 404 errors thrown on the page change the server performance?

Is a 404 error more resource-intensive then a normal request?

Any info re: the way Apache handles 404 errors would be appreciated.

Thanks, Sj

Sj Sros
  • 41
  • 1
  • 2
  • depends on whether you are throwing 404 document or not? – ajreal Dec 10 '10 at 18:00
  • the html page loads fine, there are two images and a css file called from the page that do not exist on the server. The errors are invisible to the end user. – Sj Sros Dec 10 '10 at 18:58

3 Answers3

2

as per the yahoo instructions on website front end engeneering

*No 404s

HTTP requests are expensive so making an HTTP request and getting a useless response (i.e. 404 Not Found) is totally unnecessary and will slow down the user experience without any benefit. Some sites have helpful 404s "Did you mean X?", which is great for the user experience but also wastes server resources (like database, etc). Particularly bad is when the link to an external JavaScript is wrong and the result is a 404. First, this download will block parallel downloads. Next the browser may try to parse the 404 response body as if it were JavaScript code, trying to find something usable in it.*

Coming back to your question : error 404 can in worst cases cause your machine to stop.

Andro Selva
  • 53,910
  • 52
  • 193
  • 240
Tore
  • 21
  • 2
0

As commenter ajreal said served documents will increase the resource allocation, but any server responding to a request will acquire some resources.

John Giotta
  • 16,432
  • 7
  • 52
  • 82
  • no 404 document is thrown, the user never sees any error. The 404 error only appears in our apache logs. – Sj Sros Dec 10 '10 at 18:56
0

Any action done by Apache as response to any user interaction will result in decreased performance for other users even if it is so small - it is almost invisible.

Both the software (memory wise) and hardware (physical limitations and capacity wise) will perform several actions to properly answer and/or log user interaction.

In this case when your users (30,000-40,000) visited the site, Apache still needed to log all of these errors in order to fulfill the logging preferences set by configuration. As the log file is just a text file on the gard disk, the hard disk was too busy to perform reads and writes on the actual site content.

The load of the server would be even higher, if Apache would trow 404 error as a document. In any case, the best way to improve server performance is to get rid of any 404 errors completely.

You can also increase the MaxRequestsPerChild parameter in Apache config file to something around 1000 to decrease the stress from one central Apache process, to multiple subprocesses. This will allow multiple Apache processes to span new ones at the same time increasing the stress to all CPU cores not only one.

With so many visitors I would also recommend changing the MPM of your Apache config to something like this :

<IfModule prefork.c>
StartServers       2
MinSpareServers    2
MaxSpareServers    5
ServerLimit        100
MaxClients         100
MaxRequestsPerChild  4000
</IfModule>
Cninroh
  • 1,796
  • 2
  • 21
  • 37