1

I have noticed decrease in server performance (page load speed become lower), then I went to investigate and have found in munin graphs (which tracks server's resources) that there was big changes on disk IOPS, disk latency, MySQL queries quantity. It was happened without any changes by myself to system or to programs/websites code that runs on server.

What does it mean? What trigger these changes? How to fix?

More info about issue: Order is by munin graphs sequence. Resource (approx. change in times to previous monitoring);

  1. Disk IOs per device (increased x10); enter link description here
  2. Average request size (decreased x2-4);
  3. Disk latency per device (avg. write IO wait time decreased x10); enter link description here
  4. MySQL queries (decreased x2); enter link description here
  5. CPU usage (increased x1.5); enter link description here
  6. Interrupts and context switches (increased x10). enter link description here

Thank you in advance for solving this out.

Michael Hampton
  • 244,070
  • 43
  • 506
  • 972
Arturas
  • 41
  • 5
  • Now, after deeper analysis, I am guessing that there was a mysql restart and configuration have changed. So, it is very possible that this is the reason of such uncommon activity and performance downgrade. If you still have any thoughts, please share it. Thanks. – Arturas Jan 11 '13 at 18:12
  • You're missing a graph; you duplicated the Disk I/O's graph and skipped the average request size graph. – Michael Hampton Jan 11 '13 at 18:15
  • Michael, Average request size is in 1st image http://i.stack.imgur.com/oaiSd.png (blue line is showing Avg Req Size) – Arturas Jan 11 '13 at 18:27

1 Answers1

2

I was trying to find an answer and finally I've reached my point.

Solution was to change MySQL tmpdir parameter in my.cnf from /tmp/ (which is allocated on disk) to /dev/shm (this one is on RAM)

Because of 48.7% [Created_tmp_disk_tables / (Created_tmp_tables + Created_tmp_disk_tables) * 100] of all temporary tables are being written to disk and when tmpdir is allocated on disk, then IOPS increased.

Arturas
  • 41
  • 5