6

I'm getting strange usage numbers on MongoDB server when it is in almost idle state: http://pastebin.com/cKVyGMts (no queries) DB size is about 3.3 Gb.

I can see in mongodb.log that snapshotthread is eating too much CPU:

Wed Nov 28 19:02:14 [snapshotthread] cpu: elapsed:4000 writelock: 0% Wed Nov 28 19:02:18 [snapshotthread] cpu: elapsed:4000 writelock: 0% Wed Nov 28 19:02:22 [snapshotthread] cpu: elapsed:4000 writelock: 0% Wed Nov 28 19:02:26 [snapshotthread] cpu: elapsed:4000 writelock: 0%

And data from top confirms it: http://pastebin.com/d57ybpBJ (iotop shows that there are no high disk activity present).

I'm running high-cpu instance on EC2, mongo confing and startup information is here: http://pastebin.com/YZyyq0gc.

How I can disable snapshotting or what can cause such a high CPU usage?

Roald Nefs
  • 426
  • 5
  • 13
Andrei Mikhaltsov
  • 3,027
  • 1
  • 23
  • 31
  • Whet version of MongoDB are you using? – FINESEC Nov 28 '12 at 15:12
  • db version v2.0.8 from 10gen repo Centos 6.2 – Andrei Mikhaltsov Nov 28 '12 at 15:14
  • did you try turning off snapshotting? cpu = false or just comment out cpu=true. Also check this bug out: https://jira.mongodb.org/browse/SERVER-2114 – Asya Kamsky Dec 05 '12 at 22:29
  • Thanks, turning off snaphotting didnt cause any effect. I've straced mongo and found that log is filled with select and setsockopts: select(10, [8 9], NULL, NULL, {0, 10000}) = 0 (Timeout) select(10, [8 9], NULL, NULL, {0, 10000}) = 0 (Timeout) select(10, [8 9], NULL, NULL, {0, 10000}) = 0 (Timeout) select(10, [8 9], NULL, NULL, {0, 10000}) = 0 (Timeout) – Andrei Mikhaltsov Dec 06 '12 at 10:11
  • How is "Cpu(s): 3.0%us, 1.0%sy, 0.0%ni, 95.5%id, 0.3%wa, 0.0%hi, 0.1%si, 0.0%st" high load? That's 95+% idle. – David Schwartz Dec 01 '15 at 06:31

1 Answers1

1

According to JIRA some changes where made to memory allocation algorithm in version 2.2.0, which possibly could fix this issue.

MongoDB high CPU usage, low disk IO, very slow response

FINESEC
  • 1,371
  • 7
  • 8
  • thank you for the answer!! unfortunately, this didn't work ;/ upgraded to db version v2.2.2, pdfile version 4.5 git version: d1b43b61a5308c4ad0679d34b262c5af9d664267 – Andrei Mikhaltsov Nov 28 '12 at 15:54
  • ok, did you try to disable NUMA? http://www.mongodb.org/display/DOCS/NUMA – FINESEC Nov 28 '12 at 16:28
  • restarted server with command: numactl --interleave=all /usr/bin/mongod --config /etc/mongod.conf (cat /proc/sys/vm/zone_reclaim_mode 0 - no luck;/ – Andrei Mikhaltsov Nov 28 '12 at 16:43
  • ok, try to increase syncdelay: http://docs.mongodb.org/manual/reference/mongod/#cmdoption-mongod--syncdelay – FINESEC Nov 28 '12 at 17:20
  • no luck;/ i now think the problem is more OS specific http://pastebin.com/yaGL4cYm because i have 8 cores and mongod CPU usage is 2200% (!!!??) i'll try to update CentOS to the lastest version. – Andrei Mikhaltsov Nov 29 '12 at 10:22
  • Any useful feedback on the subject would be great. Did you find the root cause of the problem? – Ostati Jul 10 '15 at 16:29
  • You are misreading the output. You'll notice that only one core is spending significant time in user space (18.2%) and the others are almost completely idle (over 95% idle). You only have 8 cores, so clearly that 2200% can't be CPU usage. – David Schwartz Dec 01 '15 at 06:28