0

I'm running a rather intensive grep search, and even though the CPU is running around 95% and there are 7 others under 3%, the system threw it on there and its now competing for CPU (fast cgi which is also locked to that CPU is timing out frequently during the grep).

How would I go about finding where this is configured/setup.

1 Answers1

2

grep is single threaded.

You can try to split (man split) the file in to lots of smaller files to add parallelism.

If this is a linux host, you should look in to processor affinity settings. (man taskset)

Side note: Is this a true 8-core system? Or is it a 4-core with hyperthreading?

What do you mean when you say "The CPU is timing out". Do you mean fast-cgi responses time out?

Joel K
  • 5,853
  • 2
  • 30
  • 34
  • It is linux. I mean grep is getting thrown on the already-full cpu. Is there a way to list all currently setup CPU affinities, or pinpoint where they are coming from (in the config files), –  Jun 30 '10 at 04:47
  • Its 2 quad cores. –  Jun 30 '10 at 04:52
  • generally the linux scheduler should auto balance these for you. But if you have a case where it isn't you can force the issue using taskset. (see man taskset) By default all processes generally are set with equal affinity for all processors. Did you give taskset a try? Also doing what I mentioned (splitting the big file up in to smaller pieces) may help you load balance more effectively. (lots of simplier small jobs are less likely to compete for resources than one big long running job) – Joel K Jun 30 '10 at 04:57
  • I'm trying to push several fcgi child processes into their own cores, as they start with next to no RAM/CPU Usage, but expand as work is forked to them. I don't see a decent way to push all of them onto separate cores equally, is there? –  Jun 30 '10 at 05:00
  • You could also run concurrent fast_cgi so they load balance requests across multiple processes. The normal kernel scheduler should balance things fairly. You might have a deeper issue going on. (io contention?) – Joel K Jul 01 '10 at 05:47