2

I need to find all the instances of a given string in the whole filesystem, because I don't remember in which configuration files, script or any other programs I put it and I need to update that string with a new one.

I tried with the following command

`grep -nr 'needle' / --exclude-dir=.svn | mail myaddress@example.com -s 'References on xxx'

If I run this command on a small directory it gives me the output I need in the form

/path1/:nn:line containing needle
/path2/:nn:line containing needle

where /path1 is the full path of the file, nn is the row containing the needle and last field is the content of the line.

However when I run the command on the root directory the grep process hang after a while. I run this script about 8 hours ago and even on a small filesystem (less than 5GB) it doesn't end and if I run top or ps the process seems sleeping

root 24909 0.0 0.1 3772 1520 pts/1 S+ Feb10 0:15 grep -nr needle / --exclude-dir=.svn

Why it doesn't end? Is there any better way to do this (it's a one time job, I don't need to execute this more than once)

Thanks.

Update: I found a working solution with find and xargs which seems to be working and uses less system resources than the find -exec solution. Here is my final command line:

find /{boot,etc,home,lib,lost+found,opt,root,sbin,usr,var} -type f -print0 | xargs -r0 grep -nr 'needle' | mail myaddress@example.com -s 'References on xxx'

I used the /{dir1,dir2,...,dirn} syntax because I want the output lines with the fullpath including the leading slash, and in this way you can use the command without cd / i.e. from any directory.

Fabio
  • 1,299
  • 2
  • 13
  • 18

5 Answers5

2

There are some files in the file system that aren't real files, but are instead hooks into the kernel. Some of those can be read from forever. Try

grep foo /dev/zero

to see this in action. Get ready to stop it with ctrl-C before it takes over the whole system.

If I wanted to do what you're doing, I'd enumerate the subdirectories of / that I wanted scanned, and make sure I only checked text files, probably with

cd /
find boot etc home lib lost+found media mnt opt root sbin tmp usr var -type f -exec grep needle {} /dev/null \;

Note how the list does not contain /dev, /proc, /sys or /selinux.

MadHatter
  • 79,770
  • 20
  • 184
  • 232
2

You can try using find + xargs + grep for that, kinda:

find /there -type f :MaybeSomeRestrictingFlagsLikeSizeNotBigger500MB_or_FS_type_if_u_know_its_exactly_on_EXT3_AndSoOn: -print0 | xargs -0r grep needle /dev/null

(/dev/null makes grep printing file name even if only one file was found)

poige
  • 9,448
  • 2
  • 25
  • 52
1

I don't know why it doesn't end but "fgrep -R" usually works in those cases.

Thorsten
  • 158
  • 5
0

You might want to exclude dev and proc from that command. I would do it with find instead and exclude special files:

find . -type f -exec grep -H "My search string" '{}' \; -print
pehrs
  • 8,789
  • 1
  • 30
  • 46
  • That's very ineffective due to running `grep` every time `find` finds a file. That brings too much overhead in. – poige Feb 11 '11 at 14:33
  • @poige I found something about your point, to avoid that I should use `find . -type f -exec grep -H "My search string" '{}' + -print` i.e. `+` instead of `\;`. In that way the find program will build the command line by appending files instead of calling the program once per file. – Fabio Feb 11 '11 at 14:59
  • @Fabio, does it allow limiting number of files (arguments) to fire up grep with? xargs does. – poige Feb 11 '11 at 15:33
  • @poige, I haven't found it because I'm experiencing some problems with that syntax. I just found a working solution with xargs and I'm gonna update my question to share it. – Fabio Feb 11 '11 at 15:41
  • @Fabio, well, anyways I'm quite happy with «find × xargs × grep». It's UNIX-way. :) – poige Feb 11 '11 at 15:45
0

As some of the others pointed out, the error is occuring because you are using grep on some 'files' that will produce weird results. For example grep XXX /dev/zero will go on forever.

I would first exclude the directories such as /dev/ and /proc/ from your search. Another way is to only search for 'files' and no other types.

Also if your argument list in grep is too long, you will get this error:

bash: /bin/grep: Argument list too long

You can get around this with a loop, such as this:

for i in `find -type f /`; do grep -H "sample string" $i; done

Or by executing it from find, whatever floats your boat.

find / -type f -exec grep -H "sample string" '{}' \; -print

This should do a find and then grep each file. Although some have pointed out that it is inefficient to spawn grep each time you find a file, you will have to do something which will be 'inefficient' because you do literally have to search every file for that string.

Dave Drager
  • 8,375
  • 29
  • 45