Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I need to find all the instances of a given string in the whole filesystem, because I don't remember in which configuration files, script or any other programs I put it and I need to update that string with a new one.

I tried with the following command

`grep -nr 'needle' / --exclude-dir=.svn | mail myaddress@example.com -s 'References on xxx'

If I run this command on a small directory it gives me the output I need in the form

/path1/:nn:line containing needle
/path2/:nn:line containing needle

where /path1 is the full path of the file, nn is the row containing the needle and last field is the content of the line.

However when I run the command on the root directory the grep process hang after a while. I run this script about 8 hours ago and even on a small filesystem (less than 5GB) it doesn't end and if I run top or ps the process seems sleeping

root 24909 0.0 0.1 3772 1520 pts/1 S+ Feb10 0:15 grep -nr needle / --exclude-dir=.svn

Why it doesn't end? Is there any better way to do this (it's a one time job, I don't need to execute this more than once)

Thanks.

Update: I found a working solution with find and xargs which seems to be working and uses less system resources than the find -exec solution. Here is my final command line:

find /{boot,etc,home,lib,lost+found,opt,root,sbin,usr,var} -type f -print0 | xargs -r0 grep -nr 'needle' | mail myaddress@example.com -s 'References on xxx'

I used the /{dir1,dir2,...,dirn} syntax because I want the output lines with the fullpath including the leading slash, and in this way you can use the command without cd / i.e. from any directory.

share|improve this question

5 Answers 5

up vote 2 down vote accepted

There are some files in the file system that aren't real files, but are instead hooks into the kernel. Some of those can be read from forever. Try

grep foo /dev/zero

to see this in action. Get ready to stop it with ctrl-C before it takes over the whole system.

If I wanted to do what you're doing, I'd enumerate the subdirectories of / that I wanted scanned, and make sure I only checked text files, probably with

cd /
find boot etc home lib lost+found media mnt opt root sbin tmp usr var -type f -exec grep needle {} /dev/null \;

Note how the list does not contain /dev, /proc, /sys or /selinux.

share|improve this answer
    
I'm gonna try, I think it should work. I'll let you know. What a stupid mistake... –  Fabio Feb 11 '11 at 14:48

You can try using find + xargs + grep for that, kinda:

find /there -type f :MaybeSomeRestrictingFlagsLikeSizeNotBigger500MB_or_FS_type_if_u_know_its_exactly_on_EXT3_AndSoOn: -print0 | xargs -0r grep needle /dev/null

(/dev/null makes grep printing file name even if only one file was found)

share|improve this answer

I don't know why it doesn't end but "fgrep -R" usually works in those cases.

share|improve this answer

You might want to exclude dev and proc from that command. I would do it with find instead and exclude special files:

find . -type f -exec grep -H "My search string" '{}' \; -print 
share|improve this answer
    
That's very ineffective due to running grep every time find finds a file. That brings too much overhead in. –  poige Feb 11 '11 at 14:33
    
@poige I found something about your point, to avoid that I should use find . -type f -exec grep -H "My search string" '{}' + -print i.e. + instead of \;. In that way the find program will build the command line by appending files instead of calling the program once per file. –  Fabio Feb 11 '11 at 14:59
    
@Fabio, does it allow limiting number of files (arguments) to fire up grep with? xargs does. –  poige Feb 11 '11 at 15:33
    
@poige, I haven't found it because I'm experiencing some problems with that syntax. I just found a working solution with xargs and I'm gonna update my question to share it. –  Fabio Feb 11 '11 at 15:41
    
@Fabio, well, anyways I'm quite happy with «find × xargs × grep». It's UNIX-way. :) –  poige Feb 11 '11 at 15:45

As some of the others pointed out, the error is occuring because you are using grep on some 'files' that will produce weird results. For example grep XXX /dev/zero will go on forever.

I would first exclude the directories such as /dev/ and /proc/ from your search. Another way is to only search for 'files' and no other types.

Also if your argument list in grep is too long, you will get this error:

bash: /bin/grep: Argument list too long

You can get around this with a loop, such as this:

for i in `find -type f /`; do grep -H "sample string" $i; done

Or by executing it from find, whatever floats your boat.

find / -type f -exec grep -H "sample string" '{}' \; -print 

This should do a find and then grep each file. Although some have pointed out that it is inefficient to spawn grep each time you find a file, you will have to do something which will be 'inefficient' because you do literally have to search every file for that string.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.