up vote 1 down vote favorite
share [g+] share [fb]

I have a directory A with a large number of sub directories and files and want to get a list of all files named foo that are inside of a directory in A that matches *bar. E.g.:

  • Yes: ./goldbar/fiz/baz/foo
  • Yes: ./leadbar/foo
  • No: ./candy/figbar/foo

I have the some additional constraints:

  • I must not descend into directories that don't match the *bar (this is a necessary optimization as it would take to long to scan those)
  • I can't allow the shell to do the glob expansion because it returns to many results: (i.e. find *bar -type f -name foo fails)

I think that the -path flag would give me the results I need but I don't know if it fits the first of the above constraints.

Edit: Assume that there are n*10k directories in A that match *bar. i.e. anything that tries to build a command with all of them (as opposed to handling each one in tern) will fail.

link|improve this question
48% accept rate
3  
What Unix are you using? Each one has their own oddities in find. If you're using Linux, you'll have GNU find, which includes the -prune option which is possibly the right route to go down – Ewan Leith Apr 19 '11 at 15:36
feedback

5 Answers

up vote 2 down vote

I'm not sure how -path will handle the "do not descend into directories that don't match *bar" requirement, and I'm too lazy to build an environment to find out.
I do know the following will work on pretty much any *nix platform:

for dir in `ls -1 | grep bar`; do
   find $dir -type f -name foo
done

Additional weaseling of the ls/grep bit may be required if you have plain files, sockets, etc. in your top-level directory, or if you want to revise your conditions a little.

link|improve this answer
As I understand it, that will expand to for dir in <some long list of dirs>; and unless bash handles it as a stream, it can be expected to fail. (See edit) – BCS Apr 19 '11 at 16:10
The "expansion" here is actually handled by ls & grep -- the shell just gets a really long list to process (which can chew up RAM, but shouldn't fail) -- for example for f in `find /usr`; do echo $f; done | wc -l processed around 371,000 items on my box, no problem :-) – voretaq7♦ Apr 19 '11 at 16:26
1  
If you're worried about shell limits, you can always do something like ls -1 | grep bar$ | while read dir; do ... done. – justarobert Apr 19 '11 at 17:59
1) So for is not subject to command line length limits? 2) memory pressure is an issue. – BCS Apr 19 '11 at 20:46
feedback
up vote 2 down vote

If you're looking for a solution purely with find, then

find . \( -depth 1 -type d \! -name "*bar" \) -prune -o \( -depth +1 \! -type d -name foo \) -print

should do the trick.

link|improve this answer
I get warning: you have specified the -depth option after a non-option argument (, but options are not positional (-depth affects tests specified before it as well as those specified after it) in GNU find, you can only tell it to work at one depth or the other. – DerfK Apr 19 '11 at 17:38
Ah, that's right. GNU find and BSD find argue about -depth. I don't see a good GNU find solution with this approach. – justarobert Apr 19 '11 at 17:56
feedback
up vote 2 down vote

Try this - not sure how quick it will be though.

find -maxdepth 1 -type d -name '*bar' -exec find {} -name foo \;

NOTE this doesn't work for Solaris 10 as it's find doesn't have -maxdepth

link|improve this answer
I'm fairly sure that would do what I want, but even the first find is to slow: i.e. I have other problems. :( – BCS Apr 19 '11 at 20:49
@BCS:It works on the data you provided. You're not going to find anything that does this quickly. – Iain♦ Apr 20 '11 at 5:51
feedback
up vote 1 down vote

How about a regex version:

find -regex './[^/]*bar/.*/?foo$'

EDIT: Tweaked it with a cleaner one and terminating the line (so you're not matching fooble, foogasm, etc...)

link|improve this answer
This works, but strace shows find thinks it's necessary to stat all directories and subdirectories, so I'm not sure it's optimizing as well as it should. But perhaps it's fast enough for @BCS. – Eduardo Ivanec Apr 19 '11 at 19:34
@Eduardo: If it's looking at extra subdirs, it is most definitively NOT fast enough. – BCS Apr 19 '11 at 20:47
Good point. It'll have to stat every directory to match the regex against them. – SmallClanger Apr 20 '11 at 8:13
feedback
up vote 1 down vote

I checked with strace and -path by itself does not avoid the unneded incursion into non-matched directories. This should do the job, though:

find . \( -name '.' \) -print -o \( ! -path '*directory_pattern*' \) -prune -o -name '*file_pattern*' -print

The \( and \)s aren't actually needed on this particular case, but they add readability and would probably be needed if you used more complex expressions. Checked with strace and it seems to work, hope it helps.

link|improve this answer
Looks like a good solution if your version of find has the -prune option, but it seems to rely on having a "pattern to exclude" - that could be a long pattern/list of patterns – voretaq7♦ Apr 19 '11 at 16:28
Fixed that, my mistake. – Eduardo Ivanec Apr 19 '11 at 19:37
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.