Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a file containing a list of files that I would like to know the total files size. Is there a command to do so?

My OS is a very basic linux (Qnap TS-410).

EDIT:

A few lines from the file:

/share/archive/Bailey Test/BD006/0.tga
/share/archive/Bailey/BD007/1 version 1.tga
/share/archive/Bailey 2/BD007/example.tga

share|improve this question
    
Give us a few example lines of the file. –  EEAA Jan 19 '12 at 17:37
    
Example from the file added. –  Nicolas Jan 19 '12 at 17:42
    
That's some sort of NAS, right? Do you have busybox installed? –  cjc Jan 19 '12 at 17:45
    
Yes it is and I think it's already installed, why? –  Nicolas Jan 19 '12 at 17:46

5 Answers 5

up vote 4 down vote accepted

I believe something like this would work in busybox:

du `cat filelist.txt` | awk '{i+=$1} END {print i}'

I don't have the same environment as you, but if you encounter issues with spaces in filenames something like this would work too:

cat filelist.txt | while read file;do
  du "$file"
done | awk '{i+=$1} END {print i}'

Edit 1:
@stew is right in his post below, du shows the disk usage and not the exact filesize. To change the behavior busybox uses the -a flag, so try: du -a "$file" for exact filesize and compare the output/behavior.

share|improve this answer
    
Thanks for your input, the first command returns /usr/bin/du: Argument list too long (almost 80,000 lines in my file). You second command just gives me a prompt once I hit enter, waiting for something more? –  Nicolas Jan 20 '12 at 12:20
    
Hard to say with your environment. Is it the normal command prompt, or just a blinking prompt? If its the latter it might just be slow waiting for the result, if its an "input prompt" it might be that you missed some character? And if its a normal prompt I don't know, I tested it quite throughly before I typed it. :( –  Mattias Ahnberg Jan 20 '12 at 14:17
    
it's an "input prompt" when I do the following cat tgafiles.txt | while read file;do du "$file" done | awk '{i+=$1} END {print i}'. thanks mattias –  Nicolas Jan 20 '12 at 17:37
1  
Ah! If you put everything on one line you need another; like this: cat tgafiles.txt | while read file;do du "$file";done | awk '{i+=$1} END {print i}' (i.e. before done). –  Mattias Ahnberg Jan 20 '12 at 22:47
    
Spot on! It worked perfectly, cheers! (although I could have figured out this mistake by myself) –  Nicolas Jan 23 '12 at 17:53

Try something like this:

$ cat filelist.txt | xargs ls -l | awk '{x+=$5} END {print "total bytes: " x}' 

To deal properly with spaces in paths:

$ find /path/to/files -type f -print0 | xargs -0 ls -l | awk '{x+=$5} END {print "total bytes: " x}' 
share|improve this answer
    
thanks for your input, unfortunately I think there's an issue with the spaces in the directories within my file not being escaped with a "\"., therefore it breaks while going through the file list. –  Nicolas Jan 19 '12 at 17:46
    
Can you bypass the text file list, and just generate this off of the ouput of find? –  EEAA Jan 19 '12 at 17:48
    
unfortunately the list is too long, there are 79159 lines of files (full path), that's why I output it to a file; maybe I can add an argument about escaping the result of the find? –  Nicolas Jan 19 '12 at 17:51
    
there's no "-print0" argument with the find on my linux system –  Nicolas Jan 19 '12 at 17:54
    
@Nicolas - that's due to it using busybox's stripped-down find instead of the real find binary. –  EEAA Jan 19 '12 at 17:55

I don't know if your linux tools are capable of this, but:

cat /tmp/filelist.txt  |xargs -d \\n du -c

Do, the xargs will set the delimiter to be a newline character, and du will produce a grand total for you.

Looking at http://busybox.net/downloads/BusyBox.html it seems that "busybox du" will support the grand total option, but the "busybox xargs" will not support custom delimiters.

Again, I'm not sure of your toolset.

share|improve this answer
    
here's the result: xargs: invalid option -- d –  Nicolas Jan 20 '12 at 12:21
    
Awesome: working with a NAS's busybox linux is like a McGuyver episode, trying to build a working airplane from some canvas, sticks and twine. –  cjc Jan 20 '12 at 15:47
    
How about this, if you have the room for it on a different machine: copy all those files that you're interested in to some other, fully functional linux, and then run Stew's solution there. Doing that might be a lot easier than trying to figure out if busybox is capable of this sort of thing. –  cjc Jan 20 '12 at 15:49
    
I think answer is the best. It's concise, and is much quicker than the other answers in this thread. –  WildVelociraptor Nov 20 at 18:54
while read filename ;  do stat -c '%s' $filename ; done < filelist.txt | awk '{total+=$1} END {print total}'

This is similar to Mattias Ahnberg's solution. Using "read" gets around problems with filenames/directories with spaces. I use stat instead of du to get the filesize. du is getting the amount of room it is using on disk instead of the filesize, which might be different. Depending on your filesystem, a 1 byte file will still occupy 4k on disk (or whatever the blocksize is). So for a 1 byte file, stat says 1 byte and du says 4k.

share|improve this answer
    
Good comment about filesize vs disksize! –  Mattias Ahnberg Jan 19 '12 at 19:31
    
Very interesting comment indeed, unfortunately my linux does not know the stat command: stat: command not found –  Nicolas Jan 20 '12 at 12:12
    
You might have to say "busybox stat". –  cjc Jan 20 '12 at 15:53
    
it says stat: applet not found in this case –  Nicolas Jan 20 '12 at 17:41

Here's another solution to the problem:

cat filelist.txt | tr '\n' '\0' | wc -c --files0-from=-
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.