I have a file containing a list of files that I would like to know the total files size. Is there a command to do so?

My OS is a very basic linux (Qnap TS-410).

EDIT:

A few lines from the file:

/share/archive/Bailey Test/BD006/0.tga
/share/archive/Bailey/BD007/1 version 1.tga
/share/archive/Bailey 2/BD007/example.tga

link|improve this question

Give us a few example lines of the file. – ErikA Jan 19 at 17:37
Example from the file added. – Nicolas Jan 19 at 17:42
That's some sort of NAS, right? Do you have busybox installed? – cjc Jan 19 at 17:45
Yes it is and I think it's already installed, why? – Nicolas Jan 19 at 17:46
feedback

4 Answers

up vote 2 down vote accepted

I believe something like this would work in busybox:

du `cat filelist.txt` | awk '{i+=$1} END {print i}'

I don't have the same environment as you, but if you encounter issues with spaces in filenames something like this would work too:

cat filelist.txt | while read file;do
  du "$file"
done | awk '{i+=$1} END {print i}'

Edit 1:
@stew is right in his post below, du shows the disk usage and not the exact filesize. To change the behavior busybox uses the -a flag, so try: du -a "$file" for exact filesize and compare the output/behavior.

link|improve this answer
Thanks for your input, the first command returns /usr/bin/du: Argument list too long (almost 80,000 lines in my file). You second command just gives me a prompt once I hit enter, waiting for something more? – Nicolas Jan 20 at 12:20
Hard to say with your environment. Is it the normal command prompt, or just a blinking prompt? If its the latter it might just be slow waiting for the result, if its an "input prompt" it might be that you missed some character? And if its a normal prompt I don't know, I tested it quite throughly before I typed it. :( – Mattias Ahnberg Jan 20 at 14:17
it's an "input prompt" when I do the following cat tgafiles.txt | while read file;do du "$file" done | awk '{i+=$1} END {print i}'. thanks mattias – Nicolas Jan 20 at 17:37
1  
Ah! If you put everything on one line you need another; like this: cat tgafiles.txt | while read file;do du "$file";done | awk '{i+=$1} END {print i}' (i.e. before done). – Mattias Ahnberg Jan 20 at 22:47
Spot on! It worked perfectly, cheers! (although I could have figured out this mistake by myself) – Nicolas Jan 23 at 17:53
feedback
while read filename ;  do stat -c '%s' $filename ; done < filelist.txt | awk '{total+=$1} END {print total}'

This is similar to Mattias Ahnberg's solution. Using "read" gets around problems with filenames/directories with spaces. I use stat instead of du to get the filesize. du is getting the amount of room it is using on disk instead of the filesize, which might be different. Depending on your filesystem, a 1 byte file will still occupy 4k on disk (or whatever the blocksize is). So for a 1 byte file, stat says 1 byte and du says 4k.

link|improve this answer
Good comment about filesize vs disksize! – Mattias Ahnberg Jan 19 at 19:31
Very interesting comment indeed, unfortunately my linux does not know the stat command: stat: command not found – Nicolas Jan 20 at 12:12
You might have to say "busybox stat". – cjc Jan 20 at 15:53
it says stat: applet not found in this case – Nicolas Jan 20 at 17:41
feedback

Try something like this:

$ cat filelist.txt | xargs ls -l | awk '{x+=$5} END {print "total bytes: " x}' 

To deal properly with spaces in paths:

$ find /path/to/files -type f -print0 | xargs -0 ls -l | awk '{x+=$5} END {print "total bytes: " x}' 
link|improve this answer
thanks for your input, unfortunately I think there's an issue with the spaces in the directories within my file not being escaped with a "\"., therefore it breaks while going through the file list. – Nicolas Jan 19 at 17:46
Can you bypass the text file list, and just generate this off of the ouput of find? – ErikA Jan 19 at 17:48
unfortunately the list is too long, there are 79159 lines of files (full path), that's why I output it to a file; maybe I can add an argument about escaping the result of the find? – Nicolas Jan 19 at 17:51
there's no "-print0" argument with the find on my linux system – Nicolas Jan 19 at 17:54
@Nicolas - that's due to it using busybox's stripped-down find instead of the real find binary. – ErikA Jan 19 at 17:55
show 1 more comment
feedback

I don't know if your linux tools are capable of this, but:

cat /tmp/filelist.txt  |xargs -d \\n du -c

Do, the xargs will set the delimiter to be a newline character, and du will produce a grand total for you.

Looking at http://busybox.net/downloads/BusyBox.html it seems that "busybox du" will support the grand total option, but the "busybox xargs" will not support custom delimiters.

Again, I'm not sure of your toolset.

link|improve this answer
here's the result: xargs: invalid option -- d – Nicolas Jan 20 at 12:21
Awesome: working with a NAS's busybox linux is like a McGuyver episode, trying to build a working airplane from some canvas, sticks and twine. – cjc Jan 20 at 15:47
How about this, if you have the room for it on a different machine: copy all those files that you're interested in to some other, fully functional linux, and then run Stew's solution there. Doing that might be a lot easier than trying to figure out if busybox is capable of this sort of thing. – cjc Jan 20 at 15:49
feedback

Your Answer

 
or
required, but never shown

Not the answer you're looking for? Browse other questions tagged or ask your own question.