I have an incredibly large tarball. I'd extract several files out of many thousands within the archive. I'm on CentOS 6.10 running GPFS 4.2.3. I've seen from this answer that pigz is useful in extracting the entire tar.ball. Extracting the entire tar ball is not useful because it will take up terabytes with of space.

I've tried something like :

$ pigz -dc ../test.tar.gz | tar xf test/analysis/something/dist.txt
tar: test/analysis/something/dist.txt: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now

I'm not exactly sure how to pass test/analysis/something/dist.txt as an argument to tar in the context of piping the output of pigz. My intuition says to use xargs, but that fails also.

$ pigz -dc ../test.tar.gz | xargs -I var | tar xf var test/analysis/something/dist.txt
tar: var: Cannot openxargs: Warning: a NUL character occurred in the input.  It cannot be passed through in the argument list.  Did you mean to use the --null option?
: No such file or directory
tar: Error is not recoverable: exiting now
xargs: /bin/echo: terminated by signal 13


  1. How do I quickly extract a single file from a large tarball using pigz?

The problem with your command is, that you decompress the file to stdout, but instead of reading it from stdin with tar you tell it to extract from a nonexisting file.

The correct command would be:

$ pigz -dc ../test.tar.gz | tar xf - test/analysis/something/dist.txt
#                                  ^- this dash tells tar to read from stdin

However, basically you are decompressing the file into your memory, so unless you have terabytes of memory available, it will fill up even faster than decompressing to disk.


Agree with the author above, just mention about file navigation inside the tar archive:

pigz -dc <archive.tar.gz> | tar xf - <file-with-path-inside-archive>

try to test/list archive (tar option -t) for your file:

pigz -dc <archive.tar.gz> | tar tf -

lookup full file name in the archive:

pigz -dc <archive.tar.gz> | tar tf - | grep <file-name>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.