Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a 130GB tar file I need to untar to /storage.

Problem is, I don't have enough disk space for both to exist at the same time. How can I untar the tar file, and ensure there is enough disk space, and delete the file immediately after untar?

Thanks.

share|improve this question
3  
Seriously?! If you had just use rysnc as Chopper3 suggested (and outvoted your accepted answer by 10x) you wouldn't have had any of the problems you're encountering, including this one –  Mark Henderson Dec 5 '10 at 2:08
1  
If you checked Andrew's recent questions, you might discover that he had a problem getting rsync to work. So cut him some slack. –  Steven Monday Dec 5 '10 at 4:15
    
guys, he already has a 130GB tar archive, he needs a solution for this time, next time he will use rsync or scp. My response was for this problem, not for the one on how he can copy 130gb files over the network. –  Paul Dec 5 '10 at 20:31

4 Answers 4

This one is hard. It really depends on how much free space you have.

Others said to extract part of the files and delete them from the tar. That's the only option I can see right now.

Updating the tar requires the tar to be reconstructed without the deleted files on the same drive. That's why you have to have 2 x tar size + something more to allow extraction of the files.

HTH

share|improve this answer

Can you put the tar file on a different server? Because then, you can untar via ssh without worrying about the space taken by the original file.

share|improve this answer
1  
Oh, or you can follow the solution you've accepted in serverfault.com/questions/208300 and no space is used on the remote server! –  mattdm Dec 5 '10 at 5:42

If it just a tar file (i.e. not zipped) and if you have enough space left on the disk to store a zipped version of the tarball, you could try to zip this and then construct an unzip | tar pipe. I would test this beforehand, though.

Easier solution: Get a bigger disk.

share|improve this answer

In a word: Pain and suffering...

Understanding the file format used by tar, you could build some tools to help with what you are doing, but there are complications that may or may not be relevant to your particular file. The tar file uses headers of 512 bytes that specify the file-name and length of the file, among other things. You could use this information to build up a series of offsets within the tar file of each file entry. You could then do something like traverse the tar file backwards, and truncate the file as you extract the files.

However, there are some issues of sequence that you have to deal with. GNU tar, for example, can create some "fake" entries for file with long file names, to store additional information that can't fit in the 512 byte header. Also you may need to be careful about directory entries, which might specify permissions that would not allow you to extract files into the directory if you extract the directory entry before the contents.

The Python programming language, among others, includes a nice library for handling tar files.

However, another option would be to just split the large tar file up into many smaller files irrespective of the tar format. Split out the end and and then truncate the source file. Repeat until instead of a single 130GB file you have 130 1GB files. Obviously, getting these split/truncates right may be a little tricky. Could be done using the "dd" and "truncate" commands

Then it would be an easy matter to make a script that would "cat" the first file, delete the first file, cat the second file, delete the second file, etc... Pipe that script to "tar x" and as the tar file extracts the source files would be deleted.

Of course, these are all destructive operations, so you basically get one shot to do them right.

The easiest would be if you have a place to copy the 130GB file to and extract it from there. Say an external USB hard drive, or another machine and extract it over an SSH tunnel.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.