Skip to main content
27 votes
Accepted

Get files from server two steps away

You can create an SSH tunnel through machine2 then in another session connect to the tunnel. For example, open two CLI sessions on machine1. In the first session run the following: MACHINE1$ ssh -L ...
Gene's user avatar
  • 3,683
17 votes

Fastest way to extract tar.gz

If you want to see progress use something like pv. Here is an Example: pigz -dc mysql-binary-backup.tar.gz | pv | tar xf -
Tim Hughes's user avatar
17 votes

Compression on tape set..but at 2.27TB..end of space

In your case it is the file level encryption that is preventing compression. Encryption tries to make the data stream look as much as random "noise" as possible. Compression tries to increase the ...
John Mahowald's user avatar
13 votes

Which tar file format should I use?

Some technical comparisons among v7, ustar and pax formats: v7 The format before POSIX.1-1988. Maximum length of a file name is 99 characters. (100 bytes minus a terminating null byte.) Maximum ...
8 votes

Different md5sums for same tar contents

To make a tar file with a consistent checksum, just prepend GZIP=-n like this: GZIP=-n tar -zcf myOutputTarball.tar /home/luke/directoryIWantToZip How this works: Tar can accept gzip options using a ...
Luke's user avatar
  • 761
7 votes

Determine if file is in the process of being written upon?

A bit old, but most of the answers completely misses the point of the question: But I figured I'd try to figure out if there is simply a way to determine if the file is whole at the command line ...
Andrew Henle's user avatar
  • 1,282
7 votes

Force tar to ignore/skip errors while compressing files?

I think the correct answer to your question should be: Use tar --warning=no-file-changed which will only suppress warnings of kind "%s: file changed as we read it". A general --ignore-failed-read may ...
chwa's user avatar
  • 71
7 votes
Accepted

Backup to remote server and create tar on the fly

You could use a combination of pipes and STDOUT to send the tar file over the network to the remote server. tar zcvf - /your/directory | ssh -i private.key backup-user@backupserver "cat > /backup/...
Mike Marseglia's user avatar
6 votes
Accepted

Is it safe to create tarball while files are in use?

It depends. When you're making tarbal (or rsync copy) of changing data, files will be copied in state when copy of that one file started. And that is a trap. If you will have ie. 100 files in ...
Ondra Sniper Flidr's user avatar
5 votes

Get files from server two steps away

You can also use Master session capability of newer versions of SSH. It's described here: https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing Probably all that you need is to edit/create ...
Jaroslav Kucera's user avatar
5 votes
Accepted

Compression on tape set..but at 2.27TB..end of space

Compression assumes it can work. tar files generally can not be compressed (they already are), so yes, you may end up not getting the "average compression ratio". Pure text files may compress a lot ...
TomTom's user avatar
  • 52k
4 votes

Device busy when reading from tape drive with tar

Did you also write with the same block size? Did you rewind beforereading? Does mt -f /dev/nst0 status work? Are you chaining dd with tar or just using tar as you wrote? There is a quirk when using ...
NickSoft's user avatar
  • 288
4 votes

How can you designate an arbitrary leading path when adding files to a tar?

In BSD tar, -s pattern is the option to make name transformations. Quoting from freebsd.org documentation: -s pattern Modify file or archive member names according to pattern. The pattern ...
Patrick M's user avatar
  • 141
4 votes

mysqldump to a tar.gz

Try this: mysqldump --all-databases --password=dbpassword | gzip -c | ssh user@servername "cat >/tmp/filename_of_your_choice.gz" Please not that I am in no way good at these things, I just ...
Charlie Candergart's user avatar
4 votes
Accepted

Tar does not retain directory permissions

If selected-images-to-copy.txt is a list of files only (the last element of the path is always a file, not a directory) here's a solution to create the archive with proper directory rights: EDIT: I ...
A.B's user avatar
  • 13k
4 votes
Accepted

how do I extract a tar archive with larger block size?

I believe you're looking for the -b switch of tar: -b, --blocking-factor BLOCKS BLOCKS x 512 bytes per record Now your command will look like this: tar xfv mybigbigtar.tar --blocking-factor=...
13dimitar's user avatar
  • 2,556
4 votes

How to tar 30 Gigas of data on a 50 Giga server?

Tar it away via ssh: tar zcf - /SRC | ssh [email protected] dd of=/remote/server/path/file.tgz
Kondybas's user avatar
  • 7,054
4 votes

How do untar as the current user by default?

If you run tar extract as non-root user it gets extracted as current user by default. If you get root ownership on untared files you are running it as root user. Regular user wouldn't be able to ...
ralz's user avatar
  • 2,781
4 votes

Magnetic tape alternative

Not having business requirements is a big red flag - so I strongly suggest to understand what is required before investing in any solution. That said, to save large amount of off-site data I can ...
shodanshok's user avatar
3 votes

Different md5sums for same tar contents

I went down a rabbit-hole after the other answers failed me, and managed to figure out that my version of tar (1.27.1 from the openSUSE 42.3 OSS repo) was using the non-deterministic pax archival ...
Hart Simha's user avatar
  • 1,193
3 votes

tar: extract a member reliably with possible leading dot-slash

It looks like the command line switch --no-anchored may do what you want. From the tar(1) man page (they really are very useful to read or a least scan) --no-anchored patterns match after any '/'...
user9517's user avatar
  • 116k
3 votes
Accepted

Find files and tar.gz the resulting list

Replace -name '*"$DATETIME".log' with -name "*$DATETIME.log" The single quotes are very strong, the double quotes allow $VARIABLE inside.
kubanczyk's user avatar
  • 14k
3 votes

Why tar is incapable of changing current directory?

What happens here is that -C option tells tar to change directory first before creating an archive. So in the first example what you are doing is this pretty much: cd /home/whatever/temp/; tar -zcvf ...
Danila Ladner's user avatar
3 votes

Apache server causes tar.gz files to download as uncompressed tarballs still named .gz

I had to add the following directive to get intentionally gzipped files to download without silently decompressing: AddEncoding x-identity .tar.gz .gz .tgz
Mike Godin's user avatar
3 votes
Accepted

How to untar a very large file with missing space

You can use netcat. Stream the file from a source server with cat and netcat and untar on the destination server using netcat, lz4 and tar. Connect the commands using pipes. See http://toast.djw.org....
hargut's user avatar
  • 3,938
3 votes

Extracting single file from large tar.gz file

The problem with your command is, that you decompress the file to stdout, but instead of reading it from stdin with tar you tell it to extract from a nonexisting file. The correct command would be: $...
Gerald Schneider's user avatar
3 votes

Unable to compress 18GB file on Ubuntu Linux 18.04

OK, I'm answering my own question which may serve someone else for realizing if he/she is actually having hardware problems. After trying multiple times to compress the problematic file (even with ...
Claudi's user avatar
  • 181
2 votes

cpio VS tar and cp

I understand from the comments and other background that cpio is less ubiquitous now and inconsistent between versions. But cpio has one advantage I recently found invaluable when dealing with a large ...
mdisibio's user avatar
  • 121
2 votes

How to convert a TAR file into an ISO file

archivemount /file.tar /media/ISO or uncompressed .tar and genisoimage -o myfile.iso directory_src (can be folder /media/ISO or point mount /media/cdrom)
eternoi's user avatar
  • 21

Only top scored, non community-wiki answers of a minimum length are eligible