27
votes
Accepted
Get files from server two steps away
You can create an SSH tunnel through machine2 then in another session connect to the tunnel.
For example, open two CLI sessions on machine1. In the first session run the following:
MACHINE1$ ssh -L ...
- 3,643
19
votes
Getting errors while making backup of whole centos with tar
The fact is tar is successful, but prints out warning that those socket files are ignored
GNU tar actually provides an option to suppress warnings. You could ignore the "socket ignored" errors using ...
- 191
17
votes
Compression on tape set..but at 2.27TB..end of space
In your case it is the file level encryption that is preventing compression.
Encryption tries to make the data stream look as much as random "noise" as possible. Compression tries to increase the ...
- 31k
13
votes
Fastest way to extract tar.gz
If you want to see progress use something like pv.
Here is an Example:
pigz -dc mysql-binary-backup.tar.gz | pv | tar xf -
- 333
7
votes
Determine if file is in the process of being written upon?
A bit old, but most of the answers completely misses the point of the question:
But I figured I'd try to figure out if there is simply a way to determine if the file is whole at the command line ...
- 1,262
7
votes
Different md5sums for same tar contents
To make a tar file with a consistent checksum, just prepend GZIP=-n like this:
GZIP=-n tar -zcf myOutputTarball.tar /home/luke/directoryIWantToZip
How this works: Tar can accept gzip options using a ...
- 641
7
votes
tar leading period slash
There are a number of ways to accomplish this, many of which have already been hinted at in this question, some of which have not. This is my attempt to compile them all (with references) and show ...
- 171
7
votes
Accepted
wget and pipe to tar in the background
You will have to instruct wget to write to STDOUT, you do this by passing -O - to wget, otherwise it will save the file to disk. tar is failing because as executed it's expecting something from STDIN.
...
- 246
6
votes
Force tar to ignore/skip errors while compressing files?
I think the correct answer to your question should be:
Use tar --warning=no-file-changed which will only suppress warnings of kind "%s: file changed as we read it". A general --ignore-failed-read may ...
- 61
6
votes
Which tar file format should I use?
Some technical comparisons among v7, ustar and pax formats:
v7
The format before POSIX.1-1988.
Maximum length of a file name is 99 characters. (100 bytes minus a terminating null byte.)
Maximum ...
6
votes
Accepted
Why am I getting different SHA-1 checksums for tar of identical mongodump backups?
This issue isn't specific to MongoDB. Like most programs for file archival, the tar format stores metadata about the archived files including timestamps. If you take a database backup at two different ...
- 1,260
6
votes
Accepted
Is it safe to create tarball while files are in use?
It depends. When you're making tarbal (or rsync copy) of changing data, files will be copied in state when copy of that one file started. And that is a trap. If you will have ie. 100 files in ...
- 2,633
5
votes
Would rsync be better than tar, wget and untar?
You will have to test to find out. There are many variables such as the speed of your storage system.
Consider restoring your tar archive or backup while the old system is up. Then during the ...
- 31k
5
votes
Get files from server two steps away
You can also use Master session capability of newer versions of SSH. It's described here:
https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing
Probably all that you need is to edit/create ...
- 1,525
5
votes
Accepted
Compression on tape set..but at 2.27TB..end of space
Compression assumes it can work. tar files generally can not be compressed (they already are), so yes, you may end up not getting the "average compression ratio". Pure text files may compress a lot ...
- 51.4k
4
votes
How can you designate an arbitrary leading path when adding files to a tar?
In BSD tar, -s pattern is the option to make name transformations. Quoting from freebsd.org documentation:
-s pattern
Modify file or archive member names according to pattern. The
pattern ...
- 141
4
votes
mysqldump to a tar.gz
Try this:
mysqldump --all-databases --password=dbpassword | gzip -c | ssh user@servername "cat >/tmp/filename_of_your_choice.gz"
Please not that I am in no way good at these things, I just ...
4
votes
Accepted
Tar does not retain directory permissions
If selected-images-to-copy.txt is a list of files only (the last element of the path is always a file, not a directory) here's a solution to create the archive with proper directory rights:
EDIT: I ...
- 9,670
4
votes
Accepted
how do I extract a tar archive with larger block size?
I believe you're looking for the -b switch of tar:
-b, --blocking-factor BLOCKS
BLOCKS x 512 bytes per record
Now your command will look like this:
tar xfv mybigbigtar.tar --blocking-factor=...
- 2,428
4
votes
How to tar 30 Gigas of data on a 50 Giga server?
Tar it away via ssh:
tar zcf - /SRC | ssh user@domain.tld dd of=/remote/server/path/file.tgz
- 6,904
4
votes
Magnetic tape alternative
Not having business requirements is a big red flag - so I strongly suggest to understand what is required before investing in any solution.
That said, to save large amount of off-site data I can ...
- 45.5k
3
votes
Different md5sums for same tar contents
I went down a rabbit-hole after the other answers failed me, and managed to figure out that my version of tar (1.27.1 from the openSUSE 42.3 OSS repo) was using the non-deterministic pax archival ...
- 1,203
3
votes
Create tar file on Solaris and untar the file on Linux is it right?
Sure, that will be OK - tar files are not tied to a single platform.
- 31.7k
3
votes
tar: extract a member reliably with possible leading dot-slash
It looks like the command line switch --no-anchored may do what you want. From the tar(1) man page (they really are very useful to read or a least scan)
--no-anchored
patterns match after any '/'...
- 115k
3
votes
Accepted
Find files and tar.gz the resulting list
Replace
-name '*"$DATETIME".log'
with
-name "*$DATETIME.log"
The single quotes are very strong, the double quotes allow $VARIABLE inside.
- 13.7k
3
votes
Why tar is incapable of changing current directory?
What happens here is that -C option tells tar to change directory first before creating an archive. So in the first example what you are doing is this pretty much:
cd /home/whatever/temp/; tar -zcvf ...
- 5,311
3
votes
Accepted
How to untar a very large file with missing space
You can use netcat.
Stream the file from a source server with cat and netcat and untar on the destination server using netcat, lz4 and tar. Connect the commands using pipes.
See http://toast.djw.org....
- 3,878
3
votes
Extracting single file from large tar.gz file
The problem with your command is, that you decompress the file to stdout, but instead of reading it from stdin with tar you tell it to extract from a nonexisting file.
The correct command would be:
$...
- 21.7k
Only top scored, non community-wiki answers of a minimum length are eligible