tar has 3 types of syntax (according to this ):
long options (--file)
short options (-f)
old options (f)
For the old option syntax, all the letters must follow "tar" and must all fall into a single clump with no spaces. The order of the letters doesn't really matter, so long as the arguments to those letters follow the same order after the clump of options....
Summing up previous answers and adding some important information:
When creating archives, tar will always preserve files' user and group ID, unless told otherwise with --owner=NAME, --group=NAME. But still there will always be a user and group associated with each file.
GNU tar, and perhaps other versions of tar, also store the user and group names, unless ...
You can create an SSH tunnel through machine2 then in another session connect to the tunnel.
For example, open two CLI sessions on machine1. In the first session run the following:
MACHINE1$ ssh -L 2022:MACHINE3:22 <user>@MACHINE2
In the second session run the following:
MACHINE1 $ ssh -p 2022 <user>@localhost
What's happening with the ...
In your case it is the file level encryption that is preventing compression.
Encryption tries to make the data stream look as much as random "noise" as possible. Compression tries to increase the data "density" which has a similar effect of limiting further compression.
The fact is tar is successful, but prints out warning that those socket files are ignored
GNU tar actually provides an option to suppress warnings. You could ignore the "socket ignored" errors using the below command which avoids running a find command that could take a long time to complete.
tar --warning='no-file-ignored' -cpzf backup_name.tar.gz /...
Your best bet is to use lsof to determine if a file has been opened by any process:
# lsof -f -- /var/log/syslog
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
rsyslogd 1520 syslog 1w REG 252,2 72692 16719 /var/log/syslog
You can't easily tell if it's in the process of being written to, but if it is being written to, it MUST be open.
GNU tar has a -C option for this.
change to directory DIR
So you could do something like:
tar -C /var/www/website -zcvf /tmp/mytar.tar.gz .
Untarring (you will do that eventually) is the same:
tar -C /var/www/website -zxvf /tmp/mytar.tar.gz
Note that your tar command is completing successfully; it's just complaining about the socket entries. Tiffany is suggesting a mechanism for filtering out those particular error messages, although tar will still exit with an error code.
You could also feed a list of sockets to tar's -X option to have them excluded from the backup, e.g:
# find / -type s -...
You are on the right track, renaming the file is an atomic operation, so performing the rename after upload is simple, elegant and not error prone. Another approach I can think of is to use lsof | grep filename.tar.gz to check if the file is being accessed by another process.
Use the mysqldump utility to direct the file wherever you like:
mysqldump -A -u[username] -p[password] > /path/to/dest/backupname.sql
If you need to, you can pipe the output through gzip:
mysqldump -A -u[username] -p[password] | gzip -c > /path/to/dest/backupname.gz
Further, you can send the output from gzip to another server via ssh:
mysqldump -A ...
I think this has been answered here:
There is the nocache utility, which can prepended to a command like
ionice and nice. It works by preloading a library which adds
posix_fadvise with the POSIX_FADV_DONTNEED flag to any open calls.
In simple terms, it advises the kernel that caching is not needed for
that particular file; the kernel will then ...
You probably want to buy more disk space, but assuming you don't, you could...
pipe the tarball around rather than downloading it.
newserver# ssh olduser@oldserver "cat /path/to/tarball" | tar xf -
or if you don't have SSH access to your old server
newserver# wget -O - http://oldserver/path/to/tarball | tar xf -
or use rsync like Dennis said.
You could use SquashFS for such archives. It is
designed to be accessed using a fuse driver (although a traditional interface exists)
compressed (the larger the block size, the more efficient)
included in the Linux kernel
stores UIDs/GIDs and creation time
endianess-aware, therefore quite portable
The only drawback I know of is that it is read-only.
tar records permissions based on the UID and GID, not on the string associated with them. So if the UID on one server was 3300 and that was linked to 'bob', on the new server the file will be owned by the user who has the UID 3300.
Virtual everything (I want to say everything, but you can never be 100% sure) on UNIX uses the UID:GID values, because that's ...
Try the --xform option, which uses a sed-like replace expression. This allows you to change the path references going into the archive rather than handle it during the extraction (such as with the --strip-components).
tar --exclude=.gitignore -cz --xform s:'./':: -f dist.tgz ./
To list the files as they are processed, add the -v and --show-transformed-name ...
Sockets are zero level files that are used by daemon processes to communicate with each other. They are created and destroyed as necessary when the daemons start and stop. They can safely be ignored.
You can always get rid of them with tar <my_options> 2> >(grep -v 'socket ignored' >&2)
In case anyone else comes across this (as I just have) there are some reasonably detailed (and mostly correct) steps over here.
The key point is to unpack all of the duplicity-full.*.difftar.gz files in the same place, so that you're left with just two snapshot/ and multivol_snapshot/ directories.
If your file is in snapshot/ then you're done. ...
Is there some parameter that allows my tar file to be resumed after
What you should do, though, is run your command from within a terminal multiplexer like screen or tmux. That way if your connection drops, the process keeps running.
To make a tar file with a consistent checksum, just prepend GZIP=-n like this:
GZIP=-n tar -zcf myOutputTarball.tar /home/luke/directoryIWantToZip
How this works: Tar can accept gzip options using a temporary GZIP environment variable, as above. Like Valter said, tar uses gzip, which by default puts a timestamp in the archive. This means you get a ...
A bit old, but most of the answers completely misses the point of the question:
But I figured I'd try to figure out if there is simply a way to determine if the file is whole at the command line first...
In general, there isn't. You simply don't have enough information to determine that.
Because determining that the file is closed is not the same as ...
It's possible some blocks from the tars will be the same, but very unlikely. ZFS deduplicates at the block level (called the recordsize in ZFS parlance), so individual blocks need to be identical. The tars are essentially guaranteed to have runs of the exact same content, but whether that will compose a whole block and be block aligned is highly unlikely.
I think I would actually do this using find and then pass that input into tar. Using your example, let's assume you want files that are between 60 and 90s days old.
find /home/public_html/images -type f -daystart -mtime -90 -and -mtime +60 -print0 | xargs -0 tar -Ajf images_60-90.tar.bz2
This will list all the files that were last modified more than 60 ...
You're compressing (gzip, even) 62 GB of data. At 40 MB/s, that transfer would take ~30 minutes on its own. However, gzip involves the CPU greatly. If you're pinning both cores at 100%, then there's your answer.
By default tar does not provide a means of skipping empty directories. That being said, it is not too much work to pull a simple shell-script out of thin air to do what needs to be done.
Here is a forum discussion where I found the solution to the exact same problem, when the need arose.
First I need to seriously apologize for my sloppy copy-...
Your assumption is correct, "File changed as we read it" is a notice, usually related to files in use (i.e. written to during the creation process) while tar is creating the archive. If consistency is vital, you're better off rsyncing the contents elsewhere i.e.
rsync -avz /my/home/ /somebackupdir/my/home/ # initial sync, followed by
rsync -avz /my/...