rsync is your friend.
rsync -ru /source/directory/* email@example.com:/destination/directory
If you want it to delete files at the destination that no longer exist at the source, add the --delete option.
rsync over ssh is probably your best bet with the --remove-source-files option
rsync -avz --remove-source-files -e ssh /this/dir remoteuser@remotehost:/remote/dir
a quick test gives;
[tomh@workstation001 ~]$ mkdir test1
[tomh@workstation001 ~]$ mkdir test2
[tomh@workstation001 ~]$ touch test1/testfile.1
[tomh@workstation001 ~]$ ls test1/
Rather than using root create an account just for this job. Use public keys without a passphrase instead of passwords.
scp -i /home/backupuser/.ssh/id_rsa firstname.lastname@example.org:/backup
By using a special account for the backup on the destination system you are not exposing your root password.
Two problems: First, the * does not go on the destination side. Second, -r is for copying an entire directory and subdirectories.
pscp -i C:\sitedeploy\abt-keypair.ppk includes\* email@example.com:/usr/local/tomcat/webapps/ROOT/includes/
Will copy all of the files in the local includes\ directory to the .../includes/ directory on the server.
pscp -r -i C:\...
Try to use rsync, it has a lot more benefits besides keeping ownership, permissions and incremental copies:
rsync -av source 192.0.2.1:/dest/ination
Besides that, since rsync uses ssh, it should work where scp works.
If you have a fast wide-area network you will find that sftp and scp are about the same speed, which is slow. They both suffer from performance problems in the underlying openssh. With modern hardware, this is not due to encryption overhead, but rather due to problems with the openssh implementation - it implements its own internal windowing mechanism ...
With more recent versions of ssh on the server near (B) machine the following will work without netcat:
ProxyCommand ssh near -W distant:22
It will however require AllowTcpForwarding to be yes (the default) on the near (B) machine
edit: requires OpenSSH 5.4+ on B
One of your login scripts (.bashrc/.cshrc/etc.) is outputting data to the terminal when it shouldn't be. This is causing scp to error when it is connecting and getting ready to copy as it starts receiving extra data it doesn't expect. Remove output that is generated here.
You can check if your terminal is interactive and only output text by using the ...
This question's been answered just fine, and the answer accepted, but since it's floated to the top of the front page, I thought I'd at least try to answer it more precisely, if less elegantly. Yes, you can use the return code from scp, and I do it often. In bash:
scp foo user@server:/destination && rm foo
I take your point about multiple files ...
You can use rsync to copy your file from one computer to the other. rsync can use ssh as its underlying transport. Combine rsync --partial with a script such as this one to try again in case of network failure, and you should be able to move your files even in the face of network errors.
Another way to do it would be to mount the remote filesystem on your ...
Your scp command sends the file to the root of the target server - you're scp:ing to /myfile. When you later look at the file, you're not giving the full path. It looks as though you have once tried to copy the contents of a file using an editor configured to insert indentation from the previous line, and that's the file you're looking at.
That is correct. "-p" does not do that. See the man page:
-p Preserves modification times, access times, and modes from the
Notice it says times and modes, NOT user/group ownership. You will have better luck with "rsync", as it has various capabilities around preserving permissions when copying between disparate ...
If you have 9,000 servers under the same management, you'd likely have some form of configuration management in place. That could be in the form of Puppet, Chef, Ansible, etc. You can distribute public keys that way.
For 30 users, this is also something that could be handled via central directory authentication (LDAP, Active Directory).
There are many limits for transferring many small files. Some have already been mentioned: network latency, disk write speed, etc. However most of those can be optimized best by using "rsync". If the files don't exist on the destination, and you are pretty sure the process won't be interrupted, using tar piped to tar will be very efficient:
cd /SOURCE/DIR ...
There are three common factors that affect a transfer speed:
Bandwidth - An obvious factor that's apparently not your trouble.
Network delay/latency - The SFTP is packet oriented-protocol. When downloading, the SFTP client sends a "read" request to the SFTP server, waits for a response, appends returned data to a local file; and repeats, until the end of ...
The best solution would be to ask the person who disabled publickey authentication on that server why and then go fix that. Using publickey authentication is more convenient and more secure than passwords.
The reason authentication fails for you is that no authentication method is enabled on both client and server. The authentication methods enabled on the ...
This problem can be solved with rsync. At least this solution should be competitive in terms of performance.
First, rsync can be called from one of the remote systems to overcome the limitation in the inability to copy between two remote systems directly.
Second, encryption/decryption can be avoided by running rsync in Daemon Access mode instead of Remote ...
You probably do it wrong.
Each user (of the 30) has to create his own (thus private!) key. The public key of each user goes to every server (of the 9000) he should have access to.
Don't do it the other way around.
I haven't got Vim's netrw plugin, but try the following.
Create an ssh client config file for your host. Put the following in $HOME/.ssh/config:
Replace the values according to your host of course. Then login using an scp URL like this:
Quoting man ssh (which is the base used by scp):
Compression is desirable on modem lines and other slow connections, but will only slow down things on fast networks.
The problem is that compressing the data takes more time then just sending it over the network.