Tag Info

Hot answers tagged

18

You could use rsync(1): rsync --remove-source-files /path/to/source /path/to/destination This will remove successfully transferred files from the original path.


16

Ugh. Your teacher is not correct at all. HTTP transfers files - that's how it works. It doesn't require anything special to do so, even if an Apache directory listing kinda looks like an FTP server listing. Look at your browser URL. Use a tool like Wireshark or Fiddler to actually look at the traffic. You'll see that if you're browsing via HTTP, and the ...


9

You want for. An example (this will just show what will be done): for item in *; do echo mv "$item" /destination/directory done When you're happy, remove echo to do it for real.


5

During an rsync transfer are you CPU bound, or is your link saturated. If your link is saturated, but your CPU is idle, then compress. If your CPU is maxed, and your link isn't, then do not compress.


3

I work this quite often, migrating data between servers and during impossibly-long data transfers... The short answer, of course, is to test with your specific data... This is really easy to do, right? Try a LAN transfer with compression off, then try it with compression on... In my experience with production data sets across a few environments, on a GigE ...


3

Using the mv command to move files from one volume to a different volume is a copy operation. But how would you run out of space on the source volume? You would only run out of space on the target volume if that volume is smaller than the total size of the files you're moving. But either way, you're only deallocating space on the source volume, not ...


3

Your network card uses the Intel igb driver. Under VMware and Linux, that device/driver can be troublesome. I'd try the following: Your ESXi build number is 1331820 and includes version 4.2.16.8 of the igb NIC driver. The current ESXi build has version 5.0.5.1.1 of the igb driver. An update of ESXi may be appropriate. The defaults on the igb driver don't ...


2

These answers didn't help me out. I really didn't need any crazy scripts. I had created a public key on my client machine in git bash and was trying to copy it to a VPS. After creating your public key, the key should be stored as "(whatever folder you started in)/.ssh/id_rsa.pub" So use this command: cat ~/.ssh/id_rsa.pub | ssh user@123.45.67.89 "cat ...


2

It's likely the most simple solution would be for you to use rsync over ssh. This requires that each user have an account on the server. For this example, assume that they're wanting to copy files from their local machine, /home/user1/folder to /home/user1/foo on the server. To do this, they'd run the following from their workstation: $ rsync -avz ...


2

For a complete retention of everything, try: rsync --archive --hard-links --sparse-files --acls --xattrs source dest Then to verify correctness, you can do something like (assuming no spaces in your file names): for file in *; do md5sum $file >> /tmp/sums.txt; done and then diff sums.txt on both servers. The md5summing is going to take a while, ...


2

Sorry I dont have a new enough cred to just comment. But something like this should help you with the zipping of the folders. #! /bin/bash DIR=$1 i=0 arr=() zipnum=0 cd $DIR for item in * do if [[ -d $item ]]; then if [ $i -lt 2 ]; then arr+=($item) ((i++)) else ...


2

You have virtually innumerable options for copying files between servers. FTP, SMB (drag and drop, or a command line copy like robocopy), BITS, HTTP, even SCP. Which one is "best" depends entirely on your preferences and specific situation (which you haven't told us enough about to make a determination on). As to speed, you're transferring 200GB over the ...


2

ls -1 | xargs -n1 -i echo mv '{}' destination Just remove the echo when happy.


1

Note: the following is all going off theory -- the real right way to make sure this is correct in your situation is to run tests on various combinations of options. The data connections in an rsync operation look something like this: Source disk <-> rsync instance <-> other rsync instance <-> destination disk In general, rsync is ...


1

Like Zoredache said, if your link isnt saturated no -z. Also, something else you can consider to help manage long transfers is the bwlimit flag.


1

The below script was tested on CentOS 6.5. Walks through the files under a specified folder and adds them one by one to zip file(s) no bigger than a specified max-size. Adding files to a zip one at a time can cause CPU or Storage I/O spikes: #!/bin/bash files_folder="/home/username/public_html/" zip_folder="/home/username/" zip_maxsize=524288000 # ...


1

Depends on what you want to do.It's almost always faster to copy a single 10 gig file than 10 gig with 1000's of small files (because of connections etc) although you could speed up the small file size by using concurrent connections. SFTP/SCP are good bets for general use. Rsync and Robocopy (windows only) are more specific tools.Torrents are a very ...


1

Just for reference, I believe the mv command doesn't do a copy then delete operation, ie having mv /home/foo /home/bar exist simultaneously for a short period. It operates more like renaming. Essential changing the filesystem accounting so that what pointed at /home/foo now points to /home/bar without actually physically moving the data on disk from one ...


1

Check out WatchDox I actually went through what you're looking at, this looked to be the most secure of all the Enterprise solutions I could find. Alternately, if you can get the encryption working, OwnCloud might work well. As was said in the comments though, those are some ridiculous requirements. With no IT staff I wouldn't even look at an on-prem ...


1

A simple alternative, still using FTP, might be to use an FTP client which manifests as a filesystem - for example I use curlftpfs under Linux. Never looked at them, but Webdrive and FTPDrive apparently do something similar under Windows. A couple of alternative solutions which might do something similar - Setting up the server with SAMBA (if the clients ...


1

If you are using cmder (or msysgit/mingw that has scp & ssh), I just wrote a simple python script for this. It can be found here: https://gist.github.com/ceilfors/fb6908dc8ac96e8fc983 Sample usage: python ssh-copy-id.py user@remote-machine. Password will be prompted upon running the script.


1

Inspired by zoredache's answer, I've created a bunch of scripts that are the windows version. However they all depend on plink. Please take a look here https://github.com/VijayS1/Scripts/blob/master/ssh-copy-id/ I also have a winscp script that can be used as per another answer. :) Excerpt from the readme: Attempted methods so far: DOS(.cmd) - Success ...


1

You can use this simple windows FTP tool, in command line; Check wiki page for manual. http://sourceforge.net/projects/ftpandfilesystemsynchronizer/



Only top voted, non community-wiki answers of a minimum length are eligible