Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I need to transfer webserver-log-like-files containing periodically from windows production servers in the US to linux servers here in India. The files are ~4 MB in size each and I get about 1 file per minute. I can take about 5 mins lag between the files getting written in windows and them being available in the linux machines. I am a bit confused between the various options here as I am quite inexperienced in such design:

  1. I am thinking of writing a service in C#.NET which will periodically archive, compress and send them over to the linux machines. These files are pretty compressible. WinRAR can convert 32 MB of these files into a 1.2 MB archive. So that should solve the network transfer speed issue. But then how exactly do I transfer files to linux? I could mount linux drive on windows server using samba, or should I create an ftp server, or send the file serialized as a POST request. Which one would be good? Also, I have to minimize the load on the windows server.
  2. Mount the windows drive on linux instead. I could use the mount command or I could use samba here (What are the pros and cons of these two?). I can then write the compressing and copying part in linux itself.

I don't trust the internet connection to be very stable, so there should be a good retry mechanism and failure protection too. What are the potential gotchas in these situations, and other points that I must be worried about?

Thanks, Hari

share|improve this question

migrated from superuser.com Nov 27 '10 at 5:19

This question came from our site for computer enthusiasts and power users.

    
Did you not like my answer? –  Ignacio Vazquez-Abrams Nov 26 '10 at 19:44

5 Answers 5

You could transfer using sftp (SSH). Simple script can be created via windows to linux - this is probably the easiest way, with some added security.

share|improve this answer

Since you are looking for reconnect and resume, I would use FTP or SFTP, together with one of the many commercial products that can easily be googled. Samba is less of a good solution, since disconnecting might cause problems.

My first find on google was BatchSync, which supports both FTP and FTPS/SFTP, but I don't personally know this product. No doubt that it is not the only one of its kind.

share|improve this answer

Install Deltacopy on the Windows machine and setup a scheduled rsync job to sync the folder with the logs. Instructions on the site also show you how to run the transfer via ssh - just one note that after setting up Deltacopy you should do a one-off run of the Deltacopy-generated rsync command manually at a command prompt so you can reply to the prompt to save the remote system's key.

share|improve this answer

You can also try WinSCP, this is the easiest way I found to transfer from Linux/ESX machines to Windows, provided they can be connected. WinSCP is free, safe, easy to use, no installation required, very less resources required and real quick in work.

share|improve this answer
    
I too dont agree for using DropBox. Its easy and tempting to use, however, its neither safe nor reliable. –  whizkid Jan 17 '11 at 8:48

The very easiest way is install dropbox on the windows server and the linux server and just save your log files to your dropbox: http://db.tt/Jw6XNac

share|improve this answer
    
Transfer production log files through DropBox... Not a good idea –  sebthebert Nov 29 '10 at 0:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.