Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a scheduled task that runs every night and copies a file of around 70-80gb from one server to another on our network. For some reason it has been taking ~8 hours to do this, which is a problem because it doesn't finish before our nightly backup tape operation runs and this file doesn't make it to the tape.

Any suggestions to make this run quicker?

Here's the batch file

if not exist g:\corp-prod-02\ihub\ihub.bkp goto backup
del /Q g:\Corp-prod-02\ihub\old\ihub.bkp
move g:\Corp-prod-02\ihub\ihub.bkp g:\corp-prod-02\ihub\old
:backup
call probkup online D:\ihubdb\live-new\ihub D:\ihubdb\ihub.bkp
robocopy D:\ihubdb G:\corp-prod-02\ihub ihub.bkp /Z /MOV /LOG:c:\scripts\logs\ihub.log
copy c:\scripts\logs\ihub.log g:\corp-prod-02\ihub
share|improve this question
    
batch file formatting got messed up, sorry about that –  Mike Feb 24 '12 at 20:55
2  
I suggest you use a tool like iperf and see what your maximum transfer rate is between the two systems. If you only had 10mb, connectivity, then it wouldn't be surprising that it isn't completing. Then double check the speed you can read/write to the drives each side. Once you know your transport speed and the actual drive speeds figure out if robocopy is being unreasonable. –  Zoredache Feb 24 '12 at 20:58
    
ahh you reminded me of something I forgot to mention, we have another scheduled task that runs 5 minutes earlier, and copies a different file between the two servers. This file is only about 1 gb yet it transfers at 40mb/sec while the one in question transfers at 2.75mb/sec –  Mike Feb 24 '12 at 21:01
    
Out of curiosity, what OS is running on each end? Also, does your robocopy log file indicate any failures followed by retries? Default robocopy behaviour is to wait 30 seconds b/w retries, which certainly wouldn't help your duration. –  JamesCW Feb 24 '12 at 21:08
    
That is concerning. Is there some reason why you are using the /z switch? There seems to be many hits on the Interwebs about drastically slowing things down. –  Zoredache Feb 24 '12 at 21:09
show 6 more comments

1 Answer

up vote 0 down vote accepted

My first thought was that it was a network issue, but then your comment explaining that you don't have any issues with smaller files, reminded me of a problem I've seen in the past when transferring large files around. It took me a while to work out what was going on, but I ended up tracing it to an exhaustion of the kernel's non paged memory pool.

It might be worth reading these articles and using poolmon.exe (specifically the MmSt pool tag) to see if you are experiencing the same issue.

Edit:

This article is aimed at NT4 and Windows 2000, but probably still relevant.

share|improve this answer
    
I'll look into that, thanks for the suggestion –  Mike Feb 24 '12 at 21:45
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.