2

While debugging a problem with slow hg cloning from Windows clients, I found that hg clone performance could be significantly improved (10x faster) on EC2 2012 virtual instances by reducing the default os page size from 8gb to 1gb to force utilisation of (abundant) physical memory.

For my tests, I run this clone command:

hg clone -U --traceback --verbose --profile --time https://hg.mozilla.org/mozilla-central

On Windows Server 2012:

  • with default settings (OS manages the page size):

    time: real 653.507 secs (user 115.891+0.000 sys 486.922+0.000)
    
  • with manual settings (page file max size 1024mb):

    time: real 179.646 secs (user 56.594+0.000 sys 58.234+0.000)
    

On Windows Server 2008:

  • with manual settings (page file max size 1024mb):

    time: real 1273.845 secs (user 98.905+0.000 sys 592.585+0.000)
    

I have not been able to replicate the performance increase from 2012 on 2008 where the same command is always slow regardless of the OS page size.

I'm pretty confident that we can rule out things like software, network and hardware considerations because I've done my best to replicate the environments as accurately as possible (virtual hardware is identical: c3.2xlarge, 16gb ram, 80gb ssd, same network subnets, etc). I think (because of a comment from Matt Mackall on the hg bug I raised, on the significance of the user/sys timing ratios), that we're really looking at an OS level io or memory configuration problem.

Any ideas what else I should look at, besides the page file size?

1
  • Have you seen any difference using MDT?
    – Jim B
    Sep 22, 2015 at 11:49

1 Answer 1

0

I can't think of a reason that the page file should make a difference other than simply a size difference (and if you are doing non-wim based cloning of course you will have perf issues with the page file). Eliminating or reducing the page file size doesn't influence how much ram is used. See https://serverfault.com/a/75027/3528

All that being said for default windows setups could EC2 be doing some sort of memory balloning?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.