1

We are setting up a new server for integration build. The original integration server can finish the "mvn install" in 3 minutes in a Linux box. Now by moving to a Windows 2012 server on VM, it's down to 9 minutes. The "mvn test" part alone costs 7 minutes. Lots of those tests are following the same pattern: 1. Start a fresh JVM 2. Load test data from files/excel spreadsheets into in-memory database 3. Run queries against that database, validate results

In your opinion, how can we improve this VM's performance? The server already has 16GB of Memory, 256GB of space, 8 vCPUs.

And in general, as a build server, most of work are like this: 1. Read the thousands of files from the workspace, 2. Compile the above source code, 3. Generate thousands of .class files. 4. Jar/Zip these thousands of files. 5. Copy the jars zips files to somewhere else.

These seems more workload on CPU and Disk I/O rather more than RAM, right?

Thanks Jirong

Improve this question
1
  • as far as i know, javac can only run single-threaded... so nothing like in gcc with the -j switch. so, this leaves you with your common bottlenecks, disk i/o, cpu performance, and plenty of fast ram. SSD's will help, so will a fast network connection if you checkout fresh for each build. Also, why did you move to windows for your build server? windows has a lot more "stuff" going on in the background that may be effecting performance for your builds. your linux box was probably lean-n-mean.
    – SnakeDoc
    Mar 4, 2014 at 5:44

2 Answers 2

Reset to default
2

Our build server performance was increased by increasing the disk subsystem performance. We now run 4 fast SSD's in hardware RAID 10.

To confirm disk is your performance bottle neck watch the disk subsystem IO compared to your theoretical and look at the latency because when the latency is constantly high you have a problem.

In Windows you can use Perfmon or Windows Resource Monitor. In Linux use iostat. VMWare vshpere client has graphs on performance tab for disk subsystem.

Improve this answer
14
  • 1
    +1. Not doing jave but C#/C++ - same here. Build servers are like databases - the LOVE IO subsystems and gladly suck up whatever budget you deploy there. SSD are nearly a must due to low latency that is stable with parallel accesses. And IO is normally also the weak point of any virtualization setup ;)
    – TomTom
    Mar 3, 2014 at 15:58
  • +1: I'd emphasize disk latency. Regardless of the theoretical limits of your system, if a single 4K read or write is taking longer than 10ms, then you have hit the practical limit of your disk system.
    – surfasb
    Mar 3, 2014 at 16:00
  • @surfasb, Thanks for the information. I kind of know how to read the CPU and RAM usage, but don't know how to interpreter the disk i/o info in Windows Performance Monitor. While i am looking into it now. Is there a simple way to benchmark the disk i/o? e.g.how can I do your "single 4K read or write is taking longer than 10ms" test?
    – user2784896
    Mar 3, 2014 at 16:13
  • @sufasb, does this IO looks ok to you? tinypic.com/r/51t64i/8
    – user2784896
    Mar 3, 2014 at 16:19
  • 1
    @user2784896: blogs.technet.com/b/askcore/archive/2012/02/07/…. The Disk Sec/ per Read/Write Performance Counter tells you how many seconds it takes to complete a 4K operation.
    – surfasb
    Mar 3, 2014 at 17:11
1

The answer to your question can be generalized to just about any situation:

  1. Start collecting metrics (RAM, IO, CPU, network, etc.)
  2. Apply load to system
  3. Analyze metrics to identify bottlenecks
  4. Mitigate bottlenecks
  5. GOTO 1
Improve this answer
4
  • Thanks for your information. The problem is this "Analyze metrics to identify bottlenecks" requires knowledge.
    – user2784896
    Mar 3, 2014 at 15:54
  • @user2784896 As does programming. If you are not willing to learn basics, there is always McDonalds waiting for some burger server. Seriously - it DOES require basic knowledge, and as someone having a build server (i.e. not a total amateur) take some pride and do not opppose learning, especialyl when basically told where to look (by another answer).
    – TomTom
    Mar 3, 2014 at 15:57
  • 1
    Yes indeed it does. Sounds like something you should research, try out, and then if you have questions, come back and ask a well-scoped, well-researched question about it.
    – EEAA
    Mar 3, 2014 at 15:57
  • Up to now, I get almost the same result form both my pC and the server. This give me a hint that the spec of the machine is no longer an issue. I see 80% of time spent in the Test phone of a particular module. Lots of those tests are following the same pattern: 1. Start a fresh JVM 2. Load test data from files/excel spreadsheets into in-memory database 3. Run queries against that database, validate results. This phase takes almost the same time in my pC and server. They told me it's faster in a Linux box. So it's the JVM performance on different platform?
    – user2784896
    Mar 6, 2014 at 16:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .