2

I am running a large scale ERP system on the following server configuration. The application is developed using AngularJS and ASP.NET 4.5

Dell PowerEdge R730 (Quad Core 2.7 Ghz, 32 GB RAM, 5 x 500 GB Hard disk, RAID5 configured) Software: Host OS is VMWare ESXi 6.0 Two VMs run on VMWare ESXi .. one is Windows Server 2012 R2 with 16 GB memory allocated ... this contains IIS 8 server with my application code Another VM is also Windows Server 2012 R2 with SQL Server 2012 and 16 GB memory allocated .... this just contains my application database.

You see, I separated the application server and database server for load balancing purposes.

My application contains a registration module where the load is expected to be very very high (around 10,000 visitors over 10 minutes)

To support this volume of requests, I have done the following in my IIS server -> increase request queue in application pool length to 5000 -> enable output caching for aspx files -> enable static and dynamic compression in IIS server -> set virtual memory limit and private memory limit of each application pool to 0 -> Increase maximum worker process of each application pool to 6

I then used gatling to run load testing on my application. I injected 500 users at once into my registration module.

However, I see that only 40% / 45% of my RAM is being used. Each worker process is using only a maximum amount of 130 MB or so.

And gatling is reporting that around 20% of my requests are getting 403 error, and more than 60% of all HTTP requests have a response time greater than 20 seconds.

A single user makes 380 HTTP requests over a span of around 3 minutes. The total data transfer of a single user is 1.5 MB. I have simulated 500 users like this.

Is there anything missing in my server tuning? I have already tuned my application code to minimize memory leaks, increase timeouts, and so on.

3
  • What is the exact ram provided to each server, I don't understand if you have a server of 32GB and two VM's running on it. You need base server OS needs memory too right? What is the allocation on the both server, you see under the computer property screen.
    – Mac
    Apr 12, 2016 at 14:04
  • You should move these questions to a comment.
    – pat o.
    Apr 12, 2016 at 15:15
  • Each of my two VMs have an allocated RAM of 16 GB. The total RAM of my server is 32 GB. @Mac
    – Shuaib
    Apr 12, 2016 at 16:25

1 Answer 1

0

Place your SQL Server on a dedicated host. You have two generations of guest operating system running here before you get access to a physical resource. You have the hypervisor as the base OS, Child Guest is the Windows Server host, the Grandchild OS is the SQL Server which grabs blocks of disk and RAM and manages it with it's own namespace. You can mitigate some of this if you have dedicated disks and dedicated network. Do not allow the VM to be relocatable.

http://www.tpc.org/tpctc/tpctc2009/tpctc2009-13.pdf

You are going to have some arbitration on shared resources as you have 32GB allocated between your two hosts, which leaves precisely zero for the hypervisor, so the hypervisor will begin arbitrating access to share segments while it leaves a block for itself. You would actually have far better performance if you installed both components in the same OS instance and used a local named pipe for all communication to and from SQL Server. As it stands you are cultivating lots of overhead for an oversubscription of the hypervisor physical host, two OS instances and the grandchild operating system running.

"A single user makes 380 HTTP requests..."

Presumably these are not all dynamic elements and there is a lot of static fluff here in the form of images, style sheets, JavaScript, fonts and the like. Get a CDN in front of these static elements, so the load drops to almost NIL on the origin for the static components. You want the resources and the load to go exclusively to the one dynamic piece, the form submitted data and not to the general file handling. Heck, I would go far to suggest that you want the cache age at the client to be at least as long as your build deployment duration. Deploy on Saturday and Midnight every week, then make the cache age to expire at Midnight on Saturday plus one week with every build. This will keep your CDN seeded well and you can avoid the hits, the network load and the server load.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .