Slow disk performance on Azure Virtual Machine

Question

Okay, so first of all, allow me to say that I'm not an operations person, but a developer. So I'm going into a bit of unknown land here, so please bear with me.

I would like to use a Azure Virtual Machine for extracting a 50 GB XML file from a 1.9 GB zip file. So I've been testing which instance size on Azure I should use to get good performance while not paying for more than I need.

However, the disk performance of Azure VM's have not been amazing, and I would like to know if it's me who's doing something wrong, or if my results are what can be expected.

First of all, what I have been testing with? I have a custom .NET console application that does nothing but take a zip file as an argument and immediately starts extracting the zip file to the same directory that the zip file resides in. While the extraction is going on, the application calculates how many megabytes the application have written to the target file per second and outputs it.

On my local development machine, I get pretty good performance with this application, 160-210 MB/s written. So the whole extraction process takes around 8 minutes. The specs of my local machine is Intel Core i7 950, 3 GHz, 4 cores (8 logical), 12 GB RAM, Samsung SSD 830 series 250 GB.

Okay, so I started testing different instance sizes, and here is my results.

• On a A4 instance with Windows Server 2012 Datacenter R2 (8 cores, 14 GB RAM) with a striped RAID of 4 virtual disks using the same storage account, without host caching, I got steady 30-35 MB/s, which means the whole extraction took 24 minutes and 48 seconds. I also tried enabling host caching, but it didn't really make any difference.
• On a D4 instance with Windows Server 2012 Datacenter (8 cores, 28 GB RAM, 500 GB local SSD disk) i got really good performance (150+ MB/s) for the first minutes, and then varying performance with peaks at 200 MB/s and valleys at 9 MB/s. Average performance was between 70 and 100 MB/s. The extraction took 9 min and 40 seconds.
• On a D3 instance with Windows Server 2012 Datacenter (4 cores, 14 GB RAM, 250 GB local SSD disk) i got really good performance (150+ MB/s) the first minute, but then the performance declined to steady 20-40 MB/s, making the extraction process take 21 minutes and 49 seconds.

On a D2 and D1 instance, the disk performance is worse than on the D3.

And this really surprises me. How can a local SSD disk perform so badly, as it does on the D1, D2 and D3 instances? And does anyone know why the disk performance differs so much between D1 to D4? Is it a memory issue? When I look at the task manager while the extraction os going on, the memory usage is exploding. I suspect it's because Windows is caching the written data, but when it runs out of memory, it has to flush the data to disk. When this happens, the disk performance declines. But this doesn't happen on my local machine, so why is such aggresive caching necessary on these VM's?

I know there's differences between my local machine and a virtual machine hosted in Azure, but is the disk performance I'm experiencing really to be expected?

(I originally posted my question on Stackoverflow, as I suspected it was my application that was the cause. But I'm not so sure anymore)

Answer 1

Bit late to the party here, but for what it's worth "attached" SSDs in Azure have their IOPs throttled based on the machine size. It wasn't mentioned in the pricing anywhere but I raised this issue as a ticket with technical support when they referred me to the blog post below.

See this link: http://azure.microsoft.com/blog/2014/10/06/d-series-performance-expectations/

Answer 2

Maybe it's not related to the memory. "Physical" disks on Azure uses what's called Blob storage and that's the reason the performance is not the same as in your local computer even with similar hardware. You can find more information on this link. This seems to be a common problem for the SQL Server community who relies heavily on disk performance. I'll quote an article i found about it

The Performance Guidance for SQL Server in Azure VMs whitepaper mentions (on pages 15 and 26) that in Azure VMs, Storage Spaces can be used to improve storage performance for SQL Server workloads

Storage Spaces is a built in feature of Windows Server 2012 that lets you use several physical disks to create one single virtual disk, hence as a work around for the I/O limitations of the VM disk itself. Quoting again:

With larger VMs that support many (up to 16) physical disks, a very significant increase in storage performance is possible. This can enable Azure as a viable platform for many SQL workloads that would be too constrained by the I/O performance of a single Azure disk otherwise.

This article will show you the steps to create a storage space.

You should try this first. Probably you can even use a smaller instance as the storage service is a separate service from the instance itself.