0

We've got a project on the go to migrate much of our infra to GCP, and a major component of that is our monitoring system. Currently the storage system for perfdata is a 6-node Carbon Storage cluster, and each node pins its underlying physical host's IO at ~2000 write IOPS.

So my baseline requirements for porting over this system is to have a number of storage nodes that can accommodate 16k write IOPS, 4k read IOPS, and contain 600GB of data. According to the docs the amount of IO throughput a GCE disk can handle will scale according to its size.

In order to determine the most economical solution I created the following spreadsheet:

|               | $/GB   | Write IOPS/GB | Read IOPS/GB | Size for Writes | Size for Reads |  Required Size  | Price   | 
|---------------|--------|---------------|--------------|-----------------|----------------|-----------------|---------| 
| PD            | $0.040 | 1.5           | 0.75         | 10666.7         | 8000.0         | 10666.7         | $426.67 | 
| SSD PD        | $0.170 | 30            | 30           | 533.3           | 200.0          | 600.0           | $102.00 | 
| Local SSD     | $0.218 | 186.7         | 266.7        | 85.7            | 22.5           | 600.0           | $130.80 | 
| NVMe SSD      | $0.218 | 240           | 453.3        | 66.7            | 13.2           | 600.0           | $130.80 | 

|               | Write  | Read 
| Required IOPS | 16000  | 6000

| Min Disk Size | 600    | 

But I'm still a bit unclear on a few things:

  1. Are the listed IOPS/GB seriously 1.5/30 and not 1,500/30,000 or some other scaling factor?
  2. Why are the Standard PD write IOPS higher than read IOPS?
  3. Is there am actual listed price for NVMe SSD somewhere, or is it considered the same as Local SSD? [I've assumed the latter]
  4. Does this all look reasonable as far as calculations and assumptions?

Thanks in advance.

1

I will answer in the same order as the questions:

  1. Yes according to this Help Center article without any other scaling factor
  2. Standard Persistent Disk sustained performance caps increase with the size of the volume until reaching the maximum performance available. These performance caps are for sustained activity to disk, not peak IO rates. IO requirements are bursty for many applications, so Compute Engine allows short bursts of IO above the documented caps for volumes less than 1 TB in size. After that burst is exhausted, the IO rates return to the documented caps. This bursting capability can enable you to select your volumes based on their sustained rate rather than the peak rate. This can result in substantial cost savings for workloads where the IO is bursty, which explains the results from the this formula IOPS * TransferSizeInBytes =BytesPerSec (with the answer typically converted to MegabytesPerSec)
  3. Local SSDs are available through both SCSI and NVMe interfaces and the pricing is the same for both.
  4. You can have a cost estimation from the Google Cloud Platform Pricing Calculator here.
1
  • @kamran can you clarify this "Compute Engine allows short bursts of IO above the documented caps for volumes less than 1 TB in size." Is the 1TB limit a per-day burst limit - or is it limited by another factor? – Lucas Oct 30 '17 at 5:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.