I just want to double check my findings here, hope someone out there is able to weight in on some of this.

Let's say I have a SSD storage array capable of delivering 25,200 IOPS and it is connected up to my server using 8Gb/s FC link, with an average IO size of 75kb.

The 8Gb/s FC link after 8b/10b encoding overhead gives you 6.8Gb/s usable bandwidth, or in storage terms 870MB/s and 500,000 IOPS theoretically (I am going off by various online blogs I've read).

Using the MBps = (IOPS * KB per IO) /1024 formula with my numbers above, if I was to run a loadgen on the server to max out the 25k IOPS at 75k avg size, this would need 1,845MBps of bandwidth.

The 8Gb/s FC only provides 870MB/s, so my bottle neck in this case is the 8Gb/s FC link?

Any storage architects out there able to help me out?

  • Are you sure this is not an enterprise storage question? Please use Server Fault for this type of question; we don't do Fibre Channel storage arrays here on Super User. (Please don't cross-post, though.)
    – bwDraco
    Sep 1, 2015 at 23:47
  • No need to repost on Server Fault; we can migrate it there.
    – Arjan
    Sep 1, 2015 at 23:48
  • Ah, sorry.. Server Fault is where I need to be.. how do I migrate the question over?
    – user146882
    Sep 2, 2015 at 0:15
  • The community will vote to migrate this question. You don't need to take any action.
    – bwDraco
    Sep 2, 2015 at 0:38
  • You're misunderestimating the line speed (raw data 8.5 Gb), so your useful throughput number is incorrect. en.wikipedia.org/wiki/Fibre_Channel
    – Ecnerwal
    Sep 5, 2015 at 15:11

2 Answers 2

25000*75*1024=1920000000 Bytes/s
1920000000 Bytes/s=1875000KBytes/s=1831MBytes/s=1,8GBytes/s=14.31GBit/s

So a single 8G Fiber is too little, but a dual link should (nearly) max it out, if the multipathing is fine.

  • And assuming deep enough queue lengths, obviously, to keep the whole thing running at that capacity non stop. Which means you need a lot of separate non-sequential operations.
    – TomTom
    Sep 2, 2015 at 7:24
  • The SAN has dual controllers, so 2 FC links to the fabric and to the hypervisors. On VMware we have Fixed path selection to the VMFS datastore. So from what I understand there. So I could have some VMFS datastores on path A and others on path B to utilize both 8Gb FC links.
    – user146882
    Sep 2, 2015 at 14:28

SSDs have a maximum MB/s as well as a maximum IO/s rating. Your storage will cap on either as you reach it. If that maximum MB/s is higher than a single FC link, you'll be capped by that link.

You must log in to answer this question.