1

We have an MD3000i array accessed by three separate servers (all blade servers accessing through chassis nic's). Controller 1 (both Ports 0 and 1) are linked to Server A. Servers B and C (both with substantially less activity than Server A) are linked to Controller 0 (Ports 0 and 1). Server A is on a separate VLAN (connected to the MD3000i through a switch on the back of the blade chassis), and Servers B and C are on their own VLAN (also connected to the MD3000i through a switch on the back of the blade chassis).

The problem we are running into is that Server A has decidedly better read throughput from the MD3000i. For instance, when looking at backup jobs run through that server, the job rate is roughly 3,000MB/min. However, when looking at backup jobs run through Servers B or C, jobs rates are as low as 200MB/min.

If I view the iSCSI MAC transmit statistics through the MDSM, there are very different readings between controllers (aside from overall volume):

Mac transmit statistics

MAC transmit legend

F = Frame Count
B = Byte Count
MF = Multicast Frame Count
BF = Broadcast Frame Count
PF = Pause Frame Count
CF = Control Frame Count
FDF = Frame Deferral Count
FED = Frame Excess Deferral Count
FLC = Frame Late Collisions Count
FA = Frame Abort Count
FSC = Frame Single Collision Count
FMC = Frame Multiple Collisions Count
FC = Frame Collision Count
FDR = Frame Dropped Count
JF = Jumbo Frame Count

iSCSI Host   | Port    | F            | B               | MF     | BF     | PF   | CF     | FDF  | FED  | FLC  | FA   | FSC  | FMC  | FC   | FDR  | JF

Controller 0 |  port 0 | 440573739    | 651904980400    | 0      | 0      | 0    | 0      | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0
Controller 0 |  port 1 | 440549412    | 651892177000    | 0      | 0      | 0    | 0      | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0    | 0
Controller 1 |  port 0 | 109676946034 | 148467634266838 | 0      | 0      | 512  | 131072 | 512  | 1024 | 1024 | 1024 | 512  | 1792 | 2304 | 2304 | 589824
Controller 1 |  port 1 | 402653242    | 111669157090    | 524288 | 524288 | 2304 | 720896 | 2304 | 3328 | 3840 | 4352 | 2048 | 3584 | 5632 | 2048 | 1114112

My suspicion is that the network card utilized by Servers B and C doesn't have the capabilities needed to achieve the performance we are looking for (notice that the Jumbo Frame count is 0 -- Jumbo Frames being something I've seen recommended when multiple servers are accessing a single MD3000i controller). Server A accesses the MD3000i through a Broadcom BCM57085 NetXtreme II, while Servers B and C access the MD3000i through an Intel PRO/1000 MB Dual Port.

As of now, I don't have a firm method for isolating the cause of the slow read throughput on Controller 0 (Servers B and C).

1

Yes, the lack of jumbo frames is most likely the issue.

For a 1500 byte MTU frame, you'll probably have about 200-250 bytes of overhead between iSCSI, checksumming, TCP/IP, ethernet framing, et cetera. That leaves around 1250 bytes available for data transfer. On the other hand, a 9000 MTU jumbo frame can transmit around 7 times as much data with the same overhead.

You might also want to look to see if the cards have a TCP Offload Engine (TOE), and make sure they're enabled. That can have a decent impact on throughput, especially if CPU starts bottlenecking the process.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.