I have just configured a server-to-server 4x SDR InfiniBand link to increase bandwidth for the backend storage of my virtual machines, specifically a database server. I am using iSCSI with TGTD on CentOS 6 and I am able to achieve about 6.45Gbps using IPoIB when testing with iperf.
I have managed to tune the iSCSI initiator and target to the point where I can achieve about 260MB/s write to an SSD. However, this is still not an optimal write speed given I can write at about 500-600MB/s locally.
The parameters I am currently using are:
- MaxRecvDataSegmentLength 4194304
- MaxXmitDataSegmentLength 524288
- MaxBurstLength 4194304
- FirstBurstLength 262144
Has anyone had any experience in tuning this? Is it possible to get any higher than this with configuration changes? Even by doing a RAID 0 on two devices, I can still only achieve the same bandwidth, which I would have expected higher.