I have done some simple performance tests and it seems that reading from my RAID1 is slower than writing:
root@dss0:~# for i in 1 2 3; do dd if=/dev/zero of=/dev/sda bs=1048576 count=131072; done 137438953472 bytes (137 GB) copied, 192.349 s, 715 MB/s 137438953472 bytes (137 GB) copied, 192.851 s, 713 MB/s 137438953472 bytes (137 GB) copied, 193.026 s, 712 MB/s root@dss0:~# for i in 1 2 3; do dd if=/dev/sda of=/dev/null bs=1048576 count=131072; done 137438953472 bytes (137 GB) copied, 257.201 s, 534 MB/s 137438953472 bytes (137 GB) copied, 255.522 s, 538 MB/s 137438953472 bytes (137 GB) copied, 259.945 s, 529 MB/s
I understand that dd is not a performance test tool, but this result is still a surprise.
System was built by the vendor and has a Supermicro main board with 16 GByte RAM. RAID controller is a MegaRAID 9271-8i with 1 GByte cache. There are 8 2 TByte SAS disks on a SAS-933EL1 backplane. I am unsure of the cabling, one connector of the controller goes to the SAS backplane, the other goes to two SATA disks which hold the OS.
The RAID1 was set up with this command:
root@dss0:~# /opt/MegaRAID/MegaCli/MegaCli64 -CfgLdAdd -r1 [8:0,8:1,8:2,8:3,8:4,8:5,8:6,8:7] WB NORA Direct -a0 Adapter 0: Created VD 0 Adapter 0: Configured the Adapter!! Exit Code: 0x00 root@dss0:~# /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -LALL -aALL Adapter 0 -- Virtual Drive Information: Virtual Drive: 0 (Target Id: 0) Name : RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0 Size : 7.275 TB Sector Size : 512 Is VD emulated : No Mirror Data : 7.275 TB State : Optimal Strip Size : 256 KB Number Of Drives : 8 Span Depth : 1 Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU Default Access Policy: Read/Write Current Access Policy: Read/Write Disk Cache Policy : Disk's Default Encryption Type : None PI type: No PI Is VD Cached: No Exit Code: 0x00
I would expect that read access is at least as fast as write access, maybe even faster. The 715 MByte/sec write speed seems to be near the 6 GBit limit of a single SAS/SATA connector. Is this maybe a configuration or cabling issue with the SAS backplane? Can the SAS backplane configuration be queried with a MegaRAID command? Please advise.
As expained by poige and Peter, the slower-than-expected read performance is probably caused by caching of the Linux I/O-subsystem.
When using the direct flag in the dd command I get
root@dss0:~# dd if=/dev/sda of=/dev/null bs=1048576 count=131072 iflag=direct 137438953472 bytes (137 GB) copied, 199.862 s, 688 MB/s
which is much better but still 10% slower that the write speed. Using oflag=direct did not affect the write speed.