CentOS, Xeon 1230, 16 Gb RAM, 2x1TB SSD in raid1 (mdadm).
Read from raid1 vary from 130 to 250 MB/s depends of files etc.
But write speed very slow: 15-20 MB/s (iotop, mc etc.)! You know the real write speed to SSD. CPU load not more than 23-25% (only one core). So situation isn't normal.
What should I do to reach hiest speed?
I try increase speed_limit_min:
# sysctl -w dev.raid.speed_limit_min=20000
but without result.
What should I do in order to increase write speed?
Update 1:
Raid details
# mdadm --detail /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Mon Dec 7 14:23:49 2015
Raid Level : raid1
Array Size : 999547904 (953.24 GiB 1023.54 GB)
Used Dev Size : 999547904 (953.24 GiB 1023.54 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Sun Feb 12 13:26:17 2017
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : localhost:pv00
UUID : 0fe1355e:edd9d104:e2db49ec:bc3bb9b6
Events : 7865
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 33 1 active sync /dev/sdc1
Parted output for sda:
# parted /dev/sda print
Model: ATA Samsung SSD 850 (scsi)
Disk /dev/sda: 1024GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 538MB 537MB primary ext4 boot
2 538MB 1024GB 1024GB primary raid
Parted output for sdc
# parted /dev/sdc print
Model: ATA Samsung SSD 850 (scsi)
Disk /dev/sdc: 1024GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 1024GB 1024GB primary raid
mdstat file contents:
# cat /proc/mdstat
Personalities : [raid1]
md2 : active raid1 sdd1[1] sdb1[0]
976630464 blocks super 1.2 [2/2] [UU]
bitmap: 0/8 pages [0KB], 65536KB chunk
md127 : active raid1 sdc1[1] sda2[0]
999547904 blocks super 1.2 [2/2] [UU]
bitmap: 3/8 pages [12KB], 65536KB chunk
It is impossible...
dmesg | grep -i ahci
return nothing... the end... Can it lead to my problem?
Update 1: Turn on (bios, initramfs). No affect. Copying 20 Gbytes: firstly 130-160 MB/s and liearly slowing down to 17 MB/s after minute or two.
Update 2:
# hdparm -I /dev/sda
/dev/sda:
ATA device, with non-removable media
Model Number: Samsung SSD 850 PRO 1TB
Serial Number: S252NXAG905316P
Firmware Revision: EXM02B6Q
Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x0039)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 2000409264
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
Logical Sector-0 offset: 0 bytes
device size with M = 1024*1024: 976762 MBytes
device size with M = 1000*1000: 1024209 MBytes (1024 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: Solid State Device
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 1 Current = 1
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* 64-bit World wide name
Write-Read-Verify feature set
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Phy event counters
* unknown 76[15]
* DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Asynchronous notification (eg. media change)
* Software settings preservation
unknown 78[8]
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
* reserved 69[4]
* DOWNLOAD MICROCODE DMA command
* SET MAX SETPASSWORD/UNLOCK DMA commands
* WRITE BUFFER DMA command
* READ BUFFER DMA command
* Data Set Management TRIM supported (limit 8 blocks)
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
supported: enhanced erase
2min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 500253884009b0f0
NAA : 5
IEEE OUI : 002538
Unique ID : 84009b0f0
Checksum: correct
# hdparm -I /dev/sdc
/dev/sdc:
ATA device, with non-removable media
Model Number: WDC WD1004FBYZ-01YCBB1
Serial Number: WD-WMC6M0D492LN
Firmware Revision: RR04
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x006d)
Supported: 10 9 8 7 6 5
Likely used: 10
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 1953525168
Logical Sector size: 512 bytes
Physical Sector size: 512 bytes
Logical Sector-0 offset: 0 bytes
device size with M = 1024*1024: 953869 MBytes
device size with M = 1000*1000: 1000204 MBytes (1000 GB)
cache/buffer size = unknown
Form Factor: 3.5 inch
Nominal Media Rotation Rate: 7200
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Advanced power management level: 96
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
* Advanced Power Management feature set
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
* 48-bit Address feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* WRITE_{DMA|MULTIPLE}_FUA_EXT
* 64-bit World wide name
* IDLE_IMMEDIATE with UNLOAD
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
unknown 119[6]
unknown 119[7]
unknown 119[8]
unknown 119[9]
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Phy event counters
* Idle-Unload when NCQ is active
* NCQ priority information
* unknown 76[14]
* unknown 76[15]
* DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Software settings preservation
unknown 78[7]
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
* reserved 69[3]
* DOWNLOAD MICROCODE DMA command
* Long physical sector diagnostics
Security:
Master password revision code = 65534
supported
not enabled
not locked
frozen
not expired: security count
supported: enhanced erase
118min for SECURITY ERASE UNIT. 118min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee0aef2a46c
NAA : 5
IEEE OUI : 0014ee
Unique ID : 0aef2a46c
Checksum: correct
Update 3: smartctl
# smartctl -a /dev/sda SMART Attributes Data Structure revision number: 1 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 097 097 000 Old_age Always - 15006 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 34 177 Wear_Leveling_Count 0x0013 099 099 000 Pre-fail Always - 18 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 067 054 000 Old_age Always - 33 195 Hardware_ECC_Recovered 0x001a 200 200 000 Old_age Always - 0 199 UDMA_CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0 235 Unknown_Attribute 0x0012 099 099 000 Old_age Always - 12 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 22922238691
Update 4: Think I've found. Option "discard" in /etc/fstab. It increase write speed on testing server up to 70-80 MB/s, on target server I can't check it right now.
Thank you all, I think problem is solved.
mdadm --detail /dev/<md-device>. Did you use partitions or the whole disk. In case of partition, are those partitions properly aligned? Add the output ofparted /dev/<disks> print. Did you test each disk individually to check if both drives are OK and perform as expected?cat /proc/mdstathdparm -I /dev/sda(and all other devices)?smartctl -a /dev/sda