Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have two identical servers, running Proxmox 2. 2x AMD 6 cores 2.3Ghz 64 Gigs of RAM DDR3 ECC (8*8) ASUS KCMA-D8 2x Seagate 2TB - (st2000dm001)

One server is running perfectly fine. The other one is not. As long as something around I/O is involved, the CPU load jumps (only of iowait). Formatting the hard drive takes a LOT of time. And when running the OS it feels like it is slower.

I thought about defective hardware, so I switched everything : PSU, RAM (all of them), SATA cables, SATA hotswap backplane, motherboard, hard-drives, etc... Still I have one server running fast, the other one being slow. I noticed that when I place on the hard-drives, the system becomes slow. Whatever the server is. Issue is that neither SMART monitoring, neither Seagate Tools does reveal something.

Then I saw something about 4K block sectors. Any chance it could come from that ? How can I be sure my system is correctly set-up for 4K ?

I'm basically out of idea now :( Any idea would be greatly appreciated !!!

--- MARS server (which is OK) ---

root@mars:~# hdparm -T /dev/sda

/dev/sda:
 Timing cached reads:   6314 MB in  2.00 seconds = 3158.71 MB/sec
root@mars:~# hdparm -T /dev/sdb

/dev/sdb:
 Timing cached reads:   6524 MB in  2.00 seconds = 3263.56 MB/sec
root@mars:~# 


root@mars:~# dmesg | grep sda
sd 0:0:0:0: [sda] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 0:0:0:0: [sda] 4096-byte physical blocks
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda:
 sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
md: bind<sda1>
md: bind<sda2>
 disk 0, wo:1, o:1, dev:sda2
 disk 0, wo:1, o:1, dev:sda2
root@mars:~# dmesg | grep sdb
sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 1:0:0:0: [sdb] 4096-byte physical blocks
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1 sdb2
sd 1:0:0:0: [sdb] Attached SCSI disk
md: bind<sdb1>
md: bind<sdb2>
 disk 1, wo:0, o:1, dev:sdb2
 disk 1, wo:0, o:1, dev:sdb2
sd 1:0:0:0: [sdb] Unhandled sense code
sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor]
sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed
sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 10 c3 3f 00 00 00 80 00
sd 1:0:0:0: [sdb] Unhandled sense code
sd 1:0:0:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 1:0:0:0: [sdb] Sense Key : Medium Error [current] [descriptor]
sd 1:0:0:0: [sdb] Add. Sense: Unrecovered read error - auto reallocate failed
sd 1:0:0:0: [sdb] CDB: Read(10): 28 00 10 c3 3f 70 00 00 08 00
md/raid1:md1: sdb: unrecoverable I/O read error for block 280180480
root@mars:~# 

root@mars:~# fdisk -u -l /dev/sda

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00081a13

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1048575      523264   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2         1048576  3907028991  1952990208   fd  Linux raid autodetect
root@mars:~# fdisk -u -l /dev/sdb

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x000c9e45

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048     1048575      523264   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2         1048576  3907028991  1952990208   fd  Linux raid autodetect
root@mars:~# 

root@mars:~# uname -a
Linux mars 2.6.32-16-pve #1 SMP Mon Oct 22 08:38:13 CEST 2012 x86_64 GNU/Linux
root@mars:~# 

(I'm constructing a RAID, everything is OK .... ~100 MB / s)

root@mars:~# cat /proc/mdstat 
Personalities : [raid1] 
md1 : active raid1 sda2[2] sdb2[1]
      1952989048 blocks super 1.2 [2/1] [_U]
      [======>..............]  recovery = 34.6% (675754624/1952989048) finish=243.2min speed=87518K/sec

md0 : active raid1 sda1[2] sdb1[1]
      523252 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@mars:~# 

--- MOON server (which is NOT OK) ---

root@moon:~# dmesg | grep sda
sd 0:0:0:0: [sda] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
sd 0:0:0:0: [sda] 4096-byte physical blocks
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda:
 sdb: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
md: bind<sda1>
md: bind<sda2>
 disk 0, wo:1, o:1, dev:sda2
 disk 0, wo:1, o:1, dev:sda2
root@moon:~# 

root@moon:~# fdisk -u -l /dev/sda

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00045925

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1048575      523264   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2         1048576  3907028991  1952990208   fd  Linux raid autodetect
root@moon:~# 

root@moon:~# fdisk -u -l /dev/sdb

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0008d735

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048     1048575      523264   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2         1048576  3907028991  1952990208   fd  Linux raid autodetect
root@moon:~# 

root@moon:~# uname -a
Linux moon 2.6.32-16-pve #1 SMP Mon Oct 22 08:38:13 CEST 2012 x86_64 GNU/Linux
root@moon:~# 

Constructing the RAID ... will never work. (!) 2 MB / s (!!!!)

root@moon:~# cat /proc/mdstat 
Personalities : [raid1] 
md1 : active raid1 sda2[2] sdb2[1]
      1952989048 blocks super 1.2 [2/1] [_U]
      [>....................]  recovery =  2.8% (55216832/1952989048) finish=15690.7min speed=2015K/sec

md0 : active raid1 sda1[2] sdb1[1]
      523252 blocks super 1.2 [2/2] [UU]

unused devices: <none>
root@moon:~# 

Thanks,

Ugo

share|improve this question
    
If your partitions start on a multiple of 4K blocks then it should be fine for AF drives. – You wrote 'have two identical servers'. If the drives are identical and the partitioning is identical then both should be equally slow or equally fast. Which would rule that out. –  Hennes Nov 2 '12 at 0:35
    
Your partition starts at 2048 so your 4k alignment is okay. –  chutz Nov 2 '12 at 3:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.