Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Yesterday, OSSEC sent me an alerting email:

Jul 29 21:25:16 SVR4149 kernel: end_request: I/O error, dev sda, sector 334634969
Jul 29 21:25:16 SVR4149 kernel: sd 0:0:0:0: SCSI error: return code = 0x00040000
Jul 29 21:25:16 SVR4149 kernel: end_request: I/O error, dev sda, sector 334634977
Jul 29 21:28:28 SVR4149 kernel: sd 0:0:0:0: SCSI error: return code = 0x00040000

The surprising is at that time I have only /dev/sdb device.

# fdisk -l

Disk /dev/sdb: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          13      104391   83  Linux
/dev/sdb2              14        7662    61440592+  83  Linux
/dev/sdb3            7663        8706     8385930   82  Linux swap / Solaris
/dev/sdb4            8707       38888   242436915    5  Extended
/dev/sdb5            8707       38888   242436883+  83  Linux

After Googling, I found this link. Do the suggested commands, it bring back my /dev/sdc:

Jul 29 22:55:45 SVR4149 kernel: ata1: hard resetting link
Jul 29 22:55:45 SVR4149 kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jul 29 22:55:45 SVR4149 kernel: ata1.00: ATA-8: WDC WD3202ABYS-01B7A0, 02.03B02, max UDMA/133
Jul 29 22:55:45 SVR4149 kernel: ata1.00: 625142448 sectors, multi 0: LBA48 NCQ (depth 31/32)
Jul 29 22:55:45 SVR4149 kernel: ata1.00: configured for UDMA/133
Jul 29 22:55:45 SVR4149 kernel: ata1: EH complete
Jul 29 22:55:45 SVR4149 kernel: ata1.00: detaching (SCSI 0:0:0:0)
Jul 29 22:55:45 SVR4149 kernel:   Vendor: ATA       Model: WDC WD3202ABYS-0  Rev: 02.0
Jul 29 22:55:45 SVR4149 kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05
Jul 29 22:55:45 SVR4149 kernel: SCSI device sdc: 625142448 512-byte hdwr sectors (320073 MB)
Jul 29 22:55:45 SVR4149 kernel: sdc: Write Protect is off
Jul 29 22:55:45 SVR4149 kernel: sdc: Mode Sense: 00 3a 00 00
Jul 29 22:55:45 SVR4149 kernel: SCSI device sdc: drive cache: write back
Jul 29 22:55:53 SVR4149 kernel: SCSI device sdc: 625142448 512-byte hdwr sectors (320073 MB)
Jul 29 22:55:53 SVR4149 kernel:  sdc: sdc1 sdc2 sdc3 sdc4 < sdc5 >
Jul 29 22:55:53 SVR4149 kernel: sd 0:0:0:0: Attached scsi disk sdc
Jul 29 22:55:53 SVR4149 kernel: sd 0:0:0:0: Attached scsi generic sg0 type 0

Rechecking with fdisk:

# fdisk -l

Disk /dev/sdb: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          13      104391   83  Linux
/dev/sdb2              14        7662    61440592+  83  Linux
/dev/sdb3            7663        8706     8385930   82  Linux swap / Solaris
/dev/sdb4            8707       38888   242436915    5  Extended
/dev/sdb5            8707       38888   242436883+  83  Linux

Disk /dev/sdc: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *           1          13      104391   83  Linux
/dev/sdc2              14        7662    61440592+  83  Linux
/dev/sdc3            7663        8706     8385930   82  Linux swap / Solaris
/dev/sdc4            8707       38888   242436915    5  Extended
/dev/sdc5            8707       38888   242436883+  83  Linux

But I got the another problem from kernel log:

Jul 30 01:03:41 SVR4149 kernel: scsi 0:0:0:0: rejecting I/O to dead device
Jul 30 01:14:40 SVR4149 kernel: scsi 0:0:0:0: rejecting I/O to dead device
Jul 30 01:16:41 SVR4149 kernel: scsi 0:0:0:0: rejecting I/O to dead device
Jul 30 01:53:18 SVR4149 last message repeated 7 times

and smartd keep opening the non exist device:

Jul 30 10:00:57 SVR4149 smartd[3749]: Device: /dev/sda, No such device, open() failed

No special in my smartd.conf file:

# grep -v "^#" /etc/smartd.conf | sed '/^$/d'
DEVICESCAN -H -m root

Does my scsi0 is going to 'die'?

cat /proc/scsi/scsi 
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: WDC WD3202ABYS-0 Rev: 02.0
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: WDC WD3202ABYS-0 Rev: 02.0
  Type:   Direct-Access                    ANSI SCSI revision: 05

Any help will be appreciated.

share|improve this question
    
I got this after yet another power failure overnight and we have no UPS at current. Think this will be a kick in the balls to get one! –  CodeBlend Apr 30 '13 at 8:55

1 Answer 1

up vote 3 down vote accepted

Looks like the drive is dropping and then reconnecting. This indicates one of three things:

  1. Most likely a bad drive, I would start checking the SMART logs and see what you get there.
  2. A bad cable/SCSI controller (normally RAID cards) ... if SMART checks out and this continues, swap the cable first then the card.
  3. You are doing so much sustained disk I/O that you are overloading the disk controller ... you should be able to tell if you are overloading the I/O.

Hope that helps ... its a scary message to get.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.