6

I experienced an error in my ZFS machine. This may have happend after I lost power to my server, but I can not tell for sure. My ZFS Pool Gewuerzglas with RAIDZ-1 is no longer willing to import. I always get the error when trying to import:

cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

I already tried several things. Non seemed to work. Any suggestens how to rescue this pool? As all the drives are still online it seems to me that the data might be still there but some check sums gone bad?

What I have tried so far

root@openmediavault:~# zpool import
   pool: Gewuerzglas
     id: 15011586312885837941
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:
        Gewuerzglas  ONLINE
          raidz1-0  ONLINE
            sda     ONLINE
            sde     ONLINE
            sdj     ONLINE
            sdc     ONLINE
            sdg     ONLINE
            sdf     ONLINE
            sdi     ONLINE
            sdh     ONLINE
            sdd     ONLINE

root@openmediavault:~# zpool import Gewuerzglas
cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

root@openmediavault:~# zpool import -f Gewuerzglas
cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

root@openmediavault:~# zpool import -f -F Gewuerzglas
cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

root@openmediavault:~# zpool import -f -F -X Gewuerzglas
cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

root@openmediavault:~# zpool import -m -f -o readonly=on Gewuerzglas
cannot import 'Gewuerzglas': I/O error
        Destroy and re-create the pool from
        a backup source.

root@openmediavault:~# zpool import -f -F -n Gewuerzglas

Does not show anything

I could also make this screenshot when I connected a monitor to my server

Anyone knows how to repair or rescue data?

System information

Distribution Name       | Openmediavault
Distribution Version    | 3.0.59
Linux Kernel            | Linux 3.16.0-4-amd64
Architecture            | Debian
ZFS Version             | version:        0.6.5.8-2~bpo8+1
SPL Version             | version:        0.6.5.8-2~bpo8+2

fdisk -l output:

root@openmediavault:~# fdisk -l

Disk /dev/sdb: 55,9 GiB, 60021399040 bytes, 117229295 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3fb31afd

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdb1  *           63 112084897 112084835  53,5G 83 Linux
/dev/sdb2       112084898 117212193   5127296   2,5G  5 Extended
/dev/sdb5       112084900 113466426   1381527 674,6M 82 Linux swap / Solaris

Disk /dev/sda: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B9434DE2-9E3A-504E-B242-B789190FA040

Device          Start        End    Sectors  Size Type
/dev/sda1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sda9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdc: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: DC494CB2-6208-6B4E-ACB9-190924BCBC70

Device          Start        End    Sectors  Size Type
/dev/sdc1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdc9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdd: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0A7AAF08-3D15-5747-8188-8121A00A70C9

Device          Start        End    Sectors  Size Type
/dev/sdd1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdd9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sde: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: DDF59778-5338-C543-B96C-D5D151C953F4

Device          Start        End    Sectors  Size Type
/dev/sde1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sde9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdg: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: F4B63B28-F391-B14E-B9B9-A6CE0F4467E9

Device          Start        End    Sectors  Size Type
/dev/sdg1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdg9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdi: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E367E0F7-C817-354D-AC4A-B730F9FAAB19

Device          Start        End    Sectors  Size Type
/dev/sdi1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdi9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdj: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 93A033FA-229E-234B-8AF1-C9869F3D05FF

Device          Start        End    Sectors  Size Type
/dev/sdj1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdj9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdh: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 18213F37-6CEC-BF48-AC84-E6434144BAFE

Device          Start        End    Sectors  Size Type
/dev/sdh1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdh9  3907012608 3907028991      16384    8M Solaris reserved 1

Disk /dev/sdf: 1,8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 03F63180-AF94-E442-8758-66F78148FA82

Device          Start        End    Sectors  Size Type
/dev/sdf1        2048 3907012607 3907010560  1,8T Solaris /usr & Apple ZFS
/dev/sdf9  3907012608 3907028991      16384    8M Solaris reserved 1

lsscsi output

root@openmediavault:~# lsscsi
[0:0:0:0]    disk    ATA      HUA722020ALA330  A34H  /dev/sda
[2:0:0:0]    disk    ATA      OCZ-VERTEX2      1.11  /dev/sdb
[6:0:0:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sdc
[6:0:1:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sdd
[6:0:2:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sde
[6:0:3:0]    disk    ATA      SAMSUNG HD204UI  0001  /dev/sdf
[6:0:4:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sdg
[6:0:5:0]    disk    ATA      ST32000542AS     CC34  /dev/sdh
[6:0:6:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sdi
[6:0:7:0]    disk    ATA      Hitachi HUA72302 A840  /dev/sdj

lsscsi -c output

root@openmediavault:~# lsscsi -c
Attached devices:
Host: scsi0 Channel: 00 Target: 00 Lun: 00
  Vendor: ATA      Model: HUA722020ALA330  Rev: A34H
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi2 Channel: 00 Target: 00 Lun: 00
  Vendor: ATA      Model: OCZ-VERTEX2      Rev: 1.11
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 00 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 01 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 02 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 03 Lun: 00
  Vendor: ATA      Model: SAMSUNG HD204UI  Rev: 0001
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 04 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 05 Lun: 00
  Vendor: ATA      Model: ST32000542AS     Rev: CC34
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 06 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

Host: scsi6 Channel: 00 Target: 07 Lun: 00
  Vendor: ATA      Model: Hitachi HUA72302 Rev: A840
  Type:   Direct-Access                    ANSI SCSI revision: 05

I somehow managed to import the pool. But unfortunaly it is still not working. I get different status from zpool.

root@openmediavault:~# zpool status
  pool: Gewuerzglas
 state: UNAVAIL
status: One or more devices could not be used because the label is missing
        or invalid.  There are insufficient replicas for the pool to continue
        functioning.
action: Destroy and re-create the pool from
        a backup source.
   see: http://zfsonlinux.org/msg/ZFS-8000-5E
  scan: none requested
config:

        NAME                     STATE     READ WRITE CKSUM
        Gewuerzglas              UNAVAIL      0     0     0  insufficient replic                                                                    as
          raidz1-0               UNAVAIL      0     0     0  insufficient replic                                                                    as
            2196863600760350153  FAULTED      0     0     0  was /dev/sda1
            sde                  ONLINE       0     0     0
            sdj                  ONLINE       0     0     0
            sdc                  ONLINE       0     0     0
            sdg                  ONLINE       0     0     0
            sdf                  ONLINE       0     0     0
            sdi                  ONLINE       0     0     0
            sdh                  ONLINE       0     0     0
            8554831367157426907  FAULTED      0     0     0  was /dev/sdd1

The otherone I become is

root@openmediavault:~# zpool status
  pool: Gewuerzglas
 state: UNAVAIL
status: One or more devices are faulted in response to persistent errors.  There are insufficient replicas for the pool to
        continue functioning.
action: Destroy and re-create the pool from a backup source.  Manually marking the device
        repaired using 'zpool clear' may allow some data to be recovered.
  scan: none requested
config:

        NAME                     STATE     READ WRITE CKSUM
        Gewuerzglas              UNAVAIL      0     0     0  insufficient replicas
          raidz1-0               UNAVAIL      0     0     0  insufficient replicas
            2196863600760350153  FAULTED      0     0     0  was /dev/sda1
            sde                  ONLINE       0     0     0
            sdj                  ONLINE       0     0     0
            sdc                  ONLINE       0     0     0
            sdg                  ONLINE       0     0     0
            sdf                  ONLINE       0     0     0
            sdi                  ONLINE       0     0     0
            sdh                  ONLINE       0     0     0
            8554831367157426907  FAULTED      0     0     0  was /dev/sdd1

root@openmediavault:~# zpool clear Gewuerzglas
cannot clear errors for Gewuerzglas: one or more devices is currently unavailable

root@openmediavault:~# blkid
/dev/sdb1: UUID="883ce00c-edc5-479f-9fac-8941b0413ef8" TYPE="ext4" PARTUUID="3fb31afd-01"
/dev/sdb5: UUID="f670cf30-a948-47bc-98bf-17f0626bd9de" TYPE="swap" PARTUUID="3fb31afd-05"
/dev/sdc1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="273389127680630550" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="5503a0c7-981f-e74d-b5c9-4084eb1df641"
/dev/sdd1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="2196863600760350153" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="97861472-b1ed-954a-831c-119b0d08f45f"
/dev/sde1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="737221115472686340" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="cf5d27f2-22f6-1842-b728-50707ed22be4"
/dev/sdg1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="15202153139351954887" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="20815fac-4447-a04f-8b13-25d0d839ca90"
/dev/sdi1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="10997858781946851491" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="e123a49f-67de-d941-a02e-284ee4dc2670"
/dev/sdh1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="7665980383293405333" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="0cb1d5df-2c3d-db46-a8c5-da8cdeff94b0"
/dev/sdj1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="12174557459640099897" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="a949ef49-83d0-884d-828e-8b7cee1c8545"
/dev/sdf1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="5962312056749847786" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="840ace6e-d262-8e48-a3a5-a50ae7e3472d"
/dev/sdk1: LABEL="Gewuerzglas" UUID="15011586312885837941" UUID_SUB="8554831367157426907" TYPE="zfs_member" PARTLABEL="zfs-a70471c39f62b609" PARTUUID="2ccfee22-e90e-8949-8971-f581e13ff243"
/dev/sdc9: PARTUUID="b7e6e29d-2862-bf4a-8346-ef5566fca891"
/dev/sdd9: PARTUUID="c1ebbcb0-a458-0e47-8f9f-2f9b8bc67165"
/dev/sde9: PARTUUID="2e134ff0-1290-8c40-b50d-10b4fb022c64"
/dev/sdg9: PARTUUID="6cd1a402-0714-5a4a-acf2-9097314401f3"
/dev/sdi9: PARTUUID="739d94f8-8eb2-444c-99ff-65b6fb7e48a7"
/dev/sdh9: PARTUUID="d2f5af40-ba13-7947-a46a-23aa1b00ea8e"
/dev/sdj9: PARTUUID="840de1b5-f24d-9446-9471-bca8d4aff777"
/dev/sdf9: PARTUUID="72b593e7-e390-8540-bba0-15469cd74d96"
/dev/sdk9: PARTUUID="a8ba619d-c59e-be41-a1bc-82c81ffc01b5"

I don't get why it is telling me that these drives are currently unavailable as they seem not to. I can see them and I just did long smart check with all the drives.

  • Hello. Please post the output of fdisk -l and lsscsi -c. – ewwhite Jan 9 '17 at 2:02
  • Actually, I just need lsscsi output – ewwhite Jan 9 '17 at 12:53
  • 4
    Never create a zpool using /dev/sd* names. These can change if hard drives are moved around or replaced, and the pool can fail to be imported. Always use the corresponding names in /dev/disk/by-id/* instead, as these never change. – Michael Hampton Jan 10 '17 at 18:28
  • 1
    Have you tried importing via zpool import -d /dev/disk/by-id? Long shot, but things look pretty bleak here. Also, definitely heed the above comment. – airhuff Feb 1 '17 at 22:27
  • yes I tried that – Terrasar Feb 3 '17 at 6:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.