Questions tagged [zfs]

ZFS is a modern file system and volume manager originally developed by Sun Microsystems and licensed under the CDDL. It is a copy-on-write file system with support for large storage arrays, protection against corruption, snapshots, clones, compression, deduplication and NFSv4 ACLs. An open-source fork of ZFS can be found at http://open-zfs.org/ , which is supported by ZFSonlinux.org, illumos.org and ZFS developers in the FreeBSD & Mac OS X communities.

Filter by
Sorted by
Tagged with
190 votes
2 answers
48k views

What are the different widely used RAID levels and when should I consider them?

This is a Canonical Question about RAID levels. What are: the RAID levels typically used (including the RAID-Z family)? deployments are they commonly found in? benefits and pitfalls of each?
MDMarra's user avatar
  • 101k
13 votes
6 answers
45k views

ZFS SAS/SATA controller recommendations

I've been working with OpenSolaris and ZFS for 6 months, primarily on a Sun Fire x4540 and standard Dell and HP hardware. One downside to standard Perc and HP Smart Array controllers is that they do ...
ewwhite's user avatar
  • 198k
23 votes
1 answer
22k views

Hosting a ZFS server as a virtual guest

I'm still new to ZFS. I've been using Nexenta but I'm thinking of switching to OpenIndiana or Solaris 11 Express. Right now, I'm at a point of considering virtualizing the ZFS server as a guest within ...
osij2is's user avatar
  • 3,885
39 votes
7 answers
65k views

ZFS best practices with hardware RAID

If one happens to have some server-grade hardware at ones disposal, is it ever advisable to run ZFS on top of a hardware-based RAID1 or some such? Should one turn off the hardware-based RAID, and run ...
cnst's user avatar
  • 14.3k
17 votes
7 answers
71k views

Disabling RAID feature on HP Smart Array P400

I'm planning to use ZFS on my system (HP ML370 G5, Smart Array P400, 8 SAS disk). I want ZFS to manage all disks individually, so it can utilize better scheduling (i.e. I want to use software RAID ...
Arie K's user avatar
  • 1,633
2 votes
1 answer
2k views

My OpenSolaris Server hangs when writing large files after upgrading zpool

yesterday I added new harddrives(four as a raidz1 and one as hot-spare) to a opensolaris server, after extending the zpool the server hangs when writing large files but not when reading large files(...
user48937's user avatar
11 votes
2 answers
4k views

ZFS - destroying deduplicated zvol or data set stalls the server. How to recover?

I'm using Nexentastor on a secondary storage server running on an HP ProLiant DL180 G6 with 12 Midline (7200 RPM) SAS drives. The system has an E5620 CPU and 8GB RAM. There is no ZIL or L2ARC device. ...
ewwhite's user avatar
  • 198k
30 votes
2 answers
58k views

Transparent compression filesystem in conjunction with ext4

I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features. Are there any production/stable solutions out there ...
user235918's user avatar
25 votes
2 answers
8k views

ZFS Data Loss Scenarios

I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
Cyclone's user avatar
  • 557
23 votes
2 answers
48k views

How can I add one disk to an existing raidz zpool?

I have an OpenSolaris server with a zpool backupz comprised of four SCSI drives: -bash-3.2# zpool status backupz pool: backupz state: ONLINE scrub: none requested config: NAME ...
Josh's user avatar
  • 9,208
12 votes
2 answers
20k views

Need to replace disk in zpool ... confused

I need to replace a bad disk in a zpool on FreeNAS. zpool status shows pool: raid-5x3 state: ONLINE scrub: scrub completed after 15h52m with 0 errors on Sun Mar 30 13:52:46 2014 config: ...
d0g's user avatar
  • 969
12 votes
3 answers
6k views

ZFS: How do you restore the correct number of copies after losing a drive?

With zfs, if you have copies=2 and then you lose a drive containing some of those copies, how do you tell the system that it should make a new copy of the data blocks for the affected files? Or does ...
James Moore's user avatar
11 votes
5 answers
12k views

Best filesystem choices for NFS storing VMware disk images

Currently we use an iSCSI SAN as storage for several VMware ESXi servers. I am investigating the use of an NFS target on a Linux server for additional virtual machines. I am also open to the idea of ...
mlambie's user avatar
  • 1,221
35 votes
3 answers
46k views

How to perform incremental / continuous backups of zfs pool?

How can zfs pools be continuously/incrementally backed up offsite? I recognise the send/receive over ssh is one method however that involves having to manage snapshots manually. There are some tools ...
Greg's user avatar
  • 1,677
17 votes
3 answers
26k views

ZFS: Mirror vs. RAID-Z

I'm planning on building a file server using OpenSolaris and ZFS that will provide two primary services - be an iSCSI target for XenServer virtual machines & be a general home file server. The ...
John Clayton's user avatar
15 votes
1 answer
3k views

Why did rebooting cause one side of my ZFS mirror to become UNAVAIL?

I just recently migrated a bulk data storage pool (ZFS On Linux 0.6.2, Debian Wheezy) from a single-device vdev configuration to a two-way mirror vdev configuration. The previous pool configuration ...
user's user avatar
  • 4,445
13 votes
1 answer
20k views

Is calculating IOPS for ZFS RAIDZ different then calculating IOPS for RAID5 & RAID6?

When calculating IOPS for traditional RAID arrays, one could use the following formula (Borrowed from Getting The Hang Of IOPS v1.3 on Symantec Connect): Ieffective = (n * Isingle) / (READ% + (F * ...
Stefan Lasiewski's user avatar
10 votes
3 answers
8k views

ZFS - Impact of L2ARC cache device failure (Nexenta)

I have an HP ProLiant DL380 G7 server running as a NexentaStor storage unit. The server has 36GB RAM, 2 LSI 9211-8i SAS controllers (no SAS expanders), 2 SAS system drives, 12 SAS data drives, a hot-...
ewwhite's user avatar
  • 198k
30 votes
1 answer
8k views

ZFS endless resilvering

I have a large (> 100TB) ZFS (FUSE) pool on Debian that lost two drives. As the drives failed, I replaced them with spares until I could schedule an outage and physically replace the bad disks. When ...
jasongullickson's user avatar
29 votes
2 answers
28k views

ZFS performance: do I need to keep free space in a pool or a file system?

I know that the performance of ZFS heavily depends on the amount of free space: Keep pool space under 80% utilization to maintain pool performance. Currently, pool performance can degrade when a pool ...
Pavel's user avatar
  • 1,058
21 votes
3 answers
15k views

Non-ECC memory with ZFS: a stupid idea?

I have a new server and am planning to upgrade the paltry 2 GB of memory to the maximum of 16 GB. (Theoretically 8 GB is the limit, but empirically 16 GB has been shown to work.) Some guides advise ...
iconoclast's user avatar
  • 1,830
10 votes
1 answer
21k views

How do I expand the root's volume size?

I have Ubuntu Server 18.04 LTS running off a 16GB SanDisk USB pendrive in my server. From what I can remember, when I installed Ubuntu on there I had enabled LVM support. For some reason, when I ssh ...
leetbacoon's user avatar
8 votes
6 answers
23k views

MegaRAID JBOD substitute

It seems that certain MegaRAID chips don't support JBOD (unfortunately). I've read online that to make these controllers act as a pass-through, I should configure the disks in RAID0. What RAID0 ...
cedivad's user avatar
  • 690
2 votes
3 answers
4k views

Solaris: detect hotswap SATA disk insert

What's the method used on Solaris to get the system to rescan for new disks that have been hot-plugged on a SATA controller? I've got an HP X1600 NAS which had 9 drives configred in a ZFS pool. I've ...
growse's user avatar
  • 8,050
76 votes
13 answers
72k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
Tamas Czinege's user avatar
46 votes
11 answers
76k views

How to delete all but last [n] ZFS snapshots?

I'm currently snapshotting my ZFS-based NAS nightly and weekly, a process that has saved my ass a few times. However, while the creation of the snapshot is automatic (from cron), the deletion of old ...
growse's user avatar
  • 8,050
39 votes
2 answers
48k views

Clear a permanent ZFS error in a healthy pool

I scrubbed my pool today, and after the scrub finished, I noticed there was an error that corrupted a file. I didn't care about the file, so I deleted it. Unfortunately, the error remains (now ...
Kevin Wood's user avatar
19 votes
2 answers
6k views

What does a permanent ZFS error indicate?

Several permanent errors were reported on my zpool today. pool: seagate3tb state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications ...
Will Haley's user avatar
17 votes
2 answers
17k views

ZFS checksum errors, when do I replace the drive?

I'm fairly new to ZFS and I have a simple mirrored storage pool setup with 8 drives. After a few weeks of running, one drive seemed to generate a lot of errors, so I replaced it. A few more weeks go ...
Dominic P's user avatar
  • 497
13 votes
4 answers
13k views

ZFS: Removing files from snapshots?

Say I have a dataset with 100 snapshots and want to rm -rf all folders named "cache" in all snapshots. I want to actually free up the space, not just hide it away in layers of snapshots, so making a ...
Tino Didriksen's user avatar
13 votes
2 answers
65k views

FreeNAS, Do I need 1GB per TB of usable storage, or 1GB of memory per TB of physical disc?

The title says it all. When setting up free NAS, do I need 1GB per TB of usable storage, or 1GB of memory per TB of physical disc?
Litzner's user avatar
  • 415
12 votes
1 answer
13k views

ZFS on Linux (ZoL) and realtime replication

I would like to understand what is the best solution for realtime replication between two ZFS on Linux (ZoL) boxes connected by a 10 GbE link. The goal is to use them for virtual machines; only one ...
shodanshok's user avatar
  • 50.5k
11 votes
4 answers
9k views

Can I trust that ZFS is consistent between Linux and FreeBSD?

I'm planning to build a FreeNAS box sometime soon, but if ZFS on Linux eventually proves to be reliable, I might want to switch, just to have a more familiar OS. So I'm wondering if I can trust that ...
iconoclast's user avatar
  • 1,830
11 votes
5 answers
18k views

ZFS Sync over unreliable, slow WAN. ZFS replication, or rsync?

I've been tasked with making an off-site backup work over the WAN. Both storage boxes are FreeBSD based NAS boxes running ZFS. Once or twice a week, 15-60 gigs of photography data gets dumped to the ...
Paul McMillan's user avatar
10 votes
4 answers
20k views

zpool status reports error ... what next?

On our FreeNAS server, zpool status gives me: pool: raid2 state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. ...
d0g's user avatar
  • 969
9 votes
3 answers
6k views

Dell R730xd - Best way to configure storage for simultaneous use with ZFS & Hardware RAID

I recently purchased a Dell R730xd that came with an H730 RAID card. I would like to bring an existing 6 disk ZFS pool to this server as well as create an additional 6 disk hardware RAID. Is it ...
TJ Zimmerman's user avatar
8 votes
3 answers
7k views

PAM: Execute a command before pam_mkhomedir?

When a user logs into a system and there is no home directory, we would like to to create the user's home directory using pam_mkhomedir. This is a common practice. Here's a quick description of ...
Stefan Lasiewski's user avatar
8 votes
2 answers
11k views

What's in my ZFS ARC and L2ARC caches?

I have a simple ZFS setup at home, four disks mirrored, 8GB ram and 120GB Intel X25-M SSD for L2ARC. I ran zpool add poolname cache cXtXdX then generated load and eagerly tracked the warming process ...
notpeter's user avatar
  • 3,525
7 votes
1 answer
5k views

FIO Benchmarking - Inconsistent and Slower than Anticipated: Are my RAIDs Misconfigured?

TL;DR: I'm having some performance issues with my hypervisor storage. here are a bunch of test results from fio. Skip to the Results section to read about them and see my questions. Summary I ...
TJ Zimmerman's user avatar
6 votes
3 answers
13k views

ZFS on enterprise RAID pass-through, and ZFS on FreeBSD root

We've been running ZFS on top of a single hardware RAID on dell poweredge for years. I know most people are against this, but the snapshot/clone, compression and flexible partition of ZFS served us ...
John's user avatar
  • 369
6 votes
1 answer
4k views

Does ZFS replace the need for hardware/software RAID?

I want to provide protection against data loss on my servers. Typically, I'd use hardware RAID 1 or 5, but I've been reading up on ZFS. Is it correct that ZFS itself provides RAID 1 or 5 like data ...
user53744's user avatar
  • 163
6 votes
1 answer
9k views

ZFS on Linux/Ubuntu: Help importing a zpool after Ubuntu upgrade from 13.04 to 13.10, device IDs have changed

I have updated Ubuntu from version 13.04 to version 13.10, only to discover that the SATA disks IDs have changed, and my ZFS pool now fails. On new Ubuntu 13.10 the disk IDs are now ata-* instead of ...
Manolo's user avatar
  • 63
5 votes
4 answers
19k views

ZFS - how to partition SSD for ZIL or L2ARC use?

I'm working with a Sun x4540 unit with two pools and newly-installed ZIL (OCZ Vertex 2 Pro) and L2ARC (Intel X25-M) devices. Since I need to keep these two pools in the near-term, I'd like to know how ...
ewwhite's user avatar
  • 198k
4 votes
1 answer
4k views

ZFS (NexentaStor) and 4k Advanced Format Partition Alignment

I have a storage system that contains 8 x 1TB drives that use the 4k sector size "Advanced Format". I'm planning to run NexentaStor on this hardware and want to ensure that I'm taking the 4k sector ...
ewwhite's user avatar
  • 198k
4 votes
1 answer
3k views

zfs send -i / receive stalling

On a Solaris 11.1 install, I see stalls when receiving a zfs incremental stream. The streams are originating from a Solaris 11.0 install, are created using zfs send -i and piped through mbuffer. At ...
the-wabbit's user avatar
3 votes
2 answers
9k views

FreeNAS confusion: not sure what happened. raidz1 "DEGRADED", disk "FAULTED"

Sorry for the long narrative, but I'm thoroughly confused. I'm using FreeNAS-8.0.4-RELEASE-p2-x64 (11367) on a box with 5x3TB SATA disks configured as a raidz volume. A few days ago, the console ...
d0g's user avatar
  • 969
3 votes
2 answers
4k views

Increase space of ZFS pool over iSCSI SAN, online

I have an iSCSI volume of 96GB on SAN, which has ZFS pool implemented on it. i have resized the size of SAN volume from 96 to 120GB, but the zpool list only shows the old space (96GB). how to achieve ...
Farhan's user avatar
  • 4,309
1 vote
2 answers
1k views

Debug Nexenta/OpenSolaris core files?

I have a Sun x4540 that was recently converted to NexentaStor Enterprise Edition from an earlier OpenSolaris. The system disks were wiped and the zpools were exported and reimported. I'm hosting ...
ewwhite's user avatar
  • 198k
48 votes
6 answers
13k views

ZFS on FreeBSD: recovery from data corruption

I have several TBs of very valuable personal data in a zpool which I can not access due to data corruption. The pool was originally set up back in 2009 or so on a FreeBSD 7.2 system running inside a ...
ssc's user avatar
  • 1,189
39 votes
9 answers
39k views

Is bit rot on hard drives a real problem? What can be done about it?

A friend is talking with me about the problem of bit rot - bits on drives randomly flipping, corrupting data. Incredibly rare, but with enough time it could be a problem, and it's impossible to detect....
scobi's user avatar
  • 879