Hot answers tagged

163

RAID: Why and When RAID stands for Redundant Array of Independent Disks (some are taught "Inexpensive" to indicate that they are "normal" disks; historically there were internally redundant disks which were very expensive; since those are no longer available the acronym has adapted). At the most general level, a RAID is a group of disks that act on the ...


65

ZFS will give you advantages beyond software RAID. The command structure is very thoughtfully laid out, and intuitive. It's also got compression, snapshots, cloning, filesystem send/receive, and cache devices (those fancy new SSD drives) to speed up indexing meta-data. Compression: #zfs set compression=on filesystem/home It supports simple to create copy-...


50

Congratulations and uh-oh. You've stumbled across one of the better things about ZFS, but also committed a configuration sin. First, since you are using raidz1, you only have one disk worth of parity data. However, you had two drives fail contemporaneously. The only possible result here is data loss. No amount of resilvering is going to fix that. Your ...


42

Also RAID ONE MILLION!!!! 128 Disks so reads would be fast, horrible writes but very reliable I'd imagine, oh and you'd get 1/128th the available space, so not great from a budgetary perspective. Don't do this with flash drives, I tried and set fire to the atmosphere...


35

I've built a number of these "all-in-one" ZFS storage setups. Initially inspired by the excellent posts at Ubiquitous Talk, my solution takes a slightly different approach to the hardware design, but yields the result of encapsulated virtualized ZFS storage. To answer your questions: Determining whether this is a wise approach really depends on your goals....


32

I've found XFS more well suited to extremely large filesystems with possibly many large files. I've had a functioning 3.6TB XFS filesystem for over 2 years now with no problems. Definately works better than ext3, etc at that size (especially when dealing with many large files and lots of I/O). What you get with ZFS is device pooling, striping and other ...


27

See this thread from one of the FreeBSD lists: http://lists.freebsd.org/pipermail/freebsd-fs/2009-June/006327.html To quote Dan Naumov, To reiterate, you cant just add a single disk drive to a raidz1 or raidz2 pool. This is a known limitation (you can check with SUN ZFS docs). If you have an existing raidz and you MUST increase that particular ...


27

I was confused by this at the beginning as well. Since the ZFS is referring to 'Storage pools', the author created the nickname 'Tank' as in a 'Tank of water' or a 'Fish tank'. It is a bit of a play on words since the English words 'Pool' and 'Tank' both refer to large containers of water. Some people find it confusing at first. Here is an old example from ...


25

For ZFS, there are number of factors which contribute to the overall cost, performance and your satisfaction with the system you've built. SUPPORTABILITY If you need to be able to call someone when you have problems don't DIY, buy a Sun 7000 Unified Storage appliance. They're a little pricey, but you get what you pay for. High quality hardware, with ...


25

Why, hello there Lars! That's a fascinating question you've asked, and after some research I may have found an answer for you. According to this and other posts out there, it may be possible to set the VFCF_JAIL attribute on the NFS filesystem provider, which would in theory allow jails to perform NFS mounts. This may, in turn, allow one to run amd inside ...


19

Design the right way and you'll minimize the chances of data loss of ZFS. You haven't explained what you're storing on the pool, though. In my applications, it's mostly serving VMWare VMDK's and exporting zvols over iSCSI. 150TB isn't a trivial amount, so I would lean on a professional for scaling advice. I've never lost data with ZFS. I have experienced ...


18

The problem was that the new motherboard's BIOS created a host protected area (HPA) on some of the drives, a small section used by OEMs for system recovery purposes, usually located at the end of the harddrive. ZFS maintains 4 labels with partition meta information and the HPA prevents ZFS from seeing the upper two. Solution: Boot Linux, use hdparm to ...


18

Most of these snapshots are copy-on-write snapshots, which are really fast and really cheap (storage-wise) on rarely-updated systems. LVM snapshots are COW snapshots, ZFS/BTRFS both have a COW-mode for snapshots, reiserfs doesn't have snapshots natively, Novell's NSS file-system is also COW, as are Shadow Copy volumes for Windows NTFS volumes. Copy-on-write ...


17

You may find something like this a little simpler zfs list -t snapshot -o name | grep ^tank@Auto | tac | tail -n +16 | xargs -n 1 zfs destroy -r output the list of snapshot (names only) with zfs list -t snaphot -o name filter to keep only the ones that match tank@Auto with grep ^tank@Auto reverse the list (previously sorted from oldest to newest) with ...


16

using lvm snapshots and xfs on live filesystems is a recipe for disaster especially when using very large filesystems. I've been running exclusively on LVM2 and xfs for the last 6 years on my servers (at home even since zfs-fuse is just plain too slow)... However, I can no longer count the different failure modes I encountered when using snapshots. I've ...


16

Offline drive #1. Replace the old drive with the newer, high-capacity drive. zpool replace the drive. Wait for the replacement to be rebuilt. Goto step 1 and repeat for each successive drive in the array. You may need to export and then import the zpool once this procedure is complete; the pool will then show the increased size. No downtime is required, ...


16

RAID-Z eliminates most of the write penalty and the data integrity issues that RAID 5/6 volumes suffer from, at the cost of some CPU time. Typically, systems have CPU cycles to spare, so spending CPU time to improve IO performance and data integrity is a good compromise vs. mirroring. Here is a detailed explanation of RAID-Z that may answer other questions. ...


16

As of April 2010 Btrfs is still in development and not ready for production use (really, it isn't - I have been bitten experimenting with kernel 2.6.32). Feature-wise btrfs contains some but not all of ZFS features. Most notably RAID-Z and data deduplication are missing. What does work with btrfs is snapshotting, copy-on-write, checksumming and using ...


16

You are thinking about things from the wrong level, is all. If you move a file within the confines of a single ZFS dataset, it will react similarly to what you're expecting. If you move a file within the confines of a pool, but between datasets, it is a real move. Yes, technically the data just went from point A to point B and both points are on the same ...


15

I hope this is gonna help a little. I tried to not let it turn into a full wall of text. :) 3Par/Isilon If you can and will dedicate a fixed amount of man-hours for someone who takes the SAN admin role and wanna enjoy a painless life with night-sleep instead of night-work then this is the way I'd go. A SAN lets you do all the stuff where a single "...


15

FreeNAS is a NAS solution, as such, some technical choices are hidden behind whatever firmware, system or GUI such appliance can use. If you get the partition schema used on a given disk inside a ZFS pool made with FreeNAS (small VM example): $ glabel status Name Status Components gptid/a699226f-bcc4-11e3-952d-...


15

One of the servers that I administrate runs the type of configuration that you describe. It has six 1TB hard drives with a LUKS-encrypted RAIDZ pool on it. I also have two 3TB hard drives in a LUKS-encrypted ZFS mirror that are swapped out every week to be taken off-site. The server has been using this configuration for about three years, and I've never had ...


14

Don't know about the speed, but here is what I believe running ZFS on RAID would means: You loose the benefits of atomic writes because now the RAID controller has the last say on when a write happens to the disk. Which means you rely on the RAID controllers NVRAM. ZFS also may get lied to if the data was written inporperly, ZFS would have to take the ...


14

This has been solved. They key is that deduplicated volumes need to have the dedup flag turned off before deletion. This should be done at the pool level as well as the zvol or filesystem level. Otherwise, the deletion is essentially being deduplicated. The process takes time because the ZFS deduplication table is being referenced. In this case, RAM helps. I ...


14

Yes, ZFS doesn't need hardware raid and is actually better when used without it. It is able to provide partial data protection even with a single device when configured to use ditto blocks but of course won't resist a full disk failure in that case. You can use it in a mirror configuration (two devices or more) or a RAIDZ one (three devices or more) to ...


14

I think you should reconsider your use of FreeNAS. You've had an uncharacteristically. large. number. of issues. with. your. FreeNAS. installation(s). over. the years. Many of these issues were planning and ZFS design problems. It may be time to refactor or rebuild your environment now that you have some knowledge of best or better-practices.


14

I use ZFS on Linux as a volume manager and a means to provide additional protections and functionality to traditional filesystems. This includes bringing block-level snapshots, replication, deduplication, compression and advanced caching to the XFS or ext4 filesystems. See: https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ for another ...


14

Don't go down the road of breaking the ZFS array to "rotate" disks offsite. As you've seen, the rebuild time is high and the resilvering process will read/verify the used size of the dataset. If you have the ability, snapshots and sending data to a remote system is a clean, non-intrusive approach. I suppose you could go through the process of having a ...


13

I build a home FreeBSD file server using ZFS. It is an AMD X2 3200+ with 3GB of RAM. It has a PCI Express Gig-E. The boot drive is an old 400GB and I have 4 750GB Seagte drives (one with a difference firmware version, just in case). Booting from ZFS would have been nice (it would make install simpler), but I used the ZFSOnRoot instructions to setup the ...



Only top voted, non community-wiki answers of a minimum length are eligible