Questions tagged [zfsonlinux]
zfsonlinux provides a native Linux kernel port of the ZFS filesystem so that ZFS may run natively on Linux. It is very different from ZFS-fuse which provides ZFS on top of a userspace-filesystem layer. The ZFS On Linux project is hosted on zfsonlinux.org.
276
questions
39
votes
2
answers
48k
views
Clear a permanent ZFS error in a healthy pool
I scrubbed my pool today, and after the scrub finished, I noticed there was an error that corrupted a file. I didn't care about the file, so I deleted it. Unfortunately, the error remains (now ...
35
votes
3
answers
46k
views
How to perform incremental / continuous backups of zfs pool?
How can zfs pools be continuously/incrementally backed up offsite?
I recognise the send/receive over ssh is one method however that involves having to manage snapshots manually.
There are some tools ...
27
votes
4
answers
5k
views
If a RAID5 system experiences a URE during rebuild, is all the data lost?
I understand the argument regarding larger drives' increased likelihood of experiencing a URE during a rebuild, however I'm not sure what the actual implications are for this. This answer says that ...
24
votes
7
answers
56k
views
Why isn't the arc_max setting honoured on ZFS on Linux?
I'm running ZoL 0.6.2 from their PPA on Ubuntu 12.04. It's on a host with 16GB of memory intended to run some VMs using KVM/Libvirt. After some time ZoL is using an insane amount of memory, reaching ...
22
votes
4
answers
44k
views
ZFS Datasets dissappear on reboot
I have installed ZFS(0.6.5) in my Centos 7 and I have also created a zpool, everything works fine apart from the fact that my datasets disappear on reboot.
I have been trying to debug this issue with ...
21
votes
7
answers
8k
views
Which filesystem for large LVM of disks (8 TB)?
I have a Linux server with many 2 TB disks, all currently in a LVM resulting in about 10 TB of space. I use all this space on an ext4 partition, and currently have about 8,8 TB of data.
Problem is, I ...
18
votes
1
answer
17k
views
Why does `zfs list` and `zpool list` report very different sizes for my raidz2 pools?
I have a ZFS server with 8 zpools. Each pool is 12 6TB disks in a 10+2 RAIDZ configuration. So, each pool has a raw storage space of 12*6=72 TB and usable space of 10*6=60 TB. However, I see different ...
18
votes
0
answers
11k
views
Restoring data after zfs destroy
On Debian 8.7 I had a zfs pool.
(obviously using ZFS on Linux, not Oracle or Solaris zfs)
It was needed to extend ZFS pool from mirror on 2 disks to raidz on 4 disks.
I did backup (one copy of data - ...
17
votes
2
answers
17k
views
ZFS checksum errors, when do I replace the drive?
I'm fairly new to ZFS and I have a simple mirrored storage pool setup with 8 drives. After a few weeks of running, one drive seemed to generate a lot of errors, so I replaced it.
A few more weeks go ...
16
votes
2
answers
32k
views
Why isn't my ZFS pool expanding using ZFS on Linux?
I have a ZFS zpool on linux under kernel 2.6.32-431.11.2.el6.x86_64 which has a single vdev. The vdev is a SAN device. I expanded the size of the SAN, and despite the zpool having autoexpand set to on,...
16
votes
3
answers
14k
views
How to 're-balance' data in zfs? (Make sure the data is spread amongst all striped mirrors)
Using a striped mirror (Raid 10) as an example, if two disks are expanded to four, how can the data from the two disks be 're-balanced' (spread out) amongst the two mirrors?
That is, how can the ...
15
votes
1
answer
3k
views
Why did rebooting cause one side of my ZFS mirror to become UNAVAIL?
I just recently migrated a bulk data storage pool (ZFS On Linux 0.6.2, Debian Wheezy) from a single-device vdev configuration to a two-way mirror vdev configuration.
The previous pool configuration ...
13
votes
5
answers
10k
views
Encryption with ZFS on linux
Does ZFS on Linux already support Encryption? If not, is it planned?
I found tons of info for ZFS+LUKS but that's absolutely uninteresting: I want ZFS encryption so that I can do replication using ...
13
votes
1
answer
2k
views
Force update checksum on zfs?
I recently changed the checksum property on one of my non-duplicated zfs filesystems to sha256 from on (fletcher4) to better support the sending of duplicated replication steams, as in this command ...
12
votes
2
answers
7k
views
Why is ZFS so much slower than ext4 and btrfs?
Problem
I recently installed a new disk and created a zpool on it:
/# zpool create morez /dev/sdb
After using it for a while, I noticed it was quite slow:
/morez# fio --name rw --rw rw --size 10G
...
12
votes
3
answers
6k
views
Why is ZFS on Linux unable to fully utilize 8x SSDs on AWS i2.8xlarge instance?
I'm completely new to ZFS, so to start with I thought I'd do some simple benchmarks on it to get a feel for how it behaves. I wanted to push the limits of its performance so I provisioned an Amazon ...
12
votes
1
answer
13k
views
ZFS on Linux (ZoL) and realtime replication
I would like to understand what is the best solution for realtime replication between two ZFS on Linux (ZoL) boxes connected by a 10 GbE link. The goal is to use them for virtual machines; only one ...
10
votes
1
answer
2k
views
Slow sequential speeds on 9x7-drive raidz2 (ZFS ZoL 0.8.1)
I'm running a large ZFS pool built for 256K+ request size sequential reads and writes via iSCSI (for backups) on Ubuntu 18.04. Given the need for high throughput and space efficiency, and less need ...
10
votes
1
answer
3k
views
Can ZFS using snaphots replace DRBD using sync protocol A?
This question is related to a former, more generic one, but this time I would like to ask more specifically if ZFS can replace DRBD in a use case like mine. That is two servers, both hosting their own ...
9
votes
2
answers
6k
views
Wait before zpool import on Ubuntu boot
I have installed Ubuntu 18.04 on a Dell PowerEdge R720xd using ZFS. There are two 1TB boot drives in a ZFS mirror configuration. I followed the instructions in the ZFS on Linux Wiki.
(Note: My system ...
9
votes
2
answers
6k
views
How to change the drive reference in a zfs pool from /dev/sdX to /dev/disk/by-id (ata-XXXXXX)
In the example below, how can the devices in the second mirror be changed so they they reference the /dev/disk/by-id instead of /dev/sdX?
user@pc:~$ sudo zpool status
pool: storage
state: ONLINE
...
9
votes
1
answer
918
views
Using ZFS head node as database server?
I'm using a dual-head ZFS-backed NAS for high availability cluster shared storage, based on Nexenta's recommended architecture as seen here:
The disks in 1 JBOD will store the database files for a ...
8
votes
5
answers
2k
views
Where is the central ZFS website now?
Oracle dumped OpenSolaris in Fall 2010, and it is unclear if Oracle will continue to publicly release updates to ZFS, except maybe after they release their next major version of Solaris.
FreeBSD now ...
8
votes
2
answers
5k
views
Docker service starts before ZFS
I am using Docker CE on Ubuntu 16.04, with ZFS as storage for Docker. Setup is pretty much standard:
There's a zpool that has multiple disks
On the zpool, there's a zfs filesystem with mountpoint /...
8
votes
5
answers
5k
views
zfs - two pools with nested mountpoints and conflicting mount order
I have a problem with automounting separate pools, that have nested mountpoints. I am using ZfsOnLinux 0.6.2.
The siuation currently is as follows:
zpool1 ---> /var
zpool1/log ---> /var/...
8
votes
4
answers
21k
views
How to set the number of snapshots zfs-auto-snapshot should retain?
Using the below zfs-auto-snapshot commands, how to specify to keep 52 weekly snapshots and infinite number of monthly snapshots?
sudo zfs set com.sun:auto-snapshot=true storage
sudo zfs set com.sun:...
8
votes
3
answers
3k
views
Upgraded Ubuntu, all drives in one zpool marked unavailable
I just upgraded Ubuntu 14.04, and I had two ZFS pools on the server. There was some minor issue with me fighting with the ZFS driver and the kernel version, but that's worked out now. One pool came ...
8
votes
1
answer
7k
views
Linux ZFS does not buffer writes to write buffer (SLOG/ZIL)?
When experiencing write I/O, the log column in zpool iostat -v does not show any ZIL activity, ever. This results in higher than expected wait times when writing data to disk (sometimes over 80ms ...
8
votes
2
answers
7k
views
Why is `zfs list -t snapshot` orders of magnitude slower than `ls .zfs/snapshot`?
With all ZFS-on-Linux versions I've ever tried, using zfs list to list all snapshots of a filesystem or volum (zfs list -r -t snapshot -H -o name pool/filesystem) always takes many orders of magnitude ...
8
votes
1
answer
3k
views
zfs - hotspare, replace, detach: ressource is busy
I'm pretty new to zfsonlinux. I've just succeeded in setting up a brand new server, with a Debian ROOT on ZFS. All is working fine, but I've got an issue with hot spare and replacing disks.
Here is ...
8
votes
1
answer
4k
views
Extreme ZFS slowdowns after several months
I've got a general-purpose server, providing mail, DNS, web, databases, and some other services for a number of users.
It's got a Xeon E3-1275 at 3.40 GHz, 16 GB ECC RAM. Running Linux kernel 4.2.3, ...
8
votes
2
answers
735
views
Why is ZFS not doing anything with my disk's duff sector?
I was under the impression that if an I/O error occurs during a read
from a ZFS pool, two things will happen:
The failure will be recorded in either the READ or CKSUM statistic of the relevant device,...
7
votes
1
answer
8k
views
How to add more physical disks to a mirror vdev (to increase redundancy)?
Can more physical drives be added to mirror vdevs after a zpool has been created?
For example, if a zpool is created using:
zpool create test_pool mirror /dev/sd0 /dev/sd1 mirror /dev/sd2 /dev/sd3
...
7
votes
2
answers
3k
views
Create zfs pool that allows replacing one of the disks with a slightly smaller disk
This is a question regarding zfs on Linux (CentOS 7). I have a very simple setup with two 8 TB disks, one disk mirroring the other.
zpool create -f -o ashift=12 $zpoolName mirror $disksById
In case ...
7
votes
1
answer
2k
views
How can I remove dedup from my pool without running out of RAM?
I have a server with 8 disk bays filled with 3 TB disks. Using 4 mirrored vdevs of 2 disks each, this gives me 12 TB of redundant storage.
Here's the issue - I read somewhere that I needed "x GBs of ...
7
votes
1
answer
5k
views
ZFS Heavy Write Amplification due to Free Space Fragmentation
I have setup ZFS RAID0 for PostgreSQL database. The Storage and Instances are in AWS EC2 and EBS volumes.
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
pgpool 479G ...
7
votes
1
answer
3k
views
Is the size reported by `zfs send` exact or estimated?
Consider the following command and the output:
zfs send -Pvi \
tank/vms/langara@zfsnap-2016-05-11_00.00.00--1w \
tank/vms/langara@zfsnap-2016-05-13_00.00.00--1w \
| dd > /dev/null
...
7
votes
0
answers
1k
views
Recovering/importing ZFS Linux pool with duplicate pool name
I did something very stupid today in attempting to add a third mirror to an existing Linux zpool named backup. Suffice it to say that I made a couple errors because I don't really do much admin with ...
6
votes
3
answers
2k
views
TRIM/UNMAP Zvol over iSCSI
I am currently setting up a SAN for diskless boot. My backend consists of ZFS-Vol shared via iSCSI. So far everything is working just fine except for TRIM/UNMAP. For test puposes I setup two VMs ...
6
votes
2
answers
14k
views
How to migrate a regular LXC container to a Proxmox LXC container?
How do you migrate a regular LXC container (from another or the same server) to a Proxmox LXC container, which can be accessed via the Proxmox web GUI?
For example, I have a container created by ...
6
votes
4
answers
9k
views
ZFS on linux upgrade from 0.6.2 to 0.6.3 made my zpool unreadable help translate CentOS to Ubuntu commands
After upgrading my ubuntu 13.10 server using sudo apt-get upgrade and rebooting, my zpool wouldn't show.
messier@ANDERSON:~$ sudo zpool status
no pools available
messier@ANDERSON:~$ sudo zpool import
...
6
votes
1
answer
6k
views
How to mount a ZFS dataset with given uid/gid?
I feel like I am missing something very simple / basic here, but I wasn't able to figure out how to mount a ZFS dataset such that the mount point (root directory) has a given uid/gid ownership? ...
6
votes
2
answers
924
views
At my wits end. What could cause my server to randomly hard reset? (Seems to be related to ZFS)
I have a server that I built years ago that has worked like a champ. But within the past few months it has started to become seriously unstable with no discernible pattern. I have been debugging it ...
6
votes
3
answers
8k
views
ZFS - Recover or repair a corrupted file in a snapshot from backup?
A pool has suffered permanent data corruption to file data that's part of a snapshot. If the file data was part of the filesystem (and not part of any snapshot), I could simply recover the file from a ...
6
votes
1
answer
9k
views
ZFS on Linux/Ubuntu: Help importing a zpool after Ubuntu upgrade from 13.04 to 13.10, device IDs have changed
I have updated Ubuntu from version 13.04 to version 13.10, only to discover that the SATA disks IDs have changed, and my ZFS pool now fails.
On new Ubuntu 13.10 the disk IDs are now ata-* instead of ...
6
votes
1
answer
319
views
Does zfs send scrub the sent data?
To my knowledge ZFS as a filesystem checks for data errors during each access to the data.
I would therefore assume that it actually performs something like a scrub before sending the data thereby ...
6
votes
1
answer
554
views
ZFS keeps faulting the same device
Our storage server has some problems, just ago we had a broken disk (WD 4TB RE SAS), though the raidcontroller (LSI MegaRAID 9271-8i) kept the disk online (status : ok). Only Media error showed 1 ...
6
votes
1
answer
1k
views
How does quota usage for ZFS on Linux work?
I have a user who according to ZFS is at their quota. However, I cannot find how they are using all of their space. This file system has no descendant file systems and according to the man page the ...
6
votes
0
answers
7k
views
What is the easiest way to encrypt an existing ZFS dataset?
I would like to convert a lot of ZFS datasets I have on a (Ubuntu 20.04) server (about 3 TB of data). I've already tested this a bit, and what I've been doing so far is:
Create a new encrypted ...
5
votes
5
answers
8k
views
MongoDB and ZFS bad performance: disk always busy with reads while doing only writes
I have huge performance issues using MongoDB (i believe it is mmapped DB) with ZFSonlinux.
Our Mongodb is almost only writes. On replicas without ZFS, disk is completely busy for ~5s spikes, when app ...