Tag Info

Hot answers tagged

26

I've built a number of these "all-in-one" ZFS storage setups. Initially inspired by the excellent posts at Ubiquitous Talk, my solution takes a slightly different approach to the hardware design, but yields the result of encapsulated virtualized ZFS storage. To answer your questions: Determining whether this is a wise approach really depends on your ...


15

The conversion of a SCSI chassis DL380 G4 to the SAS chassis is possible, but not at all practical. It requires a bezel change, backplane modification, a new drive cage, a Smart Array P600 RAID controller (or SAS/SATA HBA) and will be severely crippled in performance. In addition, it can only accommodate 2.5" small-form-factor 1st generation SAS and SATA ...


13

This has been solved. They key is that deduplicated volumes need to have the dedup flag turned off before deletion. This should be done at the pool level as well as the zvol or filesystem level. Otherwise, the deletion is essentially being deduplicated. The process takes time because the ZFS deduplication table is being referenced. In this case, RAM helps. I ...


10

Certainly ZFS is plenty stable enough to do this kind of thing, there are many very large high-profile and reliable production platforms out there based entirely on ZFS and Nexenta. That said always like to have on-site disk-based backups such as the one you're suggesting AND removable-disk or tape based backups that go off-site daily to protect against ...


10

(assuming you're referring to using dedupe within ZFS versus your backup software) I would not recommend using ZFS native deduplication for your backup system unless you design your storage system specifically for it. Using dedupe in ZFS is extremely RAM intensive. Since the deduplication occurs in real-time as data is streamed/written to the storage ...


10

ZFS does not do disk I/O, device drivers below ZFS do disk I/O. If the device does not respond in a timely manner, or as in this case, disrupts all other devices on the expander, then it is not visible as a failure to ZFS. All ZFS sees is a slow I/O. There is a bug in Intel X-25M firmware that affects their behaviour during heavy loads and can cause reset ...


10

My current recommendation is NexentaStor, available in a free community-supported edition and as a commercial offering. Also see: Anybody have experience with using Nexenta? NexentaStor CE or Openfiler? Which do you recommend?


10

FreeBSD 8.2, running ZFS. ZFS includes the following out of the box: Supports NFS & iSCSI out of the box. ZFS includes Snapshots, data checksums, multiple copies, filesystem compression RAID-Z - Similar to RAID-5, but without the RAID-5 write hole. All disk writes are atomic copy-on-write transactions, so the on-disk state is never inconsistent (No ...


7

If you're not running snapshots then restoring from backup is your only option. I would advise you to look into snapshotting, as it's extremely useful on fileservers. Users are dumb, and they overwrite/delete files way more often than you can run a backup. Edit: As mentioned by ErikA - providing snapshots on a file server also gives users a easy way of ...


7

A quick note about Openfiler (and I hear NexentaStor is the same) when used as an iSCSI target - you are almost guaranteed to see timeout errors and targets dropping offline, requiring a reboot of the server to correct. This usually happens under heavy load (though I've seen it happen under light loads, too). We went through hell with Openfiler using iSCSI ...


6

If you have some monitoring system like Nagios in place, you easily could write a check evaluating the output of zpool list and checking it against thresholds within your comfort zone. If you don't have a monitoring system, you should use this opportunity to install one - a SAN is a critical piece of infrastructure equipment which needs constant monitoring ...


6

On the Nexenta side, there's a volume-check script that's setup to run hourly by default. It will: Check volume health and capacity, clear correctable device errors, validate mountpoints. It also sends a weekly summary report via email. However, there are some things you should consider when planning a Nexenta storage solution for the purposes you've ...


5

Please see the notes about configuring an all-in-one ZFS setup in my post at: Hosting a ZFS server as a virtual guest. If you're talking about creating an SSD pool or adding drives as raw device-mappings (RDM's), disregard the rest of this answer. The preference there would be to run through the HBA versus RDM. Use SAS expanders if needed. The main reasons ...


5

There seems to be more momentum behind NexentaStor. You haven't provided much detail on the hardware arrangement other than it being old. What are the CPU/RAM numbers? However, one reason I'd go the NexentaStor route is the presence of inline compression of its storage volumes. Your setup probably isn't suitable for the deduplication features, but the ...


5

As a long time user of Sun/Oracle ZFS 7000-series appliances, I can tell you without question dedupe isn't polished. Never confuse sales with delivery! The salesguys will tell you "Oh, it's been fixed". In real life - my real life - I can tell you 24GB isn't enough to handle the "DDT tables". That is, the back end index which stores the dedupe table. That ...


5

You can make the expert_mode option persistent here. Run option expert_mode = 1 -s. The -s makes the setting permanent. From now on, you'd just need to run !bash to obtain a shell from the nmc console. You can also ssh as the admin user and use su when needed. However, NexentaStor is really meant to be an appliance managed either by the web interface or ...


5

As mentioned, you cannot "upgrade" the OS portion of the system. You can however, export your existing data zpool (tank), install Solaris on the OS disks and re-import them without any trouble. Things that may not survive are iSCSI configurations, but if you're just using it as an NFS NAS, that configuration information is contained in the zfs filesystems. ...


5

Is it strongly recommended to avoid using RDM's for this purpose. Pass your SATA controller through to the virtualized NexentaStor instance using "PCI Passthrough". This requires a reasonably-current processor to do so. See: Hosting a ZFS server as a virtual guest That said, there's no major issue running ZFS on VMDK's if you have an underlying hardware ...


5

ZFS scrub operations operate on some fairly brain-dead principles. Most notably, it only spends time scrubbing when there's nothing else going on. If you poke a pool with just a bit of data access on a fairly constant basis, scrub will effectively starve itself and do nearly nothing. Tunables to explore, with my quick notes on what it does (I last looked ...


4

8Gb Fibre Channel MPIO Enabling compression on your ZFS zvols/LUNs. Who is consuming this data? Are you actually seeing bottlenecks? And remember, it's not always about throughput. Have you taken time to understand your I/O patterns? The mix of read to write? The nature of the transfers? Can you give more information about the setup?


4

I'd suggest dedicated drives for OS and dedicated disks for data. NexentaStor is software RAID and the loss of an OS disk can be painful. You can combine L2ARC and ZIL, but the class of SSDs you're probably using won't be helpful as ZIL devices. I understand the desire to try to use all of these ZFS features, but you really have to do it the right way. I ...


4

9 times out of 10, this is not NexentaStor at all, except that the default port, 2000, is one claimed by Cisco. Login to the appliance using SSH as root, and type "setup appliance init" and then hit No to the first question then it asks for HTTP/HTTPS and the port to use - you can use HTTP or HTTPS, but change the port to something else, I use 7878. Then try ...


4

We were recently in the same situation, and no, unfortunately FreeNAS is not ready for the enterprise. With 8.0.1, we had tons of issues with Active Directory integration working intermittently, as well as periodic lockups. All you need to look at are the change logs for recent beta releases to see that there are still major bugs being addressed in the 8.0 ...


4

Let's start by saying that since this is for a home lab it really doesn't matter one way or another unless you have some business need to go with one solution over another. By business need I mean to gain experience with a particular application in order to put it on your resume. I personally do not like NexentaStor because it doesn't provide an UI based ...


3

This sounds a lot like a similar issue I experienced on my Nexenta installation. I had recently done a ZFS version upgrade across both the syspool and my other pools, but failed to re-install grub afterwards. The result was that the next power outage left grub unable to read the ZFS filesystem and the system was unbootable. There was a discussion over at ...


3

As other answers alluded to, the ideal approach is LUN-per-VDI. At first it didn't look like it was possible to do this, but there is an undocumented "iscsi" SR driver that will create a LUN-per-VDI SR (I found this when looking through the /opt/xensource/sm directory - see the ISCSISR.py file). You essentially setup one SR for each iSCSI target, and ...


3

ZFS handles 4k sectors well as long as the drive advertises them correctly. However, some drives have 4k sectors internally but present a logical 512 sector size to the operating system for backwards compatibility. If ZFS believes the drive, and writes in 512 byte chunks to 4k sectors, you'll suffer a heavy read-modify-write penalty. Have a look at the ...


3

Are you connecting the X25-M SSD to the backplane? There's a known issue with Nexenta and accessing the L2ARC over a backplane. Your best bet is to connect the SSD directly into a SATA port on the motherboard. Make sure it's configured to use AHCI as well. If you're running anything mission critical on this server I would switch to a SLC SSD (like the X25-E ...


3

If you plan to continue using your Hardware RAID, you should be aware that ZFS (on Nexenta) really needs direct access to the single disks in order to be fully operational. CIFS is somewhat limited on Nexenta CE as it currently can't work with LDAP users for access control, this only works with NFS. For CIFS, you need to create local users on the Nexenta ...


3

Yes it is possible, you just need to set up the port as a "trunk port". You can also add ACLs to permit only those two VLANs. You will also need to set up appropriate VLANs on your ESXi (In the virtual switch configuration, you'll have to set up VLANs for your virtual machines, for all of them to be in the "lan" VLAN).



Only top voted, non community-wiki answers of a minimum length are eligible