0

We've noticed a problem with one of our volumes on netapp filer. It seems that volume is full, NetApp informs, that volume is using or reserving 100% of space (0% of inodes) - this is shown as the warning. Problem is, it does not looks this way. Volume size is 190 GB. Volume is Flexible type, file space guarantee, no mirroring. We have exactly two LUNs on the volume mapped. 95 GB and 50 GB. Both of them are set to reserve 0% for snapshots. Both have space reservation. There is still much space on the volume (in theory). df -r shows:

Filesystem              kbytes       used      avail   reserved  Mounted on
/vol/BACKUP/   199229440  199229440          0  142799672  /vol/BACKUP/

There is also some free space left on aggregate. We have on the same aggregate similar volumes with LUNs (same configuration) and they are completely fine. We have new shelf and we want to migrate there some data, before we install new shelf, we want to make sure, that we have backups of all the data. However, because of this one, particular volume, backup fails (no free space for the snapshot).

added: If i check space occupied on production system, where both LUNs are mapped, it's only 94 GB.

Improve this question

2 Answers 2

Reset to default
1

Take a look at man vol and read the bit about fractional reserve - this is the root of your problem.

Specifically - when LUNs run out of space, they break horribly and can cause chaos on the host. NetApp allows you to take snapshots of volumes - a snap uses space in proportion to the changed blocks on the volume. If your volume fills up, and you cannot allocate new blocks, because there's a snapshot present... your LUNs will all break.

So in comes fractional reserve, which says 'whenever I take a snap, reserve volume space so that I don't risk running out'. Set to 100, each volume (when a snap exists) tries to reserve space equal to the sum total of the allocated LUN space - meaning the volume needs to be 200% of the size in order to be sure you don't run out.

Lowering fractional reserve is a risk, but not a big one if you don't routinely cycle all the data in your LUNs. Just bear in mind that running out will mean write failures to the LUNs, and that's generally bad news. You could also adjust the volume guarantee options - file guarantee combined with fractional reserve 100 means your volume will need to be 200% the size of the LUNs within (+some, if you've multiple snaps, although it won't be +100% per snap).

Improve this answer
4
  • Yeah, i already found out a bit about fractional reserve. However, it's not possible to change fractional reserve when my volume has file guarantee active. What i really need is a temporary solution. I just want to free enough space on the volume to make a full backup (external solution - Simpana), in order to proceed with migration. So, i believe i have to switch guarantee on the volume to none or volume and reduce fractional reserve?
    – Tomasz Szkudlarek
    Sep 16, 2014 at 12:30
  • Setting volume guarantee to none will probably do the trick, just bear in mind the risk (e.g. if there's lots of churn, it might fill). Checking your snaps and tidying them up might help too assuming by 'backup' you don't mean 'snapshot/snapvault'? file guarantee means your volume reserves space up to max possible LUN size e.g. how big they might grow. That might end up being the same as none if your LUNs have been filled at any point though. (but you should be able to drop the frac reserve in that case)
    – Sobrique
    Sep 16, 2014 at 12:36
  • Yeah, that is a little tricky, because these LUN's contains incremental online backups of our erp database. Still, there shouldn't be so much changes. By backup i mean external software, which uses ndmp protocol to backup netapp volumes.
    – Tomasz Szkudlarek
    Sep 16, 2014 at 12:47
  • You never use fractional reserve on LUNs and haven't for a few years...
    – Basil
    Sep 16, 2014 at 14:30
0

I've seen this issue. The way LUNs work, as you write to sectors, the original contents of those sectors get unallocated, but not cleaned off the volume until any snapshots using them are deleted. In my case, I wasn't snapping the LUN, but we had a power failure and our UPS only covered one of the two power circuits. In that situation, the NAS prioritizes cleaning unused blocks lower.

Best practice is to put a single LUN in each volume. Thin provision the LUN, and then set up autogrow on the containing volume. I thickly provision the volumes. This will mean that so long as the LUN is still mostly unused, you'll never get into trouble. When every sector has been written to and they start overwriting heavily, instead of going offline, the volume will grow a little to accommodate the increased footprint of the LUN. Of course, the server still sees the same size LUN, so as soon as this condition has ended, the space utilization goes back down to normal.

The command to thin provision a LUN is lun set reservation lunpath disable. The command to configure autogrow on the volume is vol autosize volname -m 100g -i 5g on (which would set the max to 100GB, and increment 5GB at a time).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .