I have an HP Microserver running OpenMediaVault 2.1.18 (Stone burner), based on Debian GNU/Linux 7 (wheezy).
A few days ago, I needed to copy a large number of files from a USB drive, formatted with NTFS. I connected it, mounted it and started the copy and left it to complete.
A day later, I find that it has copied hardly anything, and my NAS performance had tanked. Investigating, I find that page and buffer cache are now taking up over 3/4 of system memory (together these normally take less than 10%) and the ZFS ARC is now squeezed into the remaining memory (less than 1/4 it's normal size).
When I realised what was wrong, I stopped the copy, unmounted and unplugged the drive and very rapidly the caches went back to their normal size, then the ARC started to fill itself back to it's normal level.
So, if I need to do do more direct copies in the future, how do I prevent mounting ntfs drives causing large amounts of memory to be consumed, starving my ZFS ARC of the memory it needs?
Note: As shodanshok suggested, I tried running
echo 3 > /proc/sys/vm/drop_caches (to "free pagecache, dentries and inodes" according to
man 5 proc), but this also flushed my ARC (except the memory needed to cache the L2ARC header). My system thus had lots of free memory, but since things are only cached in either ARC or L2ARC, my system then spent ages thrashing the drives to repopulate the ARC, not a desirable state of affairs.
In this image, you can see the ZFS starvation at the start of the 5th, when I started my ntfs copy, things returning to normal when unmounting the ntfs drive early on the 6th, and then finally the ARC being cleared (and rebuilt) after using drop_caches, on the afternoon of the 7th.