Related to this question about using an SSD for system and HDD for data, except I would like my system to do this automatically...

Is it possible to have several layers of storage and push items automatically between them, using preferably free, open-source software?

I know of hugely expensive enterprise-class solutions like the EMC SAN->EMC Centera automatic archiving, but I was wondering if this sort of staged storage is possible to do automatically.

It would be nice to be able to have several layers in this : Memory->SSD->HDD->slower HDD or tape or some other archive solution.

Are there any filesystems which can do this automatically? (ZFS, Btrfs, HAMMER?)

Any Unix-variants are fine, as I'm interested in how this works and whether it's likely to be portable to Linux or other flavours (BSD etc).


Well, ZFS uses a storage layering called Hybrid Storage Pool (HSP):

  1. Layer: memory
  2. Layer: SSD-based read-caches (L2ARC) and write-caches (separate intent log aka slog).
  3. Layer: (cheap) harddisks

With HSP its easy to automatically benefit from the advantages of SSDs compared to a harddisk-only solution. A system using HSP can be both faster and cheaper than the latter. See this link for some nice examples and more details.

I think there are plans regarding hierarchical storage management (HSM) for ZFS (see for example the Automatic Data Migration (ADM) OpenSolaris project) but I don't know its current status.

  • This is just what I was thinking of, thanks! (Now to help on the Linux port... ;) – David Gardner Jul 23 '09 at 8:30
  • I would not call this tiering but caching. In the sense that caching means keeping a copy of frequently used blocks, tiering moving the data to slower/faster storage. I would not call the intent log in ZFS a real write cache if I understand it correctly. – mrossi May 20 '16 at 7:46

Check out the LVM based "lvmts" (LVM Tiered Storage) solution this guy is cooking up:


Pretty cool.

  • +1 This looks very interesting -- I shall be keeping an eye on it! – David Gardner Aug 6 '12 at 11:26

TIER seems to answer your needs It is a linux kernel module that can create a tiered storage. It seems to learn by itself the pattern and optimize the placement into the storage


  • Just saw the question is old ... Sorry ^^' – Dolanor Aug 8 '12 at 15:33
  • 3
    Revitalising old questions is better than having then rotting with old and out-of-date information on them! – David Gardner Aug 9 '12 at 10:57
  • I'm worried that this guy wants to reduce redundancy for the sake of extra capacity (especially when SSDs are comparatively small and unreliable). – Tobu Nov 6 '12 at 10:35
  • Also worried that he suggests raid as the solution, when experimental software needs backups most. – Tobu Nov 6 '12 at 10:54
  • Here's a bit of more positive feedback though: permalink.gmane.org/gmane.comp.file-systems.ceph.devel/8316 – Tobu Nov 6 '12 at 10:55

Answering my own question with something I just found:

I was just updating the kernel and looking at the new stuff that has been added, and there is now a 'CACHEFILES' option which allows for caching (usually remote) filesystems to a local filesystem. I guess I could use this to cache a slower storage mechanism (HDD) to a faster one (SSD), at least for one level of hierarchy.

  • Did you try this solution? How does it perform? – skolima Nov 24 '09 at 16:17
  • 1
    Nevermind - I found out myself that CacheFS for now only supports NFS and AFS. – skolima Nov 24 '09 at 16:22
  • Using network filesystems for this isn't a bad idea, they tend to mature much faster than kernel patchsets. I wonder if Ceph or DRBD have useful modes for tiered storage. – Tobu Nov 6 '12 at 10:48

Some relevant links. YMMV.

http://code.google.com/p/fscops/ -- "Online Hierarchical Storage Manager (OHSM) is the first attempt towards an enterprise level open source data storage manager which automatically moves data between high-cost and low-cost storage media.".

http://www.tack.ch/unix/dmapi/ -- XFS + DMAPI under Linux

http://jfs.sourceforge.net/ -- JFS + DMAPI under Linux

  • The OHSM project looks like an interesting one to follow. Only ext2 and the 2.6.30 kernel is supported for now, but I expect if it works out well then this will gain more filesystems. – David Gardner Jan 22 '10 at 11:41
  • That project looks abandoned, most recent activity was in 2009. – sendmoreinfo Sep 12 '11 at 22:05

RAID controllers offer some of these features.

"Leverages SSDs in front of HDD volumes to create high-capacity, high-performance controller cache pools"



LVM2 snapshots come to mind... but you can't really do more than a single snapshot.


SAM-QFS is Sun's existing product and was open-sourced last year. It's CDDL, so you could only directly port it to *BSD.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.