I'm thinking of a situation where I would have something that creates a copy of a directory, tweaks a few files, and then does some processing on the result. This wold be done fairly often, maybe a few dozen times a day. (The exact use case is testing patch submissions; dupe the code, patch it, build/test/report/etc.)

What I'm looking for could be done by creating a new directory structure and populating it with hard links from the origonal. However this only works if all the tools you use delete and recreate files rather than edit them in place.

Is there a way to have the file system do copy-on-write for a file?

Note: I'm aware that many FSs use COW at a block level (all updates are done via writes to new blocks) but this is not what I want.

  • It doesn't appear maintained, but perhaps boklm.eu/copyfs – Zoredache Apr 7 '10 at 3:07
  • So you want to do copy-on-write via the filesystem, but you don't want a filesystem that uses copy-on-write? I'm confused. – Kamil Kisiel Apr 7 '10 at 5:02
  • 1
    @Kamil Kisiel: I want file-level COW not block-level COW. File-level COW results in cp and ln doing almost exactly the same thing. Block-level COW is a trick to minimize the interval that the FS meta data is incorrect: en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model – BCS Apr 7 '10 at 15:01
  • What kind of system is this expected to be used on? – John Gardeniers Apr 8 '10 at 4:56
  • It's speculative enough that unless you know of a way to do it for some setup (in that case just post it) I wouldn't want people burning time figuring out how to do it (unless they are doing it for fun :) – BCS Apr 8 '10 at 15:19

If you have your choice of platform for your fileserver, I'd go with a recent OpenSolaris build and use the deduplication feature of ZFS. This way copies of files would take up no additional space, and even common segments between files would not be replicated.

You can read all about ZFS deduplication at this post on Jeff Bonwick's Blog.

  • 1
    ZFS is amazing; it will change the way you think about storage – tegbains Apr 7 '10 at 5:39
  • Neat, unless there's a down side to having it on for the whole drive. – BCS Apr 7 '10 at 14:54
  • I added a link to a blog post that has more technical details. You can see if it's what you need there. – Kamil Kisiel Apr 8 '10 at 3:57
  • New url for post of Jeff Bonwick: blogs.oracle.com/bonwick/zfs-deduplication-v2 – Mathieu Jun 27 '18 at 9:06

Most linux virtualization solutions offer file-level COW among virtualized OSes (copied from the WP):

  • YES:
    • Docker, Linux-VServer, lmctfy, LXC, Singularity, Virtuozzo, Sandboxie, systemd-nspawn, Solaris Containers(ZFS), FreeBSD jail(ZFS), OpenVZ
  • NO:
    • chroot, sysjail, WPARs, iCore Virtual Accounts, Turbo
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes – Romeo Ninov Feb 8 at 20:01

May be SVN or other versioning tool (git, cvs)?

  • That's even worse because it creates two copies of each file and sucks them down over a socket. – BCS Apr 7 '10 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.