11

Lets say we have one server with lxc installed, and a lxc container used for as a base img /var/lib/lxc/ubuntu_base. For simplicity let's forget the config changes after copying the base img.

some people suggest using subvolumes and snapshots for making new containers, but one could easily do cp --reflink with simmilar results.

So what is the propper way (or which is better) for managing multiple containers?

  • snapshots

This way seems best, but commands like lxc-destroy won't work since it won't be able to delete the directory.

btrfs subvolume snapshot /var/lib/lxc/ubuntu_base /var/lib/lxc/container_1
  • cp with reflink

I am not sure if there is any performance difference between this or snapshots

cp --reflink=always /var/lib/lxc/ubuntu_base /var/lib/lxc/container_1
  • or Is there any other better way of doing this that I am not aware of.

edit:

One thing I've seen with the reflink option is, that you can't delete the base container if others are running, because the /proc and /dev are mounted and never changed, se the reference is always the same. But shutting down all the coppied containers seems to help.

  • I've used the btrfs snapshot feature to create new containers - and it works well (pretty quick provisioning etc). However, btrfs has a per-subvolume page cache - so although using snapshots is quick/disk space efficient, you're likely to end up having multiple copies of what's effectively the same binary in memory. – David Goodwin Mar 10 '17 at 20:21
2

I am on Ubuntu LTS 14 and just ran the following (for first time even) and it worked like a charm:

lxc-stop -n ubuntu_base
lxc-clone -o ubuntu_base -n ubuntu_base_c1 -s
lxc-start -n ubuntu_base_c1 -d # make changes if needed
lxc-stop -n ubuntu_base_c1
lxc-snapshot -n ubuntu_base_c1

Using -s with lxc-clone will take a snapshot if backing store is btrfs (in your case).

Verify new clone/snapshots with

lxc-ls -f
btrfs subvolume list /var/lib/lxc

Hope that helps!

  • 1
    Note that these days you need to specify btrfs as the backing store when you run lxc-create or lxc-clone will give you a overlayfs instead. – Lester Cheung Mar 31 '15 at 11:13
  • To just create a new "light" container, it seems that the two first lines are enough (lxc-stop and lxc-clone), thanks to the -s option, and that the three last ones (lxc_start, lxc_stop, lxc_snapshot) are just an helper process to manage future container evolutions. Is it correct ? – lalebarde May 30 '15 at 20:43
3

if you will use btrfs subvolumes for lxc, you need add the option user_subvol_rm_allowed in your /etc/fstab. Example extracted from one fstab file:

UUID=XXXXXXXXXXXXXXXXXXXXXX / btrfs subvol=@,user_subvol_rm_allowed,defaults 0 0

the option will permit that you can remove subvol without be root, only normal user. This capability is used by lxc when the snapshots go in btrfs subvolumes

3

Guess it depends on how big your base image it is. I'd probably lxc-create a new container and use Salt/Puppet etc to provision my containers and only lxc-clone for relatively bigger containers (e.g. dev containers with lots of tools installed and configured).

Note that lxc-clone will use the same backing store as the source. So to use subvolume you will need to create your containers with "-B btrfs". For example:

lxc-create -B btrfs -n mycontainer -t ubuntu

Then clone it with:

lxc-clone -s mycontainer mynewcontainer

In case you are using zfs to store you containers, there is an extra --zfsroot option to lxc-create so you can choose a zpool other than the default "tank". For example:

lxc-create -B zfs --zfsroot=data/lxc

Share and enjoy!

  • I am on debian wheezy with BTRFS. It appears that the option -B btrfs is used by default - I assume because my debian is set-up with BTRFS. Actually, I created my first container (a 32 bit debian wheezy) without this option, and a subvolume was created for it. – lalebarde May 30 '15 at 20:21
  • @lalebarde that's probably -B best in action but good to know! – Lester Cheung Nov 12 '15 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.