using same LVM volume for devicemapper and a bind mounted volume

Question

I'm running an Amazon EC2 machine with a 512GB EBS mount. My plan is to split the EBS into two parts - one to be used by Docker 1.11 for its images (via devicemapper) and another to be bind-mounted to the docker VM (for storing some big pieces of data persistently)

admin@ip-172:~/docker-setup$ sudo lvs
  LV       VG         Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data     direct-lvm twi-a-tz-- 230.39g             4.19   10.06                           
  homedata direct-lvm -wi-a----- 256.00g                                                    
  metadata direct-lvm -wi-a-----  12.00g   


admin@ip-172:~/docker-setup$ sudo pvs
  PV         VG         Fmt  Attr PSize   PFree 
  /dev/xvdg1 direct-lvm lvm2 a--  512.00g 13.38g


admin@ip-172:~/docker-setup$ sudo dmsetup ls --tree
docker-202:2-407552-76fd7e91f9d116412221e660108dbf577ecaa301796068e63bd3007e402d2ccf (254:6)
 └─direct--lvm-data-tpool (254:4)
    ├─direct--lvm-data_tdata (254:3)
    │  └─ (202:97)
    └─direct--lvm-data_tmeta (254:1)
       └─ (202:97)
direct--lvm-metadata (254:2)
 └─ (202:97)
direct--lvm-data (254:5)
 └─direct--lvm-data-tpool (254:4)
    ├─direct--lvm-data_tdata (254:3)
    │  └─ (202:97)
    └─direct--lvm-data_tmeta (254:1)
       └─ (202:97)
direct--lvm-homedata (254:0)
 └─ (202:97)

I'm mounting /dev/direct-lvm/homedata to /bigdata on the docker VM. A docker inspect shows this:

{
                "Source": "/dev/direct-lvm/homedata",
                "Destination": "/bigdata",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },

however, the directory /bigdata is shown as 10mb udev mount.

deployer@b11853af11c3$$ df   
Filesystem                                                                                       1K-blocks    Used Available Use% Mounted on
/dev/mapper/docker-202:2-407552-76fd7e91f9d116412221e660108dbf577ecaa301796068e63bd3007e402d2ccf  10475520 6065028   4410492  58% /
tmpfs                                                                                              7849700       0   7849700   0% /dev
tmpfs                                                                                              7849700       0   7849700   0% /sys/fs/cgroup
/dev/xvda2                                                                                         8124836 1231776   6457300  17% /run
udev                                                                                                 10240      56     10184   1% /bigdata
shm                                                                                                  65536       0     65536   0% /dev/shm

I have lots of other mounts that are simply local directories and they work just fine. But when I'm trying to bind mount the lvm volume.. it simply doesnt work. What gives ? should I be doing something here with the lvm volume to make it mountable ? please help.

Answer 1

For anyone else finding this in the future. Also, apologies for dodging the question.

In the specific case of AWS EC2 and EBS volumes there are some cloud-specific changes you can make to your approach more "cloudy".

Iteration 1: Just Use Separate EBS Volumes

It is far simpler to use separate EBS volumes rather than LVM managing logical volumes on a single EBS volume. You'll also have better I/O performance by using separate EBS volumes.

Iteration 2: Use EBS Volumes 'Natively'

You can also use a volume plugin (such as Cloudstor that will allow you to allocate separate EBS volumes per container.