I am planing to move my small shared-hosting environment to Amazon EC2. It requires a central storage for the webstuff (leave databases and so on aside for now) so that i can do both: load balance a website across multiple http nodes and/or move a website from one node to another without actually moving any data.
Currently, my storage is on a NFS Server, which is made high available via pacemaker/heartbeat/drbd. Afaik, you cannot work with virtual IPs in EC2, so most HA solutions drop out. What remains is GlusterFS (and Ceph, but they themself state not to use it in prdocution). So i would setup two GlusterFS nodes, each in a separate availability zone, each mounting multiple EBS volumes (mdadm raid 10 for performance and paranoia) and replicating to each other. Each client would be aware of both exporter nodes and if one goes down, the storage itself would be still up.
So far so good (or not?). Now comes the problem: from my understanding of GlusterFS the access controls are volume based hosts.allow/.deny lists. To reduce the impact of a corrupted HTTP node, i'd like to limit the access from each node to the storage to the webroots of the websites he actually serves.
Let me clearify that with an example of my current NFSv4 + mount-bind setup:
On httpnode1
those are the websites he serves
* domain1.tld
* domain5.tld
* domain10.tld
the nfs export in /etc/fstab
10.0.0.1:/ /srv/web nfs4 rw,..
On the NFS exporter
the structure
* /srv/storage << here are all websites
* /srv/export/httpnode1/ << webroots for httpnode1
* /srv/export/httpnode1/domain1.tld/ << mount-bind of /srv/storage/domain1.tld/
* /srv/export/httpnode1/domain5.tld/ << mount-bind of /srv/storage/domain5.tld/
* /srv/export/httpnode1/domain10.tld/ << mount-bind of /srv/storage/domain10.tld/
* /srv/export/httpnode2/ << webroots for httpnode2
* /srv/export/httpnode2/domain31.tld/ << mount-bind of /srv/storage/domain3.tld/
* /srv/export/httpnode2/domain5.tld/ << mount-bind of /srv/storage/domain5.tld/
/etc/exports
# httpnode1
/srv/export/httpnode1 10.0.0.123(rw,..,fsid=0,crossmnt)
/srv/export/httpnode1/domain1.tld 10.0.0.123(rw,..)
/srv/export/httpnode1/domain5.tld 10.0.0.123(rw,..)
/srv/export/httpnode1/domain10.tld 10.0.0.123(rw,..)
# httpnode2
/srv/export/httpnode2 10.0.0.192(rw,..,fsid=0,crossmnt)
/srv/export/httpnode2/domain3.tld 10.0.0.192(rw,..)
/srv/export/httpnode1/domain5.tld 10.0.0.123(rw,..)
What i can think of, so far, there is only one solution with EC2 + EBS + GlusterFS would be to export multiple volumes from GlusterFS: one per HTTP node.
Does anybody have any experience with this (eg ressource requirements in some hundred exported volumes) and how do look your configs for this? Or even better: Is there any elegant approach to achieve my goals?
Thanks for any insight!
Greets Joerg