1

I am planning a proxmox HA configuration with two Dell R710 machines (dual 6 core processors in each) with enterprise level drive raid arrays. I would be using DRBD with a quorum disk on a third machine. I would dedicate two 1GB nics on each server to the DRBD communications. We would have approximately 12 to 14 Virtual Machines running on this pair of servers. The proxmox manual recommends creating two DRBD resources - one for the Virtual Machines that normally run on ServerA and one for the Virtual Machines that normally run on ServerB. This is because of the Primary/Primary state in which this configuration runs. If both servers have VMs talking to the same DRBD resource and a split brain situation occurs, there is potential for data corruption that must be resolved.

While I understand it would take more effort to create new virtual machines, can anybody foresee any potential problems with running a separate DRBD resource for each VM instead? Does anyone have experience running a setup that way and has it worked well? It seems to me that would allow more flexibility in moving machines back and forth.

Improve this question
1
  • From promox web site, i saw the promox ha is based on redhat cluster, so the first thing you need to configured for avoid data corruption in your cluster is fencing
    – c4f4t0r
    Dec 11, 2013 at 0:23

1 Answer 1

Reset to default
3
+50

I did not have an experience with Poxmox but configured normal pacemaker/corosync cluster on CentOS, so hope my observations are still useful and applicable here.

I am very suspicious about Primary/Primary DRBD setup. Even with Primary/Secondary configuration split brain is probable if something goes wrong. I was wondered how easily DRBD can fall into split-brain condition in not well-tuned cluster.

With Primary/Primary case special attention should be done to fencing facilities in order to reduce data loss probability. Excellent introduction to two-node DRBD cluster is here.

Primary/Primary setup is needed mainly for live migration. If you do not use live migration Primary/Secondary is enough, and much more preferable.

Concerning your question, dedicated DRBD resource is also working solution. You will probably move storage stack from DRBD/LVM to LVM/DRBD. So, cluttered LVM becomes to be needed even in Primary/Secondary setup . UPD:Clustered LVM is not needed here as well as dlm to provide It.

Main disadvantage I see: a lot of manual careful work to prepare a VM storage.

Another point to think about in advance - backup strategy. With many DRBD resources it may be a bit more complicated.

I started my first cluster setup with LVM/DRBD stack and dedicated DRBD resource for VM, but later switched to more common DRBD/LVM since new VM provisioning is much simpler in this case.

Improve this answer
3
  • Sorry about the delay - I thought I had my account set up to email me but apparently not. I think this answers my question in that you have stated you have operated a setup using both paradigms. I do use live migration so the dangers of Primary/Primary are required and I understand that. I am using fencing. I also agree that configuring it this way makes adding vms much more difficult, but with the advantage that you can live migrate any VM to any server at any time. I was just making sure there was nothing catastrophically bad about setting it up this way that I was not aware of. Thanks
    – AudioDan
    Dec 14, 2013 at 21:14
  • One other question - do you know any good writeups on the advantages/disadvantages of DRBD/LVM versus LVM/DRBD. I am not quite sure why I want want to switch the stack in this case.
    – AudioDan
    Dec 14, 2013 at 21:15
  • @AudioDan I can not recommend any paper, sory. The only aditional disadvantage of LVM/DRBD in Primary/Primary setup I can see: multiple DRBD instances increase fencing condition probability that finaly affects whole cluster. On the other hand no host-level metadata are directly affected by DRBD here that seems to significantly reduce chances of host-wide catastrophic data loss. But I personally would hold my hand from any final recomendations.
    – Veniamin
    Dec 15, 2013 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.