I am designing a MongoDB cluster on multiple data centers
The 3 data centers are located at different suppliers. They are linked through IPSEC site-to-site VPNs.
I am looking for the best design to avoid a split brain. But some situations may be problematic. For example, if I lose the site-to-site between the provider 1 and 2, and the 2 providers continue to see the provider 3, they will both have the majority and continue to write each on their own.
One solution would be to double the communication between 1 and 2. Sites 1 and 2 must be able to communicate via the site-to-site '1-2' but also through the "site-to-site 1-3 + site-to-site 2 -3". Thus, if 1 loses direct communication with 2, we do not lose it completely.
But then, how to be sure to avoid network loops?
What would be your advice and ideas regarding this type of configuration?