Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm working on a design for a Linux based appliance and looking to use KVM for the virtualisation layer. The system has 3 VM's interconnected in very specific ways. The fag packet diagram I've been given looks like this...

+------ | ------------+
|       |             |
|      |A|            |
|       |             |
|       |             |
|      |B|-----|C|    |
|       |             |
+------ | ------------+

This basically mirrors the data flow - a web server at the front with a back end server behind it, and a subsidiary service on the side. A and C must be on "separate" networks due to internet proximity etc. This is fine for the bulk of the system, but this means that all management traffic for vm's A and C would need to pass through vm B to reach an external syslog server or such. Similarily for ssh access in, we'd need something like port forwarding on B. Whilst the volume of this traffic is minimal compared to the application traffic, it feels pretty horrible, especially as the automated building of these vm's using kickstart or some such, mean that we can't start building A or C until B is finished.

So I'm thinking about making it all a lot more confusing with something like this:

+---- | ----------------------+
|     |                       |
|     |       +-----------+   |
|     |       |           |   |
|    |A|-----|B|-----|C|  |   |
|     |       |       |   |   |
|     |       b       |   |   |
|     +-----a x c-----+   |   |
|             d           |   |
|             |           |   |
+------------ | --------- | --+
              |           |
             mgt         dbs

So the data flow between A B and C for the app is largely identical (but drawn flat here) and we add in a second management nic to each VM connected to another bridge which also has an interface on the host (a, b and c). iptables and forwarding is then used between these virtual host interfaces and the outside world (d) to ensure that whilst all machines can directly reach external services lower down in the environment, they can't reach each other in a way that means compromise of vm A does not reduce the security to any larger extent. I could probably use ebtables and a single bridge in my additions but I think it'd look too subtle with A and C being on the same subnet still.

It's a fair cop to note that there's a significant angle on what things look like on a diagram for management as well as the actual technical credibility of things (hence "separate" vs separate)! Simplicity is also a huge motivator, but at the cost of chucking all additional traffic through a 2nd VM feels real uncomfortable, as well as adding maybe 5 to 10 minutes for the build in a time sensitive environment

So essentially, am I nuts?



share|improve this question
up vote 0 down vote accepted

No, not nuts. But I do think you are making this sound more difficult than it needs to be. I've read your question several times, and there are still a few things I don't understand. What does machine C actually do? Why do you need it there? What do machines A and B do?.

In any case, if I follow your first diagram, than all you need to do is apply simple IP routing and you're fine. If machine C forms a separate subnet with B (so that it is separated from the rest), then in order to get to it, routing must be set up like this:

  • on dbs: route all traffic through B (that's the default gateway, which knows how to route
  • traffic to C, because it is directly connected).
  • on www: route all traffic for C through A (only needed if A is not the default gateway)
  • on A: route all traffic for C through B (only needed if B is not the default gateway)
  • on C set B as the default gateway.

Once the routing is in place, you can add firewalls to make sure that traffic only flows where you want it to flow. Unless you are experienced with this sort of stuff, I'd recommend to get the routing going first and then add the firewalls (one step at a time, so you always know what broke it. Because you will break it).

share|improve this answer
Thanks for the response. A is just apache holding static content, B is tomcat running the main app. C is a Windows DRM server in an otherwise Linux environment as the software only runs on Windows. We'd love it to not need to exist, but it has to. There is no traffic in from the web other than the end users, and so they only need to reach A, which is all fine and dandy. Additionally databases and other services (dbs) are all called out from B inside. I'd anticipate a NAT or some form on the host so traffic only ever hits a single IP on the back, and one on the front. – Chris Phillips Sep 8 '10 at 10:48
Well, that only makes things even simpler. Obviously www is out of the question (didn't realise that was meant to mean the Internet). A is the only machine with a public IP. It therefore also needs to act as a router/gateway (unless there are other machines for that purpose).B and C share a private little network. And my above answer still stands as it is. – wolfgangsz Sep 8 '10 at 11:35
Would you believe it fair to say that the additional networking does not compromise the requirement for notional separation of A and C? If A were to be compromised then there is admittedly a shorter path through to the external environment as it no longer also needs to traverse security on B. The iptables config on the host machine would, in my mind, cover all of that off to retain a suitably clear security model. – Chris Phillips Sep 8 '10 at 13:04
If A is compromised, then it depends on the security configuration of B whether C is compromised. Since C is not on the same network as A and B, it is not immediately visible AND protected by its own firewall and the firewall on B. So in essence the advice here really is as said: get your routing sorted out and then tighten the firewalls to the point where only the traffic that you need will pass. – wolfgangsz Sep 8 '10 at 16:45

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.