Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I plan on creating a reusable template image for deploying my application to multiple virtual machines that may be on different networks and different datacenters. I plan on having full control of the host system and running KVM as the hypervisor.

The image will be made immutable when attached to the virtual machine so I can deploy changes more easily by updating a single template instead of each machine's internal software.

I understand how to configure the /etc/network/interfaces file on a regular ubuntu system, but how would you automatically assign each virtual machine 1 or more IP addresses when you are working with an immutable image for the virtual machine on the first boot?

The only way I have imagined to do this so far is to set up the template to rely on DHCP for networking when it boots. After the machine has booted, it could then run a script that verifies and re-configures the IP configuration by connecting to another server or remote filesystem that has the network configuration information.

Perhaps, immediately after the network is available, I could run a script on the guest VM that sends the MAC address of the attached network interface(s) as the unique id to retrieve the network configuration from another "master" server that has the IP assignments. This assumes that all VMs will be assigned a unique MAC address when they are created.

I might need to avoid starting services like mysql, and php until the network has been configured and verified so that the network restarting shortly after boot doesn't disrupt anything important.

Would that sound like a correct way to do it?

Do Amazon EC2, openstack or others have a different approach to network automation? Perhaps they provide an API and/or scripts that do the same thing during boot?

I'm using KVM on the Ubuntu host and the guest OS will also be Ubuntu if that matters.

share|improve this question
You mean like cloud-init? – Michael Hampton Dec 23 '13 at 22:59
btw, if you deploy VMs from a template and change the template, you are looking at trouble. Assuming you are talking about the template using a disk image and the derived VMs being based on thin snapshots from that image. – dyasny Dec 23 '13 at 23:04
@dyasny My goal is to avoid individually running software updates on every instance, but I don't have the experience to describe how to do that accurately until I've learned it. – Bruce Kirkpatrick Dec 24 '13 at 0:13
@MichaelHampton cloud-init looks like a valid answer with a broader range of features and ubuntu / KVM support. I found some examples of using it for networking configuration here: I couldn't find something like this when looking on google. Feel free to post this as an answer. – Bruce Kirkpatrick Dec 24 '13 at 0:22
@BruceKirkpatrick you cannot avoid that directly. You can automate updates (use a local repo and update commands in cron for example), but you can't expect to update one VM and all of it's "child" VMS to have the same updates magically appear. Snapshots don't support that sort of thing, and they are the commonplace templating mechanism – dyasny Dec 24 '13 at 1:06
up vote 1 down vote accepted

The only thing you can control from the host side is the MAC you provide to the VM. So the simplest approach is to reserve the IP you want a specific VM to have in DHCP for a specific MAC, and then give that MAC address to the VM in question when you set it up. This can scale to a point, but with hundreds of VMs managing MACs can become tedious.

So the next approach is to boot a VM up with an attached ISO or floppy image, where additional, per VM settings file is provided. The VM will run a script on boot which will look for the attached ISO/floppy image and apply the settings found there, if such an image is attached. Easy to use when configuring new VMs or when reconfiguring existing ones.

The best approach of course, is to use a configuration management system (puppet/chef/cfengine/ansible/saltstack/etc) or some sort of converged deployment/config system like theForeman. This is as scalable as it ever gets, and allows you lots of flexibility in managing the settings for more than just networking.

share|improve this answer
I love how I get downvoted as an act of revenge :) Some people need to grow up. – dyasny Dec 23 '13 at 23:07
Downvoted for unnecessary hackery with ISOs and puppet. – dmourati Dec 23 '13 at 23:08
@dmourati, seriously? For knowing what I'm talking about? Thanks, I'll take that for a compliment. – dyasny Dec 23 '13 at 23:09
Guess my 40+ VM OpenStack cluster is broken running on NAT? – dmourati Dec 23 '13 at 23:10
Do you think 40+ VMs is impressive? Maybe for a localhost admin who never saw a datacentre in his life... Or do you think it is normal to set up huge batches of iptables rules on every host? How do you handle live migrations? What if you need to run a low latency networking VM in that "cluster"? NAT on the host is a small solution oriented towards a single standalone host with no real networking requirements for the VMs. If you are "here to learn", then learn, don't be a child. – dyasny Dec 23 '13 at 23:15

I think you are trying to make this more difficult than it needs to be. The default KVM networking in Ubuntu is called Usermode.

This will create a NAT on your physical host allowing all your VMs to access the network over NAT. The VMs get a DHCP address as you said above and that's it.

Alternately, you can use Bridged Networking to enable external hosts to directly access services on the guest operating system.

share|improve this answer
NAT is not the way normal networking is done. Everyone uses bridges or OVS because it's simpler and doesn't enerate immense overheads on hosts and guests. – dyasny Dec 23 '13 at 22:58
NAT is probably ok for everything but SSL based domains. Some of these cloud servers may be shared with people outside of my company, and they'd need a public ip for that too usually unless I have some kind of proxy server in front of everything. – Bruce Kirkpatrick Dec 23 '13 at 23:53
That's how Amazon does everything. Private IPs in VPCs then EIPs/ELBs in front. Seems to scale pretty well. YMMV. – dmourati Dec 24 '13 at 2:25

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.