7

I'm planning on migrating a large-ish amount of websites (approx 100) to a new server and I'm in the migration planning process.

A typical DNS zone for each website has two A records pointing to the web server IP, one for example.com and one for the www subdomain.

When we're all setup and ready to launch the new server to production, changing 100x2 DNS records will be time-consuming so I'm searching a way to make this quicker. In a couple cases I read about creating a bash script that iterates the DNS records and performs a find-replace with the new IP. In other topics I've read suggestions about adding A records with the new IP so that when the current server is no longer available, the DNS server will point the requests to the next record, that containing the new IP.

Apart from these, is there any scenario in which I could replace the A records with some other type of DNS entry ie a hostname so that, when the time comes, I can only change the IP of the hostname with the new one and have all websites point to the new server? I'm sure 'hostname' is not the right term, I hope you all get the idea though.

  • 1
    What DNS server / service are you using? When you do make the change you could consider using a CNAME rather than A record then if you move server again you only have a single A record to update – Phil Sep 27 '18 at 13:21
  • We're using WHM for managing our DNS. Good point about the CNAME, I'm going to use your suggestion. – bikey77 Sep 27 '18 at 13:37
  • 3
    "the current server is no longer available, the DNS server will point the requests to the next record, " no the DNS does not work like this, it is not a failover. If a label has two A records, the DNS will always get back both, even if one IP does not work anymore. Someone or something will need to explicitely remove the defunct IP from the zone. – Patrick Mevzek Sep 27 '18 at 15:38
  • Using a CNAME record is an option for the www record but not for the apex. Also, will it be better to modify the CNAME than modifying the A record? I'm not sure. – Tommiie Oct 1 '18 at 13:09
  • Convert the old server as a reverse proxy to the new server? If you have so many sites on a server you should consider an IP failover or something similar – giammin Oct 1 '18 at 19:50
12

The term you're looking for is CNAME, and the answer to your question is both yes and no.

First, here's an example of how a CNAME works in a zone file.

example.se  IN SOA  ns1.example.se. hostmaster.example.se. (
            [....]
            )

server1    A       10.1.2.3
www        CNAME   server1

Now you just need to update the server1 record in order to move both server1 and www to the new IP address.

The CNAME doesn't have to point to an address within the same domain; it could also look like this:

example.se  IN SOA  ns1.example.se. hostmaster.example.se. (
            [....]
            )

www         CNAME   server1.example.org.

Now, when you update the A record for server1 in the zone example.org, the record for www.example.se will follow along without any further configuration.

The bad part, from your point of view, is that this does not work for the apex record - that means the "bare" domain. In other words, you can make www.example.com into a CNAME, but you can't do that with example.com. This is because when you use a CNAME record, you can't have any additional records for that entry - meaning that you can't have mail server records, or name server records... meaning that the domain will stop working.

The best-practice solution is to use some kind of configuration management software, such as puppet, chef or ansible, to generate the zone files from a template. If, for some reason, that is not possible for you, then I'd use a script to replace the IP addresses in all files.

You'll also want to reduce the TTL value for the domain in due time before the migration. (And don't forget to update the serial number of the zone file - I have, and it's very embarrassing...)

  • But isn't this problematic in case DNS caches ignore the TTL and you want to avoid downtimes? Doesn't routing the traffic from the old to the new IP or from the new IP to the old IP until the sites are moved (depending on the approach) make more sense? When relying on DNS for this you basically have no control over when your clients get served the new IP. With routing it doesn't matter, you can just keep your old server and route to the new one for a few days/weeks. – Broco Sep 27 '18 at 14:00
  • 1
    @Broco yes, depending on constraints, you either have to maintain both servers live for some time during handover, or redirect traffic (like HTTP redirection or email forward, etc.) from old to new. – Patrick Mevzek Sep 27 '18 at 15:43
  • "You'll also want to reduce the TTL value for the domain in due time before the migration." if you want to be absolutely fool proof you will have other knobs to adjust and more steps. You need to decrease the SOA REFRESH value too (depending on how all nameservers are provisioned). If you "convert" an A to a CNAME you may first need to publish the A with reduced TTL and then change it for CNAME. You may need to change the minimum/negative TTL too. In short it is complicated (if there is a need for fast convergence). – Patrick Mevzek Sep 27 '18 at 15:45
  • @Broco "DNS caches ignore the TTL" they should not. Or at least can clear things before it but not after. Except that some will indeed clamp very small TTL values like few seconds. – Patrick Mevzek Sep 27 '18 at 15:46
  • @PatrickMevzek I know from experience that they do (not all, but some). In the past I had issues when moving sites with the DNS approach. Even though we set the TTL to 5 minutes two days before we moved the sites (it was 24 hours before), some providers didn't care and only served the new IP after 24 hours, some took an hour, some took the 5 minutes. – Broco Sep 27 '18 at 22:56
3

First, the answer depends on how your DNS is implemented. Are you self-hosting using something like bind, unbound, or various other DNS servers? Are your DNS servers using text files for configuration, are they using a GUI interface (such as Windows servers), are they using a scripting API (such as PowerShell), or are they using a Web interface (common if you outsourced DNS). Outsourced DNS also sometimes has a REST API you can use.

You probably will want to use either a scripting API to do the updates, or edit a text file, since both can be automated easily. If it's a text file, you can even prepare it in advance and just copy it in place when doing the migration.

Incidentally, making this type of change on your kind of scale is why many people are now using DevOps tools such as Chef, Ansible, Puppet, Salt, ... Obviously these are big tools that require planning, so this is not going to be of immediate help for this specific need, but it may make your life easier in the long term.

A few other important considerations:

  • Check your TTL. About a day or week before your migration, change the TTL to the shortest one you can do. Increase the TTL after the migration is complete.

  • Don't forget to increment the serial number in he SOA record! APIs and Web interfaces may or may not do this automatically for you. If your DNS server uses text files, you must remember to do it.

  • Depending on the nature of the server, see if you can run the old and new side-by-side for a bit. If you can, you may not have to complete all DNS updates at once.

2

If I get this right the old server has a single IP address and the new one has only one, too.

So what you could do is to set up the new server and on it just route all relevant http/https/ftp/whatever traffic to the old IP and then update the DNS records to show the new IP.

If you only have one IP you can use a simple find/replace to change all IPs in the DNS (don't forget to update the serial number) at once and wait for your current TTL to run out. Once the TTL has run out you can just remove the routes on the new server and adjust your other settings, so that it serves the sites directly instead of redirecting.

Before removing the routes the websites should have been moved to the new location obviously. This has the advantage that you don't have to rely on other DNS servers to update their records in time you can just "wait it out". To shorten that period you can reduce the TTL but be aware that some DNS caches out there ignore the TTL settings, that's why I don't rely on TTLs any more.

Edit/further explanation: To make it somewhat easier to understand:

You can also just install your new server right now, configure everything and move all your sites to the new IP and THEN route all traffic from the old server to the new server.

E.g. your old server has the IP 192.0.2.1 your new server has the IP 198.51.100.2

Your DNS points to 192.0.2.1 for all sites(?) and the webserver decides which content to serve by the name of the domain requested, e.g. example.com -> /var/www/sites/example.com and www.example.com -> /var/www/sites/example.com.

So you just turn this:

example.com -> 192.0.2.1

Into this (as soon as you moved the sites to 198.51.100.2):

example.com -> 192.0.2.1 -> 198.51.100.2

And then change the DNS records to point to 198.51.100.2:

example.com -> 198.51.100.2

The problem with DNS is that the update is never instant, so for some of your clients example.com will still point to 198.51.100.2 for a variable amount of time (either the TTL you set or, if they ignore it, who knows how long).

So my point is: Instead of relying on DNS, redirect the traffic on the IP layer so you reduce downtime. I hope this makes it a bit more clear.

On Ubuntu you can do it with forwarding and NAT (e.g. routing http and https for your old destination 198.51.100.2 to your new destination 198.51.100.2; these settings done on your old server):

echo 1 > /proc/sys/net/ipv4/ip_forward

iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -p tcp --dport 80 -m state --state NEW -j ACCEPT
iptables -A FORWARD -p tcp --dport 443 -m state --state NEW -j ACCEPT

iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 198.51.100.2:80
iptables -t nat -A PREROUTING -p tcp --dport 443 -j DNAT --to-destination 198.51.100.2:443
iptables -t nat -A POSTROUTING -j MASQUERADE
  • 1
    Care to explain a bit more about the routes you mentioned? What type/form do they have? – bikey77 Sep 27 '18 at 13:40
  • I had to explain it a bit broader since you didn't mention which type of server you're using to serve your sites. E.g. in iptables (Linux) you could use Prerouting and masquerading to redirect traffic to a new location, see here: debuntu.org/… The problem with a DNS only approach is that you can't rely on when the DNS caches of your site's visitors update. With the routing approach you determine the time of the switch, minimizing potential downtimes. – Broco Sep 27 '18 at 13:45
  • 2
    Please do not use frivolous IP addresses for examples, especially not ones really used right now on the Internet for big global services. I edited your post using RFC5737 IP addresses as reserved for documentation – Patrick Mevzek Sep 27 '18 at 15:41
1

A quick solution (not the most elegant one, but probably working) would be just replacing the old ip by the new one with a simple find-replace.

First backup the zone files.

Then check for problems:

grep 'ol\.d\.i\.p' *.zone

Check if there are any lines, which are no records you want to replace

Next replace the ip:

sed -i 's/ol\.d\.i\.p/ne.w.i.p/g' *.zone
grep 'ne\.w\.i\.p' *.zone # check if the new lines look correct

This is nothing you should do in a script which runs every day, but for a one-off migration this may be just enough to solve the problem without starting to program sophisticated migration tools.

  • The grep command just searches for lines containing the regular expression (here: a literal match of the ip, the . need to be escaped because they have special meaning in regex).
  • sed is used for line based operations on streams or files (using the -i option). s/regex/replacement/g replaces the regular expression (same as used in the grep command) with the replacement (the new ip). /g means global, which makes sure that the ip is replaced multiple times, if there are multiple matches on a single line.
  • Have a look at RFC5737 for examples on IP addresses to use in documentation. – Patrick Mevzek Sep 27 '18 at 15:38
  • 2
    And you search and replace will need to be more precise. Because if you have both 192.0.2.1 and 192.0.2.100 in your zonefile, the pattern 192\.0\.2\.1 will affect both... – Patrick Mevzek Sep 27 '18 at 15:48
  • 1
    As said, it is a quick & dirty solution and you absolutely need to backup the files and test what will be matched before using it. Indeed you may even want to match \s before and possibly \s or $ after the ip, but it looked like the OP only needs to replace one IP anyway. If you want to manage a whole subnet or other bigger use cases, look for a proper tool and do not rely on simple shell scripting. – allo Sep 27 '18 at 22:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.