Fleet
DNS failover and GSLB
A load balancer can only move traffic that reaches it. If a whole node is gone, nothing reaches it, so nothing it might have done matters. Only the DNS lookup can send people somewhere else.
What this is called elsewhere
If you have used other products, you may know this as GSLB, or global server load balancing. F5 call it GTM or BIG-IP DNS, Infoblox call it DTC, and Amazon Route 53 does the same thing with its health checks. It is all the same idea.
Being honest about the limits
It cannot be instant, and nothing can make it instant
How long a computer remembers an answer is a request from you, not a rule. Some providers refuse very short values, browsers keep their own copy, and some programs remember an address for as long as they are running. Expect most visitors to move within a minute or two and expect a small number to take much longer. Anybody who has promised you instant failover through DNS has promised something DNS cannot do.
What it is genuinely good at is the case where nothing else works at all: two nodes in different buildings, on different networks, with different addresses. Two nodes on one network can share a floating address and move it in a second. Two nodes in different places cannot, and then this is the answer.
What you need first
| You need | Why |
|---|---|
| Each node on its own public address | The whole point is handing out different addresses. |
| A part of your domain you can delegate | Such as gslb.example.com. Not the whole domain. |
| Access to your domain host | To point that part at these nodes. |
Delegate a piece, never the whole thing
Delegating example.com itself moves every other record you have, including mail, onto these nodes. Delegating gslb.example.com moves nothing except what you put there, and you point your real names at it with a CNAME.
Setting it up
- Add a zone, using the piece of your domain you are delegating.
- Give one nameserver name per node, such as
ns1.example.comandns2.example.com. You are inventing these now. - Add a name, such as
www, with a type and a TTL. - Add one answer per node. Pick the node from the list and its address is filled in for you.
- Create the records the page prints at your domain host. This is the step people miss.
The two kinds of record
| Record | What it does |
|---|---|
| NS | Says who answers for this part of your domain. One per node. |
| A, called glue | Gives the address of each nameserver. Needed because a nameserver named inside the zone it answers for cannot be looked up without it, which is a loop. |
The glue records are the ones people forget, and forgetting them looks exactly like the feature being broken: everything on the page reads correctly and the name still does not resolve.
What the answers do
| Setting | Options |
|---|---|
| When a node is down | Hand out all the healthy ones spreads visitors across everything that is up. Hand out the best one only keeps everybody on one node until it fails, which is what you want when the nodes are not equal. |
| If every node is down | Hand them out anyway is usually right. If the health check itself is what broke, an empty answer takes the site down when it did not have to be. |
An answer tied to a node is only given out while that node is healthy. An answer left untied is always given out, which is what you want for an address that is not one of these nodes.
A note about port 53
Lookups arrive on UDP port 53, and fall back to TCP port 53 when the answer is too big for one packet, so both need to be open. The part that answers them runs without root on purpose, and a program without root cannot bind port 53. So it listens on 5353 instead, on UDP and TCP both, and a firewall rule the software manages sends port 53 to it.
You do not set any of this up. It is mentioned for one reason: if you test using the high port you will get an answer whether or not the public path works. Always test with no port on the end, because that is the path your visitors use.
What failover actually looks like
| What happened | How long it took |
|---|---|
| Dead node removed from the answer | About 30 seconds |
| Node restored, back in the answer | About 5 seconds |
| A visitor whose computer had already asked | Up to the TTL on top, so about a minute |
Removal is deliberately slower than restoring. A node is only taken out after it has failed for a while, so a brief hiccup does not move everybody, and it is put back as soon as it is well.
Common questions
What TTL should I use?
Thirty seconds is a sensible low number and is what the software suggests. Going much lower mostly buys extra lookups without moving anybody faster, because resolvers and browsers have their own ideas.
Can I use this for the apex of my domain?
The name at the very top of a domain cannot be a CNAME, so an apex needs addresses rather than an alias. Either delegate the apex, which moves everything including mail, or use your provider's own failover feature for that one name.
What if my domain host will not let me create glue records?
Then delegation will not work, and this feature cannot help. Most hosts do allow it. It is worth checking before you start.
Step by step instructions
The how to section has searchable, task shaped answers. Search it for dns failover.