The bare minimum a distributed system developer should know aboutDNS

time to read 4 min | 737 words

DNS is used to resolve a hostname to an IP. This is something that most developers already know. What is not widely known, or at least not talked so much is the structure of the DNS network. To the right you can find the the map of root servers, at least in a historical point of view, but I’ll get to it.

If we have root servers, then we also have non root servers, and probably non root ones. In fact, the whole DNS system is based on 13 well known root servers who then delegate authority to servers who own the relevant portion of the namespace, you can see that in the diagram below. It goes down like that for pretty much forever.


Things become a lot more interesting when you start to consider that traversing the full DNS path is fast, but it is done trillions of times per day. Because of that, there are always caching DNS servers in the middle. This is where the TTL (time to live) aspect of DNS records come into play.

A DNS is basically just a distributed database with very slow updates. The root servers allow you to reach the owner of a piece of the namespace and from that you can extract the relevant records for that namespace. All of that is backed with the premise that DNS values change rarely and that you can cache them for long durations, typically minutes at the low end and usually for days.

This means that a DNS query will most often hit a cache along the way and not have to traverse the entire path. For that matter, portions of the path are also cached. For example, the DNS route for the “.com” domain is usually cached for 48 hours. So even if you are using a new hostname, you’ll typically be able to skip the whole “let’s go to the root server” and stop at somewhere along the way.

For developers, the most common usage of DNS is when you’ll edit the “/etc/hosts” to enable some scenario (such as local development with the real URLs). But most organizations has their own DNS (if only so you’ll be able to find other machines on the organization network). This include the ability to modify the results of the public DNS, although this is mostly done at coffee shops.

I also mentioned earlier that the the map above is a historical view of how things used to be. This is where things gets really confusing. Remember when I said that a DNS is mapping a hostname to IP? Well, the common view about an IP being a pointer to a single server is actually false. Welcome to the wonderful world of IP Anycast. Using anycast, you can basically specify multiple servers with the same IP. You’ll typically route to the nearest node and you’ll usually only do that for connectionless protocols (such as DNS). This is one of the ways that the 13 root servers are actually implemented. The IPs are routed to multiple locations.

This misdirection is done by effectively laying down multiple paths to the same IP address using the low level routing protocols (a developer will rarely need to concern themselves with that, this is the realm of infrastructure and network engineers). This is how the internet usually works, you have multiple paths that you can send a packet and you’ll chose the best one. In this case, instead of all the paths terminating in a single location, they’ll each terminate in a different one, but they will behave in the same manner. This is typically only useful for UDP, since each packet in such a case may reach a totally different server, so you cannot use TCP or any connection oriented protocols.

Another really interesting aspect of DNS is that there really isn’t any limitation on the kind of answers it returns. In other words, querying “” will give you back, even though this is an entry that reside on the global internet, not in your own local network. There are all sort of fun games that one can play with this approach, by making a global address point to a local IP address. One of them is the possibility of issuing a SSL certificate for a local server, which isn’t expose to the internet. But that is a hack for another time.

More posts in "The bare minimum a distributed system developer should know about" series:

  1. (20 Nov 2017) Binding to IP addresses
  2. (15 Nov 2017) HTTPS Negotiation
  3. (06 Nov 2017) DNS
  4. (03 Nov 2017) Certificates
  5. (01 Nov 2017) Transport level security
  6. (31 Oct 2017) networking