I often run into a lot of misunderstandings when it comes to domain registrations, DNS, and website launches. Why does it take 24 to 48 hours to change websites over sometimes? Why can’t I transfer this domain?
Domain registration (who owns the domain)
Domain registration is the issue “Who owns and controls domain xyz.com”. You can buy a domain at a staggering number of places – they are called “domain registrars” and there are thousands of them. The most important domain extension, “.com”, can be bought and sold by anyone, and it is. But other extensions (the technical name for them is TLD or “top level domain”) are more restricted. The two-letter extensions represent countries, and each country can treat it however they want, so it’s decided by national politics how this domain can be registered. Some are very restrictive – you can only register a .fr domain if you live in France or are French. This is circumvented by countless domain registrars who will happily register your .fr domain on their French address for an additional fee.
Others, like the “cole.st” domain that I registered as a joke years ago (Cole Street –> cole.st, get it?), choose to let anyone buy, but you have to do it on their website (no resellers allowed) and they charge a ton for it. The .st domain is some banana republic, so it makes sense they try to make a buck on an asset that has global value.
In practical terms, owning a domain means two things: that you can sell it, by transferring it to someone else, or you can use it, by setting up the DNS records to point to your web servers and mail servers. More about that in a moment. Again, each domain extension is allowed their own rules, but the .com domain has a restriction on transfers – you can only do it 90 days after obtaining it. This prevents some of the worst speculative trading in them, as a lot of people register .com domains by the boatload, in the hope they will be worth something one day.
Owning a domain is controlling a domain
Owning a domain means you can set up the DNS. So what does that do? DNS, simply put, is turning a hostname like www.colestreet.com into an IP address.
So a domain can contain hostnames (hostname1.colestreet.com, hostname2.colestreet.com), subdomains (hostname1.media.rudebaguette.com), and MX records for email, which I will talk about later.
The IP address is often compared to a phone number – it has four numbers between 0 and 255 (for now!) that uniquely identifies a physical server somewhere on the Internet. In the 60’s, when this part of the Internet infrastructure was conceived, it was already seen that phone numbers (and IP addresses) were a ridiculously cumbersome thing, and there should be something more elegant. DNS was the (60’s) answer to that problem. Instead of typing in 18.104.22.168, you can type in www.colestreet.com, and your computer knows what IP address to connect to. The ‘lookup’ (translation from hostname to IP address) happens invisibly. So DNS can do several things: it can identify a hostname on a domain (like www.colestreet.com but also ftp.colestreet.com ), but it can also tell you where to deliver email! Those records are called ‘MX’ records (normal host lookup records are A records).
So to have a website and email running on your own domain, you need the following things:
- Ownership (or at least control) of the domain
- DNS records hosted somewhere, that the domain registration will point to (usually offered for free by your domain registrar and your web hosting place, take your pick)
- A web server hosted somewhere, that the DNS records will point to (can be GoDaddy or anything else)
- A mail service that the DNS records will point to (can be Google Apps or something inferior)
So when you own a domain, you can specify where the DNS records for your domain are located, by entering (usually two) “name servers”, who are the DNS servers who contain your records. Where are those for most people? If you have a website, in 90% of the cases the hosting company that hosts your website, also hosts (and manages) your DNS records, without you knowing it. In other cases, the domain registry place you used to buy the domain will have it. This is very common for “mail only domains”, where you own a domain only to use it for email.
A bit of Internet history
Back in the day, before the Web existed, the Internet was mainly text-based services, such a Usenet news groups (who were legendary for their high concentration of smart and knowledgeable people on them before they were overrun by mainstream internet users). Email was already very popular, even if today you would barely recognize it – mail clients like Pine were text-based, a bit like Minitel, and half of the time the backspace key didn’t work. But there were standards for domain hostnaming – news.domainname.com for the Usenet, mail.domainname.com for email, gopher.domainname.com (Gopher was an early text-only predecessor to the Web that died a quick death when the Web became popular). When the Web was conceived, it was the norm that the web server would be www.domainname.com . So these days, when you set up a web server for a domain, you point the www record to the web server.
So how do we set up a website?
So, what happens when you buy a domain? It will be registered in your name and you can set up the name server records. But where to point those to? Well, odds are you’ve already identified where you will host your website. Log into the Control Panel that they gave you, and identify the name server information – usually two pairs of hostnames and IP addresses. You plug that into the domain registration service, and wait the famous 48 hours! That’s all it takes.
So how do we migrate a website from one server to another?
Now we’re in a more fun situation. We’re trying to repoint the “www” record in our DNS records from one IP address to a new one. But – there’s a delay on making the changes visible to the world. Those DNS values, imagine how many of those lookups are made at any given time. Every time some types in www.colestreet.com, the computer that that address is typed into, gives the command to translate that. But who knows that info? Only the DNS server that we have at our cheap hosting place. So every time, that server would have to be consulted. This would be undoable of course, and back in the 60’s, they decided that a copy of those value can be stored on any system that gets their hand on it. So the second time you type in www.colestreet.com, your computer doesn’t have to go look up the translation anymore! It’s already stored on our computer, saving massive amounts of internet traffic. This is a storage mechanism like that is called “caching” and it’s an incredibly important concept in computer networking.
How does caching work and why do we need it?
But what if the value changes? How will my computer know not to use that locally stored copy? This is why we remove cached value after a certain amount of time. We call that “expiration”. But how much time? This is defined on the place where the DNS records live, and is called the “TTL value” (time-to-live). So you get to define it yourself, which is great. The default setting is 86400, which is 24 hours expressed in seconds. So when you want to change your records, you simply change that value to 5 minutes (300 seconds). But hey – all the cached copies around the world still remember 86400! The solution for that is to change your TTL value 24 hours in advance from 24 hours to 5 minutes. By the time you make the actual change, every cached copy will only live for 5 minutes. So when you change over that www record to the new IP address, the whole world will know in 5 minute! After the migration, you change it back to 24 hours.
Why even change it back to 24 hours afterwards? Because if the value is 5 minutes, it results in more “real” lookups, that cost internet bandwidth, and the user experiences a slight delay when “hitting” your website. This delay is around 50 milliseconds on a modern high-speed internet connection, but on 3G and other crap connections can be a full second. Also, your ISP is not a fan of handling the extra traffic, so they often monitor the DNS records of their customers, and send warnings to customers with short ones.
So now your website migration plan looks like this:
- 48 hours in advance: Change TTL value to 5 minutes
- Migration time: Change DNS record ‘www’ to the new IP address, test, confirm
- Post-migration (usually a few days after): Change TTL value back to 24 hours
How to set up email for my domain
As I mentioned before, the DNS records control where email is delivered. They are called the MX records. Simply put, if your mail provider has two servers where incoming mail can be delivered, with one as the primary, and the other one as a backup, you would have two MX records. How do we know which one to deliver to? Each MX record has a priority associated with it. We start with the lowest value (highest priority), and if that server does not respond, we move on to the next.
I’ll give one practical and relevant example: Using Google Apps for your email. First of all, I can’t recommend this enough – Google rules the email universe, and is FREE(!!) and awesome, and very flexible – you can still use your cruddy old email client if you insist.
So the first step is to sign up for the free Google Apps. Careful not to get suckered into the 30 day trial of the paid edition – they try. They ask you for the domain name when you sign up. Enter your newly purchased domain. Google will then prompt you to set up the MX records, with detailed instructions on how to set it up. And sure enough, they show a set of mail servers, and their priorities. You plug that into your DNS records, and you’re good to go! Google will verify the records automatically, and before you know it, you’re set up and ready to look like an internet professional!
This concludes our article about domain registration and DNS. There are many more intricate details, and there are excellent books written on the subject. The native DNS server built into every UNIX (including Linux) server is called BIND, and the O’Reilly book ‘DNS and BIND’ about has been the standard reference work for many years now, and makes an excellent read for anyone who wants to understand one of the most critical parts of the Internet infrastructure.