Websites use a system called DNS to tell users how to access their website. It’s like a giant phonebook of all the websites in the world. When you create a new website, you put an entry in the phonebook, and then when someone wants to visit a website, they go to the phonebook for information on where that website is located and then can access it.
Although it sounds like a great system, it means that there is a single point of failure. If your phonebook is not accessible, no one can access your website (unless they know the server it’s located on – which is very unlikely).
Building a super-reliable system means that you want to avoid single points of failure.
Unfortunately last night, our DNS provider was the victim of a DOS attack. A DOS attack is a Denial of Service attack. It’s where an attacker organises or imitates a huge number of visitors all trying to access the same thing at once. Just like a motorway, the system becomes clogged up if there are enough happening at once. You may remember that a group of hackers managed to do the same thing to some US Government websites (including the Department of Justice and the FBI) in January 2012 (info).
As DNS is a single point of failure, the attack on our DNS provider brought our websites down because the DNS provider wasn’t able to deal with it effectively. The larger the company, the more attractive a DOS attack is for hackers, so the more systems they have in place to deal with one, and the harder it is to successfully bring their systems down. Our DNS provider was not that large, so was brought down last night, and is continuing to have issues now.
As soon as this began to happen, we were notified of the issue and began switching to a significantly bigger DNS provider which a much more stable system. As it happens, I was in the Olympic Stadium watching the last night of the Paralympics, but our server admin genius Callum was working hard to resolve the problem.
We learnt an important lesson last night and I feel confident now that our DNS service is significantly less likely to be brought down again.