The Facebook, Instagram and WhatsApp outage

I was trying to understand how ‘Facebook disappeared from the internet’ here: https://blog.cloudflare.com/october-2021-facebook-outage/

Anyone’s got links to even simpler posts to understand what really goes on?

5 Likes

There are conflicting theories about what happened (hackers, Facebook taking it down for back-end work etc). Not sure if more information has been released/found!

2 Likes

I think we should all keep copies of our Facebook data with us now :smiley:

2 Likes

See this tweet :slight_smile: @ananditam https://twitter.com/sidagarwal/status/1445078327639691270

1 Like

I’d suggest checking this video out, I found it interesting.
Why Did Facebook Go Down? - Computerphile

3 Likes

I will watch this, thank you!

So, I will try to explain what happened in as layman terms as possible. I have studied a few blog posts to understand what happened.

The crux of the problem lies with 2 protocols that mostly are responsible for running our internet as we know it - BGP and DNS. BGP stands for Border Gateway Protocol. You can think of BGP as a protocol that connects multiple autonomous systems. You can think of an autonomous system as a post office. When you send a letter through a post office, what happens is that the letter will be routed through multiple intermediary post offices, before it reaches the destination post office. Similarly, what happens is when a network packet actually goes out of the sender, it typically goes through a network of autonomous systems before it reaches the final destination. Now, each of these autonomous systems must continuously advertise its existence to other autonomous systems so they know if they can send packets to the other autonomous system. (Analogous to post offices, post office A must know if post office B is operational, otherwise it will send the post to post office C). Typically these autonomous systems are owned by big ISPs.

Now coming to DNS (Domain Name System), what happens is when you type http://www.facebook.com, it has to resolve to an IP address since IP addresses are what differentiates an internet service from another. IP addresses are composed of four individual numbers (ranging from 0 to 255) connected by a ‘.’ (like 2.2.2.2). Because it is humanly impossible to remember IP addresses, the DNS protocol allows you to assign human readable names to services. Think of the DNS as another computer that exists only to perform this translation of human readable names to IP addresses for you.

Typically, this DNS computer also must notify of its existence to the Autonomous system that will communicate to it. Basically, this DNS ‘post-office’ will tell the Autonomous system post-office that asks if it is open for sending posts or not through an advertisement. This is what you call DNS advertisement through BGP. If the advertisement is successful, the autonomous system will know that the DNS computer is up. One important point to note here is that it isn’t enough for just the DNS computer to be up, it is equally important for the advertisement to be successful. Only if the advertisement is successful, only then will autonomous systems will know where to route the incoming packet that asks “resolve www.facebook.com to an ip address”.

In the case of Facebook what happened is that a network of backend services that house (facebook, whatsapp and instagram) suddenly went down due to some misconfiguration. So this is internal to Facebook and we don’t entirely know what that misconfiguration was. Now when that misconfiguration happened, Facebook decided to stop the DNS advertisement to autonomous services as well. Note that the DNS service/computer was up. But it stopped advertising its existence to the autonomous system that is responsible to communicate with it (the Cloudflare autonomous system). Now because the advertisement stopped, the Cloudflare autonomous system didn’t really know where to send the network packet asking “resolve www.facebook.com to an IP address”. So basically when you were typing www.facebook.com to your browser you were not able to connect to it because the IP address that www.facebook.com represents wasn’t being identified.

3 Likes

Facebook is a profoundly ‘evil’ company and the way they vehemently push advertisements down our throats is pretty well known. I’m almost certain they’ve sold all of our data already and maybe given it off on the black market.

Woah that is some explanation! Thanks, Arnab!

1 Like