Rick Reed of WhatsApp explains at Erlang Factory 2014 how the service scaled to billions of messages with a very small engineering team. He covers the FreeBSD and Erlang stack, Mnesia usage, server counts and bottleneck work.
Why Hacker News points people here: The go-to talk showing how Erlang let a tiny team run a massive messaging system.
“watch this talk and imagine trying to recreate that system with only a handful of server engineers”HN comment, 2017
“how small their team was compared to the volume and complexity of the operation.”
Tiny team handling enormous volume and complexity
Full comment
My other favorite "scalability" study is from WhatsApp: this video That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp (that walk title was create before the acquisition and was mean to imply message count, after the acquisition it got a secondary meaning). The one thing that is fascinating about it, is how small their team was compared to the volume and complexity of the operation.
Strongly recommendskornishJul 2017
“watch this talk and imagine trying to recreate that system with only a handful of server engineers”
Specialized tech enabling a handful of engineers to run huge system
Full comment
Not sure why this comment saw a couple downvotes earlier. mbesto is correct: for most startups, most of the time, competitive advantage doesn't come from the underlying tech stack. To make a general statement, most things could be done similarly on any of several platforms. However, when product requirements match exceptionally well with a specialized technology, you can see things that would simply be infeasible or extremely tough using a different stack. WhatsApp + Erlang was one of those cases (watch this talk and imagine trying to recreate that system with only a handful of server engineers using any other tech: this video ). Discord + Elixir appears to be another. Curious if anyone has any examples that spring to mind from outside the highly concurrent messaging space.
Strongly recommendskornishSep 2020
“One of my favorite talks is by Rick Reed about scaling Erlang at WhatsApp.”
In-depth system performance and bottleneck mitigation curriculum
Full comment
One of my favorite talks is by Rick Reed about scaling Erlang at WhatsApp. What an absolute savage. He flies through an articulate and in-depth curriculum on system performance and bottleneck mitigation. The talk is called "That's 'Billion' with a 'B'" and makes for a great lunchtime watch: this video
RecommendsrdtscOct 2014
“Here is a video of Rick Reed talking about how they did it”
How WhatsApp made Mnesia work with a small team
Full comment
Unrelated, but maybe interesting: WhatsApp was running Mnesia and made it work fantastically. Especially with such a small team of engineers. Here is a video of Rick Reed talking about how they did it (hint it is not a one monolithic Mnesia cluster). this video
RecommendsrbsnNov 2014
“here is a presentation given at Erlang Factory 2014 about their goals with scaling WhatsApp”
WhatsApp's goals for scaling to billions of users
Full comment
Just to follow up on topic of WhatsApp and Erlang, here is a presentation given at Erlang Factory 2014 about their goals with scaling WhatsApp to billions of simultaneous users. this video
RecommendsrdtscSep 2015
“He explains how the product works”
Rick Reed explains how the product works
Full comment
Not "parent" but search for Erlang Factory talks they talk about it. Here is one from Rick Reed. He explains how the product works : this video
RecommendsddeckMay 2016
“Their technology stack (FreeBSD/Erlang) is pretty interesting, more info here: 2014 talks by Rick Reed”
Interesting tech stack with huge scale on limited hardware
Full comment
It's required because your phone is where your messages are stored. Whatsapp don't retain messages/media after they've been delivered to your phone, which is a compelling privacy feature for many. It's also what allows them to serve such an enormous user base with limited hardware. Their technology stack (FreeBSD/Erlang) is pretty interesting, more info here: 2014 talks by Rick Reed: this videolink Slides: link There's likely no technical reason why you couldn't use a pc instead of a phone for users that want to use the pc as the primary client (with the phone optionally accessing the DB on the pc in the same way that the desktop client does for the phone). Perhaps they've decided that this is a small and declining market. Edit: Slides for second talk link 440k connections/sec, 1.1 million msgs/sec, 1 billion images/day, and that was in 2014...
RecommendsotterleyMar 2017
“Here's a great talk from Rick Reed”
Great talk on scaling WhatsApp's Erlang infrastructure
Full comment
Here's a great talk from Rick Reed (a former mentor of mine at Yahoo!) about how WhatsApp scaled their Erlang infrastructure: link Here's another talk: this video
RecommendsrdtscSep 2017
“Here is the description of their stack and how Erlang helped them by Rick Reed at Erlang Factory”
Description of the WhatsApp stack and how Erlang helped
Full comment
Here is the description of their stack and how Erlang helped them by Rick Reed at Erlang Factory: this video Here is Jamshid Mahdavi talking about their stack as well but focusing on development, testing and shipping code. Some of the stuff they do will surprise you if you've been at a shop with a large QA team and deployment pipelines with many stages. link
RecommendsdpeckAug 2022
“is a good glimpse into their own view of it”
Glimpse into WhatsApp's own view of its early tech decisions
Full comment
WhatsApp. Rick Reed’s “That’s Billion with a B” presentation is a good glimpse into their own view of it this video
Cites itddeckJan 2015
“Details on their architecture”
Details of WhatsApp architecture
Full comment
I believe the issue is that unlike web chats and apps like Telegram, Whatsapp doesn't store conversations on their servers beyond what is required to deliver messages to the required destination device(s). This presents obvious problems with having not just a web client, but any client other than the main device, since the main device is the only store of the existing conversations. New messages sent also need to be synchronized across devices, which I presume is why it's required to keep your phone connected to the Internet. Otherwise Whatsapp would need keep conversations on their servers until they could be synced, which is very much not their model. Details on their architecture: this videolink
Cites itrdtscJul 2015
“Here is an talk on how WhatsApp uses Mnesia”
How WhatsApp uses Mnesia
Full comment
Klarna (the European payment processor) and WhatsApp architectures are built on Mnesia. Mnesia is Erlang's built-in distributed database. Due to its age it has a few bad corner cases, that could be fixed if it just had a better backend (the front-end API is nice, well designed, supports transactions, is built-in to the lanauge). So this effort brought in the ability to have a better backend, and make Mnesia a better option as a general purpose distributed database. Here is an talk on how WhatsApp uses Mnesia: Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp this video Here is an example of using Mnesia: link
Cites ithellbannerAug 2017
“WhatsApp Scaling to 1 Billion users in erlang”
Evidence Erlang is proven at massive scale
Full comment
Yes, I understand, I was just wondering if Erlang would be considered a "proven language". From my (limited) Erlang experience, Erlang & Elixir have the same core functionality. Elixir has a "nicer" syntax (I prefer Erlang's syntax but most devs I've talked to like Elixir's). So if Elixir isn't proven but the concurrency model & programming paradigms make sense, maybe Erlang is a good choice. * WhatsApp Scaling to 1 Billion users in erlang: this video * link
Cites ithellbannerAug 2017
“I haven't watched but this would be a good guess”
Guess at relevant talk, unwatched
Full comment
I haven't watched but this would be a good guess this video "Scaling with A B: Erlang and WhatsApp"
Cites itkbensonMay 2018
“he talks about ~550 servers, which includes 250 multimedia servers and 150 chat servers.”
Server counts and architecture figures from the talk
Full comment
There's a talk one of the engineers gives[1] (that someone else posted here so I'm watching it) about their architecture that's published in March of 2014, and in it he talks about ~550 servers, which includes 250 multimedia servers and 150 chat servers. The only places he mentions 16 is when he talks about the "multimedia database". I think there were 16 sharded database servers. 1: this video
