Discord Blog

I absolutely agree, Rust and Elixir are an amazing couple of languages! I recommend reading https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-…

Discord | Senior Engineers | San Francisco | ONSITE Discord is building a platform dedicated to connecting people through games. We grew from 45 million to 150 million users in the last year. We have over 20 million daily active users and that number is growing every day. Discord is just 3 years old. We're hiring senior engineers in the key roles of backend, native, data science, and data infrastructure. Join an amazing team solving interesting scaling problems and creating next generation features. https://discordapp.com/jobs Tech: Rust, Elixir/Erlang, React, Javascript, Python, Go, C++, Cassandra, GCP Engineering blog: https://blog.discordapp.com/tagged/engineering Feel free to message us directly at [email protected].

They have some fantastic dev blogs[1]. Their backend seems to mostly be elixir. 1: https://blog.discordapp.com/tagged/engineering

I think while this is great, it is good to remember that your current tech stack maybe just fine! after all, Discord start with mongodb[0]. [1]. https://blog.discordapp.com/how-discord-stores-billions-of-messages-7f…

Stuff like this is great! And you said you mentioned you are primarily doing it for timeseries data - then you can easily batch writes to handle high volume throughput too! We did this with S3 as the storage engine, 100M+ records for $10/day: https://www.youtube.com/watch?v=x_WqBuEA7s8 And discord had a very nice article on this as well: https://blog.discordapp.com/how-discord-stores-billions-of-messages-7f… Great work, I think there is a lot of exciting stuff you can add to it!

My vote would go to Discord's blog -- https://blog.discordapp.com/

https://blog.discordapp.com/how-discord-handles-push-request-bursts-of… Erlang has its uses and Elixir has its uses. Dismissing Elixir's userbase as 'pathetic' is pretty rude and a disservice to a pretty incredible language. See the Discord article where they clearly demonstrate Elixir and Genstage are ready for production and can be used to write performant services. I get that you really like Erlang and its quirks, but maybe there are more mature ways of expressing this love than calling Elixir users 'pathetic'.

The unofficial API community is also really great, and there are a plethora of libraries to choose from to get something working quickly. https://blog.discordapp.com/the-robot-revolution-has-unofficially-begu…

It's interesting to compare Slack's tech stack vs Discord. The tech stack of slack is pretty common, while the tech stack of Discord is very geeky: Elixir/Rust + Cassandra for crazy high concurrency, and React/React Native for fast developing and consistent experience. Like LAMP for Slack, Discord also used MongoDB prototyping in the first few years, but while they prototyping they prototype in a manner they can easily migrate to Cassandra later. Then they successfully migrated to Cassandra. This is brilliant. I remember the React community migrated from Slack to Discord[0], I guess one of the reason could be the performance. According to the blog, at that time the team only have "14 people full time", which means they've got a very high velocity, amazing. I didn't find any summary of their whole tech stack but there are several insightful articles from their blog like[1]. [0] https://reactjs.org/blog/2015/10/19/reactiflux-is-moving-to-discord.ht… [1] https://blog.discordapp.com/how-discord-stores-billions-of-messages-7f…

Disclaimer: Using the following methods could be seen as bypassing the intent of the port restrictions set by your org. I commit to github using ssh on port 443. # git remote set-url origin git@<host-in-ssh-config>:<username>/<repo> From ~/.ssh/config Hostname ssh.github.com Port 443 User git IdentityFile ~/.ssh/.github Consider using Dovecot on your mail server for port 443. I was not aware that Discord stopped supporting websockets over 443. I run my own Mumble voice chat server so I can put it on 443, but I understand people have a lot of friends on Discord. I am just not happy with their chat logging policy [0]. You could also run a proxy or VPN on port 443. You can also use sslh [1] to have SSH+SSL+Proxy+VPN all on the same port 443. You can do similar things using HAProxy + SNI + SSL, but that is a bigger subject. For other things that must use other ports or protocols, you can sometimes wrap them using tsocks [2] in your VPN or SSH tunnels to port 443. [0] - https://blog.discordapp.com/how-discord-stores-billions-of-messages-7f… [1] - https://github.com/yrutschle/sslh [2] - https://www.redpill-linpro.com/sysadvent/2015/12/13/socks-proxy-as-poo…

This blog post from 2018 from Discord indicated that the browser implementation of WebRTC wasn't as good: https://blog.discordapp.com/how-discord-handles-two-and-half-million-c… Is this still the case? Would you still hypothetically see better audio performance of something like Discord in a standalone app?

I guess this mentality leads to the current state of play where all UI related latencies are out of the roof compare to the 80s. At work, I deal with systems that require 16G of heap as the minimum. Funnily when things get rewritten in Rust providing the exact same functionality and the same or better performance the memory requirement goes down 10x (or more). It is up to us how much garbage our systems producing, how much CO2 is wasted on this. I guess many of us are ok with it. While some of us are not. https://blog.discordapp.com/why-discord-is-switching-from-go-to-rust-a…

Discord is a heavy Elixir user, and even uses it with Rust via NIF: https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-…

It should also be noted that Rust interoperates extremely well with Erlang, which is the basis of Discord (via Rustler). https://github.com/rusterlium/rustler https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b

>Are there any engineering blogs that talk about using BEAM/Elixir/et al in production? Blogs from the Discord team pop up occasionally. They're primarily an Elixir shop with some Rust thrown in (via NIF w/ Rustler) https://blog.discordapp.com/tagged/elixir

Yep. The Financial Times: https://github.com/Financial-Times/tapper Discord: https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-… Pintrest, who seems to really like it (note the highlighted comment): https://medium.com/pinterest-engineering/introducing-new-open-source-t… And looking on https://elixir-companies.com/en I see two in the UK, two in Brazil, Amsterdam, Ukraine, Canada...Plenty there.

They already tried[0], but the dedicated store page got removed (now devs. can allow people to download games via their discord server/guild). 0: https://blog.discordapp.com/the-discord-store-beta-9a35596fdd4

This is common and necessary for WebRTC SFUs, which perhaps is why Discord does it to support the least common denominator of their web browser based clients. Edit: Yep, I thought I remembered reading this. Their voice servers are WebRTC SFUs. So this is basically state-of-the-art when it comes to voice over WebRTC. End to end encryption in WebRTC is not possible if you are using a SFU. https://blog.discordapp.com/how-discord-handles-two-and-half-million-c…

They were announced within months of each other [1][2]. That means both companies were planning their stores at the same time. I find it hard to believe Discord did not have any idea about Epic's store, considering they were simultaneously raising a funding round and talking to VCs about their plans (who you would think would be aware of Epic's move). I suppose it's possible Epic played their cards close to the chest though. [1] Discord Announcement https://blog.discordapp.com/the-discord-store-beta-9a35596fdd4 [2] Epic Announcement https://www.unrealengine.com/en-US/blog/announcing-the-epic-games-stor…

I find the "high throughput" comment a bit weird, but i've seen it a lot of times. Do you mind drilling into it a bit more? I see posts like this https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b where they are talking millions of events per second. I also work in the distributed messaging space and so far my experiments have shown elixir to be able to keep up with our scala stack with the advantages of no thread tuning and much easier debugging. So I'm curious why this keeps popping up.

WhatsApp stored no data and ran everything as through service, that's why they did it with so few engineers. Almost no games ever reach scale close to what you just wrote. The few that do are known with large technical teams. Same goes to everything else. The reason why you can name them almost all is because it's so rare. You are grossly overblowing the numbers. Only a very few services ever get to these numbers and once they do, they talk about the challenges of scale constantly [0] [1] [2] [0] https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-… [1] https://blog.roblox.com/2012/04/supporting-millions-of-players-at-robl… [2] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-fa…

Discord uses Rust via Elixir/BEAM NIFs to scale to 11 million concurrent users. Big enough for you? https://blog.discordapp.com/using-rust-to-scale-elixir-for-11-million-…

>How do you make optimal use of a computer's memory hierarchy? Apparently more easily, with a faster time to market, and with less errors than alternative languages. E.g: https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b https://www.wired.com/2015/09/whatsapp-serves-900-million-users-50-eng… But the idea is not that you as the end programmer "make optimal use of a computer's memory hierarchy". It's that you write in a way that makes it easy for the runtime to ensure what you do is solid and correct and scalable -- and it's up for the runtime makers to make sure it makes optimal use of the computer's memory hierarchy. And it's also that you sacrifice some of that "optimal use", to get the "solid and correct and scalable" part.

> cost nothing Nobody thinks it cost nothing, it's the draconian 30% they object to. Other app stores are offering similar services, with a much lower cut: https://www.unrealengine.com/en-US/blog/announcing-the-epic-games-stor… https://blog.discordapp.com/why-not-90-10-3761ebef4eab

I'm happy for Discord since I know they're a big user of Elixir [0]. I'm sure it will make the Elixir language stronger and will make it a strong candidate for apps in the future. Much like how Twitter/GitHub did for Rails. [0]: https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b

They give it rave reviews: https://blog.discordapp.com/why-discord-is-sticking-with-react-native-…

That’s what I’ve read too: https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b

The Discord engineering blog post this week has a fascinating sentence on this topic, perhaps much more interesting to me than the engineering itself: > Every audio/video communication in Discord is multiparty. Supporting large group channels (we have seen 1000 people taking turns speaking) https://blog.discordapp.com/how-discord-handles-two-and-half-million-c…

This is how we will make money. https://blog.discordapp.com/the-discord-store-beta-9a35596fdd4 Aside from the money we already make from https://discordapp.com/nitro :)