Nick Craver

Their best engineer went to work for Microsoft https://nickcraver.com/ As did their SRE SQL Server DBA. I don't know if their replacements are up to speed.

OIDC integration, RBAC, data persistence/replication across nodes, observability, mTLS. And yes, you're right, all these things are complex in any situation, except when you simply use a load balancer and two servers. There are companies estimated to be worth close to US$2B using with sub 30 servers using a service you may have heard of: Stack Overflow (https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-201… && https://nickcraver.com/blog/2016/05/03/stack-overflow-how-we-do-deploy… (As an aside: K8s does not free you from your Cloud vendor - after floating around my city working on various "platforms", all locked in due to IAM, for example.)

For people who enjoyed this post but want to see the other side of the spectrum where self hosted is the norm I'll point to the now classic series of posts on how Stack Overflow runs its infra: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… If anyone has newer posts like the above, please reply with links as I would love to read them.

Minor correction but that’s Nick Craver https://nickcraver.com/

To add to OP, Nick Craver (formerly @Stack Overflow) wrote an insightful series [1] (2016) on Stack Overflow’s architecture. [1] https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture…

I guess I am missing something, why does this cause dependency hell? I guess microsoft is basically recommending against targeting netstandard now anyways so there must be some problem with it. But I don't think I have run into any problems with the situation you described. edit: Ok I see, it seems in that scenario loading the correct assembly is more difficult on dotnet framework and usually requires binding redirects to be specified. On dotnet core/.NET 5+ there are no issues though, which is why I haven't experienced this problem myself. https://nickcraver.com/blog/2020/02/11/binding-redirects/#the-fix-if-y…

Back to on prem: https://www.economist.com/business/2021/07/03/do-the-costs-of-the-clou… Battle of cloud is intensifying: https://www.economist.com/business/the-battle-of-the-computing-clouds-… HIGHLY recommend looking at Stackoverflow architecture: https://stackexchange.com/performance https://nickcraver.com/blog/2016/02/03/stack-overflow-a-technical-deco… Technology and society has ways to build up layers of abstraction. It takes some realization, hindsight, genius and humbleness to cut down the overgrowth and pick the fruits, plant new seeds.

Most of my favorite blog posts are from individuals rather than companies, but here are a few that are fully or basically from tech companies: - https://engineering.linkedin.com/distributed-systems/log-what-every-so… (pretty long but very lucid) - https://web.archive.org/web/20170612053125/https://blogs.dropbox.com/t… (can't find it on their blog anymore) - https://nickcraver.com/blog/2016/05/03/stack-overflow-how-we-do-deploy… (personal blog but clearly about Stack Overflow)

IMO your assertion is validated by the excellent overview of Stack Overflow's infrastructure given here: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… Very few web apps will ever serve as much traffic as SO.

I loved the latency numbers posted by one stackoverflow engineers Nick Craver: https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-ca… L1: 1.3ns L2: 3.92ns (3x slower) L3: 11.11ns (8.5x slower) DDR4 RAM: 100ns (77x slower) NVMe SSD: 120,000ns (92,307x slower) SATA/SAS SSD: 400,000ns (307,692x slower) Rotational HDD: 2–6ms (1,538,461x slower) Microsoft Live Login: 12 redirects and 5s (3,846,153,846x slower, approximately) “How many Microsoft Live Logins before this thing loads?”

I think there are plenty of examples of teams that hit that cliff and had to rewrite. I am trying to remember specific articles. Counter-argument: I think stackoverflow hit a cliff, and remained monolithic, but that was several years ago [1] (sure they have a ton of servers, but they appear to be classic cache+web-server+database). There was another, maybe Instagram, that was monolithic and had to figure out how to shard their DB overnight. [1] https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-201… Also, I don't think your post deserved a down-vote.

Maybe just a couple. If you google the StackExchange Hardware Setup, you will find they use surprisingly little Hardware. Back in 2013, they could go with 2 single webservers not accounting for failover reduncancy [0]. They have upgraded since, but still, if you consider the amount of hits they get, they run a pretty humble hardware [1]. One of the arguments I make when people say microservices + cloud built to the core is the only way to scale - clearly a cleverly architected approach can save you lots on hardware/hosting money. [0]: https://nickcraver.com/blog/2013/11/22/what-it-takes-to-run-stack-over… [1]: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture…

[Nick Craver](http://nickcraver.com/) and on twitter [@Nick_Craver](https://twitter.com/Nick_Craver)

I know Stackoverflow and all their related services are running bare-metal and has done so for a long time. They have written a very detailed series of blog posts about their whole infrastructure and hardware which I really recommend that you read if you haven't already. Here's Part 1 of the series: http://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-… (the other parts are linked from there)

Ironically, in another "universe", StackOverflow doesn't have either part of this problem - they use the DB recommended by Microsoft and the OS recommended by Microsoft :) http://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-…

If you haven't seen stackoverflows server architecture posts you'd probably enjoy them: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… I still think containers are great, and I really like the abstractions kubernetes provides, but if I ever had enough traffic to worry about scaling, I envision running a small cluster of very powerful computers rather than 100s of weak ones.

Before anyone wants to criticize Stack Overflow, consider their server footprints as of 2016 [1]: 4 Microsoft SQL Servers (new hardware for 2 of them) 11 IIS Web Servers (new hardware) 2 Redis Servers (new hardware) 3 Tag Engine servers (new hardware for 2 of the 3) 3 Elasticsearch servers (same) 4 HAProxy Load Balancers (added 2 to support CloudFlare) 2 Networks (each a Nexus 5596 Core + 2232TM Fabric Extenders, upgraded to 10Gbps everywhere) 2 Fortinet 800C Firewalls (replaced Cisco 5525-X ASAs) 2 Cisco ASR-1001 Routers (replaced Cisco 3945 Routers) 2 Cisco ASR-1001-x Routers (new!) Assuming they aren't lying, this is the entire list of machines needed for a site used by the world's developers. That's no small feat. Many companies whose developers blog about their "best practices" would struggle to achieve 1/10th as much efficiency. [1] https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture…

See https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… for an overview of their stack. They do NOT need a stack of daily traffic analytics hardware. As I said above, remote RPCs take an order of magnitude more resources than function calls. If you are used to using various distributed tools to do analytics, you are used to requiring an order of magnitude more hardware than a simpler solution. And if there are inefficiencies in your solution, it is easy for a second order of magnitude to sneak in. Therefore your experience is going to seriously mislead you about the likely limits of a simpler solution. Making this concrete, if your job requires a Hadoop cluster of less than 50 machines, odds are good that it could be run just fine on a single machine with a different technology choice. But if you are fully utilizing hundreds of machines in your cluster, then you actually need to be distributed. (Though you could achieve considerable savings for some of your jobs could be run in a more efficient architecture. Amazon in particular makes excellent use of this kind of hybrid architecture in places.) Google is making excellent decisions for the scale that they operate on. You just are unlikely to operate at a scale that makes their decisions make much sense for you.

I'm not sure how many full handshakes/sec we're running - the logging isn't as readily available there because we're explicitly not logging the listeners involved on separate proc binds. We are logging the traffic on the port 80 front end they feed. We have tuned the performance there via best practices and taking into account very recent HAProxy changes for various SSL windows added by Willy, Emeric and help from Ilya Grigorik. They're all awesome and tremendously helpful. We are on a version slightly newer than 1.5.2 (we build straight from the repo). Even with all that SSL we're only running around 15% CPU at peak so it's not having any trouble. Most of that CPU usage does come from the SSL termination though - it ran around 3-5% CPU before. We're also working on much larger infrastructure changes that mean SSL termination much more local to our users, which means the HAProxy load will drop to just about nothing again. I'm working on a followup to my previous SSL post: http://nickcraver.com/blog/2013/04/23/stackoverflow-com-the-road-to-ss… that will explain why it's not already on for all hits...I think hacker news will have fun with hat one. All that being said, there's no reason we can't forward that syslog traffic from the listeners to our logging infrastructure to get at least a counter out of it. If you're curious how we log, I'll explain a bit. We feed it into a logging display called Realog built by Kyle Brandt and Matt Jibson in Go. Here are a few snapshots of…

Blog post from 2016, detailing the setup: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… It's a bit of a shame, but I guess also with declining traffic and revenue, they're also downsizing.

The details of their architecture is documented in a series of blog posts: https://nickcraver.com/blog/2016/02/03/stack-overflow-a-technical-deco… I get what you're saying, they didn't do dynamic and "wild" horizontal scaling, they focused more on having an optimal architecture with beefy "vertically scaled" servers. Very much something we should focus on. These days horizontal scaling, microservices, kubernetes, and just generally "throwing compute" at the problem is the lazy answer to scaling issues.

stack overflow has no problem serving entire planet from just four SQL Servers (https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… There is really nothing wrong with a large vertically scaled up SQL server. You need to be either really really large large scale - or really really UNSKILLED in sql as to keep your relational model and working set in SQL so bad that you reach its limits

It's not linked there (or on any Wikitech pages I can find), but I can imagine there's a secondary concern of *.wikimedia.org cookies getting sent to third parties - e.g. Stack Overflow has separate second-level domains (stackoverflow.email/stackoverflow.blog) for their 3rd-party-hosted email service and blog for exactly this reason (cf. https://nickcraver.com/blog/2017/05/22/https-on-stack-overflow/#certif…

> Sub 20ms response time with zero caching. I mean... to be clear, they do tons of caching[0], which is certainly critical for their ability to have a non-cached response time of 20ms. Most of their responses should be coming from a cache, given the type of site they run, otherwise they would need a lot more servers. [0]: https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-ca…

Back in 2016 at least, Stack overflow was container free https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture… No idea how much has changed since then

StackOverflow is known to operate their own servers on their own datacenters and as far I recently read, they are still tuning with no container tech. https://nickcraver.com/blog/2016/05/03/stack-overflow-how-we-do-deploy…

>You're looking at simple, static pages. Stackoverflow pages have many dynamic components like vote counts, reputation points, sidebar related-questions-links, new comments, etc. An excerpt from their blog explains they can't cache the output : https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-ca… Even though Stackoverflow's website io access pattern has higher reads than writes, the resultant generated html is still not as static as Wikipedia pages. Even if they're efficiently using cpu to generate the dynamic elements, the cost of egress traffic may also be a factor. All that said, I don't have any insight into what heuristics they use to block certain ip addresses.

I remember seeing that Stack Overflow runs on half a rack of servers, plus a redundant copy. https://nickcraver.com/blog/2016/03/29/stack-overflow-the-hardware-201… I bet most people are running sites smaller than Stack Overflow (Alexa rank #65 [did you know Alexa is getting permanently shut down in a few months?!]).