Poul-Henning Kamp

It sucks when this happens, but it's easily avoidable by using a caching frontend of some sort. My favorite is Varnish,[0] which I have used with great success for _many_ web sites throughout the years. Even a web site that 10+ millions of requests per day ran from a single web server for a long time a decade-ish ago. [0] https://varnish-cache.org/

I guess that signals it's time for nginx users to check out possible alternatives - just in case, if things turn out for the worse. I can recommend having a look at https://varnish-cache.org/ - while its performance might not be 100% up to par with nginx in some (very, very high-end) scenarios, it has many other fortes that nginx (at least in its FOSS release version; I've never used nginx plus) just cannot match in my experience. Having seen `varnishlog` and `varnishtest` in action alone are worth spending a day or two exploring it.

You should see Varnish [1] and VCL language. Also recommends to review offers / availability from major players. fastly, keycdn, cloudflare w/o railgun, cdn77 (and the list go on) [1] https://varnish-cache.org/

You can not accurately estimate how much network traffic will be generated. There is nothing from stopping it from becoming immediately viral. Sites have had hundreds of thousands of visitors increases daily. Be prepared. Host the static on s3 etc... Engineer for it. Varnish: https://www.varnish-cache.org/ A decent primer. Failover database servers? Auto scaling on amazon? A Fortune 500 is a big deal. Don't worry. This is a good kind problem to be having. Regardless, the exposure will continue and be referenceable which is great for your site.

It's cached on my server using varnish http://varnish-cache.org, which uses LRU caching with ttls. A very cool open source web caching project. I use a configuration very similar to media wiki where I set long ttls and then have the backend send a PURGE http message followed by a GET to varnish when a file is updated to refresh the cache.

W3 Total Cache really is a first rate plugin, we've had great success throwing Varnish https://www.varnish-cache.org/ infront of WordPress

By the way, the Rails Tutorial site itself is also hosted on Heroku. By setting the appropriate header, I've arranged for Heroku to cache the site's static pages, which on subsequent requests get served with blazing speed (see, e.g., http://docs.heroku.com/http-caching). (Because of its read-only file system, Heroku doesn't support Rails page caching, but that's OK: their Varnish cache (http://varnish-cache.org/) is even faster. :-) Thus, though designed principally for dynamic applications, Heroku is also great for static content.

It's not the same - you have to compile Nginx with mod_wsgi support. It's not as simple as doing an "apt-get install nginx" and it may compile with your version, or it may not compile. Also, read this: http://blog.dscpl.com.au/2009/05/blocking-requests-and-nginx-version-o… mod_wsgi is an interesting example. Surely it has the wrong architecture for Nginx and something better could surely be written. But mod_wsgi for Apache simply kicks ass, being a server that manages everything for you with low overhead and good performance caracteristics. No need for supervisord or any of that crap. Some people go as far as recommending running Nginx in front of Apache, with all the overhead that brings, just for mod_wsgi (although the same could be said about mod_php). That's the thing about Apache - it's so mature and has so much traction that you can find quality plugins for anything and everything, being an all you can eat buffet. But then if you're going to put Nginx in front of Apache, why go with Nginx instead of Varnish: http://www.varnish-cache.org/ -- which is designed from the ground-up to function as a high-performance proxy cache (the Nginx plugin for proxy caching sucks donkey balls IMHO) What I'm trying to say here is that Nginx is very far from a one-size-fits-all. It's awesome for at least one website I've got written in Ruby on Rails and I'm using it in combination with Passenger (Ruby's mod_wsgi). But for all my other use-cases, I'm starting with Apache…

Needs (2023) That services layer reminds be of a big, fancy, distributed Varnish Cache... they don't mention caching and they chose the word coalesce so I assume it doesn't do much actual caching. But made me think of Varnish's "grace mode" and it's use to prevent the thundering herd problem (which is where I first heard of 'request coalescing') https://varnish-cache.org/docs/6.1/users-guide/vcl-grace.html#grace-mo… Also love to see consistent hashing come up again and again. It's a great piece of duct tape that has proven useful in many similar situations. If you know where something should be then you know where everything is gonna come look for it!

> Is there some document somewhere that goes over the choices, overlaps, etc? I believe Apache Traffic Server doesn't really compete with the likes of nginx or traefik. Apache Traffic Server is a HTTP caching server/web accelerator, thus it's specialized for caching HTTP requests and consequently it's simpler to deploy and configure. With that in mind, it competes with the likes of Squid[1] or Varnish[2]. [1] http://www.squid-cache.org/ [2] https://varnish-cache.org/

If performance is your concern, check out https://varnish-cache.org/

The general attitude here reminds a bit of the following post from the architect of the Varnish proxy. I think the attitude comes down to the fact that modern kernels and in general the foundations of network programming are pretty strong. We should trust them more. https://varnish-cache.org/docs/5.2/phk/notes.html

https://queue.acm.org/detail.cfm?id=1814327 followed by http://varnish-cache.org/docs/trunk/phk/notes.html have completely transformed how I view and architect software. I basically started to strive to leave as much memory management as possible to the kernel thus not only simplifying my code but also making it behave better as citizens in the country that the operating system is, if you will.

I think phk said it[1] best [1] https://varnish-cache.org/docs/trunk/phk/http20.html#beating-http-1-1

Title is misleading. When I read "go dark" I think of a service doing the clever thing and burrowing itself in the darkweb where it's nearly impossible to take down (unless it gets DDOSed of course). But even DDOS'ing can be mitigated by hardening your server, and doing rudimentary things like rate limiting and filtering out bots with captchas. Things are made even easier by using a tricked out NGINX[0] server or even Varnish cache[1]. [0] https://en.wikipedia.org/wiki/Nginx [1] https://www.varnish-cache.org/

Moving memory management into the OS often makes sense, because the OS has the big cross-application picture, knows the system-wide memory pressure, and most importantly, already manages the memory of applications, by swapping between RAM and disk. For the same reason, OS X and Android Linux both have systems for OS managed caches, and AFAIK Firefox already uses these: https://bugzilla.mozilla.org/show_bug.cgi?id=748598 The status on mainline Linux is a bit more nebulous (seems Android's ashmem has been upstreamed, but it's not directly usable on GNU/Linux systems?), and other efforts have stranded: https://lwn.net/Articles/602650/ For some more thoughts about memory management on OS level vs. application level, I can recommend this "random outburst" from the designer of the Varnish HTTP cache: https://www.varnish-cache.org/docs/trunk/phk/notes.html

Hopefully Google considers contributing a Brotli vmod to Varnish[0]! 0. https://www.varnish-cache.org

Look at Varnish Cache https://www.varnish-cache.org/ and Google's PageSpeed module on the server. https://developers.google.com/speed/pagespeed/?hl=en

They are talking about Varnish: https://www.varnish-cache.org Or, you can start with this to get the general idea first: https://en.wikipedia.org/wiki/Varnish_software

Good read about Varnish and SSL(again). https://www.varnish-cache.org/docs/trunk/phk/ssl_again.html

Why are you connecting to a Redis box across the internet? There's a great (and after Heartbleed, prophetic) post on the Varnish web site about why they don't implement SSL, I imagine Redis would be similar: https://www.varnish-cache.org/docs/trunk/phk/ssl.html

"Varnish allocate[s] some virtual memory, it tells the operating system to back this memory with space from a disk file. When it needs to send the object to a client, it simply refers to that piece of virtual memory and leaves the rest to the kernel. If/when the kernel decides it needs to use RAM for something else, the page will get written to the backing file and the RAM page reused elsewhere. When Varnish next time refers to the virtual memory, the operating system will find a RAM page, possibly freeing one, and read the contents in from the backing file. And that's it. " https://www.varnish-cache.org/trac/wiki/ArchitectNotes I'll try and hold back the snark but I find it interesting that after attacking '1975 programming' and Squid's deficiencies, here we are 8 years later and maybe the kernel doesn't know best.

I'd add that edge side includes (https://www.varnish-cache.org/trac/wiki/ESIfeatures) can be amazingly useful. Even if you've offloaded all of your static content to a CDN, the ESI features of varnish alone can be a huge win.

Trust the OS, and don't use 70's technique like separating RAM and disk. Here's a really interesting piece on how PHK built varnish by using "modern" software techniques: https://www.varnish-cache.org/trac/wiki/ArchitectNotes And here's a Key-Value db you can use in your own programs that reuses the same idea, ie ask the OS for some space, trust it for RAM/Disk allocation; it turns out it is excellent at what it does thanks to careful design: http://symas.com/mdb/

> Is ther any CMS/blog/... type of thing that would create flat files, but keep all the content also in DB? Both Drupal and Wordpress have modules/plugins that produce static pages: * Drupal: https://drupal.org/project/boost * Wordpress: https://wordpress.org/plugins/wp-super-cache/ They're both quite mature and the standard practice for websites that get a lot of anonymous traffic using their respective CMSes. An alternative (or supplement) to them would be something like Varnish: https://www.varnish-cache.org

I recently upgrade from version 3 to 4, please note there are several changes on the VCL. Read carefully the upgrade docs: https://www.varnish-cache.org/docs/trunk/whats-new/upgrading.html Also, despite it's not on the manual, it's worth noting that req.grace and beresp.grace were deprecated.

You need to read the documentation on this and understand it: https://www.varnish-cache.org/docs/4.0/whats-new/index.html Otherwise you should just be using a popular file from around the web. Take a look at the Varnish Utilities Directory: https://www.varnish-cache.org/utilities There you can find two of the best resources for ready to go VCL (for now Varnish 3 only): https://github.com/slashsBin/nuCache https://github.com/mattiasgeniar/varnish-3.0-configuration-templates I am sure that if you ask nicely, the project members of these efforts might just give you a hand and upgrade to VCL 4.0 syntax a usable Wordpress template. Be willing to test and contribute back with whatever you can though.

I agree. There are a few other pieces of software I'd put in the same class... here's two examples I can think of now: * Varnish (the "Notes from the architect" are a great read: https://www.varnish-cache.org/trac/wiki/ArchitectNotes) * QEMU (actually, pretty much anything Fabrice Bellard has touched)

You know PHP, so use PHP. There's going to be a mountain of business challenges between 0 and 100K daily visitors - free up your mental bandwidth for that. Also, something as read-heavy as a blog is well suited for something like Varnish (https://www.varnish-cache.org/). Regardless of the backend technology you choose, a caching http proxy (or whatever) should be absorbing most of the traffic.

Not being sure about the rest of HN, I definitely recommend reading what FreeBSD contributor Poul-Henning Kamp has to say regarding another great project of his, Varnish: https://www.varnish-cache.org/docs/trunk/phk/index.html