Varnish Software Blog

I borderline love Varnish cache, but the way the open source and commercial versions have diverged and all the nice features only go into the $10k/yr+ commercial version sucks. Features like the memory governer [0], because trying to predict how much memory (open-source) Varnish will use is an absolute PITA and a sure-fire way to run out of memory if you're not careful. My clients can't justify the commercial license costs (as a sibling comment says, CDNs eat Varnish's lunch in that market) and yet what I can do with Varnish and the power it gives me makes it real magic. It would be nice to see a modern reimplementation of Varnish, open-sourced, but I doubt that would ever happen. 0. https://info.varnish-software.com/blog/two-minute-tech-tuesdays-memory…

Excellent tour through this topic, here's my best attempt at a tl;dr summary: 1. write a 'Hello World' HTTP server to show how Tokio's async-Rust primitives work for epoll-based I/O; 2. show how the tracing crate ecosystem works to inspect and debug async-runtime tasks; 3. play with converting traces to OpenTelemetry, tokio-console, etc. for different views of the collected data; 4. play with hyper and axum to build a simple HTTP service that makes an upstream HTTP request to the YouTube API; 5. cache results by ID so it can shared across all request-handlers; 6. add 'request coalescing' to prevent multiple concurrent requests from making upstream requests in parallel; 7. fix a bug in the request-coalescing implementation that caused coalesced requests to hang when the in-flight request raised an error. One word of caution about using any request-coalescing pattern like this on a highly-loaded server: in the case where every request fails to populate the cache (e.g., timeout / not found), you end up in a situation where all requests end up _serialized_ and can quickly lock up a server under constant load. You need to either add a circuit-breaker, or otherwise ensure that _something_ always gets cached in any otherwise-uncacheable edge case (e.g., hit-for-pass in Varnish [1].) [1] https://info.varnish-software.com/blog/hit-for-pass-varnish-cache

How wrong can one comment be? You might want to read this post on the founding of Varnish https://info.varnish-software.com/blog/history-varnish-cache-10-years And Fastly would certainly disagree that it’s only useful for PHP as they built a whole company based on Varnish

Closed source, write up would be here: https://info.varnish-software.com/blog/introducing-varnish-total-encry…

And no support for HTTP/2 until recently. They're suggesting Hitch for SSL termination now. [1]: https://info.varnish-software.com/blog/varnish-cache-5-http2-support Edit: Apache also has caching/does it all in one package; they're even contemplating a module for certbot (in addition to or as replacement for the official certbot/letsencrypt Python package for Apache)

Interesting; I'd love to read _what_ exactly they send in those long HTTP requests. I'd think something along the lines of Surrogate Keys/Cache Tags would make sense: - https://www.drupal.org/developing/api/8/cache/tags - http://info.varnish-software.com/blog/advanced-cache-invalidation-stra… (ugh, they broke their site, cached at https://pinboard.in/cached/a31c4630dd2d/) - https://www.fastly.com/blog/surrogate-keys-part-1 - https://blog.cloudflare.com/introducing-a-powerful-way-to-purge-cache-…

I'll echo what Simon said; we share some experiences here. There's a potential footgun, though, anyone getting started with this should know about- Request coalescing can be incredibly beneficial for cacheable content, but for uncacheable content you need to turn it off! Otherwise you'll cause your cache server to serialize requests to your backend for it. Let's imagine a piece of uncacheable content takes one second for your backend to generate. What happens if your users request it at a rate of twice a second? Those requests are going to start piling up, breaking page loads for your users while your backend servers sit idle. If you are using Varnish, the hit-for-miss concept addresses this. However, it's easy to implement wrong when you start writing your own VCL. Be sure to read https://info.varnish-software.com/blog/hit-for-miss-and-why-a-null-ttl… and related posts. My general answer to getting your VCL correct is writing tests, but this is a tricky behavior to validate. I'm unsure how nginx's caching handles this, which would make me nervous using the proxy_cache_lock directive for locations with a mix of cacheable and uncacheable content.

That's really neat that you can do the authentication/authorization headers within Varnish. I'd still use CloudFront instead, which gives you these features plus the advantage of having global edge servers, and support for TLS/SSL (not sure about the http/2 part). Although the one advantage this has is the cache can stop you from having to leave your external network to return a file (vs going to CloudFront or S3). That can be useful. Another commenter pointed out that Varnish won't support TLS either, which is correct. The funny part about that is while they have long had the "why not SSL?" FAQ page espousing the terribleness of OpenSSL, they DO offer that feature on their paid product Varnish Plus. That not being a basic feature is a tad crazy. [1] Why no ssl? https://www.varnish-cache.org/docs/4.1/phk/ssl.html [2] http://info.varnish-software.com/blog/how-implement-ssltls-varnish-plu…

I was spec-ing out Varnish for a project recently and was really impressed with what it could do. The stale-while-revalidate approach (immediately serve data past the `ttl` for a `grace` period while asynchronously refreshing it) keeps data flowing. It can `keep` data past the `grace` period and serve that when the origin goes down or use that in conditional requests (If-Modified-Since) to the origin to prevent fetching unchanged content. Coalescing requests mitigates thundering herds that reach stale caches. VCL scripting makes the whole thing very configurable. I think Varnish needs better marketing, but I suppose Fastly does that now. [1] https://www.varnish-software.com/developers/tutorials/http-caching-bas… [2] https://info.varnish-software.com/blog/request-coalescing-and-other-re… [3] https://docs.varnish-software.com/tutorials/object-lifetime/ [4] https://www.varnish-software.com/developers/tutorials/varnish-builtin-…