Patrick McManus

bitsup.blogspot.com · Personal blog · on HN since 2010

Patrick McManus, a Mozilla engineer working on HTTP and networking, writes about caching, SPDY, TCP, and DNS over HTTPS.

What HN saysHN commenters link his Cache-Control: immutable post and the Firefox behavior tests, and discuss his DNS and TCP congestion posts.
32HN people shared it
20linked it in comments
17stories submitted
1.0Kstory points
7 years agolast shared
2010people per year2026

Most-shared posts

  1. 1
    Cache control immutable
    5 people · first shared 2016
  2. 2
    SPDY: What I Like About You
    4 people · 219 points · first shared 2011 · 38 comments on HN
  3. 3
    HTTP Pipelining aint that Great
    3 people · 1 points · first shared 2012 · discussion on HN
  4. 4
    Google fixes nearly decade-old Linux kernel TCP bug
    2 people · 355 points · first shared 2015 · 46 comments on HN
  5. 5
    Cache-Control: immutable
    2 people · 197 points · first shared 2016 · 71 comments on HN
  6. 6
    Opportunistic Encryption for Firefox
    2 people · 109 points · first shared 2015 · 82 comments on HN
  7. 7
    The Benefits of HTTPS for DNS
    2 people · 2 points · first shared 2018 · discussion on HN
  8. 8
  9. 9
    Trekking the Road from SPDY to HTTP/2 in Firefox
    1 people · 31 points · first shared 2012 · 13 comments on HN
  10. 10
    On the Merits of QUIC for HTTP
    1 people · 26 points · first shared 2017 · 1 comments on HN
  11. 11
    HTTP/2 is Live in Firefox
    1 people · 16 points · first shared 2015 · discussion on HN
  12. 12
    Smarter Network Page Load for Firefox
    1 people · 4 points · first shared 2012 · discussion on HN
  13. 13
    Making Firefox Search Snappier
    1 people · 3 points · first shared 2012 · discussion on HN
  14. 14
    HTTP/2 Dependency Priorities in Firefox 37
    1 people · 2 points · first shared 2015 · discussion on HN
  15. 15

What HN people wrote when they linked it

patrickmcmanus · 2018-05-31
Everybody is right in this thread :) First, just to avoid confusion, the post linked to this HN article is just about the classic recursive resolver model. That's the scope of what is being experimented with actively. Second, the notion of resolverless dns (where dns records are obtained from somewhere other than your recursive resolver) is indeed something DoH contemplates but does not yet allow. That's because issues around tracking, correctness, and attacks haven't been fully explored. So unsolicited DNS is interesting but its not something any browser would accept yet. There are some other opinions on how HTTPS matches the needs of DNS here: https://bitsup.blogspot.com/2018/05/the-benefits-of-https-for-dns.html
vasilvv · 2020-04-02
> By the way, does anyone know why QUIC specifically put NewReno as its official congestion control [0]? Different environments benefit from different algorithms so I don't see why. It's a "reasonable default", in the sense that it normally works quite well, and is easy to implement. The dangerous thing about congestion control is that it's really easy to get it wrong and not notice (as noticing requires performance testing and monitoring). Google QUIC uses BBR on the server side. I wrote the initial version of said BBR implementation, and it took a lot of effort to get it to a production-ready state, so I would not generally recommend writing BBR from scratch. CUBIC is much simpler than BBR, but is also prone to subtle bugs like [0]. > 2. Not everything that needs encryption needs TLS specifically. TLS is not a one-size-fits-all solution. I'm actually curious what use cases you have in mind. TLS definitely does not cover 100% of all possible cases, but I've seen a surprising number of cases where people rolled their own thing when using (D)TLS would have just worked. [0] http://bitsup.blogspot.com/2015/09/thanks-google-tcp-team-for-open-sou…
theandrewbailey · 2018-09-04
> The heaviest page, the marketing page, today weighs 110 KB with a cold cache, and 4.6 KB warm. (more comparisons between cold and warm loads) I set HTTP cache-control immutable on my blog's image/JS/CSS URLs this weekend. Cache-Control: immutable tells the browser that this URL will never change, so don't check to see if it has. It should cut down on the warm loads more. If it does change, the timestamp in the URL path will change, so browsers will check the new URL. https://bitsup.blogspot.com/2016/05/cache-control-immutable.html I suggest caching the main HTML, too. It would get the total warm load down to less than 1 KB.
renesd · 2017-04-05
Yeah, that would be cool. They could also even potentially save JIT typing, specialization, and even parsing. This is a big deal on mobile. There is Cache-Control: immutable https://hacks.mozilla.org/2017/01/using-immutable-caching-to-speed-up-… https://bitsup.blogspot.de/2016/05/cache-control-immutable.html
tyingq · 2017-01-26
Firefox themselves appear to have tested and confirmed that regular F5/Refresh (not Ctrl+F5) causes revalidation: http://bitsup.blogspot.de/2016/05/cache-control-immutable.html
renke1 · 2016-12-19
There is also Cache-Control: immutable [1] which prevents browsers requesting the given file even when the site is refreshed. It's experimental though. [1]: http://bitsup.blogspot.de/2016/05/cache-control-immutable.html
cpeterso · 2016-12-09
You can also serve the font files with the `Cache-Control: immutable` HTTP response header so the client never needs to revalidate with the server (even when the user forces a page reload). Use versioned filenames for immutable resources if you need to change them later. http://bitsup.blogspot.de/2016/05/cache-control-immutable.html
ryao · 2016-12-01
I wouldn't say 8 to 10 years. A major bug in Linux's default congestion control algorithm was only fixed just last year: https://bitsup.blogspot.com.au/2015/09/thanks-google-tcp-team-for-open… Linux is the platform of choice for bufferbloat research, although FreeBSD isn't far behind in adopting the results of it: https://lists.freebsd.org/pipermail/freebsd-ipfw/2016-April/006154.htm…
xzyfer · 2016-07-15
Browser do currently do this. H2 has two types of prioritisations: weighted, and dependency. All browsers implement weighted resource prioritisation and weigh resources by content type. This is a hold over from what they do for HTTP/1 connections. Firefox has dependency resources prioritisation. https://bitsup.blogspot.com.au/2015/01/http2-dependency-priorities-in-… The spec purposely leave how these heuristics should work to the implementor. Things will change and implementations will diverge over time. The server ultimately being in control means we can tell the server what resources are important for specific pages with absolute knowledge of the page.
jor-el · 2016-04-15
The quotes in the paper are interesting: "Nobody actually creates perfect code the first time around, except me. But there’s only one of me.” Linus Torvalds, 2007 And another, which highlights why there might be many more unearthed bugs, and would probably go unnoticed. "I suspect that making the scheduler use per-CPU queues together with some inter-CPU load balancing logic is probably trivial . Patches already exist, and I don’t feel that people can screw up the few hundred lines too badly" Looking at the bigger picture in general, this again shows that getting software right is not easy. Now and then you still hear about the bugs popping up in the code which is very core to an OS. One I can recall is a decade old TCP bug which Google fixed last year[1]. [1] http://bitsup.blogspot.sg/2015/09/thanks-google-tcp-team-for-open-sour…
vtlynch · 2015-04-07
Well one of the problems is that SSL does two things: Server Authentication and Encryption. Since there is nothing backing the identity of a self-signed certificate, it can easily masquerade as whatever it wants. Yes, a CA-issued cert and a self-signed are equilvent when it comes to encryption. But with Server Authentication, its too easy for self-signed certificates to mislead people, and that is deserving of the red-X in my opinion. You may be interested in the Opportunistic Encryption that Firefox has just enabled: http://bitsup.blogspot.it/2015/03/opportunistic-encryption-for-firefox… It allows for SSL encryption with a self-signed, and ignores authentication. So not red-x, but no green padlock either. EDIT: Opportunistic Encryption seems to have caused some problems: https://www.mozilla.org/en-US/security/advisories/mfsa2015-44/
mbrubeck · 2015-02-24
Yes. According to Patrick McManus (owner of Gecko's networking stack), 9% of Firefox requests are already using HTTP/2 with the draft implementation enabled for Google (and now Twitter). With Google alone, HTTP/2 usage is already higher than SPDY. http://bitsup.blogspot.com/2015/02/http2-is-live-in-firefox.html
magicalist · 2012-02-04
which is to be expected with a TLS connection. OTOH you get to drop all the extra TCP handshakes, you get better congestion control, you get header compression, and that encrypted connection is often a really good thing. http://bitsup.blogspot.com/2011/09/spdy-what-i-like-about-you.html But yes, there remains no silver bullet, and you'll have to pick the right tool for the job.
justinschuh · 2011-10-10
The points you're using to substantiate your argument are not really in line with the facts. Many browsers broadly share a single implementation for huge chunks of the web. Examples off the top of my head include NSS, WebGL, LibPNG, LibJPEG, and WebM. And it's inaccurate to claim WebM is "licensed" from Google; that's no more true than claiming that NSS is "licensed" from Mozilla. Both are open source projects accepting third-party contributions. People can happily use the code as is, contribute to changing it, or simply fork it off. Finally, you seem to have missed the fact that Mozilla is already working on an experimentatl implementation of SPDY in Firefox: http://bitsup.blogspot.com/2011/09/spdy-what-i-like-about-you.html
pdovy · 2011-07-12
Has anybody used the __builtin_expect feature to good effect? We had been using this extensively in our codebase in places where there is an obvious common "fast path". I tried an experiment where I did a comparison of the __builtin_expect version against a version with our likely/unlikely macros defined to the identity, and I actually saw no difference, which was disappointing. When I did some digging I found this blog post: http://bitsup.blogspot.com/2008/04/measuring-performance-of-linux-kern… which suggests the same result for the Linux kernel. FWIW GCC also has a reasonable set of PGO options (not mentioned in this article), and those actually can (anecdotally) make a noticeable difference, although you're subject to the overtraining problem if you're not careful.
edmorley · 2017-09-15
For more on the rationale behind this feature, see: https://www.ietf.org/mail-archive/web/httpbisa/current/msg25463.html https://bitsup.blogspot.co.uk/2016/05/cache-control-immutable.html Rough summary: > At Facebook, ... we've noticed that despite our nearly infinite expiration dates we see 10-20% of requests (depending on browser) for static resource being conditional revalidation. We believe this happens because UAs perform revalidation of requests if a user refreshes the page. > A user who refreshes their Facebook page isn't looking for new versions of our _javascript_. Really they want updated content from our site. However UAs refresh all subresoruces of a page when the user refreshes a web page. This is designed to serve cases such as a weather site that says <img src="" ... > Without an additional header, web sites are unable to control UA's behavior when the user uses the refresh button. UA's are rightfully hesitant in any solution that alters the long standing semantics of the refresh button (for example, not refreshing subresources).
the8472 · 2017-01-06
Just code your site to do something sane with 3rd-party content blocked. E.g. handle load errors with fallbacks. That way people with µMatrix or similar blockers can use the control tools they have instead of needing to do something site-specific. Also, such decisions can't be remembered if cookies/localstorage are disabled. So prompting over and over again could also be annoying. > 1. May we retrieve common libraries from third party CDNs? Doing so helps support this site by saving on our bandwidth costs, but may expose information about you to those third parties. In an ideal world browsers would never send a cache-refresh request for resources tagged with SRI[0] because the hashes would guarantee that the content is 100% stable. Alas, it has non-trivial privacy implications, so they don't do that. Maybe it could be implemented as a privacy addon with a whitelist for CDN domains, but then sites would still have to adopt SRI for that addon to do its work. Or maybe an addon that injects cache-control: immutable[1] to CDN could work too, but that's limited to https. [0] https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Inte… [1] https://bitsup.blogspot.de/2016/05/cache-control-immutable.html
acdha · 2015-08-20
Your theory that it was just malware is easily disproven by reading the bug reports and blog posts from the years people spent trying to make this work. Firefox found far from universal success even after years of testing and blacklisting known-noncompliant servers: http://bitsup.blogspot.com/2012/11/a-brief-note-on-pipelines-for-firef… (the actual blacklist at the time http://hg.mozilla.org/mozilla-central/file/1d122eaa9070/netwerk/protoc… Opera's implementation apparently relied on some non-trivial heuristics but they apparently weren't well documented and were discontinued with the transition to Blink. Similarly, it's easy to find cases where people discovered real problems in the wild with the iOS implementation: http://tech.vg.no/2011/12/14/safari-on-ios-5-randomly-switches-images/ (This also affected Mozilla: https://bugzilla.mozilla.org/show_bug.cgi?id=716840) https://github.com/AFNetworking/AFNetworking/issues/528 It's easy to understand why people decided that it wasn't worth investing so much time in this when HTTP/2 would deliver significant additional benefits and by using TLS as a starting point could provably avoid the worst tampering proxies entirely.
geofft · 2015-04-07
Treating self-signed HTTPS as HTTP would be a behavior change with subtle implications that are hard to get right. For instance, let's say that I log into https://example.com today, and it has a valid cert (but no HSTS). It sets a cookie with the "HTTPOnly" and "Secure" flags, indicating that it should only be sent over HTTPS, and not exposed to e.g. JS. I come back tomorrow, and https://example.com is now sporting a self-signed certificate. There are now three possibilities for what's going on here. One is that this is an attack, so the browser should by default block the site. One is that the site downgraded to a self-signed certificate because they want to be treated as HTTP. One is that the site downgraded to a self-signed certificate because of misconfiguration, but you know out-of-band that it should be treated as HTTPS (the fingerprint matches what the support phone number tells you, or you're on a secure network, or something). Those last two situations are different. In the former, that cookie should no longer get sent, just as it would not be sent to http://example.com. In the latter, it should be sent. Right now, on the assumption that "intended to downgrade to HTTP with opportunistic encryption" is rare, the browser will put up a scary alert to make sure you're verifying the security of the connection out-of-band, and then send the cookie if you click through the warning. You'd have to somehow find a way for site owners to signal that they intend to do OE,…
akalin · 2013-07-07
I didn't have to look hard to find bug reports for pipelining. An example is https://bugs.launchpad.net/ubuntu/+source/apt/+bug/948461 for Amazon's S3. I'd be interested if the problem is still reproducible now. Also, one of the comments mentions Squid 2.0.2 as being buggy. Also, see https://insouciant.org/tech/status-of-http-pipelining-in-chromium/ for a link to Firefox's blacklist of buggy servers (and a good discussion of pipelining in Chromium). Most of the improvements in SPDY are latency improvements, so if you're downloading sites with netcat and then viewing them in a browser, I'm pretty sure the overhead of that would dwarf anything SPDY would save. That having been said, there's ample evidence of SPDY improving things. From http://bitsup.blogspot.com/2012/11/a-brief-note-on-pipelines-for-firef… : "Also see telemetry for TRANSACTION_WAIT_TIME_HTTP and TRANSACTON_WAIT_TIME_HTTP_PIPELINES - you'll see that pipelines do marginally reduce queuing time, but not by a heck of a lot in practice. (~65% of transactions are sent within 50ms using straight HTTP, ~75% with pipelining enabled).... Check out TRANSACTON_WAIT_TIME_SPDY and you'll see that 93% of all transactions wait less than 1ms in the queue!"

Keyboard shortcuts

/Search
j kNext / previous blog
EnterOpen selected blog
vVisit the blog site
fFollow / unfollow
lSwitch grid / list view
dSwitch dark mode
g hHome
g pTop posts
g tTopics
g fFollowing
g lLiked
g yHistory
g qFAQ
EscBack / close