Colin Scott

This gets posted here frequently - I submitted it a year ago - but imo that's not a problem because it's insightful and useful. Should have a (2012) tag though. I also ran across this version that updates the numbers to 2020 values: https://colin-scott.github.io/personal_website/research/interactive_la… edit: it appears they're just estimating for a given time above, not measuring...

I don't know if the book goes into this, but one thing I've noticed about brilliant software people like Jeff Dean or Martin Thompson is that they appear to love doing simple "back-of-the-envelope calculations" (and lots of them). They can iterate designs rapidly in their head or on paper, evaluate the cost or value of a design, know when a design has promise or when it won't work, and where the bottlenecks or low-hanging fruit might be, because they have a good feel for the performance/scaling cost of all the resources involved along the axes of things like branch mispredicts, main memory references, per core memory bandwidth, disk seeks, disk bandwidth, network latency, network bandwidth, expensive memory vs inexpensive storage tradeoffs. Numbers that programmers should know [1] but often don't. In other words, before they even write specifications or start coding, they've probably done tens to hundreds of back-of-the-envelope calculations to put their design in the "roughly right" ballpark. This drastically increase the chances that the design is going to work, and work really really well. And I think this habit is also what tends to set them apart. [1] https://colin-scott.github.io/personal_website/research/interactive_la…

It’s just a copy of https://colin-scott.github.io/personal_website/research/interactive_la… ?

Obligatory link to Latency numbers every programmer should know. https://colin-scott.github.io/personal_website/research/interactive_la…

it is easy to assume most programmers today know roughly how long things take since this has existed in various forms since Jeff/Peter first published it: https://colin-scott.github.io/personal_website/research/interactive_la… (This version is more useful since it shows change over time)

> Reading a file takes a couple hundred nanoseconds on a modern SSD. According to "Latency Numbers Every Programmer Should Know (2020)" [1] reading even a single page from a modern SSD takes over 10 microseconds. [1] https://colin-scott.github.io/personal_website/research/interactive_la…

The original is honestly better: https://colin-scott.github.io/personal_website/research/interactive_la…

So you know the latency numbers that every programmer should know? (I hope you do: https://colin-scott.github.io/personal_website/research/interactive_la… Is there an equivalent for software org spend? Obviously it couldn't be as principled as the above but if you look at engineering at StackOverflow and their compute / requests ratio it's orders of magnitude better than companies I've worked at. I just want numbers in spend on compute and headcount related to operational requirements. Has someone compiled a set of simple heuristics that says "If you are serving thousands of requests per hour to hundreds of enterprise clients your AWS bills shouldn't be more than thousands per month?" Something which can be used to say very clearly "we are incompetent" to upper management?

What-ifs really need to be dated. Some of those assumptions are enough to date it; 160tb/kg is kind of laughable now, when talking about micro SD cards. Reasonably guessable using https://colin-scott.github.io/personal_website/research/interactive_la…

I like the interactive version: https://colin-scott.github.io/personal_website/research/interactive_la…

No, because at 8 nines (99.999999%) you're responsible for the 310ms of downtime per year, aka ping time. 100% uptime would be all of Slack running in memory. See "Latency Numbers Every Programmer Should Know" https://colin-scott.github.io/personal_website/research/interactive_la…

While the numbers are changing, there are still 2-3 orders of magnitude between RAM and SSD latency: https://colin-scott.github.io/personal_website/research/interactive_la…

I really found this valuable, particularly the slide at the top that enables you to visualize low level latency times (Jeff Dean numbers) over the years. tl;dr: not much has changed in the processor hardware numbers since 2012. So everything right of the processor is where the action is. And sounds like people are starting to actually make progress. https://colin-scott.github.io/personal_website/research/interactive_la…

That’s true, but also networks can be fast. https://colin-scott.github.io/personal_website/research/interactive_la… It can be faster to read over the network than to read from a hard drive.

Per year version https://colin-scott.github.io/personal_website/research/interactive_la…

The original source linked from this post [0] is using models that assume exponential growth of bandwidths over time (see the JavaScript at the bottom of the page): this is fun, but these figures are real things that can be measured, so I think it’s very misleading for the site in this link to present them without explaining they’re basically made up. The 1Gb network latency figure on this post is complete nonsense (I left another comment about this further down); looking at the source data it’s clear that this is because this isn’t based on a 1Gb network, but rather a “commodity NIC” with this model, and the quoted figure is for a 200Gb network: function getNICTransmissionDelay(payloadBytes) { // NIC bandwidth doubles every 2 years // [source: http://ampcamp.berkeley.edu/wp-content/uploads/2012/06/Ion-stoica-amp-… // TODO: should really be a step function // 1Gb/s = 125MB/s = 125*10^6 B/s in 2003 // 125*10^6 = a*b^x // b = 2^(1/2) // -> a = 125*10^6 / 2^(2003.5) var a = 125 * Math.pow(10,6) / Math.pow(2,shift(2003) * 0.5); var b = Math.pow(2, 1.0/2); var bw = a * Math.pow(b, shift(year)); // B/s * s/ns = B/ns var ns = payloadBytes / (bw / Math.pow(10,9)); return ns; } [0] https://colin-scott.github.io/personal_website/research/interactive_la…

I put together this tool to estimate runtime complexity given the input size and max time the algorithm has available. I was motivated by competitive programming. Sometimes the problem size is sufficiently small and your algorithm has enough time to be naive. I lacked an intuition for how input size and complexity translates into time. My intuition was limited to: O(N^2) is bad unless the input size was ~10,000. Putting this tool together, I feel like I have a better intuition now. Hopefully exploring the tool will improve your intuition. If you didn't enable javascript, take a look at the table at the bottom of the page to see how input size and complexity translate to time. I was hoping for a resource similar to "Latency Numbers Every Programmer Should Know"[1] to help me ballpark how long my algorithm would take before implementing it. I put together a table at the bottom of the page to explore the powers of 2, time, and the runtime complexity. Eventually I realized I could take a step further and ask for a deadline and input size and I could estimate the required runtime complexity. [1] https://colin-scott.github.io/personal_website/research/interactive_la…