Frank McSherry

Always worth giving a shoutout to the "Scalability! But at what COST?" paper (pdf https://www.frankmcsherry.org/assets/COST.pdf) We offer a new metric for big data platforms, COST, or the Configuration that Outperforms a Single Thread. The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation. COST weighs a system’s scalability against the overheads introduced by the system, and indicates the actual performance gains of the system, without rewarding systems that bring substantial but parallelizable overheads.

I recommend this table of latency figures for any software engineer: https://gist.github.com/jboner/2841832 Essentially IO is expensive except within a datacenter but even in a data center, you can do a lot of loop iterations in a hot loop in the time it takes to ask a server for something. There is a whitepaper which talks about the raw throughput and performance of single core systems outperforming scalable systems. These should be required reading of those developing distributed systems. http://www.frankmcsherry.org/assets/COST.pdf A summary: http://dsrg.pdos.csail.mit.edu/2016/06/26/scalability-cost/

"Scalability! But at what COST?" is another classic in the category (though it's not precisely about on-box parallelism if I recall; the authors' acronym "COST" is the "Configuration that Outperforms a Single Thread"). https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-m… http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… "Rather than making your computation go faster, the systems introduce substantial overheads which can require large compute clusters just to bring under control. In many cases, you’d be better off running the same computation on your laptop." My limited experience fits this in that a bit of smarts on a single box beats a bunch of boxes. (the link is a very good read BTW)

Have you seen Frank McSherry's COST paper? (and follow ups) http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… Really fun reading on exactly this issue - definitely changed my view on Spark and compute clusters. People definitely don't realise just how powerful a modern computer is, and how far you can get on a single one.

A classic from 2015 along the same lines: Scalability, but at what COST? http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

"Big data" frameworks are very good at throwing a lot of hardware at problems. See eg the classic big data vs laptop treatment: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

> Almost by definition, any company that runs a cluster is probably dealing with TBs of data (otherwise they could probably run it on one machine). This doesn't follow: 1) One reason to run a cluster is to use flaky commodity hardware instead of high-reliability specialized hardware. Note the transition from specialized hardware and computer rooms of the '80s and '90s to cloud computing on preemptible instances. 2) They might just be straight-up wrong / misguided. http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… As a professional developer I've seen tons of distributed systems that could be replaced by a well-designed non-distributed system, but ELK and Mongo and etcd and Kafka and more generally cloud servers are easy off-the-shelf tools.

On another note - these kinds of "let's go back and check what we can do with a single node today" approaches are valuable to recalibrate our thinking once in a while. I didn't interpret this post as a case against distributed processing - there are many things going for it - but just to avoid falling into hype. A couple of great posts by Frank McSherry along the same lines a few years ago, doing page rank for a 128bil edge graph on a single laptop (in the second post below) - The first post - "scalability! But at what cost" http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… The second post - "bigger data, same laptop" http://www.frankmcsherry.org/graph/scalability/cost/2015/02/04/COST2.h…

Brilliant writing style! Also check out his work on Naiad, which is IMO by far the most interesting work in its area, and check out the Scalability! But at what COST? paper: http://www.frankmcsherry.org/assets/COST.pdf which showed that a program running on a single thread often outperforms distributed systems running on 100+ cores.

Well, first, a lot of this stuff is actually written in Scala, which is like Java's younger and more fun cousin. Still runs on the JVM but has many fun language aspects I won't get into because there's already a billion posts online about that. But second, not quite a "platform" but I've been enjoying Frank McSherry's blog posts in which he builds up a complete distributed dataflow engine in Rust: http://www.frankmcsherry.org/

His blog posts are easier to follow and aren't forced to be in the academic format :) http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… http://www.frankmcsherry.org/graph/scalability/cost/2015/02/04/COST2.h…

A single server is much faster than most people think, too! In the microservice or serverless arrangements I've seen, data is scattered across the cloud. It's common for the dominant factor in performance to be data locality, most times this talk about data locality is about avoiding trips to RAM, or worse disk. But in our "modern" distributed cloud things, finding a bit of data frequently involves a trip over the network. In the monolith world what was once invoking a method on an account object, has become making a HTTP POST to the accounts microservice. What might have been a microseconds operation in the single server world, might become hundreds of milliseconds in the distributed cloud world. While you can't horizontally scale a single server, your 1000x head start in performance might delay scaling issues for a very long time. A most excellent paper related to this topic that I think should be mandatory reading before allowing anyone an AWS account is http://www.frankmcsherry.org/assets/COST.pdf :)

We are actually talking about orthogonal aspects. You are right that being able to run the system on a smaller number of nodes reduces inefficiencies and reduces the "hit" of diminishing returns stemming from Amdahl's law. However, to do that, you need to speed up your code and I argue that unless you speed up the non-parallelizable part together with the parallelizable part, you won't be able to achieve that system speedup that allows you to remove a number of servers from your cluster in the first place. However, I think you are right that on small number of servers speeding up parallel part can have a more immediate effect. I did a small calculation just to check the numbers. Assume you have a server that can do 100 RPS. You can either speed up the parallel code 2x (let's be realistic here and stop this 100x nonsense) and bring the RPS almost to 200 or you can take that serial code that is just a fraction and reduce it by half, so that f=0.995 and not 0.99 (and adding a marginal RPS increase to 102 RPS). Here is the plot: https://imgur.com/a/7w1hms9 Edit: read http://www.frankmcsherry.org/assets/COST.pdf as recommended in the upvoted comment above!

That post from Frank McSherry is excellent. His earlier post [1] is also worth reading. I was curious to read more about the Hilbert ordering of edges, and the delta-encoding. There is a related article from Yzelman & Bisseling titled "A cache-oblivious sparse matrix–vector multiplication scheme based on the Hilbert curve"[2] which discusses Hilbert ordering combined with a delta-encoded version of the usual CSR encoding for sparse matrices [3]. > A Hilbert curve thus can be projected on any [2^m * 2^m] matrix, which in turn can embed the sparse matrix A, thus imposing a 1D ordering on its nonzeroes. The largest sparse matrix considered for the experimental results from this article was "wikipedia-20070206" with 45,030,389 nonzeros (aka weighted edges). Ignoring the weights, this is about 1/3000th of the size of the 128 billion edge graph that McSherry mentions in his post. Yzelman & Bisseling also measure the additional preprocessing time required to convert their sparse matrices using the Hilbert-ordered delta-encoded CSR scheme -- this is roughly 30x the time for a single matrix-vector product evaluation [4]. Clearly this approach is only going to pay off if one plans to do a bunch of analysis using the single matrix / graph, without changing the matrix / graph. McSherry mentions a similar issue but points out that he could preprocess the graph using his approach faster than he could download the data, so the preprocessing speed wasn't really an issue. [1]…

Missing from the title is "enough" for what? For any trivial personal project? Python, Ruby, PHP are each enough. There's not a need for literature about personal low-risk development or deployment. That said, Scalability? But at what COST? [http://www.frankmcsherry.org/assets/COST.pdf] was a good note that orienting around scaleable technologies can be very inefficient- scalable does not mean either fast or economical.

The canonical reference for this: Scalability! But at what COST? https://www.frankmcsherry.org/assets/COST.pdf

With regards to Spark, consider if you could possibly do the processing on a single machine. For many workloads, a single threaded program running on a laptop will beat an entire Spark cluster. https://www.frankmcsherry.org/assets/COST.pdf

More entertainment than how-to guide, and oriented more towards developers than ops, but if you haven't read "Scalability! But at what COST?" [0], I think you'll enjoy it. [0] https://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.h…

“the performance of the underlying hardware has increased so much in the last decade that the simplicity of SQLite has become a more important tradeoff” Thanks for articulating this so clearly! It's going to be interesting also to see how much simplicity is realized from new thread-per-core designs leveraging io_uring to get rid of multithreading, locks and memory barriers scattered throughout the control plane. The performance dynamics are all changing and it's easier than ever to be writing simple, fast systems, that hit the 90% sweet spot. You've probably also seen McSherry et. al's “Scalability! But at what COST?” [1] [1] http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

If you're allowed to use a second computer, you need at least 4 to match the performance you previously had on the single machine. http://www.frankmcsherry.org/assets/COST.pdf

If I am not misremembering, "all those blog posts" are all written by Frank McSherry, one of the cofounders of materialize materialize and developer of differential dataflow :-) http://www.frankmcsherry.org/assets/COST.pdf

Very nice, reminds me of a quote from this very related paper: You can have a second computer once you’ve shown you know how to use the first one. –Paul Barham Scalability! But at what COST? https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-m… http://www.frankmcsherry.org/graph/scalability/cost/2015/02/04/COST2.h… Interesting it also involves graph processing, which Bane's comment did. The point is that distributed graph processing frameworks were mostly parallelizing their own overhead, not solving the problem, which could be done on a single machine!

McSherry's COST paper (http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht… is a good reference on the advantages of SISD graph processing.

This one? http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

Scalability! But at what COST?: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…

See also: Scalability! But at what COST? [1] [2] > The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation. COST weighs a system’s scalability against the overheads introduced by the system, and indicates the actual performance gains of the system, without rewarding systems that bring substantial but parallelizable overheads. [1] http://www.frankmcsherry.org/assets/COST.pdf [2] https://news.ycombinator.com/item?id=11855594

Is this http://www.frankmcsherry.org/assets/COST.pdf the paper you referenced?

There's even a standard model for that kind of thing: http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.ht…