Emery Berger

It is well accepted that GC-based runtimes greatly increase memory footprint. A tracing GC might need upto several times the memory to match manual memory management. Nothing interesting about the "viewpoint". https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf https://blog.metaobject.com/2020/11/m1-memory-and-performance.html

Some people refer to only tracing algorithms (mark-sweep, copying, etc) as "garbage collection". The best reason I can think of is marketing. There are generational reference counting systems e.g. ulterior reference counting [1] and age-oriented collection [2]. "Tracing" usually refers to scanning a heap to work out liveness all at once, which reference counting doesn't do; though deferred reference counting scans the roots to collect, and ulterior reference counting traces a nursery. Deferring these (more common) updates to reference counts improves throughput considerably. [1] https://people.cs.umass.edu/~emery/classes/cmpsci691s-fall2004/papers/… [2] https://users.cecs.anu.edu.au/~steveb/pubs/papers/aogc-cc-2005.pdf

I often think about Proebsting's Law: Compiler Advances Double Computing Power Every 18 Years. Sure, optimizing compilers are nice to have, but maybe their complexity is disproportionate to their benefit? I love the way Dynamo [1] is able to reproduce many of the benefits with a fraction of the complexity by doing some of the optimizations at runtime with simpler algorithms. Can we use this approach to "garbage collect" some of the complexity embodied in humongous projects like LLVM? [1] Dynamo: https://people.cs.umass.edu/~emery/classes/cmpsci691s-fall2004/papers/…

So we are agreed that mixing tracing GCs with any other form of memory management (including another GC) is an unsolved problem. Good. Swift is a disaster, also agreed. So? For the rest: actual research disagrees with your forceful but unsubstantiated assertion: https://2013.splashcon.org/details/oopsla-2013-papers/21/Taking-Off-th… And once again, tracing GCs do well in microbenchmarks where you only check the cost of local operations. They are horrible when you take the global effects into account, with those local/benchmarking advantages not translating into real world use. https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf Very similar to JITs, which also do massively better in microbenchmarks than in production code: http://blog.metaobject.com/2015/10/jitterdammerung.html Oh, and integrating well and without high cost to fast languages where you have even more control is actually an important feature.

2005: "Quantifying the Performanceof Garbage Collection vs. Explicit Memory Management" (https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf) [Edit] Also, 1993: "The Measured Cost of Conservative Garbage Collection" (http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.1816) [Anyone seen any work on conservative GC for existing languages since Boehm?] Throughput-style garbage collection has mostly been solved; I believe the only outstanding problem is to reduce the necessary memory footprint towards the minimum required by the application. (On the other hand, the best memory management scheme is still an old-style Unix proggy that does bump-allocations in an arena and never frees memory until it terminates. Just don't let it run too long.) Most of the work in GC has been towards reducing latency, which typically means different GC algorithms for different tasks. The ideal of a fast, real-time GC is still an open research project. Explicit heaps are still not a solved problem, either, and they've been around longer than GC work (ahem*). With no static checking for leaks, use-after-free, and other issues, C memory management can be a nightmare unless you have iron-willed development discipline. The only real tools for dealing with those issues are run-time testing tools like valgrind, which among other things are horribly slow if you want to run them against a real workload. Rust is an interesting case: it does an amazing job, if you use it for C++-like RAII…

When talking about Garbage Collection they claim that "Even languages with highly optimized garbage collectors can’t match the performance of non-GC’d languages" and link to this paper: http://greenlab.di.uminho.pt/wp-content/uploads/2017/09/paperSLE.pdf However I cannot see where this paper supports the assertion (although Rust comes out well). The particular problem is that malloc/free is not free of charge. Different allocators have complex internal implementations, plus you lose easy sharing of complex structures and compaction. So if you're using a GC you probably program in a different way, eg using more shared immutable structures, exploiting the advantages of GC. Edit: A bit later they brush off this old favourite for GC-haters: https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf claiming that GC requires 5x as much memory. The problem with this paper is it assumes that malloc/free is cost-free and instantaneous, and that your programmer always calls free at the exact moment that the memory is no longer needed, both of which are extremely unrealistic.

I think the real problem is the move from manual memory management to garbage collection. Garbage collection has immense advantages, and is indisputably the right choice for most application programming, but it only goes fast if you feed it lots of memory. See figure 3 of this wonderful but terrifying paper: http://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf (Caveat: that was published in 2005, and garbage collection has improved since then. However, i doubt it has improved enough to invalidate its conclusions.) The best collector there needed ~3x as much memory as a (admittedly superhumanly perfect) manually memory management to get to a competitive speed. There are plenty of problems where that memory is available. But there are plenty where it is not. I can justify giving loads of memory to the application that makes my company money. I can't justify giving it to some tiny little log shipper or message forwarder or other random system utility that i want to run on every machine in my infrastructure.

Paper link (2011): https://people.cs.umass.edu/~emery/pubs/dthreads-sosp11.pdf > DTHREADS works by exploding multithreaded applications into multiple processes, with private, copy-on-write mappings to shared memory... Experimental results show that DTHREADS substantially outperforms a state-of-the-art deterministic runtime system, and for a majority of the benchmarks evaluated here, matches and occasionally exceeds the performance of pthreads.

> Why do I need to? Why can't I let the garbage collector deal with it? Determinism. With Rust lifetimes, you can statically prove when resources will be released. Garbage collectors provide no such guarantees. It has been shown that garbage-collected languages have a space-performance tradeoff: you need five times as much RAM to achieve the same performance, even with a "good" GC, as the same program with explicit memory management: https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf

I thought comparing reachability to "true" liveness had been studied in the past. Ben Zorn may have worked on it in the 1980s: https://www2.eecs.berkeley.edu/Pubs/TechRpts/1989/5313.html Also: https://people.cs.umass.edu/~emery/pubs/04-17.pdf

Also, in practice, -O3 doesn't necessarily lead to faster code. https://people.cs.umass.edu/~emery/pubs/stabilizer-asplos13.pdf https://m.youtube.com/watch?v=r-TLSBdHe1A

Reference counting cannot be faster than naive manual memory management because it does the same thing with the additional overhead of reference counting. Even a very simple generational GC beats naive explicit memory management on most nontrivial workloads given enough memory. https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf

here is a related paper: https://people.cs.umass.edu/~emery/pubs/stabilizer-asplos13.pdf

It seems to be part of a larger framework which looks something like an in-browser OS. Here's their paper, mentioned in the GitHub page (PDF format): https://people.cs.umass.edu/~emery/pubs/doppio.pdf

Here's the corresponding papers: CheckCell - https://people.cs.umass.edu/~emery/pubs/CheckCell-preprint-OOPSLA2014.… ExceLint - https://arxiv.org/pdf/1901.11100.pdf

All modern allocator implementations I am aware of (including Hoard and Mesh) use per-thread caches (of a limited size), and thus avoid the use of atomic operations in the common case while also avoiding blowup (wasted memory caused by using a memory allocator that "leaks" -- see the Hoard paper (ASPLOS 2000, https://people.cs.umass.edu/~emery/pubs/berger-asplos2000.pdf) for a full discussion).

Good throughput with GC seems to come with significant extra memory use: "We compare explicit memory management to both copying and non-copying garbage collectors across a range of benchmarks using the oracular memory manager, and present real (non-simulated) runs that lend further validity to our results. These results quantify the time-space tradeoff of garbage collection: with five times as much memory, an Appel-style generational collector with a non-copying mature space matches the performance of reachability-based explicit memory management. With only three times as much memory, the collector runs on average 17% slower than explicit memory management. However, with only twice as much memory, garbage collection degrades performance by nearly 70%." https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf So you're still paying for it, just in a different dimension.

I saved a few papers on deterministic multithreading both to knock out concurrency errors and support formal verification. I'm not sure how helpful they are in your use case. I'm just dropping them in case they give you ideas: https://people.csail.mit.edu/mareko/asplos073-olszewski.pdf https://people.cs.umass.edu/~emery/pubs/dthreads-sosp11.pdf

> Yes -- five times as much RAM: http://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf That article was presented at OOPSLA '05 which ran from October 16–20, 2005. I don't know when it was written, nor when the data for it was worked out. That said, it seems safe to say that it's more than twelve years out of date.

But in-place sort doesn't use malloc() at all. Problem solved. Custom allocators are nice in theory but come with practical costs. And it's easier to do worse than your system malloc() these days. Counterpoints: - Why custom allocators/pools are hard: http://yosefk.com/blog/why-custom-allocatorspools-are-hard.html - Reconsidering Custom Memory Allocation: https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf I think it's good to test your assumptions and this article is a nice example of that.

It's nonsense, the only way to make a GC fast is to use many times as much memory as manual management (about 6x to reach parity[1]). [1] https://people.cs.umass.edu/~emery/pubs/04-17.pdf

>Custom allocators have the disadvantage of not knowing how the rest of the heap is managed. Sure, but they can work out of a fixed region statically allocated on startup. >Feel free to examine the many papers put out by Emery Berger on the subject of memory allocation as well as the design documents of jemalloc. https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf Do you have links to more? I only found one paper from 14 years ago. It is interesting but it comes with a huge YMMV since they only examined specific projects which might not have a pattern that _requires_ a custom strategy anyway. A distribution of the objects along with time distribution would have been nice to include.

Well, another solution (which I believe is common) is to just tenure the young object (and recursively tenure objects reachable from it) if the write barrier trips, which preserves the precision. There isn't anything fundamental to generational GC that requires imprecision. It's an implementation detail. In fact, David Ungar's original generational GC paper from 1984 [1] uses precise remembered sets. GHC's GC [2] among others uses precise remembered sets. [1]: https://people.cs.umass.edu/~emery/classes/cmpsci691s-fall2004/papers/… [2]: https://ghc.haskell.org/trac/ghc/wiki/Commentary/Rts/Storage/GC/Rememb…

For memory allocation you often don't need to use malloc. Many (most?) programs can mmap a large region of memory and allocate from that. Then when you're done you just unmap the region. This is much easier to use since you only have to remember to unmap once as opposed to handling each allocated pointer individually. Edit: Here is a paper that touches a bit on what I'm describing: http://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf Also, to clarify, I'm not saying allocate yourself a big piece of memory and then reimplement malloc on top of that. You literally just get a big piece of memory and every time you want to allocate something new from it add the offset to the base address, return that to the user, and store the new offset.

> How can a GC'ed language be faster than a non-GC'ed language? Why should a GC'ed language be slower than a non-GC'ed language? The evidence [0,1,2] seems more like: Given enough memory, GC is faster. Non-GC means explicit deterministic free at specific points in the source code. A garbage collector has the freedom to delay the free, which means potential for optimization. [0] https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf [1] http://www.cs.princeton.edu/~appel/papers/cmljava.html [2] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.1816

You could do better with a probabilistically secure allocator instead: http://people.cs.umass.edu/~emery/pubs/ccs03-novark.pdf Randomized allocation makes it nearly impossible to forge pointers, locate sensitive data in the heap, and makes reuse unpredictable. This is strictly more powerful than ASLR, which does nothing prevent Heartbleed. Moving the base of the heap doesn't change the relative addresses of heap objects with a deterministic allocator. A randomized allocator does change these offsets, which makes it nearly impossible to exploit a heap buffer overrun (and quite a few other heap errors).

No, it integrates HITs into a programming language (e.g. Amazon Turk tasks can be called dynamically through function calls). See the paper: http://people.cs.umass.edu/~emery/pubs/res0007-barowy.pdf

This has been shown not to work: http://people.cs.umass.edu/~emery/pubs/TFS.pdf (section 5.1) I wonder how they think they're getting around the upstream replication bottleneck.

> Because about 99% of the time the garbage collect is a negligible portion of your runtime lol .. reality disagrees with you. https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf#:~:text=We%20sh… On page 3 they broadly conclude that if you use FIVE TIMES as much memory as your program would if managed manually, you get a 9% performance hit. If you only use DOUBLE, you get as much as a 70% hit. Further on, there are comprehensive details on the tradeoffs between style of GC vs memory consumption vs performance. --- Moving a value from DRAM into a CPU register is an expensive operation, both in terms of latency, and power consumption. Much of the code out in the "real world" is now written in garbage collected languages. Our datacenters are extremely power hungry (as much as 2% of total power in the US is consumed by datacenters), and becoming more so every day. The conclusion here is that garbage collection is fucking expensive, in real-world terms, and we need to stop perpetuating the idea that it's not.

I think using a different language, so a different compiler with a different level of maturity introduces too many conflating variables to decisively answer this question. I don’t know the answer, either, but believe (corrections welcome) https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf hasn’t been refuted. It says GC needs 5 times the memory as manual memory management for equivalent performance, 3 times the memory for about equivalent performance. Of course, that paper isn’t about reference counting. I think reference counting should see the same alloc/free patterns as manual memory management, but with larger allocation sizes, to accommodate for a reference count in every object allocated. Even if that, on average, doubles the allocation size, that 5 and 3 still would be 2½, respectively 1½. That logic may be broken, but I also notice that iPhones typically have less memory than Android phones, yet seem to do well in reviews. That, too conflates variables, so GCvs Reference Counting may not be a big factor, but if not, what is the reason?