Per Liden

You're right; Java has ZGC, a best-in-class garbage collector with average pause times of 50 microseconds while still having great throughput. https://malloc.se/blog/zgc-jdk16#:~:text=With%20concurrent%20thread%2D…

>I didn't find any other measurements on the internet, please point me to them if you find them. It has been years since I have written thousands sockets servers (used in forex), yet even a couple milliseconds per write would have made the entire operation useless. >another explanation could be false sharing effects that I didn't know well at the time. False sharing sucks, of course, but milliseconds seems way way too much again penalty. You'd need all cores updating the same cache line, even then I doubt it'd be that bad. Could it be the virtualization layer? (again I have run 'that' on bare metal only) >I think it should be possible to do some similar stuff with multiple readers and/or writers Multiple writes should be avoided entirely, contention on writing is what prevents scaling - false sharing is pretty much that. A writer honoring the readers is pretty simple - each reader has its own pointer and the write should fail (or spin) when the buffer is full. I think that's quite a classic structure and indeed - it requires no extra memory to communicate/hand-off. About latency - this is java's current GC that does still has a compaction phase. https://malloc.se/blog/zgc-jdk16

> How do you find a reasonably small Xmx that doesn't lead to OutOfMemoryError exceptions? That's tough to figure out. In new versions of Java, I think 14+, if you use ZGC collector it will return unused memory to the OS. Memory options vary depending on collector, but new versions of ZGC support "soft max" heap size and uncommit. Together it might be close to what you're looking for https://malloc.se/blog/zgc-softmaxheapsize I should mention the GC situation was worse until the last few years. Until ZGC and Shenandoah came around, Java still didn't collect frequently but when it did there were long pauses. This is what V8 and Go's collectors were designed to avoid. They have more overhead from collecting frequently, but low pauses. With the new Java collectors you get the best of both worlds. You can set heap ratios and such for older collectors to decrease Java memory use with those, but IMO you're better off using ZGC and uncommit these days

ZGC (JVM) now has guaranteed maximum pause times of 500 microseconds, with average pause times of just 50 microseconds: https://malloc.se/blog/zgc-jdk16 Pause time is not the same as cycle time, but it’s what’s important for reactivity.

Low pause collectors exist explicitly for this purpose. ZGC and Shenandoah both trade throughput (more concurrent collections) for extremely low pause times: https://malloc.se/blog/zgc-jdk16

> - no global gc pauses, low-latency ZGC has sub-millisecond pauses https://malloc.se/blog/zgc-jdk16 And afaik azul's C4 collector has no global pauses, only per-thread pauses (which are also short)

Max pause times of 0.5ms is what got me really interested. It feels like a huge trade-off of GCs is almost completely gone. https://malloc.se/blog/zgc-jdk16

Enjoy, https://malloc.se/blog/zgc-jdk16 I guess Oracle, IBM, Azul, PTC, Microsoft can provide the necessary data about their pauseless GC implementantions to any relevant customer.

From what I understand Python uses reference counting, which is a type of garbage collection that allows you to have deterministic performance, almost no "pauses" but at the cost of performance. About the consistency, I think that's not a GC problem in general. For example, Java's ZGC has constant pause times: https://malloc.se/blog/zgc-jdk16.

C# has added a lot of control over memory layout. Span [0], structs etc. I am hoping they add more D like features (@nogc) to the language. Java itself has just made strides on reducing GC pause times [1]. Granted, this isn't optional GC, but does D really give you that? Don't you lose the entire ecosystem and stdlib with -asBetterC? Why would I choose that over rust/zig/c++? FWIW, I am a big fan of D. [0] https://docs.microsoft.com/en-us/archive/msdn-magazine/2018/january/cs… [1] https://malloc.se/blog/zgc-jdk16

Hotspot has sub-millisecond GC pause times. "But as long as your system isn’t heavily over-provisioned, you can expect to see average GC pause times of around 0.05ms (50µs) and max pause times of around 0.5ms (500µs)." https://malloc.se/blog/zgc-jdk16

You're mixing two things here: memory management and memory layout. This "pointer-heavy programming" is, indeed, a bad fit for modern hardware in terms of processing speed due to cache misses, which is why even Java is now getting user-defined primitive types (aka inline types, aka value types), but in terms of memory management, in recent versions OpenJDK is pretty spectacular, not only in throughput but also latency (ZGC in JDK 16 has sub millisecond maximum pause time for any size of heap and up to a very respectable allocation rate: https://malloc.se/blog/zgc-jdk16 and both throughput and max allocation rate are expected to grow drastically in the coming year with ZGC becoming generational). As far as performance is concerned, GC can now be considered a solved problem (albeit one that requires a complex implementation); the only real price you pay is in footprint overhead.

ZGC in its current form trades a bit of throughput performance for better latency. This presentation provides some more details and some performance numbers (a link to the slides is also available there). https://malloc.se/blog/zgc-oracle-developer-live-2020

Try ZGC, sub 1ms pauses https://malloc.se/blog/zgc-jdk16

There is a worst case that is much worse then the now mentioned 1ms. Namely really big change in allocation rate in which case the gc cycle does not finish before running out of memory (ZGC uses allocation rate to determine when to start next gc cycle). In that case ZGC stops allocations ("Allocation Stall" in gc logs). But this is basically a failure mode and should not happen during normal operations at all. Though you can configure for this in a couple ways if you run into this issue 1. By telling it to treat some amount of heap as the max that is not actual max when it comes to it calculating when it should start next gc cycle. (-XX:SoftMaxHeapSize) https://malloc.se/blog/zgc-softmaxheapsize 2. Increasing the amount of concurrent gc threads so they will finish their work faster (-XX:ConcGCThreads) 3. Just run really large heap so the "run a gc cycle even if we don't need to based on allocation rate" gc cycle keeps the heap in check. Though after JDK 15 we have not had to mess with any of these. Prior to that we had to adjust soft max heap size a bit. With JDK 16 it should be even better I guess (should be upgrading sometime next week)

The HotSpot JVM does return memory to the operating system. https://stackoverflow.com/a/30464183/839689 , https://www.stefankrause.net/wp/?p=14 , https://shipilev.net/jvm/anatomy-quarks/21-heap-uncommit/ , https://openjdk.org/jeps/346 , https://openjdk.org/jeps/351 , https://malloc.se/blog/zgc-jdk15#incremental-uncommit