Clojure Goes Fast

Like others said, Clojure does not reflect on every call. A good resource if you really want to push the perf limit http://clojure-goes-fast.com

What sort of scale are you working at? At our scale, it is normal to look into these sorts of things. Reading and understanding and applying articles like the following is absolutely necessary. http://clojure-goes-fast.com/blog/shenandoah-in-production/

Line count is not very useful to compare without the context of standard library size, third-party dependencies, etc. The code in TFA depends[1] on a JSON library[2] that is about a thousand lines of code (excluding tests) wrapping a Java library for JSON decoding. Then there's other things to consider, like the fact that this LSP client, while succinct, pays not only the cost of loading Jackson, but also the cost of loading clojure.core, which is quite non-trivial[3]. Startup time for LSP servers and clients definitely matters to some, considering that e.g. even clojure-lsp recommends running native executables over JAR files[4]. Can't find documentation proving it's for quick startup, but it's a plausible rationale for their recommendation of a binary over a JAR. Note: I have used Clojure professionally and in hobby projects. I think it's nice that one can interactively develop a minimal LSP client and the resulting amount of work is roughly 200 lines of code. I say "minimal" because it's unclear how this client deals with offsets reported by LSP servers, which are all given as offsets in a UTF-16 encoded string. In any case, I still think advertising "LSP client in 200 lines of code" hides valuable information regarding functionality, implementation, "actual" code size, and trade-offs made in the choice of technology stack. [1] https://github.com/vlaaad/lsp-clj-client/blob/a567e66/deps.edn [2] https://github.com/metosin/jsonista/blob/c8f2b62/project.clj#L20-L22 [3]…

See also http://clojure-goes-fast.com

Lazy sequences reduce the size of intermediate collections but they “chunk” - you get 32 items at a time, multiply that by however many transformations you have and obviously by the size of the items. There are some additional inefficiencies in terms of context capturing at each lazy transformation point. The problem gets worse outside of a tidy immediate set of transformations like you’ll see in any example. This article gives a good overview of the inefficiencies, search on “thunk” for tldr. https://clojure-goes-fast.com/blog/clojures-deadly-sin/ (I don’t agree with its near condemnation of the whole lazy pattern (laziness is quite useful - we can complain about it because we have it, it would suck if we didn’t).)

In Clojure, where lazy evaluation is also the default (although only for sequences), someone recently wrote an in-depth article about the issues caused by this choice: https://clojure-goes-fast.com/blog/clojures-deadly-sin/ I haven’t read the whole article yet, but maybe it could be interesting to compare it with Haskell and see if some of the same problems occur or if Haskells compiler or overall design around (pervasive) lazyness makes it easier to work with and more performant.

Just to add some reference to why it can take a long time to start a REPL (for example): > As it turns out, Clojure start time is a complicated and multi-dimensional topic. Clojure projects are slow to start not only because of JVM — JVM itself starts in ~50 ms — but because of JVM specifics the classes are loaded slowly. Clojure projects are slow to start not only because of Clojure — Clojure itself starts in ~1 second — but because of Clojure specifics, the namespaces, especially not AOT-compiled one, are loaded slowly. And so on. http://clojure-goes-fast.com/blog/clojures-slow-start/ A more recent comparison of Clojure, Java and some other Clojure-inspired languages: https://stuartsierra.com/2019/12/21/clojure-start-time-in-2019

Whenever "Clojure startup time" comes up, people are quick to point to one thing: either it's the JVM "bloat," or that it's nothing to do with the JVM and is actually all Clojure. Here's a pretty good (if a bit outdated) analysis on Clojure startup time: http://clojure-goes-fast.com/blog/clojures-slow-start/ The question is, could Clojure be rewritten in a way that loads significantly faster with the same behavior on the JVM? I'm not sure. And especially on big projects, the lack of DCE on the JVM can lead to loading a ton of useless code that significantly impacts startup time outside of the Clojure language.

This is what I am referring to: http://clojure-goes-fast.com/blog/clojures-slow-start/ > As it turns out, Clojure start time is a complicated and multi-dimensional topic. Clojure projects are slow to start not only because of JVM — JVM itself starts in ~50 ms — but because of JVM specifics the classes are loaded slowly. Clojure projects are slow to start not only because of Clojure — Clojure itself starts in ~1 second — but because of Clojure specifics, the namespaces, especially not AOT-compiled one, are loaded slowly. And so on.

Ah nice, we do want to switch Nextjournal to Shenandoah eventually after reading http://clojure-goes-fast.com/blog/shenandoah-in-production/ So is graal polyglot already compatible with newer JDKs and it's just native image generation that's blocking the upgrade? https://github.com/oracle/graal/issues/651

Clojure startup time is only partly related to JVM startup time, which is ~60ms[1][2]. The reason Clojure startup is slow is because Clojure requires a very elaborate initialization. OTOH, the Erlang VM (BEAM) is much, much, much slower than the JVM (Hotspot, at least). [1]: https://blog.ndk.io/clojure-bootstrapping.html [2]: http://clojure-goes-fast.com/blog/clojures-slow-start/

> - Incredibly slow Almost as fast as Java, and for performance critic processes 99.9% of the time we bump on infrastructure performance problem BEFORE hitting the wall with Clojure (see https://clojure-goes-fast.com/blog/ for tips and tooling about performance) > - Hosted on the JVM, so you're going to be dealing with Java eventually For me it's a major selling point: JVM is battle tested and a wonder of engineering. The Java ecosystem richness is incredible (see tooling above for monitoring and profiling for an example). And the platform is constantly moving forward (see latest JDK with virtual threads, generational ZGC, etc.) > - No automatic tail call optimization because of the limitations of the JVM, so it feels like you're writing a mess of macros rather than real functional code Never have been a problem and I don't see the point with macros, and code we write looks _very_ functional...

Based on the facts stated in the article (part 2), it looks like (years later) Java now has a new garbage collector available (ZGC, still labeled experimental, not used by default) [1]. It's an order of magnitude lower latency than current default collector, but it still has an order of magnitude higher latency target (10ms vs 1ms) than Go. I'm sure there are many good reasons for the trade offs made and that's a huge improvement, but in light of the basic numbers I'm not sure that "the Java guys are winning the low latency game" is, in fact, a fair conclusion. Go introduced a new GC a while back with dramatically lower latency numbers, not only compared to the previous Go GC but any other commonly available GC in any language (excluding exotic commercial ones like Azul). Neither the old or new Go GCs are particularly sophisticated compared to the highly developed ones in the Java ecosystem. The author of these articles seems to take objection that the latency numbers were achieved not by magic or pure GC implementation genius but by simply optimizing for latency, which involves some trade offs. I don't think this negates the usefulness of having a low latency GC available though, although I can understand how it might be frustrating to see a project getting a lot of attention for what feels like a lesser intellectual achievement. Obviously not all projects even require low worst-case latency, monolithic apps will be less sensitive, applications waiting on 10 other…