Stefan Marr

There aren't many people looking at these JITs at the moment. Stefan Marr[1]'s group[2] is, I believe, the where most of the research is currently done. A recent paper[3] compares performance of interpreters in RPython and Graal. Their baseline performance is Java, and they achieve performance close to V8, which itself is about 2x slower than Java. My summary is you can write fast interpreters + get JIT for free, but fast JIT for dynamic languages still means 2x slower than JIT for statically typed languages (and Java definitely leaves some performance on the table due to how it represents data). [1]: https://stefan-marr.de/ [2]: https://research.kent.ac.uk/programming-languages-systems/ [3]: https://dl.acm.org/doi/10.1145/3622808

> Or, in other words, the performance difference comes somewhere else than the choice of programming language. We're comparing the code generated by a compiler or VM/JIT of a specific programming language, not the programming language itself. I'm amazed that this is even up for discussion in the article. It is clear that even with the same language there can be differences in implementation. Here is e.g. a measurement series I did to compare different Smalltalk implementations (LjSOM is mine): http://software.rochus-keller.ch/are-we-fast-yet_crystal_lua_node_som_… Smalltalk is a tough nut to crack because nearly all statements are closures; the Open Smalltalk VM team invested two decades to achieve a decent performance, but it's still quite behind e.g. V8 or a statically typed AOT compiled language. The Are-we-fast-yet suite (https://stefan-marr.de/papers/dls-marr-et-al-cross-language-compiler-b… gives more credible results than CLBG (referenced in the article) from my point of view because of the stricter rules the implementations have to adhere to (in favour of fair comparisons) and the larger benchmark apps. In return, the implementation gives significantly more work, but it is doable and worth it; here is one of my implementations that helped me significantly improve the compiler (still in progress): https://github.com/rochus-keller/Oberon/tree/master/testcases/Are-we-f… this also demonstrates that -…

Please see Figure 2 in the blog post https://stefan-marr.de/2023/10/ast-vs-bytecode-interpreters/#fig-inter… or Sec. 5.2 in the paper https://stefan-marr.de/papers/oopsla-larose-et-al-ast-vs-bytecode-inte… We report on results for both, with and without just-in-time compilation. The specific focus for this work was pure interpreter performance in the context of metacompilation systems, but before compilation had a chance to kick in. For both RPython and Truffle/Graal, it's possible to disable the JIT compilers and measure pure interpreter speed.

The key problems with this approach is not mentioned in the blog post, but is shown in figures 6 and 7 of the paper - https://stefan-marr.de/downloads/acmsac23-huang-et-al-optimizing-the-o… Basically, the code handler ordering does not generalise well across benchmark nor processor, so to get the speedup they see, you'd need a specialised interpreter for your specific benchmark and processor. That puts this into the "interesting, but not very practical" category.

If Rust wasn't able to elide bounds checks, that is most likely where the perf difference is from. Analyzing the output the assembly output from goldbolt or looking at the MIR can help. https://stefan-marr.de/2022/10/cost-of-safety-in-java/ Unsafety buys you a little more performance, but the baseline should the safe version, not the unsafe version. It is like having to explain on a case by case basis why you aren't using lead pipes for this application. Always default to safety. The difference between safe and unsafe native code is usually single digit percentage points. Or weeks on a Moore scale.

The 2008 RoarVM project allowed Squeak to run on a 56 core Tilera chip and was later ported to run on multiprocessor x86 machines: https://stefan-marr.de/renaissance/ https://github.com/smarr/RoarVM This was a research project so performance on a single core was poor compared to the official Squeak virtual machine, but it was an interesting exploration of the natural fit between objects/message passing and multiple cores.

The Truffle based implementations of dynamic languages get close depending on what you’re benchmarking. https://stefan-marr.de/papers/dls-marr-et-al-cross-language-compiler-b…

Note that PyPy does something similar, but based on (meta-)tracing instead of partial evaluation. See https://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-part… for a discussion.

Saw this benchmark comparison between RPython vs Graal the other day: http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-parti… Is there a scheme implementation on graal vm?

This is super cool. It looks like you are in the process of porting the .js to .lua? Wasm will exist in many more places than Js will. I think I would have started with many of the Lua on Js implementations and used zero Js. Have you seen http://stefan-marr.de/2015/10/jit-data-structures-fully-reflective-vms… Check out this paper, https://www.cs.purdue.edu/homes/rompf/papers/amin-draft2017a.pdf Collapsing Towers of Interpreters

> Just fix the VM then No the point isn't that the VM is broken. The point is that the Ruby language has semantics and is used in idioms that nobody knows how to optimise well yet. > tinyrb based on potion based on lua with classes is ~200x faster in method calls tinyrb is interesting, but it just isn't the same semantics as Ruby: https://github.com/macournoyer/tinyrb#what-wont-be-in-tinyrb-tiny-patc… tinyrb doesn't have a better VM - they're implementing a different programming language with all the hard bits left out! In my project we've been having to do new original research into how to optimise Ruby - introducing concepts such as dispatch chains which haven't been needed more because people don't try to push metaprogramming in most language like they do in Ruby. http://stefan-marr.de/downloads/pldi15-marr-et-al-zero-overhead-metapr…

RPython is very similar to Graal+Truffle, except that RPython is based on tracing and Graal+Truffle is based on partial evaluation. There's a nice paper comparing the two approaches and their relative merits: http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-parti…

Research on partial evaluation (PE) was fashionable in the 1990s, but largely fizzled out. I was told that was because they could never really get the results to run fast. I'm trying to understand why. Clearly meta-tracing and PE have a lot of overlap. Truffle is based on some variant of dynamic PE if I understand what they do correctly. Most of the 1990s work in PE was more about static PE I think. The paper [1] touches on some of these issues, but I have not studied it closely yet. [1] http://stefan-marr.de/downloads/oopsla15-marr-ducasse-meta-tracing-vs-…

Check out the work on roar vm https://github.com/smarr/RoarVM if you are interested in such things. There are rad videos here too: http://stefan-marr.de/renaissance/

We compared the runtime performance of GraalSqueak with OpenSmalltalkVM (formerly Cog) and RSqueak (RPython-based VM) in Figure 4 of our MPLR’19 paper [1] for the first time. Since then, we've worked on a couple of performance optimizations. So at the moment, I think GraalSqueak is only slower in the DeltaBlue benchmark... in all others, it's (often significantly) better than OpenSmalltalkVM. However, please also note its limitations which we discussed in section 5.2, especially with regard to Smalltalk's interrupt handler and Partial Evaluation performed by the Graal compiler. So far, I'd say our observations wrt RPython match the ones discussed in "Tracing vs. Partial Evaluation" [2]: Language implementers have to do more work in Truffle, but probably get better peak performance in return (not talking about warmup or memory consumption here). RE "a more optimized bytecode" set: Sista [3] (an extension of the OpenSmalltalkVM) is doing something like that, too. I'd guess performance would be more or less the same using GraalVM as Truffle produces highly specialized code. But, of course, this would need to be benchmarked. The advantage of the Sista approach is that it's managed on the image level, so specialized versions of methods are persisted as part of the image. AFAIK the GraalVM team is working toward persisting compiled code caches, which is kind of similar but on the level of the language implementation framework. Hope this answers your questions! [1]…

It's interesting to read the OP's response to partial evaluation (employed by Truffle) and why he switched to meta-tracing in RPython. Basically, the inlining/specialization decisions in PE-based JITs were harder to control than just letting the VM observe loop iterations. Here's a paper regarding a pair of language implementors' experience using both techniques: http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracing-vs-parti… By changing the abstract syntax tree during execution, type-generic nodes in the AST can be replaced with type-specialized nodes. This carries more overhead than tracing through the interpreter. It also requires more explicit work from the user to achieve optimized results. The authors of the above study conclude that the resulting performance was similar, but that meta-tracing was an easier technique to use than PE. This agrees with the OP's assessment.

Particle systems [1]. AR. VR [2][3][4][5]. Image processing, neural nets. But not in programming languages like C, Java or Linux. Only in massively parallel message passing languages[4]. I suggest a Smalltalk message passing scalable cluster VM like the RoarVM [6][7][]8]. [1] Shadama 3D. Yoshiki Oshima. https://www.youtube.com/watch?v=rka3w90Odo8 [2] OpenCobalt Alpha. https://www.youtube.com/watch?v=1s9ldlqhVkM&t=13s [4] OpenCroquet Alan Kay Turing lecture. https://www.youtube.com/watch?v=aXC19T5sJ1U?t=3581 [5] OpenCroquet Alan Kay et al. https://www.youtube.com/watch?v=XZO7av2ZFB8 [6] RoarVM paper. David Ungar et al. https://stefan-marr.de/renaissance/ungar-adams-2009-hosting-an-object-… [7] RoarVM Demo. https://www.youtube.com/watch?v=8pmVAtk612M [8] RoarVM Demo. https://www.youtube.com/watch?v=GeDFcC6ps88