Qin Sibo

There's a retrospective here which addresses quite a lot of it: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html Partly it sounds like performance-critical code at Google tends to get written in a lower level language instead, so Python performance isn't as critical, even though they use it quite a bit. Agree that it'd be great if they helped out the PyPy team though, since it's showing such great promise and has some pretty impressive numbers already.

A lot of compilers will optimize out a NULL pointer check because dereferencing a NULL pointer is UB. Because assert will not run the following code in the case of a NULL pointer, AFAIK this exact code is still defined behavior, but if for some reason some code dereferenced the NULL pointer before, it would be optimized out - there are some corner cases that aren't obvious on the surface. This kind of thing was always theoretically allowed, but really started to become insidious within the past 5-10 years. It's probably one of the more surprising UB things that bites people in the field. GCC has a flag "-fno-delete-null-pointer-checks" to specifically turn off this behavior. https://qinsb.blogspot.com/2018/03/ub-will-delete-your-null-checks.htm… This is an actual Linux kernel exploit caused by this behavior where the compiler optimized out code that checked for a NULL pointer and returned an error. https://lwn.net/Articles/342330/

Unladen swallow has never been an official attempt to solve that. It was the initiative of one guy (http://qinsb.blogspot.fr/2011/03/unladen-swallow-retrospective.html), during an internship, with basically zero support. And so google never even consdered as a serious solution. Beside, when they took the Go decision, they were still using Python 2.4. It had poor unicode support, bad async io, no multiprocessing pool, aweful packaging and deployment story and the project of a 3.X breaking everything. It made a lot of sense, business wise. If you have to rewrite your code base, better rewrite it in a language you control (no PSF to fight against), specifically design for your workfload, and fixing all those quirks, plus running faster and eating each memory. Today's Python unicode support is top notch, asyncio ensure easy networking code, you can use multi-core easily, deployment is pretty much a solved problem and 3.X is running pretty much for 90% of people. You even have a hook to plug in a JIT inside CPython, waiting to be used, and Type hints to facilitate the handling of huge code bases. Their decision would probably have been different then, although that wouldn't have made Python any easier to speed up. But if a few smart people managed to make it work, by rewritting python in python, no less, I'm sure a dedicated google team would have done great. They did it for the slug that was JS after all. But they didn't have the luxury to be able to change the language for…

Ah, that's cool. Thanks for the correction. I was misremembering the Unladen Swallow retrospective[1]. It's fair to say they used a lot fo their available time contributing to LLVM, but it sounds like that was feature work, not bug focused. [1] https://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

Unladen Swallow was only an internship project. http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

Yep, it was tried. See: Unladen Swallow - an experimental CPython fork trying to speed up execution by removing some of the least used dynamism. It started 10 years ago and was abandoned 8 years ago. Retrospective at http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

Parallel post from Unladen Swallow… http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

But it was no fault of Python: https://qinsb.blogspot.nl/2011/03/unladen-swallow-retrospective.html

Unladen Swallow started as an intership project and didn't get very much support at Google [1]. When US started (~2009), Google had been writing their core, high-performance stuff in Java and C++ for many years already so there was no internal or existential need to make Python faster. [1] http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

Those who do not learn history are doomed to repeat it. http://qinsb.blogspot.kr/2011/03/unladen-swallow-retrospective.html

for a moment there i thought this was a new attempt at unladen swallow... [if you've not clicked, it's a python lib to access llvm tools] http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

Indeed: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

There was a PEP/branch (p3k-jit) to merge Unladen Swallow into mainline, but the PEP was withdrawn as Unladen Swallow faded to black: http://qinsb.blogspot.be/2011/03/unladen-swallow-retrospective.html http://www.python.org/dev/peps/pep-3146/

Unladen Swallow is dead and won't be merged into CPython. http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html

See also Collin's comment on the post: http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html?s…

I'm by no means an expert on JIT compilation, but I can give a less technical answer. I think the primary reason is that most of the developers (both CPython core developers and Python developers) do not consider Python as a tool for writing perfomance critical software. So, any JIT initiative, which requires a lot of resources, doesn't get enough of them. CPython allows to write C extension modules quite easily, so if you need to write some perfomance critical piece of code , you just switch to C and call a C function from Python. Unladen Swallow was a version of CPython that added JIT support. PEP-3146 describes an unsuccessful attempt to merge it into CPython [1]. Here's a postmortem [2] Starting with version 3.6, CPython allows to set a custom frame evaluation function on the interpreter and to store JIT compiled code in a code object. This was done to support JIT in a form of a separate module and is described in PEP 523 [3]. This is very interesting topic. I should study it in detail and probably write a post. I hope I'll be able to give a more thorough answer someday. [1] https://www.python.org/dev/peps/pep-3146 [2] http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html [3] https://www.python.org/dev/peps/pep-0523/

I think mostly because of missing a Mike Pall :) Effectively though, Ruby/Perl are just much larger languages. E.g. ruby has/had at least 5 different ways of controlling flow in a block between throw/catch, next/break, raise/rescue, return/implicit-return and callcc. 99% of the code uses the same small subset, but if you want to support "Ruby" you need to implement all of those. Likewise, Perl has maybe simple semantics underneat, but the sheer amount of syntax-level elements is insane, also because of stratified version changes. e.g. did you know that declaring a variable with "local" instead of "my" makes it behave like a lisp-ish dynamic variable? I am fairly sure that does not exist in Lua. As for python, there is PyPy, but I think the retrospective view on Unladen Swallow has some good points, basically: lack of enough interest from devs, lack of enough interest from users. http://qinsb.blogspot.hu/2011/03/unladen-swallow-retrospective.html

I have some answers! * Yes, the JIT is part of the RPython translation toolchain (more on this in a few) and can be used in other interpreters, for example we have Scheme, Javascript, Prolog, and Haskell in varying level of completeness (Prolog being the most complete). * Highly optimized C? Not spectacularly, normal C, not bad, on numerical code we often hover around gcc -O1. * They weren't representative, most of that code was heavily optimized for CPython, which has very different performance characteristics. * RPython is the language the PyPy interpreter is written in, it's not really meant for general purpose code, it's designed mostly for VMs, but it does run at basically C-like speed. * Unladen Swallow was Google's fork of CPython, it's basically dead at this point, retrospective here (http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospective.html), Psyco was an extension module for CPython that added a JIT, it's no longer maintained (but still works), it's creator is the lead developer of PyPy. * Cython isn't Python (this could be a meme or something), it does not implement the entire language and thus isn't directly comparable (although we still often compare, because it is a competitor in the space of "making Python-like stuff faster").