Pavel Zemtsov

Did you consider something like the following to speed up the calculation? This method can go forward _really_ fast. Not sur e if it can also go backward... https://pzemtsov.github.io/2015/04/24/game-of-life-hash-tables-and-has…

> A compiled binary that worked correctly won't suddenly run into undefined behaviour, that can only happen if you upgrade compiler versions... Upgrading the compiler version and causing undefined behavior sounds pretty sudden. Here's my favorite story along those lines: http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html

A concrete example of undefined behavior caused by an unaligned pointer: https://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html

Interestingly enough, I've just checked the code from this article (https://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html)… linked by the OP's article, and guess what - CLANG generates code that does not crash, although it still uses SSE4.

I think syscalls are not as slow as many people imagine they are, especially with modern CPUs and kernels (there are special instruction for syscalls that are faster than the old "interrupt" approach). See here: http://pzemtsov.github.io/2017/07/23/the-slow-currenttimemillis.html ("Off-topic: A system call"). But they will be slow with this mitigation.

I believe the article was http://pzemtsov.github.io/2017/07/23/the-slow-currenttimemillis.html Although I couldn't easily find it in my history or on HN. I had to resort to the Googles.

> creating a super-optimized tight loop of a program in C++, C or (better) assembly might be much easier to do than in higher level JITted languages Sometimes, the situation is exactly opposite, though: http://pzemtsov.github.io/2014/05/15/how-to-make-C-as-fast-as-java.htm…

Oh I see you didn't get to caveat 5: you can't read anything more complicated than raw bytes, i.e. chars, because of unaligned memory access errors. Let's say you mmap a file and do something like this: char *fileContents=...mmap etc...; int headerOffset=*(int*)fileContents; int *someListOfNumbers=(int*)(fileContents+headerOffset); int importantSum= someListOfNumbers[0] + someListOfNumbers[1] + someListOfNumbers[2] + someListOfNumbers[3]; If headerOffset is not a multiple of 4, bad things will happen. On x86 you'll get away with it... Unless you were summing many more ints and the compiler decides to use aligned SSE loads for speed[1]. On ARM you'll get a SIGBUS just for trying to read unaligned ints (IIRC). You can fix that by wrapping your ints or whatever you're trying to read in a packed struct, but it is yet another thing to keep in mind. [1] http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html

> I like the idea of software, libraries and application code all being accessible to each other and have the ability to call each other. Too much functionality is siloed behind application A or B, and the interface exposed by these applications is often woefully inadequate for anything interesting. Maybe I'm skeptical, but I've gotten to the point where I believe that this specific problem is not going to be solved anytime soon. Here's how I see it. I believe that unification, specifically within software, is universally based on the idea that we can clean up the world (where "the world" refers to a given software environment) if only we can reimagine it clearly enough and, to borrow a few technical concepts, compute a delta with enough detail and resolution that it'll carry us from where we are to where we want to be. (With the implicit [mis]understanding that all we need to do is make said delta exist, and then it'll do all the work of carrying us.) I think the above is what we're trying to do - and that we're doing it because we're staring at/barking up the wrong set of trees (the associated forests of which got lost long ago). Ultimately, the world is a disorganized mess. Any security researcher could entertain you for days with highly-detailed hair-raising examples and proofs of this. I've very recently (finally) realized this is because, at a very high level, the world operates somewhat similarly to the bazaar model Linux uses. Evolution is highly organic, often…

> Optimizers don't look for undefined behaviour to then do whatever, they look for the set of defined behaviours and then try to generate the cheapest code that behaves correctly in all those cases--and if that means that undefined cases do weird stuff, that's an acceptable side effect. > if any of the bytes are not inside any valid C object, then that would produce undefined behaviour The compiler might also assume that every one of these loads will be exactly 4 bytes apart. Even worse, it might know that int32_t allocations will always be aligned, so it will assume that every load will be 4-byte-aligned. It then might output code that loses track of the bottom two bits, or SSE instructions that only work with certain alignments. This isn't theoretical, this exact kind of bug has happened before, on a normal non-malicious compiler: http://pzemtsov.github.io/2016/11/06/bug-story-alignment-on-x86.html https://news.ycombinator.com/item?id=12889855 It's true that the buffer overrun issue is the same for either version; I misread what was being replaced. But the alignment issue is specific to this version, and can't be ignored.