Dougall Johnson

> store-store dependency You can make it an exchange rather than a plain store. Questionably worthwhile but you can do it. swp is only 3 cycles on firestorm https://dougallj.github.io/applecpu/firestorm-int.html

> Out-of-order pipelines actually have a really elegant way of handling flags, they just store a copy of the flags register on every ROB entry. And yet, flag writing instructions are usually half the throughput on regular ALU instructions, on modern wide ooo designs. [1, 2] But I generally agree, there is a good way of handling them, it's hust unclear to me how expensive it is. It can't be that expensive, but apparently it is expensive enough, to not put a mask register write port on all integer execution units. [1] https://dougallj.github.io/applecpu/firestorm-int.html (see how ADD is 6-issue and ADDS 3-issue) [2] https://developer.arm.com/documentation/111027/4-0/ (again 8-issue ADD, but 4-issue ADDS)

> There are only a handful of different instructions that account for 90% of all operations executed, and, near the top of that list are addition and subtraction. On ARM these can optionally set the four-bit NZCV register, whereas on x86 these always set six flag bits: CF, ZF, SF and OF (which correspond well-enough to NZCV), as well as PF (the parity flag) and AF (the adjust flag). > Emulating the last two in software is possible (and seems to be supported by Rosetta 2 for Linux), but can be rather expensive. Most software won’t notice if you get these wrong, but some software will. The Apple M1 has an undocumented extension that, when enabled, ensures instructions like ADDS, SUBS and CMP compute PF and AF and store them as bits 26 and 27 of NZCV respectively, providing accurate emulation with no performance penalty. https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

The breakout engineering to exploit Dolphin has already happened, see for example: * https://dougallj.wordpress.com/2016/11/13/exploiting-dolphin-part-1/ * https://gist.github.com/hthh/502ae16db55612f64d3966769a154c3e * https://github.com/dolphin-emu/dolphin/pull/4447

https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ > Apple M1 has an undocumented extension that, when enabled, ensures instructions like ADDS, SUBS and CMP compute PF and AF and store them as bits 26 and 27 of NZCV respectively, providing accurate emulation with no performance penalty.

Apple Arm CPUs have some special tricks to make x86 sogtware emulation faster. https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

Not really. A RISC design can have very complex timing and pipelines, with instructions converted to uOPs (and fused to uOPs!) just like X86: https://dougallj.github.io/applecpu/firestorm.html Caches can be fast and very expensive (in area & power)! I have an HP PA-RISC 8900 with 768KB I&D caches. They are relatively fast and low latency, given the time-frame of their design. They also take up over half the die area.

Apple CPUs do decode instructions into micro-ops. https://dougallj.github.io/applecpu/firestorm.html

https://dougallj.github.io/asil/ is like officedaytime but for SVE.

This post was very helpful when I tried to figure it out: https://dougallj.wordpress.com/2021/10/30/bit-twiddling-optimising-aar…

Ah, your gcc example does have the branchful branch that could be done branchlessly; I was thinking about my original example with a load, which can't be transformed back. On fusion, https://dougallj.github.io/applecpu/firestorm.html mentions ones that Apple's M1 does - arith+branch, and very specialized stuff.

I found this blog post reverse engineering Rosetta 2 translated code: https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

See “flag-manipulation” at https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

Besides TSO there are a bunch of ARM extensions designed to match x86 behavior. https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ I'm curious whether Oryon and X925 implement these instructions and whether other emulators like FEX use them.

They are pretty good: https://dougallj.github.io/applecpu/measurements/firestorm/LD4_multi_1… Gathers on AVX2 used to be problematic, but assume it shouldn't be the case today especially if the lane-crossing is minimal? (if you do know, please share!)

M1 has CRC32 acceleration intrinsics. https://dougallj.wordpress.com/2022/05/22/faster-crc32-on-the-apple-m1… https://github.com/corsix/fast-crc32?tab=readme-ov-file#apple-m1-perfo…

Here’s more information: https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

This is essentially what Apple did with Apple Silicon to make Rosetta fast: https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ There’s a number of CPU features that make running translated x86 code much faster than it otherwise would be, like “Alternate Floating Point Mode” that happens to exactly match the x86’s floating point behavior.

The flags get saved and restored on task switches and interrupts. I would think deciding whether those bits actually get used across that is infeasible, but yet, https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ says: “This almost entirely prevents inter-instruction optimisations. There are two known exceptions. The first is an “unused-flags” optimisation, which avoids calculating x86 flags value if they are not used before being overwritten on every path from a flag-setting instruction.” Does this mean the emulator is buggy (if I write a loop that sets and clears flags, but doesn’t do anything with them, an interrupt handler that samples the flag values would need to see both values), but in a way that no sane code would notice?

In my experience the M1 does have competitive SIMD performance? https://dougallj.wordpress.com/2022/04/01/converting-integers-to-fixed… https://dougallj.wordpress.com/2022/05/22/faster-crc32-on-the-apple-m1… https://lemire.me/blog/2020/12/13/arm-macbook-vs-intel-macbook-a-simd-… (I later optimised the slower benchmark in that post: https://github.com/simdjson/simdjson/pull/1708 ) Obviously the GPU will be better, but at one point I compared the M1 CPU to other ARM GPUs (in laptops at that time) and found it had both better memory bandwidth and compute throughput, which is quite funny.

The Firestorm cores are capable of issuing two 128bit stores per cycle [1], giving a bandwidth of 102.4 GBps. This matches the experiments by Andrei with mixed reads/writes [2], and I have personally verified that it is attainable for pure write workloads to L1 as well as L2. This does not mean 102.4 GBps is achievable on a single core, but if L2 bandwidth is the limiting factor it very well could be. [1] https://dougallj.github.io/applecpu/firestorm.html [2] https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2

Wow, I would buy every book written by this person, as the amount of knowledge and experience required to pull off a stunt like that feels overwhelming Does anyone know if https://dougallj.github.io/applecpu/firestorm.html is the "Dougall Johnson's work if you can find it" reference, or is there some dark web version?

>Division is slow I wondered if that's really still true, since I haven't done much assembly language programming since PowerPC was new. Here, it says the M1 has 7-9 cycles latency for division instructions, but throughput of 2 cycles per. https://dougallj.github.io/applecpu/firestorm-int.html "The M1 is 10x faster than the Xeon at 64 bit divides. It’s…just wow." So, given all of the other things that can slow you up, I wonder if it really makes sense to avoid division any more? (I guess the energy efficient "Icestorm" cores have throughput equal to latency, so it's only the "Firestorm" ones where it's super fast)

> to lack of documentation about speculation barriers Can you elaborate on this? I was having a look at the M1 instructions here [1] and it seems that they implement at least one of the barriers, CSDB, which is actually documented by ARM[2]. [1] https://dougallj.github.io/applecpu/firestorm-int.html [2] https://developer.arm.com/documentation/ddi0596/2020-12/Base-Instructi…

Apple M1, can do four fused multiply-adds per cycle with latency of 4 cycles. Interestingly enough it seems that the latency on the vector FMA is even lower. So it’s 16 float FMA per cycle. Source: https://dougallj.github.io/applecpu/firestorm-simd.html

There was just an article about exploits in Dolphin published yesterday that I found pretty interesting: https://dougallj.wordpress.com/2016/11/13/exploiting-dolphin-part-1/ I agree this will probably catch on as a popular new vector, especially given the popularity of emulators on mobile devices.

Oh, I see what you are saying. I don't consider it to be a heuristic because it's simply bruteforcing it. IMO a heuristic needs improve over brute force. > And 4 vs 8 is a pretty sizeable difference. True, but x86 was doing four instruction 20 years ago. As I mentioned the current state of the art (in a shipping product) is 9, and 9 is larger than 8. Importantly, this Skymont of leapfrogging decoders approach is scalable. > whereas Apple can just decode each op independently & only decodes 8. Apple isn't as free from serialisation as you suggest. Like X86, many instructions decode to multiple uops. According to research [1] instructions which decode to two uops are common and a few decode to as many as 12 uops. It also does instruction fusion, two neighbouring instructions can sometimes decode into a single uop. This all means that there is plenty of serialisation within Apple's decoder. And branching also creates serialisation. It's just not as simple as independently decoding eight instructions into eight uops every cycle. Simpler than what x86 implementations need to do, but not as brain-dead simple as you suggest. Actually, Skymont's approach has an advantage over Apple here, because it only needs to serialise within each 3-wide decoder. [1] https://dougallj.github.io/applecpu/firestorm.html

> I believe Rosetta 2 also has a JIT mode It does have JIT translation (not a JIT "mode" though, as it always use AOT translation, only relying on JIT translation at runtime for the parts that need it) > which is a bit more like conventional emulators Not at all†, Rosetta 2 does the same†† translation step on dynamic Intel code, whose arm64 output can be reused afterwards > But it's used infrequently, eg when dealing with x86_64 apps that themselves use a JIT Yes, although it's more like "exceedingly rarely" in practice since usually those interpreters are up to date enough to have a native arm64 release. See there for details: https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ † Unless you've been meaning dynarec, but I would not call that "conventional" although it is a well-known technique https://en.wikipedia.org/wiki/Dynamic_recompilation †† IIUC minus a few things that can't be done when just-in-time because some assumptions are not guaranteed to be satisfied.

> (even if they can actually execute all instructions concurrently, which is not clear to me and unlikely - Intel can also only execute some instructions on certain ports) It's pretty symmetrical, moreso than Cortex-X2's 4 pipelines; there's analysis that on M1 only some floating point and crypto instructions can't execute on all 4 pipelines. [1] (TP in that table is inverse throughput) Which means that, for example, byte permutes from tables 256b or less can actually achieve the same throughput on M1 as with Intel's AVX-512, since M1 can sustain 4x 2-register TBL per cycle. And doing the exact equivalent of a 512b vpermb (3 cycle latency, 1/cycle throughput) can be done with 5 cycles latency and 0.33/cycle throughput on M1, via 4x 4-register TBL. Well, a vpermd in NEON would need an extra MLA to convert indexes, and vpermi2* equivalents fall off a cliff. And Intel still has p01 free, and COMPACT is SVE. But in general, a lot of the parallelism that enables AVX-512 implementations will convert directly into ILP across 128b vectors. > This seems unlikely because we're sorting 8 MB and my understanding is that cores (unless L2==LLC) generally have private, partitioned L2 caches, so 3 MB in the case of M1. Is that incorrect? Anandtech [2] measured the same L2 latency up to about 8MB single-core, so regardless of the details, 8MB is a pretty significant cliff on M1. Regardless, RAM bandwidth is ~60GB/s, and unlike Intel, can be just about saturated by a single core. [1]…

If I'm reading this analysis [1] correctly, M1 has only one integer divider, but it appears to be almost fully pipelined as it can indeed start a new divide every other clock cycle. The latency is also impressively good even at 64 bits, but the benchmark should not be div-latency bound in any case as, each division can be started without waiting for the previous division to finish. edit: of course it is possible that internally the divider is not fully pipelined, but there are actually multiple dividers. It is only exposed as a single unit (i.e only a single port) because it wouldn't be able to start more than one operation per clock cycle in any case. If I had to guess, the additional SIMD dividers are repurposed for this, but that doesn't explain why 64 bit still has the same throughput as 32bit. edit2: SIMD is on the same port as fdiv, not integer. fdiv has a throughput of 1 div per clock cycle! [1] https://dougallj.github.io/applecpu/measurements/firestorm/SDIV_fast_3…