Travis Downs

For low-level software, these are great: https://eli.thegreenplace.net/ https://travisdowns.github.io/ https://preshing.com/

Here's some reading that I personally have found helpful for optimizing parallel programs: The best I've found so far: https://cdn.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbo… And some other good reading: https://www.amazon.com/Systems-Performance-Brendan-Gregg/dp/0136820158 https://fgiesen.wordpress.com/2014/08/18/atomics-and-contention/ https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html

If you want to deep dive there Travis Downs has written an amazing thing about it https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html and you should follow the links down to his github avx-turbo project that actually runs all sorts of 'frequency reducing' benchmarks and it's just clearer like this. For example I noticed that on some Intel Gold 6230 processors (2000+ USD each at the time), not only avx512 did bring down frequency, but also 'dense avx2' (sequences of heavy FMA). Explained a lot about measured latencies for some real-time processes. Can't thank Travis enough for cutting through all the vagueness there.

Yes, but I also mean that parallelism to be atomic/non-locking. afaik only Java/C# can do this on a high-level (without going back to atomic arrays in C) but I still can't explain why, the best I found so far is: "Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html But it seems there is also some benefit to have a VM in addition to a GC for this joint/atomic parallelism.

For those interested in the number of physical registers on a modern CPU, Henry Wong and Travis Downs have done some great work deducing the physical implementation of recent Intel cores: https://travisdowns.github.io/blog/2019/06/11/speed-limits.html#ooo-ta… http://blog.stuffedcow.net/2013/05/measuring-rob-capacity/ https://travisdowns.github.io/blog/2019/12/05/kreg-facts.html https://travisdowns.github.io/blog/2020/05/26/kreg2.html

There is more recent work. This blog post by Travis Downs is the most detailed analysis of transition behavior I've seen: https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html For general guidelines on when to use AVX-512, this (older post) remains best guide I've seen: https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-ne…

Absolutely. It's useful in text processing, it's useful in emulation, it's useful for doing bit-math for various encodings and checksumming, etc. The newer revisions don't have any downclocking either. https://old.reddit.com/r/emulation/comments/lzfpz5/what_are_the_implic… https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html People have made some incredibly broad assertions about the technology as a whole on the basis of a first-gen product that lingered on the market for far too long because of the 10nm woes. People also hang on Linus's words far, far too much. He's incredibly toxic and willing to opine about stuff he's been disconnected from for 20 years. Lisa Su is putting her money where her mouth is, AMD is making a billion-dollar bet on AVX-512 at a time when Intel has actually abandoned it in the consumer market entirely. Maybe she's doing it just for the server market, but that still means she thinks server-market customers are going to see an actual benefit on normal server tasks, such that it will be worth making every single customer pay more for a larger piece of silicon. It's not the kind of thing you do "just to win a couple HPC benchmarks". Sorry, I trust the person with the financial stake in this discussion a lot more than I trust Linus's spit-takes.

Note that the AVX-512 code we're referring to is the code that Intel also contributed to OpenSSL. As a side-note, I believe the CPU we tested this on does not suffer from the AVX-512 power limits reported with earlier AVX-512 parts. https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html#roc… seems to confirm that.

Not sure what the parent meant but I always think of https://travisdowns.github.io/blog/2019/08/26/vector-inc.html

Some processors have “hardware store elimination” that makes writing all zeros a bit faster than writing other values. https://travisdowns.github.io/blog/2020/05/13/intel-zero-opt.html

It was Travis Downs: https://travisdowns.github.io/blog/2019/08/20/interrupts.html

Thanks for that summary. I'm working on a system where references can be passed around at runtime. Can references in Pony be passed around at runtime between threads. I'm working on ideas such as: - all threads reading or writing at the same time. This is how I think GPUs work. - I am working how to parallelise the bank transfer problem. You have bank accounts of money and want to transfer money between them but never deduct below 0. So you need consistency on an accounts balance. I was looking at sharding the data. I want to get more than 1 million transfers a second. This article is good. https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html

They do not translate x87 to SSE/AVX under the hood. It's goofy enough, (not just with the extra precision, but the status word needs to be renamed too) that it has dedicated hardware. Therefore there's a seperate register file that stores x87/mmx state (and avx-512 k mask registers). https://travisdowns.github.io/blog/2020/05/26/kreg2.html

Wonder if this is because of the zero-value optimizations on Ice Lake: https://travisdowns.github.io/blog/2020/05/18/icelake-zero-opt.html https://news.ycombinator.com/item?id=23225260

Big fan of your site. Travis Downs has done some work on characterizing hardware transition latency. I believe his tools are on GitHub. https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html

This feels like the database locking version of https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html, which instead covers in-process concurrency speed.

>Because I've actually benchmarked it: https://quick-bench.com/q/ISEetAHOohv-GaEuYR-7MajJgTc 18.5 nanoseconds fits under no reasonable definition of "crazy expensive", not when a regular increment clocks in at 5.9 nanoseconds. And there is extremely few situations where you increment a reference count more than, like, 5 times. It's just not an issue. Congratulations. You tested a construct meant for multicore/threading in a single threaded benchmark and then marvel at the low overhead. Of course you will only start seeing the cost if there is actually contention to operate on the value between multiple threads running simultaniously. See.: https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html

>On Ice Lake Xeon the penalty for using the AVX-512 features on a single core is -100MHz. That is a penalty due to licensing [0], not thermal throttling. As I wrote elsewhere, I’ve seen my clockspeed get cut in half across all cores on a physical die when running AVX-heavy operations for a sustained period of time, due to thermal throttling. [0] https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html

It's likely that it's going to need a post on its own since it's an extremely complicated topic. Someone else wrote an awesome post about this for the Neoverse 2 chips [1] and they found that with LSE atomics, the N2 performs as well or better than Icelake. Given gravitron3 has a much wider fabric, I would assume this lead only improves. [1] https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html

> Additionally, 512-bit wide vectors are just too big - the resulting vector units take up too much die space even on big cores, and the power consumption causes issues causing said dies to downclock. This is no longer true, citing [0]: > At least, it means we need to adjust our mental model of the frequency related cost of AVX-512 instructions. Rather than the prior-generation verdict of “AVX-512 generally causes significant downclocking”, on these Ice Lake and Rocket Lake client chips we can say that AVX-512 causes insignificant (usually, none at all) license-based downclocking and I expect this to be true on other ICL and RKL client chips as well. And we still have to see AMDs implementation of AVX512 on Zen4 to know what behavior and limits it may have (if any). [0] https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html

I think this is less of a problem on newer CPUs: https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html

I think this one helps: https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html

> Instead of just the one status register, to be renamed at need the way x86 and ARM do lately, I would append these bits to every register, carried along with the value. Is there any difference? I doubt that Intel and AMD rename the status flags separately. For example, Skylake-X has the same number of physical flags “registers” as physical GPRs.[0] [0]: https://travisdowns.github.io/blog/2020/05/26/kreg2.html (the skinny register file next to the GPR ones)

From the post about the original finding (https://travisdowns.github.io/blog/2020/05/13/intel-zero-opt.html), it looks like Haswell did not have this optimization to begin with (find section Hardware Survey and look at the different architectures - specifically looking for divergence in performance at L3 and RAM between orange and blue dots).

The operations are somewhat different though. Store-to-load forwarding is more complicated and doesn't completely eliminate the operation, it just significantly reduces the cycle count when successful. Although apparently Zen 2 changed this and can pull off zero latency. (https://www.agner.org/forum/viewtopic.php?t=41) Some general background: (https://travisdowns.github.io/blog/2019/06/11/speed-limits.html#ooo-ta…

4.6 GHz all core with AVX512 surprised me. At least AVX512 downclocking seems to be a Skylake problem. Ice Lake (which Rocket Lake is a backport of) similarly doesn't downclock much: https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html The 100+C temperatures suggest it probably should downclock a bit, at least on 14nm.

There's good evidence for mask registers and x87 registers sharing the same hardware on recent Intel chips: https://travisdowns.github.io/blog/2020/05/26/kreg2.html

https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html > There is a transition period (the rightmost of the two shaded regions, in orange14) of ~11 μs15 where the CPU is halted: no samples occur during this period16. For fun, I’ll call this a frequency transition. It stops executing for 35 thousand cycles. I call that a "lockup" "as it shifts".

The three links in the third paragraph of the article describe the meaning of license in this context: https://stackoverflow.com/a/56861355 https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-ne… Excerpted from the Stack Overflow answer: There are three frequency levels, so-called licenses, from fastest to slowest: L0, L1 and L2. L0 is the "nominal" speed you'll see written on the box: when the chip says "3.5 GHz turbo", they are referring to the single-core L0 turbo. L1 is a lower speed sometimes called AVX turbo or AVX2 turbo, originally associated with AVX and AVX2 instructions. L2 is a lower speed than L1, sometimes called "AVX-512 turbo".