Matt Pharr

> Intel ISPC is a great project btw if you need high level code that targets SSE, AVX, AVX512 and even ARM NEON It's pretty funny how NEON ended up in there. A former Intel employee decided to implement it for fun and submitted it as a pull request, which Intel quietly ignored for obvious reasons, but then another former Intel employee who still had commit rights merged the PR, and the optics of publicly reverting it would be even worse than stonewalling so Intel begrudgingly let it stand (but they did revoke that devs commit rights). https://pharr.org/matt/blog/2018/04/29/ispc-retrospective

Some of my favourite reading on this topic is Matt Pharr's retrospective on ISPC: https://pharr.org/matt/blog/2018/04/30/ispc-all

As Bryan mentions in https://news.ycombinator.com/item?id=37880669, I would read Matt's post-Intel and post-Google write up of why Intel didn't really invest. https://pharr.org/matt/blog/2018/04/30/ispc-all if you don't want to follow the comment link again :).

Matt Pharr’s series of blogs on ISPC are worth reading: https://pharr.org/matt/blog/2018/04/30/ispc-all

Haven't seen it mentioned in here: Matt Phar is also behind ispc, a subset of C (?) in the spirit of SPMD, and an optimizing compiler (based on llvm iirc). It can target Intel AVX (up to avx512, I think?) and is quite fun to work with. It can help out if you're in a rut of using intrinsics and cursed cuda kernels. A fresh take on a programming model. I recommend taking a look at the amazing story behind it https://pharr.org/matt/blog/2018/04/30/ispc-all . It's very interesting to see not only the pure tech part, but also the disheartening lack of interest at Intel on the subject. Missed opportunity I think... You can feel a lot of fun turning into a lot of... Sadness? in the end. Really a big fan of all things Matt Phar, I hope he has the time of his life at nvidia and that they put him to good use!

Some favorites: - Bruce Dawson - Chrome engineer (https://randomascii.wordpress.com/) - Matt Pharr - Graphics researcher (https://pharr.org/matt/blog/) - Pete Shirley - Graphics researcher (http://psgraphics.blogspot.com/)

I love Matt Pharr's blog [0]. He's one of the authors of the Physically Based Rendering book [1] and his blog is mostly centered around that topic, with really well written and interesting articles. [0] https://pharr.org/matt/blog/ [1] https://www.pbrt.org/

Matt Pharr wrote a series of posts telling the story of ispc: https://pharr.org/matt/blog/2018/04/18/ispc-origins.html I found them extremely interesting - highly recommended.

Physically Based Rendering looks great. I know that one of the authors - Matt Pharr - also has some great blog posts about "ispc", a compiler he wrote while at Intel: https://pharr.org/matt/blog/2018/04/30/ispc-all.html. If he was involved in a book in an area I was thinking about studying I would buy it in a heartbeat

Zig has extensive metaprogramming support. For SIMD I'm using ispc which is a shader-like language/compiler from Intel. You write your code from the point of view of a single SIMD lane and the compiler takes care of the rest. Intel kind of screwed the project over with their stupid naming and terrible website but it's a very cool project. I take the resulting object file and link it with the rest of the project. This is very different from trusting a compiler to vectorize your code, with a shading language you know it's going to vectorize. What's absolutely brilliant is when you start to use divergent control flow and see how well it handles it. https://pharr.org/matt/blog/2018/04/18/ispc-origins.html

There was also this fantastic series of articles by Matt Pharr (of Physically Based Rendering fame) - http://pharr.org/matt/blog/2018/04/30/ispc-all.html While it's focused on ISPC (https://ispc.github.io) it does go into a lot of detail about what happened at Intel during the Larrabee & post Larrabee days.

There are several other parts too: Part 1: http://pharr.org/matt/blog/2018/04/18/ispc-origins.html Part 3: http://pharr.org/matt/blog/2018/04/20/ispc-volta-going-all-in.html Part 4: http://pharr.org/matt/blog/2018/04/21/ispc-volta-c-and-spmd.html In general, anything Matt writes is a worthwhile read. =)

> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand. Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins > I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model. > The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit. > And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation. > With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.

I think one of the best ways to understand the "nice property" of compilers we like isn't necessarily determinacy, but "programming models". There's this really good blog post about how autovectorization is not a programming model https://pharr.org/matt/blog/2018/04/18/ispc-origins The point is that you want to reliably express semantics in the top level language, tool, API etc. because that's the only way you can build a stable mental model on top of that. Needing to worry about if something actually did something under the hood is awful. Now of course, that depends on the level of granularity YOU want. When writing plain code, even if it's expressively rich in the logic and semantics (e.g. c++ template metaprogramming), sometimes I don't necessarily care about the specific linker and assembly details (but sometimes I do!) The issue I think is that building a reliable mental model of an LLM is hard. Note that "reliable" is the key word - consistent. Be it consistently good or bad. The frustrating thing is that it can sometimes deliver great value and sometimes brick horribly and we don't have a good idea for the mental model yet. To constrain said possibility space, we tether to absolute memes (LLMs are fully stupid or LLMs are a superset of humans). Idk where I'm going with this

> ISPC: https://ispc.github.io/ an open source compiler for a SPMD language which compiles it to efficient SIMD code I've been learning ispc lately and it does seem like a wonderful solution, you avoid having to build separate implementations for every instruction set and/or worrying about per-compiler massaging to get it to recognise the vectorisation opportunities. The arguments for having a domain-specific language variant and why it was written (https://pharr.org/matt/blog/2018/04/30/ispc-all.html is a good read) seem like persuasive arguments. However, outside of the projects in the above list - it doesn't seem to have very wide usage. There are still commits coming in/responding to some issues so it doesn't seem dead, but there are many issues untouched or just untriaged. There isn't much discussion about using it, or people asking for advice. The mailing list has about a message a month. Is it merely just an extremely highly specialised domain? Is it just that CUDA/OpenCL is a more efficient solution for most cases where one would otherwise consider it? Are there too many ASM/intrinsic experts out there to bother learning?

> I wonder what the overhead of rounding every operation is? Comparable to the cost of using a proper Decimal class? For what it’s worth, for the major ICs (Intel, NVIDIA, etc.) there is zero extra overhead. A choice of rounding modes is part of the floating point operation’s instruction. And keep in mind that a floating point op is always rounding no matter what you do, the question is whether it’s always using the same rounding strategy consistently, whether you can control it, and whether it has what you need. > I remain skeptical You are correct. There are good reasons not to use float for money that the article didn’t discuss, and perhaps the author isn’t aware of. You run out of integer precision at 2^24, which is only 16 million. If you process a 20 million dollar payment in units of dollars, you might be off by at least a dollar. That error will multiply with every floating point op done on the result. If your units are pennies, the largest safe value is only 160,000 dollars. If you ever subtract floats, like say make a payment or withdrawal, you can run into catastrophic cancellation without knowing it. Deposit $200k and then withdraw $199k, suddenly you have a small balance with large error that could remain in your account and continue to grow until the balance is zero. https://pharr.org/matt/blog/2019/11/03/difference-of-floats.html

OpenMP is a bad approach for the types of problems commonly encountered in games and graphics programming in my experience. Matt Pharr's excellent series of articles on the history of ISPC gives some good explanations of what programming models actually work well for graphics particularly: http://pharr.org/matt/blog/2018/04/30/ispc-all.html At the time I was doing most of my SPE work (helping to optimize launch titles at EA prior to the launch of the PS3) most titles weren't taking much advantage of them at all. We were a central team helping move some code that seemed like it would most benefit over, I was particularly involved in moving animation code to the SPEs. There weren't really any options for libraries to help at that point, other than things we were building internally, so it was almost all manual work. Later on in the PS3 lifecycle people moved more and more code to the SPEs. To my knowledge most of that work was largely manual still. For a while I was project lead on EA's internal job/task management library which had had a big focus on supporting use of the SPEs but my involvement in it was mostly during the early part of the Xbox One / PS4 generation. The Frostbite graphics team in particular did a lot of interesting work shifting GPU work over to the SPEs (I think some of it they've talked about publicly) but I wasn't directly involved in that.

Larrabee influenced ispc but was dead by that time. https://pharr.org/matt/blog/2018/04/18/ispc-origins

> not software per [se] and more so Developer Experience This aligns with my experiences trying to use Intel software. The software was often technically very good, but the user experience trying to use it was horrible, and god help you if you got an error. Larrabee all over, really. I suppose linking to [1] and [2] is probably germane for this discussion. I swear there was another blogpost explaining the tradeoffs made in the Larrabee/Xeon Phi development process in exacting detail, and confirming that it ran gaming workloads pretty well before being shelved, but I can't find it again. [1] - https://pharr.org/matt/blog/2018/04/18/ispc-origins [2] - http://tomforsyth1000.github.io/blog.wiki.html#%5B%5BWhy%20didn%27t%20…

From a cursory look at the code, I don't see any use of fused-multiply-add (FMA) which would likely help with precision issues in a number of places with the float version. The "problem in more detail"[1] readme specifically calls out computation of `x^2 - y^2` as a source for error, and that has methods that dramatically reduce error with FMAs[2]. [1] https://github.com/ProfJski/FloatCompMandelbrot/blob/master/FloatCompM… [2] https://pharr.org/matt/blog/2019/11/03/difference-of-floats

Also related as far as the programming model: C* on Connection Machines and the C dialect used on MasPar systems in the 1980s. There is some discussion in the ispc paper: https://pharr.org/matt/assets/ispc.pdf

> things that cannot be expressed that way tend to not be a good fit for GPU computing anyway I'll have to disagree with you a little bit here. SIMT model of GPUs are quiet a bit more expressive than the numpy's SIMD model. As an obvious example, you'll have to manually maintain a mask to implement if/else i.e. code path divergence in SIMD. GPUs automatically does this and many more to make your life easier. And frankly, I find it lot more easier to reason about what should happen to one data point than a bunch of them together. An interesting article I read recently that has some relevance to this discussion. https://pharr.org/matt/blog/2018/04/18/ispc-origins

The problem with Intel is: 1. They don't pay - Nvidia/Google/Apple easily pays 1.5-2x Intel before appreciattion. 2. They're cheap/beaurcratic. The office sucks, your laptop sucks. 3. They suck at software. https://pharr.org/matt/blog/2018/04/18/ispc-origins 4. They can't develop/retain talent. Half the ML-HW/FW teams at AMD/Google/Nvidia/Apple are ex-Intel.

See the history of ISPC, and how it was rescued, it touches on some of those points. https://pharr.org/matt/blog/2018/04/18/ispc-origins

ispc is an open source high performance compiler by Intel and guess what happened? https://pharr.org/matt/blog/2018/04/29/ispc-retrospective#the-unwanted…

Matt Pharr recently released an early version of PBRTv4, which has been rewritten extensively to make heavy use of Optix and the GPU. Given that he previously wrote a series on rendering Moana using PBRTv3, I wonder if v4 could be used as a comparison? https://pharr.org/matt/blog/2020/08/19/pbrt-v4-released.html

A couple of other attempts at rendering the Moana scene with something other than Hyperion: - Matt Pharr (author of pbrt) -- 6 part series: https://pharr.org/matt/blog/2018/07/08/moana-island-pbrt-1.html - Joe Schutte (WDAS): https://schuttejoe.github.io/post/disneybsdf/

Not on company time, but after leaving intel, some former employees added ARM support to ispc[0]. In the years since then, current Intel employees have maintained and even expanded the ARM support and Intel’s website even mentions it now[1]. [0] https://pharr.org/matt/blog/2018/04/29/ispc-retrospective.html [1] https://devmesh.intel.com/projects/intel-ispc-in-unreal-engine-4-a-pee…

Here's more on the problem of SIMD and C compilers: https://pharr.org/matt/blog/2018/04/18/ispc-origins.html#auto-vectoriz… > as long as vectorization can fail (and it will), […] you must come to deeply understand the auto-vectorizer. […] This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation

Link: https://pharr.org/matt/blog/2018/04/18/ispc-origins.html