Chips and Cheese

Chips and Cheese is Anandtech's spiritual successor and I think its patron model is probably better overall. https://chipsandcheese.com/ David Kanter doesn't write articles very much these days, but Real World Tech has always had top-shelf stuff and it's one of the few places where all the comments are worth reading too. https://www.realworldtech.com/

There's Chips and Cheese[0], but not sure if "layman terms" is accurate. They're really great, though. [0]: https://chipsandcheese.com/

It's also worth nothing that the 128GB of "VRAM" in the GB10 is even less straightforward than just being aware that the memory is shared with the CPU cores. There's a lot of details in memory performance that differ across both the different core types, and the two core clusters: https://chipsandcheese.com/p/inside-nvidia-gb10s-memory-subsystem

> "ARM or x86? ISA Doesn’t Matter" > * https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter Some people, for some strange reason, want to endlessly relitigate the old 1980'ies RISC vs CISC flamewars. Jim Kellers interview above is a good antidote for that. Yes, RISC vs CISC matters for something like a simple in-order core you might see in embedded systems. For a big OoO core, much less so. That doesn't mean you'd end up with x86 if you'd design a clean sheet 'best practices' ISA today. Probably it would indeed look something like aarch64 or RISC-V. So certainly in that sense RISC won. But the win isn't so overwhelming that it overcomes the value of the x86 software ecosystem in the markets where x86 plays.

https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter Basically, x86 uses op caches and micro ops which reduces instruction decoder use, the decoder itself doesn't use significant power, and ARM also uses op caches and micro ops to improve performance. So there is little effective difference. Micro ops and branch prediction is where the big wins are and both ISAs use them extensively. If the hardware is equal and the designers are equally skilled, yet one ISA consistently pulls ahead, that leads to the likely conclusion that the way the chips get designed must be different for teams using the winning ISA. For what it's worth, the same is happening in GPU land. Infamously, the M1 Ultra GPU at 120W equals the performance of the RTX 3090 at 320W (!). That same M1 also smoked an Intel i9.

Chips & Cheese is a good current alternative. https://chipsandcheese.com/

Where to go now? Chips and Cheese https://chipsandcheese.com/ Serve the Home https://www.servethehome.com/ Tom's Hardware https://www.tomshardware.com/

Great post as it is also directly applicable to invalidate the myth that the arm instruction set somehow makes the whole cpu better than analogous x86 silicon. It might be true and responsible for like 0.1% (guesstimate) of the total advantage; it's actually all RISC under the hood and both ISAs need decoders, x86 might need a slightly bigger one which amounts to accounting noise in terms of area. c.f. https://chipsandcheese.com/2021/07/13/arm-or-x86-isa-doesnt-matter/

It's worth noting that AMD had worked with Microsoft on Zen-specific optimizations in the past, too. Ahead of the Zen 2 launch, they touted improvements of up to 15% from Windows scheduler changes for CPU topology awareness [1]. And with Zen 5 having higher cross-CCD communication latency than the previous generations [2], it probably gets an even harsher penalty from poor scheduling. While the less-than-stellar reviews for the 9950X are already out, AMD's recent strategy of staggering releases for desktop parts (first the regular, then the X3D SKUs) can improve this generation's perception, assuming there are meaningful performance gains they can reach in software for Zen 5 prior to the X3D release. [1] https://borncity.com/win/2019/06/30/windows-10-v1903-optimization-for-… [2] https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5-on-deskto…

That seems to not matter much nowadays. There's another great(according to my untrained eye) writeup of the lack of importance on chips and cheese. https://chipsandcheese.com/2021/07/13/arm-or-x86-isa-doesnt-matter/

VLIW, they’re still out there, doing mad stuff.. VLIW is cool and as a DSP programmer I love/live that time a VLIW comes my way :) https://chipsandcheese.com/2023/10/04/qualcomms-hexagon-dsp-and-now-np…

I largely agree with you, but funnily enough the very same blog has a great post on the x86 decoding myth https://chipsandcheese.com/2021/07/13/arm-or-x86-isa-doesnt-matter/

Chips and Cheese reviewed it quite favourably, worth a read: https://chipsandcheese.com/2024/03/13/loongson-3a6000-a-star-among-chi…

Just to note: m6i instances are Intel-based. As for getting information on AWS instances, the best way in my opinion is just to spin up the instance and look up which exact CPU model it uses. Then you can go for example to WikiChip (https://en.wikichip.org/wiki/WikiChip) to see more information about the CPU. Other good sources include Anandtech (for example https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-… and Chips and Cheese (for example https://chipsandcheese.com/2022/05/29/graviton-3-first-impressions/). Things like NUMA configuration can be inspected with tools like numactl.

Agreed. This is very fascinating stuff to help understand what's really happening at that low level in these new chips and how this can impact software performance, the intricate details of which are sometimes only sparsely available online and usually not as approachable. They also had an excellent one posted just a few weeks ago on the Intel Skylake architecture in case you hadn't yet seen it: https://chipsandcheese.com/2022/10/14/skylake-intels-longest-serving-a…

Just sticking to the hardware predictions, I am unsure if things will really go as planned. The thing with classical computing is that it's an easy-to-understand model, compilers and programmers can both produce good code for it. Predictions that simple, core-parallel processors will sweep away clever, ILP-based processors have been big failures historically, and I expect that to continue. The end of garbage collection also seems a step too far. It, like our current processors, is good enough that radically different successors are unlikely to succeed. EDIT: I should clarify this is for general-purpose architectures. For domain-specific purposes, we definitely are seeing new, weird designs pop up. I liked this breakdown of the 'Dojo' architecture, to give an example: https://chipsandcheese.com/2022/09/01/hot-chips-34-teslas-dojo-microar…

This analysis from ex-Anandtech Ian Cutress and two contributors to the excellent Chips and Cheese is pretty great for processor architecture fans. See also: https://chipsandcheese.com/2022/08/26/tachyums-revised-prodigy-archite…

> The big question then is, why are ARM desktop (and server?) cores so far behind on wider SIMD support? Very wide SIMD instructions require a lot of die space and a lot of power. The AVX-512 implementation in Intel's Knight's Landing took up 40% of the die area (Source https://chipsandcheese.com/p/knights-landing-atom-with-avx-512 which is an excellent site for architectural analysis) Most ARM desktop/mobile parts are designed to be low power and low cost. Spending valuable die space on large logic blocks for instructions that are rarely used isn't a good tradeoff for consumer apps. Most ARM server parts are designed to have very high core counts, which requires small individual die sizes. Adding very wide SIMD support would grow die space of individual cores a lot and reduce the number that could go into a single package. Supporting 256-bit or 512-bit instructions would be hard to do without interfering with the other design goals for those parts. Even Intel has started dropping support for the wider AVX instructions in their smaller efficiency cores as a tradeoff to fit more of them into the same chip. For many workloads this is actually a good tradeoff. As this article mentions, many common use cases of high throughput SIMD code are just moving to GPUs anyway.

I feel like you are pulling all sorts of nonsense out of nowhere. Your numbers seem all made up. 6GB/s seems outlandishly tiny. Your justifications are not really washing. Zen4 here shows single core as, at absolute worst behavior, dropping to 57GB/s. Basically 10x what you are spinning. You are correct in that memory limits are problematic, but we also have had technology like Intel's Direct Data IO (2011) that lets the CPU talk to peripherals without having to go through main memory at all (big security disclosure on that in 2019, yikes). AMD is making what they call "Smart Data Cache Injection" which similarly makes memory speed not gating. So even if you do divide the 80GB/s memory speed across 16 chips on desktop and look at 5GB/s, that still doesn't have to tell the whole story. https://chipsandcheese.com/p/amds-zen-4-part-2-memory-subsystem-and-co… https://nick-black.com/dankwiki/index.php/DDIO As for SSD, for most drives, it's true true that they cannot sustain writes indefinitely. They often write in SLC mode then have to rewrite, re-pack things into denser storage configurations that takes more time to write. They'll do that in the background, given the chance, so it's often not seen. But write write write and the drive won't have the time. Thats very well known, very visible, and most review sites worth a salt test for it and show that sustained write performance. Some drives are much better than others. Even still, an Phison E28 will let you keep…

hello, imho. (!) i think this would be great!! personally i totally understood why AMD gave up on its last attempt - the A1100 opterons - about 10 years ago in favor of the back then new ryzen architecture: * https://en.wikipedia.org/wiki/List_of_AMD_Opteron_processors#Opteron_A… but what i would really like to see: an ARM soc/apu on an "open"*) (!) hardware-platform similar to the existing amd64 pc hardware. *) "open" as in: i'm able to boot whatever (vanilla) arm64 linux-distribution or other OS i want ... i have to add: i'm personally offended by the amount of tinkering of the firmware/boot-process which is necessary to get for example the raspberry pi 5 (or 4) to boot vanilla debian/arm64 ... ;) br, a..z ps. even if its a bit o.T. in this context, as a reminder a link to a slightly older article about an interview with jim keller about how ISA no longer matters that much ... "ARM or x86? ISA Doesn’t Matter" * https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter

I'm not aware of any GPU that implements this. Even the interleaved execution introduced in Volta still can only execute one type of instruction at a time [1]. This feature wasn't meant to accelerate code, but to allow more composable programming models [2]. Going of the diagram, it looks equivilant to rapidly switching between predicates, not executing two different operations at once. if (theradIdx.x < 4) { A; B; } else { X; Y; } Z; The diagram shows how this executes in the following order: Volta: ->| ->X ->Y ->Z|-> ->|->A ->B ->Z |-> pre Volta: ->| ->X->Y|->Z ->|->A->B |->Z The SIMD equivilant of pre Volta is: vslt mask, vid, 4 vopA ..., mask vopB ..., mask vopX ..., ~mask vopY ..., ~mask vopZ ... The Volta model is: vslt mask, vid, 4 vopA ..., mask vopX ..., ~mask vopB ..., mask vopY ..., ~mask vopZ ... [1] https://chipsandcheese.com/i/138977322/shader-execution-reordering [2] https://stackoverflow.com/questions/70987051/independent-thread-schedu…

In my eyes, it is less risky with AMD. When you're rooting for the underdog, they have every incentive to help you. This isn't a battle to "win" all of AI or have one beat the other, I just need to create a nice profitable business that solves customer needs. If I go with Nvidia, then I'm just another one of the 500 other companies doing exactly the same thing. I'm a firm believer that there should not be a single company that controls all of the compute for AI. It would be like having Cisco be the only company that provides routers for the internet. Additionally, we are not just AMD. We will run any compute that our customers want us to deploy for them. We are the capex/opex for businesses that don't want to put up the millions, or figure out and deal with all the domain specific details of deploying this level of compute. The only criteria we have is that it is the best-in-class available today for each accelerator. For example, I wouldn't deploy H100's because they are essentially old tech now. > Are you not interested in AI/ML customers? Read these blog posts and tell me why you'd ask that question... https://chipsandcheese.com/2024/06/25/testing-amds-giant-mi300x/ https://www.nscale.com/blog/nscale-benchmarks-amd-mi300x-gpus-with-gem…

I got tired of the video because they never got to the point. Here’s the article: https://hackaday.com/2024/03/21/why-x86-needs-to-die/ The article is, uh, well it’s bad. I’m a little hesitant to respond point-by-point to an article like this because it just sucks up so much time, and there’s just so much wrong with the article on so many levels. Here’s the response article: https://chipsandcheese.com/2024/03/27/why-x86-doesnt-need-to-die/ The video appears to go into detail about things like how pipelines but it’s a long video that just doesn’t get to the point fast enough. I think the video is somewhat tearing into the article and ripping it apart, but I hear some comments in the video which I think are not that great, like at 1:01:25: > RISC is reduced instruction set computer, just means not very many instructions RISC is better understood not in terms of the acronym itself, but as a label for the way CPUs were designed post 1980 or so.

> Cheese is extra https://chipsandcheese.com/

> C++ is Ahead of Time, by design; there is nothing to "just in time" compile. Can I talk to you about our Lord and Savior the CPU trace cache[1]? That is to say, I know next to nothing about how modern CPUs are actually designed and hardly more about JITs, but a modern CPU’s frontend with a microop cache sure looks JITy to me. The cache on NetBurst even more classically JITy, but by itself it was a miserable failure, so meh. In any event, a printf invocation seems like it should be too large for the cache to come into play;—on the other hand, all the predictors learning stuff over the iterations might make a meaningful impact? Seems to me like that learning, if present, would make the benchmark less interesting, not more, as an actual prospective application of string formatting seems unlikely to go through formatting the same (kind of) thing and nothing else in a tight loop. [1] https://chipsandcheese.com/2022/06/17/intels-netburst-failure-is-a-fou…

I don't think a next-gen APU for a Steam Deck 2 is going to be that big a problem. The Steam Deck's APU is officially designated: "AMD Custom APU 0405"[1], basically a mix and match of AMD's same-generation parts. With the success of the Steam Deck, a followup custom APU is all but assured, although I don't think it'll show up before next year. If I had to guess, I'd expect a 4C Zen5c and 12 CU RDNA 3.5 on 4nm in late 2024 which should offer a big bump (assuming they can improve memory bandwidth). Note, we're seeing a lot of Chinese handhelds using 7840s already that while a bit behind at 10W, start beating the Steam Deck's performance (sometimes dramatically) at 15W+. Personally, I'd be pretty excited for a Strix Point handheld next year - AMD seems like they're finally getting serious on the iGPU side of things again (with Meteor Lake looking to be competitive). [1] https://chipsandcheese.com/2023/03/05/van-gogh-amds-steam-deck-apu/

Thanks for the response, this made me dig around some more in this fascinating topic. I found multiple papers that also simply use 100ns for RAM latency (e.g. this one[1]) but in reality it is a lot more complicated than one number. E.g. the time to first word for DDR3 is just above 6ns[2]. I'm not a CPU engineer but it looks like CPUs don't necessarily wait for the entire cache line and can utilize the first word straight away. Also, the signal needs to travel both ways and the traces are not straight lines. I'm not an electronics engineer either so the best number for the maximum DRAM trace lengths I could find are 12-30cm, which would be 1-2ns indeed. This means that there is some room for improvement (in theory). But even with ideal RAM it is physically impossible to beat register access at 0.17ns (i9-13900KS @ 6GHz) or even L1D cache at 0.7ns[3]. What is causing the extra 4-5ns latency of the RAM? Would it be physically possible to remove it? [1] https://www.usenix.org/system/files/conference/hotcloud18/hotcloud18-p… [2] https://en.wikipedia.org/wiki/CAS_latency [3] https://chipsandcheese.com/2022/11/08/amds-zen-4-part-2-memory-subsyst…

You say that like it's bad, but that's just life. We're trying to fit a whole universe into 3 pounds of meat. If we stopped to truly understand anything before taking action, we'd never get anywhere. That's especially true in technology, where whole armies of people are working to complicate things as fast as possible. For the best of reasons, of course. But when I started out, I could read one short book and have a pretty good understanding of my computer's processor, and from there it wasn't much further to understanding the whole computer. Now I could spend days just understanding, say, processor cache strategies [1]. A field that is super interesting, but if I am to get any of my actual work done, I can't afford to dig into that and the many, many other similar subtopics. I'm going to get a casual understanding of something, do my best to make something useful, and only dig in further when I'm forced to. When I do have to dig in, it comes in two cases for me. One is where there is something necessary complexity that I would have to learn about regardless. E.g., if something is too slow, I need to learn about what happens under the hood to do proper performance tuning. Great, fine, I will learn it. But then there's the other bucket, which includes unnecessary complexity, bad abstractions, poorly considered UX, and the like. For me, git is clearly in that bucket. I intentionally have very simple development flows. Git can do a great deal, 98% of which I not only don't…

Thanks for correcting my hand-wavy-ness on that. The thrust of my comment was that blog posts like this need to be handled with some care: this is just enough detail to be dangerous without enough context to be truly helpful. Getting a bunch of people excited to go start relaxing their loads and stores with mostly-truths like "The CPU's store buffer must be flushed to L1 cache to make the write visible to other threads through cache coherency." is just leaving loaded guns lying around. I'm only mostly sure that some uarchs will read from the store buffer without engaging the cache-coherency stuff at least in an HT/ILP world, but that is kind of the point: playing intrinsics jazz is a very asymmetrical bet. You can win big if you absolutely nail it for your particular chip, but you usually lose it all when it's not correct. For a slightly more detailed treatment I like this SO answer: https://stackoverflow.com/a/62480523/19734375. It sketches out a better intuition about how the store buffer and speculative stores play out in practice and is very well footnoted. For those that want to go a level deeper I can't recommend Chips and Cheese highly enough. This treatment of Golden Cove is typical of their rigor in understanding what really pushes architecture-specific performance on cutting-edge gear: https://chipsandcheese.com/2021/12/02/popping-the-hood-on-golden-cove/… Edit: Even this comment is self-contradictory. I pulled that SO answer out of my `x86_64 arcana` bookmark…