Bryan Cantrill

I don't disagree that Sun was a company run aground by engineers -- though I certainly like to think of myself as one of the engineers trying to navigate us around the rocky shoals! For whatever it's worth, I broadly stand by my analysis on HN fourteen years ago (!!) of Sun's demise[0] -- which now also stands as clear foreshadowing for Oxide eight years before its founding.[1] [0] https://news.ycombinator.com/item?id=2287033 [1] https://bcantrill.dtrace.org/2019/12/02/the-soul-of-a-new-computer-com…

Read Bryan Cantrill on how unimpressive Dorsey is to anyone with a clue: "I, like many people, have a complicated relationship with Twitter. As Adam and I regaled in a recent Twitter Space, it started when debugging the Twitter fail whale in the offices of Obvious in 2007, where I became thoroughly unimpressed with their self-important skipper, Jack Dorsey. In part because I thought he was such a fool, I refused to join Twitter out of principle." https://bcantrill.dtrace.org/2022/11/05/twitter-when-the-wall-came-dow… It seems to me letting Dorsey speak and ridicule himself in front of an audience of technically savvy people would actually be the best way to discredit him.

I believe that macOS still ships with DTrace; Xcode Instruments was originally built on top of it. https://dtrace.org (Some people find it easier to write a one-line script that reports the timings that they need; I don't know if it helps you.)

Iirc in part 9 of this and the talk linked, Cantrill did that http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ Basically he moves from his homegrown pointer chasing looking a lot like a doubly linked list to just using a hashmap and putting the key in his structs. As a naive first approach to satisfy the borrow checker, thinking he would sacrifice performance but at least the compiler would be happy. Spoiler: it got faster. Far faster. To his surprise.

When you get sufficiently close to the metal, the performance gains seem to be coming from better algorithms rather than from the programming language. Like you are saying: > The reasons for these gaps can include things like it being harder to use the optimal data structure for your application (eg C code using red-black trees instead of btrees in 2023) Bryan Cantrill described this experience[1] and maybe that is exactly what you are referring to. But if you're just looking for a language to get work done with, does it matter if Rust is faster because of better off-the-shelf data structures and algorithms or because of some inherent magic in the programming language? [1] See point 9 here: http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/

Not everyone agrees with this characterization, including some well known (now former) C developers: http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/

Two long-form examples for programs I don't see here yet. Note that they won't necessarily be great for other projects (e.g. for a library, where you as the user will probably need/expect "on-point documentation" for specific details while working with the product in question), but as a wholistic introduction to a specific tool, I've yet to see much better examples. - https://dtrace.org/guide/preface.html - https://archive.org/details/pdfy-MgN0H1joIoDVoIC7

I’m surprised I haven’t seen this linked yet, but Bryan Cantrill of dtrace, Sun, lawnmower, Joyent, etc, fame gave an amazing talk for Monktoberfest 2016, titled “Oral Tradition in Software Engineering”, which features The Story of Mel [1]. Highly recommend checking it out — there are loads of little gems and stories like this throughout. All of his other presentations are great too and definitely worth a listen if you like this sort of thing [2]. A couple of my favorites are “Fork Yeah! The Rise and Development of Illumos” [3] and “Debugging Under Fire: Keep your Head when Systems have Lost their Mind” [4]. [1]: https://youtu.be/4PaWFYm0kEw?t=644 [2]: http://dtrace.org/blogs/bmc/2018/02/03/talks/ [3]: https://youtu.be/-zRN7XLCRhc [4]: https://youtu.be/30jNsCVLpAE

Do you also think skip lists would work better than B-trees for in-memory maps and sets, like Rust's BTreeMap and BTreeSet? From what I've read, the performance of those B-trees in the Rust standard library is already great. http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-a…

> The more I hear about how Zig implements it, the more it is exactly like exceptions. It's exactly like exceptions aside from all the problematic bits of exceptions: * it is very explicit * it is much easier to interact with * it can be abstracted over (since a "result" is a reified value of a known type) * it supports but is not over-optimised for happy path or split-path scenarios * it doesn't require separate allocations or RTTI * it has a much more uniform cost model That doesn't necessarily mean it's the right system for what you usually work with, but it's a really nice way to do error handling. For an expansion of this, Brian Cantrill has a paean to this error handling style in his "falling in love with rust"[0], I can't direct-link the section but it's "1. Rust’s error handling is beautiful". [0] http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/

You should read the author's other post: http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ They weren't trying to game benchmarks in their analysis.

The opposite also holds true. Because Rust supports much higher level abstractions than C, it can be much faster and easier to experiment with different designs than with C. Parallelism is the classic example, but I also really like Bryan Cantrill's blog post from 2018 [1], where he was able to use a BTreeMap as a drop-in replacement for a HashMap, and gain a lot of performance that way. [1] http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-a…

Bryan Cantrill: http://dtrace.org/blogs/bmc/2004/08/28/the-economics-of-software/#comm… >> software does not wear out. That isn’t to say that software never breaks (or isn’t broken to begin with), but software that works can work in perpetuity. A favorite example of mine is troff. The source for troff is some of the nastiest stuff ever written — but it works. It hasn’t been touched in years, and probably will never be: it’s written in a portable language (C) and relies only on the most basic OS facilities. troff will work indefinitely — it will never wear out. >> (The tragic footnote to troff is that its author, Joseph Ossanna, died tragically in 1977; the very fact that his software is humming along perfectly more than a quarter of a [century] after his death is a testament to software’s unique imperviousness to wear.)

Follow up: "Rust after the Honeymoon" http://dtrace.org/blogs/bmc/2020/10/11/rust-after-the-honeymoon/ Previous discussion: https://news.ycombinator.com/item?id=24747492

FOSS is the start. Then comes governance, culture, etc. Bryan Cantrill's "Power to Fork" thesis is relevant. http://dtrace.org/blogs/bmc/2010/08/03/opensolaris-and-the-power-to-fo… Sadly, I lived that experience on another FOSS project. Wish I had heard Cantrill before I committed to that effort. I'm also deeply influenced by Pieter Hintjens. https://en.wikipedia.org/wiki/Pieter_Hintjens#Bibliography Ditto Clay Shirky, eg Here Comes Everyone. Nadia Eghbal's research is on my reading list. https://nadiaeghbal.com/oss/ Having just skim read, it looks very insightful, cogent.

It seems like in your case the easiest path might be to learn how the language works and see it for yourself. Otherwise, I think these posts are roughly in the genre you want: * http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ * http://dtrace.org/blogs/bmc/2020/10/11/rust-after-the-honeymoon/ * https://gregoryszorc.com/blog/2021/04/13/rust-is-for-professionals/

He also wrote a couple of blog posts called "Falling in love with Rust" and "Rust after the honeymoon": http://dtrace.org/blogs/bmc/2018/09/18/falling-in-love-with-rust/ http://dtrace.org/blogs/bmc/2020/10/11/rust-after-the-honeymoon/

I came to AWK via DTrace http://dtrace.org/blogs/about/ When processing event streams, it's very natural to express transformation as FILTER + ACTION rules. AWK is an embodiment of this idea for the domain of text processing where events are lines (or multi-line records). DTrace uses the paradigm to process low-level system events (like scheduling events, or (kernel) function calls). It's a good paradigm, worth using. However, it's easily replicated in every other language with a loop: while true { rec = read() if( filter1(rec) ) { action1(rec) } if( filter2(rec) ) { action2(rec) } ... } You don't need a special language to do event processing like this.

>> * Many people, early on, suggested moving to an FFI (namely Cantrill) and regrettably I ignored them. A bit off-topic but Dahl referenced Cantrill here, who I figured to mean one of the authors of DTrace, Bryan Cantrill, who I then found from his Twitter (https://twitter.com/bcantrill) just last month started a new "computer company" which sounds super interesting, especially with his past experience and the passion he seems to have for attempting to solve a tough, bold problem: http://dtrace.org/blogs/bmc/ Edit: which apparently was posted on HN with 500+ upvotes https://news.ycombinator.com/item?id=21682360, regardless it's a good read worth re-mentioning here Edit 2: Also thank you for the excellent write up, I wish every tech talk had a text TLDR

I am not a fan of Go, and I also wish these things were true (and more[1], actually), but I find it hard to agree that its priorities are "terribly misplaced." Inside the context of Go's goals (e.g., "compile fast") and non-goals (e.g., "make it easy to attach debuggers to apps replicated a zillion times in Borg") these trade-offs make a lot of sense to me. Like: Go rewrote their linker, I think, 3 times, to increase the speed. If step 1 was to wade through the LLVM backend, I am not sure this would have happened. Am I missing something? I love Rust, but Go is focused on a handful of very specific use cases. Rust is not. I don't know that I can fault Go for choosing implementation details that directly enable those use cases. [1]: http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/

Not to mention the numerous times where TM casually asks to wipe your backup history because the integrity test didn't pass without errors. Usually this can be fixed with a few commands copied off a blogpost that must be getting thousands of hits every day. It seems obvious to me that TM was intended to run on a snapshotting filesystem such as ZFS* and when this didn't materialize the engineers tried their best to make it work using the tools available to them. Unfortunately thsoe were sparsebundles and hardlinks. I would hope that TM will get updated to use APFS snapshots sooner rather than later, assuming the feature is proven and stable. iOS makes use of APFS snapshots for rolling back failed updates, so one would assume it is. * http://dtrace.org/blogs/ahl/2016/06/15/apple_and_zfs/ and https://arstechnica.com/staff/2006/08/4995/

> Fun fact: we've a system where Rust is faster than C despite still using more instructions I think Bryan Cantrill did a pretty good explanation on differences you might see when rewriting something in Rust[1], and one of the things he looked at to see what was going on was Cycles Per Instruction. Instruction count itself means little if the instructions themselves have very different performance profiles and require different amounts of cycles to complete. Edit: You are tracking and reporting that, so it's not like I'm telling you anything you don't know. I still think the included article is well worth reading though. 1: http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-a…

Godbolt Compiler Explorer: https://godbolt.org/ DTrace: http://dtrace.org/

Anyone operating a cloud of remotely interesting size will need DTrace, or, if they use Linux, eBPF tracing. It was a foregone conclusion that Microsoft needed DTrace. Microsoft surely also needed it for other reasons. Developers, sysadmins, and power users -- all need DTrace or DTrace-like tools. We're finally leaving the dark ages of unobservable operating systems! Hooray! We should all give huge thanks to the creators of DTrace, Mike Shapiro, Bryan Cantrill, and Adam Leventhal, as well as Brendan Gregg and all of the crew at http://dtrace.org.

"We had made the product so easy to use and so important, that we had out-engineered ourselves." Yeah, so "easy" that we had to debug and extend microdhcp code to properly support PXE booting and add option 150 because you guys offered nothing with which to properly boot XEN VM's; XEN was so woefully unfinished that we had to finish it for you and now you're telling us how complete of a product it is and patting yourself on the back. "Details-schmetails", it's "the big picture" that's important, which is that someone cashed out, am I right? This is one of the reasons why my passion and love for computers turned to bitter disappointment: the lies and really bad, half-cooked software. Damn it all, Keith Wesolowski was so right[1]. [1] http://dtrace.org/blogs/wesolows/2014/12/29/fin/

Bryan Cantrill. http://dtrace.org/blogs/bmc/ Overzealous love of systems, I don't necessarily always agree with him but I always learn something when listening. Brenden Gregg. http://www.brendangregg.com/blog/ If I could import someones brain to my own, it would be his. Kyle Fuller https://github.com/kylef Guy is like a UNIX programmer for the modern age.

It seems to me that the workloads on Digital Ocean are not fully utilizing the block device. FIO shows about 50% util for DO, and 81% for Linode on the queue depth=1 test, and 70%/82%, respectively, on the queue depth=8 test. Which seems to indicate a bottle neck somewhere else (at least on DO), and that seems to align with OP's statement at the end, where it seems DO is being capped. Also as it was pointed out, the sizes of these VPS are not even mentioned, and most likely the working set size of 128MB, might be either too small or too big for any of these VPS, making even harder to see the value of these results. Brendan Gregg has excellent info on these topics, http://www.joyent.com/blog/benchmarking-the-cloud http://dtrace.org/blogs/brendan/2011/05/11/file-system-latency-part-1/ http://dtrace.org/blogs/brendan/2012/10/23/active-benchmarking/

Thanks for the praise! As far as pricing goes, you're charged for the wall clock time spent running inside the zone, so if your task queues up because the system is jammed, you're not paying for that. Task runtime can be affected on busy systems, but this has been in the noise for the jobs we've looked at. We're also looking to see how the system gets used in order to understand whether it would be useful to have other pricing options (e.g., tiers of service with guaranteed resources). As for performance: we're using the system internally every day, and we've found the performance very good. Of course, the best way to find out is to evaluate it on your own workload. :) Finally, you'll probably be interested to read Keith's summary of our hardware choices here: http://dtrace.org/blogs/wesolows/2013/06/25/manta-what-lies-beneath/

Pinkie Pie is helpful, but the DTrace Laser Pony is the best debugging pony :) http://dtrace.org/blogs/about/dtracepony/

Are you writing code that gets reviewed by other people? Were code reviews hard in the past? Do your coworkers care about "code quality" (I mean this in scare quotes because that means different things to different people). Are you working more on operational stuff or on "long-running product" stuff? My personal headcanon: this tooling works well when built on simple patterns, and can handle complex work. This tooling has also been not great at coming up with new patterns, and if left unsupervised will totally make up new patterns that are going to go south very quickly. With that lens, I find myself just rewriting what Claude gives me in a good number of cases. I sometimes race the robot and beat the robot at doing a change. I am "cheating" I guess cuz I know what I want already in many cases and it has to find things first but... I think the futzing fraction[0] is underestimated for some people. And like in the "perils of laziness lost"[1] essay... I think that sometimes the machine trying too hard just offends my sensibilities. Why are you doing 3 things instead of just doing the one thing! One might say "but it fixes it after it's corrected"... but I already go through this annoying "no don't do A,B, C just do A, yes just that it's fine" flow when working with coworkers, and it's annoying there too! "Claude writes thorough tests" is also its own micro-mess here, because while guided test creation works very well for me, giving it any leeway in creativity leads to…