Manish Goregaokar

Worth highlighting: library-level GC would not be convenient enough to use pervasively in Rust anyway. library-level GC does not replace Rust's "point". It's useful to have when you have complex graph structures. Or when implementing language runtimes. I've written a bit about these types of use cases in https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe-tracing-g… And there's a huge benefit in being able to narrowly use a GC. GCs can be useful in gamedev, but it's a terrible tradeoff to need to use a GC'd language to get them, because then everything is GCd. library-level GC lets you GC the handful of things that need to be GCd, while the bulk of your program uses normal, efficient memory management.

To elaborate on this point, you could consider reading https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t… and the followup post https://manishearth.github.io/blog/2017/01/15/breaking-our-latin-1-ass…

I appreciate the honesty in the FAQ. No thread safety and still some open questions about iterator invalidation. I look forward to seeing what they come up with! Borrow checker style "Aliasability XOR Mutability" does restrict a lot of valid programs, but has some nice properties for scaling large codebases besides the obvious performance advantage of ARC. I can see how this kind of big "down payment" to restructure programs might go against the incremental/fun approach embraced for SerenityOS. More reading on AXM and it's benefits for correctness: * https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-… * http://smallcultfollowing.com/babysteps/blog/2013/06/11/on-the-connect… * https://manishearth.github.io/blog/2015/05/03/where-rust-really-shines…

Manish Goregaokar has a wonderful blog post on garbage collectors in Rust, the current implementations, the trade-offs and where they might be worthwhile: https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe-tracing-g…

Funnily enough, the situation on FreeBSD is reversed: cut(1) indexes by Unicode codepoints and awk(1) indexes by bytes. But, more importantly, just indexing by codepoints isn't enough. Grapheme clusters are the real unit of user-perceived characters [1][2]. Technically and practically (just replace the anime music with Indian classical music and you're sure to find some artists with names that can't be split into codepoints), both implementations of cut(1) are broken. [1]: https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t… [2]: https://unicode.org/reports/tr29/

My favourite example of the thousand-yard-stare horror of web specs is Manish's "Font-size: An Unexpectedly Complex CSS Property". It's awful and hilarious and just keeps getting worse and worse and worse. https://manishearth.github.io/blog/2017/08/10/font-size-an-unexpectedl…

I'd love to read a book on git wizardry that goes beyond the git book. Manishearth, a Mozillian and Rust contributor, wrote this post on splitting apart a repo. He's good at explaining all sorts of things. https://manishearth.github.io/blog/2017/03/05/understanding-git-filter…

Reference counting [1] and atomic reference counting [2] are supported within Rust, and are sufficient for any persistent data structure without cycles. There's also a prototype crate for a mark and sweep GC [3], which would likely work for anything else. Writing such data structures using these features might be a bit more tedious than writing them in a GC language, but I suspect the increased explicitness provides a worthwhile tradeoff. See the crossbeam library [4] for some examples of lockless concurrent data structures implemented in Rust. [1] https://doc.rust-lang.org/std/rc/struct.Rc.html [2] https://doc.rust-lang.org/std/sync/struct.Arc.html [3] https://manishearth.github.io/blog/2016/08/18/gc-support-in-rust-api-d… [4] https://github.com/crossbeam-rs/crossbeam

Most guys I follow are either rust or haskell developers. @manishearth - rust - https://manishearth.github.io/ @bitemyapp - writer of haskell: first principles @bartoszmilewski - haskell - https://bartoszmilewski.com/ @jdegoes - writes informative FP blog posts - http://degoes.net/ @aaron_turon - rust - https://aturon.github.io/blog/ @pcwalton - rust @nikomatsakis - rust - http://smallcultfollowing.com/babysteps/ @paf31 - purescript creator @nick_r_cameron - rust @kmett - famous haskeller @steveklabnik - rust - http://words.steveklabnik.com/

I can actually help you out on this one. I spent the summer hacking on the Rust compiler for my internship. If I'm understanding the code correctly, this is the enum of elements represented in the dependency graph. https://manishearth.github.io/rust-internals-docs/rustc/dep_graph/enum… Note that as per: https://manishearth.github.io/rust-internals-docs/rustc/dep_graph/stru… The DepGraph that we actually care about is specialized to talk about DefIds. In rustc, DefIds are attached to the following things: https://manishearth.github.io/rust-internals-docs/rustc/hir/def/enum.D… This means dependencies are analyzed down to local variables and uses. But even beyond that it tracks things such as borrow checks, linting and more!

It’s worth noting, given the aspirational conclusion, that @T managed pointers were actually equivalent to Rc<T>, since they were only ever implemented with reference counting without cycle collection. (This would have changed had they stayed in the language, but winds changed instead.) The standard library doesn’t ship tracing GC, and it’s still not altogether clear how it would be best to do it. https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe-tracing-g… is useful further reading. I think it’s fair to say that the “ship tracing GC as part of the standard library” ship has sailed, but power and convenience could still warrant language/standard library features to support it, though I doubt anything will ever happen, or at least in the next decade. The focus of the language shifted and was clarified quite a lot between the time of this article and even Rust 1.0 just under two years later.

Reference counting is a type of GC [0]. Just not a very good one in many cases. I think it's a fair assumption to say that the author is aware of what Arcs are and how they work. I believe their point is more so that because of how async works in Rust, users have to reach for Arc over normal RAII far more often than in sync code. So at a certain point, if you have a program where 90% of objects are refcounted, you might as well use a tracing GC and not have the overhead of many small heap allocations/frees plus atomic ops. Perhaps there are in fact ways around Arc-ing things for the author's use cases. But in my (limited) experience with Rust async I've definitely run into things like this, and plenty of example code out there seems to do the same thing [1]. For what it's worth, I've definitely wondered whether a real tracing GC (e.g. [2]) could meaningfully speed up many common async applications like HTTP servers. I'd assume that other async use cases like embedded state machines would likely have pretty different performance characteristics, though. [0] https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)#Re… [1] https://tokio.rs/tokio/tutorial/shared-state [2] https://manishearth.github.io/blog/2015/09/01/designing-a-gc-in-rust/

Out of the box, Rust encourages almost everything to be stack allocated, imposes a read-write lock on every value (i.e., you can have multiple immutable readers, but only one mutable writer at a time), and uses move semantics for variable assignments (i.e., values aren't copied but rather ownership of the value changes from one variable to some other variable). Sometimes you need to drop one or several of these properties. It's very easy to do that -- Rust just makes you be explicit about it. If you want something to be heap allocated, you can use a Box/Rc/etc. If you want a value shared among multiple writers, there are several options; a common one is Arc<Mutex<T>>, which is also thread-safe. If you don't want move semantics, you can just copy the value using .clone(). A few lessons to take away, though: * Because of that read/write lock I mentioned, it follows that you generally want to avoid taking a mutable reference to an entire data structure, because doing so effectively puts that writer lock on the whole thing. If you can take a mutable reference only to the piece you actually care about, that can help out a lot in some cases. * Rust makes you feel like you should never clone, but really it's up to you. Exercise basic judgment that you'd use in any other language, and don't let perfect be the enemy of the good. In many cases it's probably not nearly as expensive as you might feel it is, and sometimes it'll just be optimized away in the end anyway. * Try to…

>First, we'd love it if you would try our early container prototype on your build. Contact us and we'll add you to our closed alpha test. If the code is available; sure. If it's not then sadly this isn't interesting. >Also, we can send you the full paper (preprint) on the Debian case study, with the gory details. This would be more interesting as I'm doing a master thesis regarding reproducible builds and rebuilders for debian at this moment, along with contributing to the effort in general. Feel free to send a copy to [email protected] >The compiler can inject malware when it recognizes that its compiling itself, but we run arbitrary build systems, compiler tool chains, JITs. The compiler can inject anny backdoor into any code as long as its invoked. We are not strictly limited to it compiling itself. I'm unsure how appealing to the diversity of the things you compile is an argument. A lot of things utilize `ld`, `gcc` or `tar` somewhere in their build chain regardless of what they build. It's not going to be harder to backdoor something if you have enough diversity. This is a neat example of the possibilities you have with the rust compiler as an example: https://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-t… >The DRB website lists a Ph.D. dissertation on improving this situation, but I haven't read it, and I don't think it's been implemented in DRB Davids PhD hasn't really been implemented to any large extent in the real-world, beside smaller…

The borrow checker allows for one mutable reference, or multiple immutable references. There are also primitives for allowing interior mutability through shared references, which enforce single access at a time by other means (mutexes, copying data in and out, etc). References are always guaranteed to point to something that lives longer than them. What they refer to could either be on the stack, or it can be allocated and deallocated on the heap via smart pointers. Box<T> is similar to C++'s unique_ptr; there is a single owner, and when that owner is dropped, the referenced value is dropped. Arc<T> is similar to C++'s shared_ptr; it's an atomically reference counted pointer, so the value will be deallocated when the last clone is deleted. These combine in ways that allow you to use most of those "best practice" patterns from C and C++, but with actual language guarantees that you will follow them and the compiler will catch it if you don't. In the internals of data structure implementation, you may need to go beyond what this allows with use of unsafe, but you can limit that just to a few small critical pieces of code that can be more thoroughly audited and tested. In a large project "try to avoid doing that" doesn't really fly; in a large codebase, with dozens of developers, developed over multiple years, people are going to make mistakes. They will need to do some kind of refactoring, and one of the guarantees will get missed. If you've done a decent amount of work in…

It's both faster and you have more control over it; the two are linked in this case. This system lets the compiler do the hard work to figure out when it's ok to deallocate some piece of memory, unlike a GC, which figures it out at runtime. One you've determined that some memory can be deallocated, the actual process is very quick, usually just updating a list of free blocks in the memory allocator and maybe some statistics. Rust does also support reference counting when you want it, which can bring its own set of performance issues in certain circumstances. Think more about general slowness due to constantly incrementing and decrementing references in an atomic fashion when you pass RC-ed data between functions, rather than the pauses seen with garbage collection. But how bad that is depends greatly on how and where you're using reference counted variables, and the addition of the lifetime/borrow system makes it easier to avoid some of the more egregious cases here. Additionally, some people are doing work on GCs that work with Rust. The two links below have some good descriptions of the progress and challenges related to that. http://blog.pnkfx.org/blog/2015/10/27/gc-and-rust-part-0-how-does-gc-w… http://manishearth.github.io/blog/2015/09/01/designing-a-gc-in-rust/

Theres even more than just UAFs that you have to worry about in a single threaded context, but yes you are correct. Here's a good post that talks about why shared mutability even in single threaded contexts is dangerous: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-…

Reflections on Rusting Trust: http://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-tr… HN discussion: https://news.ycombinator.com/item?id=13091941

>such as when those references could not be shared across threads Shared references can be problematic in a single threaded context too. (You may well be aware of this, but posting for the benefit of anyone who isn't.) This blog post has a good summary of the problematic cases: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-…

Rust chose to tie these together for a reason: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-…

Give https://manishearth.github.io/blog/2017/01/15/breaking-our-latin-1-ass… a read and, ideally, the other things it mentions like https://eev.ee/blog/2015/09/12/dark-corners-of-unicode/ and https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t… (Among other things, it points out that doing that to non-Latin text is liable to change pronunciations and meanings in other languages. For example, some languages use diacritics for voiced/unvoiced indication where your "normalization" could do things like "tick→dick" or "did→tit".) (Did you ever notice that? B/P, D/T, V/F, G/K, J/CH, and Z/S form voiced/unvoiced pairs that could have been indicated with a single letter and a diacritic. Same mouth behaviour. It's just a question of whether you engage your vocal cords.)

Here is a full article of such examples: https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t… Discussion on HN: https://news.ycombinator.com/item?id=31858311

In Rust terms, the arena should own the data and each object should just borrow from the arena. In Rust, when you drop a borrow it doesn't deallocate the memory; only owned values deallocate on drop (that is, in the destructor). Besides, borrowing enforces that the arena won't drop while the borrows are still outstanding. https://manishearth.github.io/blog/2021/03/15/arenas-in-rust/

Personally I think this is much easier to digest than the above link: https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t… It starts by breaking down common Unicode assumptions folks have

I think one of the issues is that Rc and Gc are not particularly ergonomic! Used at scale, they add noise/non-readability to any screen of code (Rc::clone, the various approaches for Gc in [0], for instance) that simply wouldn't be there in a dynamic language. Now, one might argue that this explicitness is good! But it's definitely an extra bunch of things to type when you're building something where you don't care about memory management. [0] https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe-tracing-g…

Only in codepoints, but it still has the problem GP mentions of ` + e = è being two codepoints (so two elements in UCS-32), but being logically one character https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t…

The problem is that multiple callbacks might require a reference to the same state, and closing over those references and retaining that callbacks, makes two mutable references to the same state, which Rust makes an illegal pattern. That is indeed safe in single-threaded code, but Rust prevents it nevertheless; there is a famous (in Rust circles) blog post about this: https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-… There is a workaround called "internal mutability", an ability to mutate state pointed by a shared pointer. That is syntactically slightly messier, and generally frowned upon. The blog post mentions about this workaround, and also states that it's not ideal so we should strive for better.

I think the term you are looking for is grapheme cluster. https://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-t…

There are two problems here. Rust accepts a subset of correct programs that involve borrowing, that's absolutely true, but it gives you the tools to get around that: you either use Arc<Mutex<T>, where now you are in the same space that any refcounted GC language is in (with the extra flexibility that if you can "choose your guarantees"[1]), or use unsafe which lets you dereference memory freely in the same way that you would in C or C++. The unstated problem is that there are patterns from other languages that are actually invalid and Rust is correctly denying, but not properly communicating it, which causes frustration. You can always make the argument that the failure modes are benign or rare, and for those cases noone is stopping you from actually using unsafe. [1]: https://manishearth.github.io/blog/2015/05/27/wrapper-types-in-rust-ch…