Nathaniel J. Smith

Go is the poster child for not having structured concurrency, the article that started this movement was an explicit take down of go’s concurrency primitive “go func” being dangerous. https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… I think Kotlin still leads here, and even the Java changes don’t have the same nice ergonomics you get with Kotlin’s DSL. Would love to see more tenseness from Kotlin leak into Java.

Anytime this comes up I plug the excellent "Unyielding" (https://glyph.twistedmatrix.com/2014/02/unyielding.html) and "Notes on structured concurrency" (https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… as the counterpoint to "What color is your function". Being able to see structurally what effects your invocation of a function will result in is very helpful in reasoning about the effects of concurrency in a complicated domain.

dabeaz's async talks are imo the most important things you can consume to understand events loops and async, followed closely by njsmith's analysis and criticism of different methods of async invocation and call structures: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen…

I will say that structured concurrency (coroutine based) via Trio/Anyio is, in my experience, so much better for most applications that can support it. Reasoning/testing threading code is so impossible once a project gets large enough. I'd highly recommend reading https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… I work with trio/anyio on a daily basis for my job, and I'd always recommend people use it for their concurrency framework and then use await anyio.to_thread.run_sync() to spawn threads if needed. I'll also add that if you need multi-processor execution, look at https://pypi.org/project/tractor/

> I'm not sure I buy the premise for the library's existence, at least with how the author describes it. Just do a google search for "structured concurrency". These ideas did not come out of a vacuum, but instead from years of research in language communities ranging from C/C++ to Java, Swift, and Python. Effection is just these same ideas projected into the world of JavaScript. You may find the following resources are helpful - https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… - https://elizarov.medium.com/structured-concurrency-722d765aa952 - https://openjdk.java.net/jeps/8277129 - https://www.wwdcnotes.com/notes/wwdc21/10134/

TaskGroup in asyncio has been promised at least as far back as Python 3.8 [1]. They're still not in the draft "What's new in python 3.11" [2] and searching the web didn't return any official statements. I believe they're planned but don't believe they'll arrive any time soon. If you want to use structured concurrency now, IMO the best bet is to use Trio directly. Reading posts by the author makes it clear that every detail of the library had been extremely scrutinised, not just the API (e.g. see this long post on ctrl-C handling [3], or any number of long technical discussions on the issue tracker), so I think it's a better choice in any case. [1] https://twitter.com/1st1/status/1041855365745455104 [2] https://docs.python.org/3.11/whatsnew/3.11.html [3] https://vorpus.org/blog/control-c-handling-in-python-and-trio/#other-a…

The video (or blog post) below is one of the best explanations I've seen about what subtle bugs are easy to make with asyncio, why it's easy to make them, and how the trio library addresses them. But yes, consider alternatives before you pick asyncio as your approach! Talk: https://www.youtube.com/watch?v=oLkfnc_UMcE Blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen…

> not personally accounting for every possible context switch In Python land explicit is better than implicit as this is cooperative multitasking after all. It's also worth reading through https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen…

Structured concurrency is why you should use Trio. What do you get? For starters, Ctrl-C/KeyboardInterrupt just works. By restricting the design space, you end up with much more intuitive designs. Obligatory Structured Concurrency essay link: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… I recommend giving it a read.

I don't know if Python is the best tool for the job, but with modern tooling it is leagues better for complex applications than old python. https://trio.readthedocs.io is an extremely good python concurrency library based on the model of Structured Concurrency (https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… The typing issues are far improved in current Python with annotations and attrs/dataclasses.

> So the best practice is to use `asyncio.gather()` to delimitate the pyramid of the task life cycle. Unfortunatly few people kow this, and hence do it. And even then, any async function might run `loop = asyncio.get_event_loop()`, run some background processes, and return before they stopped! I actually had this exact problem with my realtime sensor data server: the background tasks were never properly closed, and the sockets remained open. The article you linked to (https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… is super interesting. I didn't get the appeal of `trio` before, but now it does seem really useful.

On why to use curio or trio. Old bust must read because mentions back-pressure which is rarely discussed at all. https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-…

> https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… Was a damn good read, Thanks!

This is good reading on the subject: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… There's a single other kind of concurrency: unstructured concurrency. Which is where you just start threads/goroutines/coroutines/etc and they just... all do their own little thing ? They can return at any time, have no parents, etc. Structured concurrency is about making a tree of concurrent processes. You may have a parent job, that has children. Cancelling this job will cancel all its children. For example, you may have a job that's linked to an UI (so it needs to stop the moment that UI is gone), to a socket (so it needs to stop the moment the socket closes), or to nothing and is a global scope (that will keep running as long as your OS doesn't exit the process). These scopes can in turn have their own children and their own scopes. And maybe you have a task way, way down in your tree that spawns two tasks: one is a long running task that must exit the moment its parent stops ("the user clicked on cancel"), one is a long running task that must stay alive as long as necessary ("the user's data is being uploaded to a server in the background"). With unstructured concurrency, you can't really express that. With structured concurrency, just pass the appropriate scope down.

> I don't care about "power" for a language construct. I need guarantees a lot more. Does using this construct allow me to put some things in the "that doesn't matter here" bin, so I can focus on some things without worrying about everything at once? Conditions and restarts aren’t much better or worse than standard exceptions in that[1] respect. You can basically implement them[2] on top of those + dynamic scope: the gist is that when an exceptional condition happens, instead of destructively unwinding the stack immediately the code only invokes a callback (selected by condition type, installed by some dynamically enclosing code), which then can choose one of the available handling strategies and unwind the stack to the requisite point (using try / throw if implementing on top of that). The fun part is that it can unwind less than to the point where the callback was installed: e.g. a REPL installs a handler that, on an undefined variable error, asks the user interactively for the value to use instead, invokes the language-provided USE-VALUE restart and proceeds; no debugger-level carnal knowledge of the language runtime is necessary at any point. (Compare also the DOS abort/retry/ignore prompt. Win32 SEH is also capable of expressing a certain degree of resumable exceptions not exposed by C++ runtime, as seen in classic VB’s On Error Resume Next—though that is admittedly a bad idea.) The advantage[3] of this construct is that it enables you to structure APIs differently,…

"Notes on Structured Concurrency, or: Go Statement Considered Harmful" [1] by Nathaniel J. Smith. I believe that structured concurrency will (slowly) cause a massive paradigm shift with regards to concurrency in the same way that structured programming [2] did, and just like "Go To Considered Harmful" kicked off structured programming, "Go Statement Considered Harmful" will be looked to as the work that kicked structured concurrency off. (It's also fun that they have similar titles!) In fact, I think that someday, someone will prove a "Structured Concurrency Theorem" that is to concurrency what the Structured Program Theorem [3] is to programming in general; I believe the Structured Concurrency Theorem will say something like: > Any concurrent program can be rewritten to an equivalent program that uses structured concurrency. I could be wrong, but I would bet heavily on this. In fact, I am betting heavily on it by making a language based on structured concurrency that would compete with Rust. I'm so confident in it that I would bet that Rust either adopts structured concurrency or will be replaced by languages that do. This is despite Rust's current popularity. Also, I've been exploring ways to prove the Structured Concurrency Theorem. I'd love to be the one to prove it. [1]: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… [2]: https://en.wikipedia.org/wiki/Structured_programming [3]:…

I'm open, as always, to refining my understanding of what structured concurrency is, what various forms it takes on, and how it came about. I don't think it's so obvious that the article has misstated, however. I was not-quite-intentionally using a broader notion of structured concurrency, including not only the initial concept (tie task hierarchy to code structure, propagate errors up the call + task stack), but also critical pieces that are built on top of that, notably cancel/timeout scopes [1]. I don't think these pieces had been assembled to form something powerful enough to make concurrency manageable, until about the time of Nathaniel J. Smith's articles and Python Trio. I'm revising the text slightly, and adding a footnote: > It achieves this in ways that seem subtle to us-- clearly so, since its utility didn't reach critical mass until around 2018† > † While it's argued that the core concept of structured concurrency has existed in some libraries and languages prior to the recent boom, these lacked significant parts of the story for making concurrency manageable. One is powerful, ergonomic control of timeouts and cancellation-- itself dependent on structured concurrency. Another is striving for a program that follows structured concurrency as a whole, including direct and transient dependencies. Nathaniel J. Smith was first to piece this all together, with a set of articles, and corresponding Python Trio concurrency library. Though I suspect this will…

(I'm the author of that piece.) It will change what some people insist is idiomatic Go, but my contention is that when this all settles out, call it a year after production release of the generics, the community as a whole will generally come down against "functional style" programming in Go, and people who want to program functionally in Go will discover that they continue to be quite frustrated because you need half-a-dozen more features. It's going to be so verbose and unfun in Go that only very stubborn people are going to stick to it anyhow. I'm not saying that because I hate the style, I'm saying that because it's still going to be a lot of boilerplate wrapped around not much payload. It really won't be fun. I like "map (f1 . f2 . f3)"; the equivalent in Go even post-generics will be crazy full of boilerplate and garbage. It will change some aspects of idiomatic Go. I don't care much at all about the "functional" aspects but I am greatly looking forward to a proliferation of more specialized data structures that are type-safe. I think one of the major weaknesses of Go in what is arguably its wheelhouse, network servers, is that network servers are often precisely where I want some obscure data structure with the performance characteristics I need to scale up some service properly. It's true that the dynamic scripting languages showed us that you can get a long way with arrays & dictionaries, but I'd like some more stuff. I've got a few places a "map" would be…

Cancellation is a little tricky. If things can cancel at any point, then it's impossible to write safe/correct code. Async has the advantage that the await calls are natural sync points, so it's (probably) safe to cancel there. But historical experience has also shown that people will forget to yield when they should, and it will lead to cooperation problems. I think the best approach has to look something like Go's, but perhaps a bit more structured (dynamic scoping[1] might help perhaps with task nurseries[2]). Unless you're writing extremely low level code, you want your language runtime to intercept all syscalls and figure out the async story for you. The language should handle making sure that the M:N mapping works out, no one opens a socket the wrong way etc. Then for you as the program writer, your responsibility is just setting explicit cancellation points as part of the general error handling approach. It's still not perfect, but I think that would be the next evolution from what exists today. [1] https://blog.merovius.de/2017/08/14/why-context-value-matters-and-how-… [2] https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen…

> This is why Golang uses goroutines. This is why Javascript made async/await. This is why project loom exists. This is why erlang uses erlang processes. JavaScript async/await is different from the others. It requires two colors of functions [1], and it conflates how the code is written with how it's executed, so it has the same problem you were talking about at the start of your comment. Also, JavaScript async/await is suboptimal in that it's ultimately built on top of unstructured callbacks. Or, as Nathaniel J. Smith put it in a post about Python's asyncio module, which has the same problem, "Your async/await functions are dumplings of local structure floating on top of callback soup, and this has far-reaching implications for the simplicity and correctness of your code." [2] That whole post is well worth a read IMO. [1]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-funct… [2]: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-…

I think a lot of projects in Rust take a hybrid approach with async, where they have some ambient "runtime" (like tokio, for example) and allow forking off futures to its executor. this is similar to the `go` statement referenced in https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… for trio. but looking at the pieces that don't rely on this ambient executor should be pretty close to how we structure our async code. in my understanding, the pure futures approach in rust goes even further than the nursery approach in trio, where there isn't even a "local" notion of forking off a task in the background. if a future wants to do an operation in the background, it's responsible for `join`ing on it itself or finding some other way to ensure it gets `poll`ed. this setup is a good fit for rust since the parent future maintains ownership over its child, and dropping the parent future immediately cancels the background tasks. but, for what it's worth, I think there's lots of tradeoffs in this space, and we have yet to find really good patterns for structuring async code. so it's good to see so much experimentation.

@steveklabnik My apologies if I appeared to make light of the Rust community’s contributions. My definition of religious war is butting heads about problems that don’t have a clear right answer (tabs/spaces). If there are new things considered by the community that make the discussion worthwhile then it’s absolutely worth having; but if it causes people to burn out on the community down the line then maybe more moderation would be helpful. I have no experience leading anything like Rust, but I talked to contributors who burned out on Python and various libraries and it’s sad to see, so ideally Rust would forge a better path. My impression of async/await is that it’s one part of making concurrency grokkable, and I’m not sure whether the debate was taking all that into account: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-…

Some specific innovations off the top of my head that are pretty firmly outside traditional academic research, and seem more revolutionary than evolutionary: - Linux's read-copy-update synchronization mechanism. It has been described in papers, but you're better off following mailing list posts or LWN writeups. - Rust's borrow checker and lifetime system. It's built on existing well-known ideas (e.g. affine types) and there's since been some academic work on formalizing it, but the specific system Rust uses has no direct precedent, is pretty novel, and was developed outside academia. (Note that Rust came out of Mozilla Research, which is far, far smaller than Bell Labs but also an organization that intentionally works on revolutionary and not evolutionary improvements.) - libdill and Trio's structured concurrency, a solid theoretical framework for handling async/await-shaped problems without turning your execution into concurrent spaghetti. The techniques are not unprecedented, but https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… is a better framing of it.

I agree, and the docs got better too. In 3.7, asyncio is usable for people that don't understand it very well. Before that, you had to learn the whole thing brick by brick before being able to do something serious. Yet, there is a missing piece I'm hoping we'll see in 3.8: a way to limit the scope of `asyncio.ensure_future()`. Indeed, right now either you `await` to get a sequential execution , or you call `asyncio.ensure_future()` to get a concurrent one. The later, unfortunatly, is the equivalent of a GOTO, and worse, it can contain a GOTO itself (see https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… So the best practice is to use `asyncio.gather()` to delimitate the pyramid of the task life cycle. Unfortunatly few people kow this, and hence do it. Plus it is not fun to do, it's boring boilerplate, something Python usually frees you of. Yuri is thinking about how to implement the trio solution (the infamous nursery) in uvloop, and if he does, we usually get the feature ported to the stdlib a year later. Meanwhile, I noted that a simple wrapper does meet the Pareto requirement: https://github.com/Tygs/ayo/ You can see it's not really hard to write your own version of it if you need to. It helped me a lot: the code is easier to reason about, and you remove a lot of edge cases. I'll have to test Janus, it seems super nice.

I think the clearest sales pitch comes from this post from the author of Trio, which is an implementation of structured concurrency for Python: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… .

Structured concurrency is also a Kotlin thing https://kotlinlang.org/docs/coroutines-basics.html#coroutine-scope-and… and a Python thing https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… and before that it was a C thing https://sustrik.github.io/250bpm/blog:71/

This blog I found seems to support your view https://vorpus.org/blog/why-im-not-collaborating-with-kenneth-reitz/

Well... maybe the people who have stopped working on it due to the mysterious disappearance of 30,000 fundraised dollars, selling paid support but "delegat[ing] the actual work to unpaid volunteers", and a pattern of other issues from other community members who have not spoken up about them. https://vorpus.org/blog/why-im-not-collaborating-with-kenneth-reitz/ https://news.ycombinator.com/item?id=19826680

Here are my favorites: - Jacob Kaplan-Moss (https://jacobian.org/) [Engineering leadership, OSS] - Anton Zhiyanov (https://antonz.org/) [SQL, Go, Python] - Julia Evans (https://jvns.ca/) [SQL, Linux, Python, Go, Web] - Brandur Leach (https://brandur.org/) [Postgres, Go, Ruby, Web] - Brandon Rhodes (https://rhodesmill.org/brandon/) [Python, Astronomy, Linux] - Nathanial J Smith (https://vorpus.org/blog/) [Python, Async, Linguistics] I also write occasionally at https://rednafi.com.

https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statemen… https://dubroy.com/blog/handling-user-input-with-structured-concurrenc… The first one is particularly good. Both of those have links to more topics.