jlouis

http://www.erlangpatterns.org/iolist.html This is also a very good introduction to how iolists work in Erlang: http://jlouisramblings.blogspot.com/2009/01/common-erlang-misconceptio… Starts with: --- There is a common saying that Erlangs I/O layer is slow. Bullshit. --- I recommend all of Jespers' blog entries. The newer ones are here: https://medium.com/@jlouis666 This is another article focused on Elixir, but same idea applies: https://www.bignerdranch.com/blog/elixir-and-io-lists-part-1-building-…

I use it more as a glorified low priority to do list. For example some page I was reading (from HN) linked to a recommended article "How to Build Stable Systems" (https://medium.com/@jlouis666/how-to-build-stable-systems-6fe9dcf32fc4… I ctrl-clicked the link to open it in a new tab, because I'd like to read the article eventually even though I don't have time at this particular moment. So when I have a little bit of time I'll work through reading/dealing with my open tabs.

Some great articles about the process of "Breaking Erlang Maps" via QuickCheck. Thanks for those! 1: https://medium.com/@jlouis666/breaking-erlang-maps-1-31952b8729e6 2: https://medium.com/@jlouis666/breaking-erlang-maps-2-362730a91400

This unfortunately breaks down with (bad acting) NIFs. Thankfully you can mark 'em as the dirty evil little things that they are (with negligible overhead): ERL_NIF_DIRTY_JOB_CPU_BOUND. [1] I implore anyone interested in Erlang or its surrounding languages, to read its source code. [2] More specifically, the BEAM. I'll warn you that it's very 80's hackeresque, but in a good way. Incredibly pragmatic. The way they achieve their world-class scheduling? Tip: not via a perfect theoretical design cooked up in some comp-sci lab. [3] If you've been following game engine design within the last 9 years or so, you'll probably have a good idea of how they do it. [4][5] [1]: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc… [2]: https://github.com/erlang/otp/tree/maint/erts/emulator/beam [3]: https://hamidreza-s.github.io/erlang/scheduling/real-time/preemptive/m… [4]: http://blog.molecular-matters.com/2012/04/05/building-a-load-balanced-… [5]: There's a hell of a lot of depth to this problem domain. Charles Bloom's guiding principles for Oodle are spot, and well worth a look.

Author here. I haven't done too much with Elixir yet, other than this Phoenix app, but I am totally enamored of it from what I've seen. First and foremost, the community is absolutely the friendliest, most helpful group of folks. I'm happy with its performance. Phoenix feels similar to rails to me (I'm primarily a rails dev), but with easily 10X performance. And the concurrency. Oh, the concurrency. This is Erlang's bread and butter, and Elixir and Phoenix are built to take complete advantage of this. I love that every request gets its own Erlang process, and you don't have to worry about it blocking anything else. The Erlang scheduler will keep it fair, cutting off any long running computation if it has to, to keep the latency down on new requests. I really like how interesting and mind bending Elixir's purely functional, process-oriented approach is. Nothing (well, very little) is mutable! You can sort of simulate mutation by spinning up a little infinitely looping tail-recursive process and sending messages to it. I encourage you to go through the excellent Elixir Getting Started guide [0], and in particular the "Processes" section for more on this. But I think the thing I like most about it so far, is the introduction to battle-tested Erlang approaches to robustness (OTP). This is a set of abstractions and libraries that have been iterated on over the years and are a fantastic way of building an app that's resilient to failure. Elixir, as it does, takes these…

I’m not expert here but I think most of that risk is mitigated these days since BEAM introduced Dirty Schedulers/Dirty NIFs for any NIFs that take over 1ms to execute… I think this isolates (or can isolate) their issues to only effecting other work on the same Dirty scheduler, aka other Dirty NIFs… Few quick links: https://www.erlang.org/doc/man/scheduler.html https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc… https://bgmarx.com/2018/08/15/using-dirty-schedulers-with-rustler/

Thanks to dirty schedulers that is not a big issue: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc…

I feel like this kind of intensely provably-correct language (this one, Coq, Agda, etc.) isn't so much for writing whole programs in, as for writing the small, very-important-to-verify modules, that will link with other, less important modules to form a whole program. When you're writing provably-correct "error kernels"[1] for your software, you probably want as little interaction with the OS to happen inside the kernel as possible. Write the rest of your program in a glue language, then hand off to the proven-correct error kernel for the hot loop. Probably over stdio, or transactional SHM IPC, or any other mechanism like that that has as little failure-surface as you can get away with. [1] https://medium.com/@jlouis666/error-kernels-9ad991200abd

You might be thinking of the optimization for constant pool references: https://medium.com/@jlouis666/an-erlang-otp-20-0-optimization-efde8b20…

Author of the repo here. Just wanted to say that 99% of this is thanks to [1] and his 2013 Masters thesis. There has been slight discussion to integrate a supercompiling pass (more involved than the ones that are in the compiler already) over at [2]. Hope to motivate enough people to see this happen, especially since we now have even cheaper constant literals [3]! Let's get on this. [1] https://github.com/weinholt [2] https://github.com/erlang/otp/pull/1080#issuecomment-314598581 [3] https://medium.com/@jlouis666/an-erlang-otp-20-0-optimization-efde8b20…

A "literal" in this case is a constant value defined in a module. Those live in a separate space in the VM and are referenced directly because they are immutable and can be shared. If you sent such a literal before OTP20, it would be copied into the heap of the target process. Not anymore. But it doesn't help with cases where you are constructing a term (dynamically) in a process and sending that term. There is more meat in the blog post of mine: https://medium.com/@jlouis666/an-erlang-otp-20-0-optimization-efde8b20…

this might be of interest: https://medium.com/@jlouis666/breaking-erlang-maps-1-31952b8729e6#.90i…

Lots of good stuff, also excited for: * dirty schedulers improved which means much better scheduling of long running NIFs from what I understand: https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc…

> Dealing with strings is not fun right now. And dealing with strings isn't fun in many other languages or runtimes or OSes. e.g.1. C# "Strings in .NET are stored using UTF-16 encoding. UTF-8 is the standard for Web protocols and other important libraries. Beginning in C# 11, you can add the u8 suffix to a string literal to specify UTF-8 encoding. UTF-8 literals are stored as ReadOnlySpan<byte> objects" - https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/bui… e.g.2. Erlang/BEAM/Elixir: "The Erlang string type is implemented as a single-linked-list of unicode code points. That is, if we write “Hello” in the language, this is represented as [$H, $e, $l, $l, $o]". The overhead of this representation is massive. Each Cons-cell use 8 bytes for the code point and 8 bytes for the pointer to the next value. This means that the 5-byte ASCII-representation of “Hello” is 5*16 = 80 bytes in the Erlang representation." - https://medium.com/@jlouis666/erlang-string-handling-7588daad8f05

> My understanding is that if I still want to ensure that the Beam VM can continue to schedule all processes efficiently It's worse than that, overlong NIFs can lead to scheduler collapse with your schedulers being put to sleep and stuck there. > these NIFs shouldn't calculate forever, but return quickly to avoid blocking all other processes. Recent BEAM (R17 and later) has the experimental feature of dirty schedulers which allows running long NIFs without risking scheduler collapse. There is one set of dirty schedulers for IO-bound NIFs and one for CPU-bound NIFs, they should be marked accordingly. > I've no idea how big the overhead actually is, I'd be interested in a rough estimate. Is it small enough that I can just ignore it entirely? https://news.ycombinator.com/item?id=10141945 and the linked article https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc… measure regular nifs at ~1.2µs and dirty nifs at ~10µs by default though they can be as low as ~3µs for core-bound schedulers and tuned wakeup options.

> Let me put it this way. In the last say five years, I can't think of a single article I've come across where someone describes DTrace making their life better (except one I wrote in 2011: http://blog.reverberate.org/2011/05/using-dtrace-on-os-x-to-...), or a single time anyone I know or work with has even mentioned DTrace. Then you haven't been looking: http://dtrace.org/blogs/brendan/2012/08/09/10-performance-wins/ http://dtrace.org/blogs/brendan/2014/02/11/another-10-performance-wins… https://www.joyent.com/blog/node-js-in-production-runtime-log-snooping https://twitter.com/b0rk/status/681536144362848257 https://speakerdeck.com/sartak/dtrace-war-stories https://www.bignerdranch.com/blog/hooked-on-dtrace-part-1/ https://medium.com/@jlouis666/erlang-dirty-scheduler-overhead-6e1219dc… Seriously, it wasn't that hard to find this stuff. I found the last one by searching HN: https://hn.algolia.com/?q=dtrace Like others who replied to you, I use DTrace every day. Very often, it solves in seconds what would have taken hours without it. Frequently, it's helped solve problems I'd never have been able to understand without it. My colleagues use it all the time as well. Sometimes, I write about the general techniques (I wrote the "runtime log snooping" blog post above), and most of our bug reports in recent years are public, but we don't write about individual bugs all that often. (Seems like you wouldn't have run into them if we had, though.)