Stressgrid Blog

That’s a great point and Phoenix 1.7 is the first release to support the usage of other web servers. 2-5x speed up would make Erlang surprise Java/Go/etc. https://stressgrid.com/blog/webserver_benchmark/

busy wait spins your cpus. I don't know for sure, but the benchmarks you linked do some more RCA, which involves looking into why cowboy2 is not great (tl;dr HTTP2 support is hard) also, here is how stressgrid can POC (possibly too dangerous for prod?) to 100k cps (which is on par with what Go can do in the article you linked), you could recompile kernel and configure ranch differently, would probably need to be proven out more: https://stressgrid.com/blog/100k_cps_with_elixir/

> I picked up _Programming Erlang_ because I like Joe. I'm trying to understand why the platform, and the accompanying language(s), aren't more widely used? People seem to speak very highly about Elixir, BEAM, and OTP. It all seems really great... Watching videos of Joe talk super impressed me. He kept the same enthusiasm for programming as any college age programmer. That's impressive! > From @hajile >>BEAM is about an order of magnitude faster than Cpython ...If you prefer Ruby syntax over Python, there is no contest here. >>BEAM is slower than the JVM, but much more stable. > The second quote seems to be a pretty reasonable trade off (my next question is, how much slower?). There was a recent web-server benchmark that illustrates the performance and stability trade-off pretty well [1] (or older web-socket one [2]). The StressGrid benchmark shows Erlang/Cowboy achieving about roughly 1/2-1/3 the speed of the compiled options (Java & Go), and roughly double that of NodeJS. Erlang/Cowboy maintained very stable latency numbers after hitting it's peak however (see graph in [3]). There's an oddity with Cowboy 2, that appears to be due to HTTP/2 support but can be disabled (though HTTP/2 in theory should support faster client responses with less multiplexing of HTTP requests). In general I'd say BEAM is on average faster than CPython, though recent CPython 3.x versions have improved a lot. Importantly BEAM is one of the few dynamic language/VM's that supports multi-core…

https://stressgrid.com/blog/cowboy_performance_part_2/ Might be interesting. Amongst other things, they experimented with a patch to Cowboy to use {active, 100} instead of the default {active, 1} and got some nice wins.

https://stressgrid.com/blog/webserver_benchmark/ These benchmarks, while dated and haven't been tested with the newly released updates to address these dynamics - might interest some folks.

It is explained in the blog post after that benchmark - https://stressgrid.com/blog/beam_cpu_usage/ . Essentially in order to optimise responsiveness the BEAM uses busy waiting, which in reality is not actually utilising the CPU as much as is reported by the OS, which results in misleading CPU usage being reported by the operating system.

Elixir and Go both have similar performance (although Elixir tends to use a lot more CPU). Here's a pretty good blog post comparing Go, Node and Elixir: https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/

We can't compare about CPU values Can you explain more why we can't compare? Looking at this chart... https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/10K-cpu.… ...shows pretty clearly how the CPU utilization grows linearly (ignoring some sawtooth) as the load increases, plateaus once the load remains constant, and then comes down linearly as the load decreases on the other end. Looks like a very clear mapping between work done and CPU load to me.

I have assumed BEAM has similar latency to Go and is garbage collected? Here is some benchmarks where Erlang beats Go in throughput: https://timyang.net/programming/c-erlang-java-performance/ https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/

huh. that seems like a lot of waste for no measurable benefit: https://stressgrid.com/blog/beam_cpu_usage/ maybe it's beneficial to someone, but I don't think that should be the default. either way, it doesn't change the picture painted by the TechEmpower benchmarks.

For what purpose? See: https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/

Erlang/Elixir's VM is gear toward high uptime, low latency, and highly concurrent. https://stressgrid.com/blog/100k_cps_with_elixir/ What it's not good at is numerical computation. As for the others I have no clue but I am sure jvm won't be as low latency as Elixir's VM since it's not a preemptive scheduler.

As far as from my tests and what I've seen reported online, Go and Rust have a substantial lead (20% ish) over Erlang for high throughput servers. EDIT: I believe this is partially due to Go being a lot more CPU efficient overall than Erlang (see below). So for simple servers, Go and Erlang will match performance, but for slightly more complex web servers that need to crunch some data, Go [and Rust] will outperform the Erlang VM. https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/

Here's a good comparison between Elixir, Go, and Node: https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/ Some of the things I like better about Elixir and Erlang vs. Go and Java are: - In Elixir/Erlang, all processes are stored in private memory and can only be accessed via defined message interfaces. In both Go and Java, processes are stored in public memory. - Elixir/Erlang processes are a lot smaller, memory-wise, than Go/Java processes. - We moved all of our development at my current company from Java to Elixir because Elixir, being functional, makes writing multi-threaded applications a lot faster/easier. Elixir is also cheaper to deploy. At least that has been our experience. - I like the syntax of both Elixir and Erlang better than Go, Java, or JS. - I don't have as much experience with Node/Express, but in my limited testing with Node/Express vs. Elixir/Phoenix, the latter is several times faster and much more scalable. It also makes better use of hardware (which I believe is discussed in the link above). I hope at least some of this helps answer your question.

I have commented this in other places but the main explanation for the benchmark you linked is that Phoenix (and its underlying Cowboy webserver) are not optimized for this particular workflow. For a plain-text benchmark, you are ultimately measuring the ability of sitting on top of a socket and reading/writing to that socket as fast as you can. However, Phoenix/Cowboy, whenever there is an HTTP connection, spawns a VM light-weight process to own that connection, and then each individual request runs in its own light-weight process too. This comes with its own guarantees in terms of state isolation, reasoning about failures, and you get both I/O and CPU concurrency for free - if you were to do any meaningful work on those requests. Even though the VM processes are cheap to spawn and lightweight, it is overhead compared to something that is just directly reading and writing to the socket. I actually wrote a proof of concept where we just sit on top of the socket without spawning processes and it performs quite well - although I don't think it has any practical purpose. There is also an interesting article from [StressGrid](https://stressgrid.com/blog/cowboy_performance_part_2/) that shows how removing the per-request process speeds it up by ~60%. But once you are going through long requests (i.e. 1ms because you need to talk to a database or API, encode JSON, etc), these differences tend to matter less and the concurrency model starts to give you an upper hand.