Tsuna

A case of only so many words in the English language. Easy to fix if we distinguish hardware fast path from the software variant. Software fast paths are worth looking into since context switches are costly to the tune of thousands of CPU cycles [0,1]. Those thousands of CPU instructions going up in smoke could've done quite a lot of useful work. [0] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… [1] https://gist.github.com/nelsnelson/3955759

There is nothing fundamentally new described in the blog post, although pinning threads to core is almost never the default operation mode (though many popular servers expose an option for turning this one). As someone else stated in another post, SO_REUSEPORT and accept4() is best, all things considered, way to accept connections across multiple threads. Again, most modern servers support this by default, if supported by the underlying operating system (e.g nginx). By the way, Aerospike accepts new connections in a single thread and then directly adds the socket FD to one of the i/o threads (round-robbing selection scheme) directly using e.g epoll_ctl(fd, EPOLL_CTL_ADD, ..). See http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… for costs of context switching, and performance improvement when pinning threads to cores (it's quite impressive). Also, note that according to the author, on average, a context switch is 2.5-3x more expensive when using virtualization. You may also want to read https://medium.com/software-development-2/high-performance-services-us… -- it's been a long time since I wrote this, but it describes how one can deal with asynchronous I/O and other operations that may block a thread.

I heard that, especially with spectre, most of the cost of context switching comes from messing up your caches + TLB (https://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-contex… There's a graph on Dan Luu's blog about how it takes 14,000 cycles to get back to full performance after a syscall: https://danluu.com/images/new-cpu-features/flexsc_ipc_recovery.png.

It's a bit dated, but here's a reference: https://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-contex…

You should clarify your terminology: -- "context switch" (expensive) is the switch among active (running on this CPU) tasks/threads/processes. -- switching between kernel and userspace would be "mode switch", much less expensive and of "constant" cost (see, for example, http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… Your phrase already carries this contradiction: "__can__ be fairly expensive" points to something depending on what is being executed, rather than a property of the mode switch itself. Then, arguably, the same need of that costly execution should also be present with unikernels?

> Context switches are really fast on modern x86. Not context switching is even faster. How fast is really fast, how modern is modern? some results from 2010 [1] show at least 1000 nanoseconds per context switch (even with optimizations for the kernel switching between threads of the same process). That's not that slow, but it is a lot of time you could be doing something else. [1] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… [1] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context…

The syscall overhead isn't the problem, that's dirt cheap as you say. The problem is the context-switch overhead. Calling into the OS flushes a lot of data and instructions from the cache, and that lost performance after returning can easily add up to around 30µs.[1] [1] http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context…

I can't explain completely, but check out http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… Relatively speaking, I don't think context switching is _that_ expensive compared to other areas you can focus on like doing smarter memory management. I don't know exactly what they mean by threads having heavy memory overhead, but possibly they mean cache interference mentioned in that link? I'd be curious if there is an actual large memory chunk other than stack/scheduling details in play here too. On modern cores too, there are a decent chunk of registers, including floating point registers. I've looked into timings before for embedded applications and the performance hit here isn't trivial when looking at interrupts and the like; but I'd be surprised if it was that overwhelming on servers.

"Context switching is expensive. My rule of thumb is that it'll cost you about 30µs of CPU overhead. This seems to be a good worst-case approximation. Applications that create too many threads that are constantly fighting for CPU time (such as Apache's HTTPd or many Java applications) can waste considerable amounts of CPU cycles just to switch back and forth between different threads." http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… On 32-bit systems, you can also easily run out of address space for your thread stacks.

I enjoyed it when I first read it a couple of years ago and the recent blog post from tsuna on context switching reminded me of it (http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context… Glad to see some other folks like it, as well.

If you read the article linked to in the article I linked to above (which is fairly out of date, from 2010): https://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-contex… you will find that on Linux a context switch takes about 30 usec. More recent measurements that take account of the effect of the TLB flush put the range at 10-300usec. That means that in 2010, on Linux, you could reasonably expect to do at least 30k/sec. In 2021, with realistic audio processing workloads, the range is probably 3-50k/sec. The 486 is a much lower register count than contemporary processors, which accounts for the faster context switching. Modern audio processing software on Linux can run with 64 sample buffers, not 1k. This recent paper on RT linux on RPi/Beagleboard single board systems concludes that on some of these relatively "low power" systems, 95% of latencies are in the 40-60usec range, which is completely adequate for the majority of RTOS tasks (but not all). https://www.mdpi.com/2073-431X/10/5/64/pdf >"The majority of Linux kernels’ measurements with PREEMPT_RT-patched kernel show the minimum response latency to be below 50 μs, both in user and kernel space. The maximum worst-case response latency (wcrl) reached 147 μs for RPi3 and 160 μs for BBB in user space, and 67 μs and 76 μs, respectively, in kernel space (average values). Most of the latencies are quite below this maximum (90% and 95%, respectively, for user space and kernel space). In general, it…