Jeremy Cole

THIS. My favorite example is Jeremy Cole’s blog [0], especially the stuff on InnoDB. You want to take a deep dive into the murky internals of MySQL? This is it. Tons of other stuff as well, but those are my favorite. He wrote tooling to visualize .ibd files FFS. [0]: https://blog.jcole.us/

Lots of people mentioning Oracle, but only one mentioning MySQL (which is famous for doing this; I wasn't even sure Oracle did ;P). This article hasn't showed this is a massive win... and frankly I doubt this is even that difficult to prototype. Before writing this article I would have loved to see them do some simple benchmark to show this was an interesting direction. As it stands I don't even see why this was published: they didn't even do a simple comparison to MySQL. Like, this is how I read this article: "PostgreSQL does something one way, which everyone is used to, and it is clearly pretty stupid; we have an entirely new way of thinking about this that we are pretty sure--but not 100% sure as we aren't that arrogant--will be better than the way it is happening now: we will just do this, this, and this, which certainly seems better. more to come!". The article I would have wanted to see: "it is well known in the world of database software that there are two ways of doing this one thing, and PostgreSQL is pretty unique in doing it this way which we know has some downsides, such as this, this, and this; of course, the other way also has known downsides, such as this, this, and this; we started to ask ourselves if it would still make sense to try that model for some workloads against PostgreSQL". Essentially, a good version of this article would have at least mentioned why MVCC is interesting at all (as this article skirts dangerously close to "clearly we should just…

You reboot a machine every time it starts to use swap?? What the hell kind of applications are you running? Even with java apps (which I grant you are difficult to estimate memory limits on without knowing the application's design) swapping can be a useful way to page out unneeded parts of memory - and more importantly, keep needed parts of memory intact. This can also mean keeping the Java processes in memory while paging out apps which are less crucial, which leaves more room for Java, etc. Swap is, for lack of a better comparison, the canvas sheet you use to catch someone jumping off a building, or a new york city sewer. In the first example you can use it to save your applications/servers so you don't need to reboot them (the higher your availability requirements, the less you can stand random reboots). In the second example it's the place you send those inhabitants you don't deem worthy of RSS. The other thing you have to consider is the idea of memory overcommit in application and kernel design. The system is built with a promise that it has way more memory than it actually physically does. Applications will always reserve a fake huge chunk of memory and doesn't care that the system is lying to them about how much is really available. Without swap, when these apps attempt to use the nonexistent available memory, they crash. With swap, they survive. Then there's apps designed to rely on swap like the Varnish malloc() and file storage methods, or database servers.…

With a 16GiB RAM laptop I often use more memory than that which then ends up on swap (on zram) With NUMA swap etc. also becomes a bit more complicated (see also https://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-numa-arch… ) I think the main question here is to use a swap partition or a swap file. The main problem of swap partitions is how inflexible they are, but lvm/btrfs/zfs make that less of an issue. On Solaris a crashdump was written to swap when the system panicked. Not sure what Linux/BSD crashdump systems use nowadays.

InnoDB does indeed use a B+ tree. A more thorough article on the topic is http://blog.jcole.us/2013/01/10/btree-index-structures-in-innodb/ or the book High Performance MySQL.

> If I were them, I'd confirm this by "perf record" or eBPF and/or by disabling transparent huge page compaction The report does look a lot like transparent hugepage defrag moving around memory randomly & system CPU spikes for no good reason. And the workaround is valid, because THP can currently only map anonymous memory regions such as heap and stack space. Literally the first thing I have to do to a Hadoop cluster is go around and turn off the defrag for the multi-threaded bits we have or system CPU goes over 20% as the system goes into workloads. The other way you end up with a thundering herd causing permanent issues is with NUMA balancing (+ false sharing). So if you scale up a system slowly, you end up allocating in the same NUMA zone for a whole section (like a 1Gb array), which makes traversing it faster, but if this happens at the same time as another thread touching the same data, it will get scheduled over in a different zone & the kernel can do a lot of busy work with zone rebalancing as you go above 50% memory usage. This used to be a big deal when the first ccNUMA Intel boxes were coming out & mysql at Yahoo used to have so much trouble with NUMA messing up memory access speed assumptions[1]. The 96 core + 235GiB of data suggests there is some NUMA messiness going on for sure, particularly because there's a pinned worker to a CPU in the design. The NUMA issues were front-and-center when the Power8 porting of Hadoop was going on , particularly because…