SoftwareMill Blog

Could you elaborate? It's a great writeup. But I'm personally curious about them hitting the limits of vertical scalability. They said they used the biggest node 'available to them' without clarifying how big that was. If the issue was RabbitMQ hitting resource limits then upgrading its hardware (or swapping it out for some other similarly featureful broker but which is faster) might have fixed some of the problems too, albeit not things like insufficient observability. I also wonder to what extent RabbitMQ suffers from being written in Erlang. Switching to a featureful broker with sharding and replication that isn't Kafka e.g. Artemis might have allowed them to avoid rewriting code to not use helpful features. This performance blog suggests RabbitMQ may top out at ~4000 messages/sec when making things durable, which isn't especially good performance: https://softwaremill.com/mqperf/

> Is java able to stackify most new Object calls and elide default initialization of object members these days? Escape analysis in OpenJDK will stack allocate values where it can show it is safe to do so. Project Valhalla is also reducing the memory footprint of objects. As for thread safety, that is more of a language concern than a runtime one. Amongst JVM languages Scala is leading here AFAIK. Its "capture checking"[1] provides thread safety (e.g. [2]) and actually covers escape analysis as well. On Scala Native (the native code backend for Scala) capture checking can be used for safe stack allocation and safe arena allocation. [1]: https://docs.scala-lang.org/scala3/reference/experimental/cc.html [2]: https://softwaremill.com/understanding-capture-checking-in-scala/

Scala is broadly used by the biggest companies in existence. https://business4s.org/scala-adoption-tracker/ The stuff currently cooking in Scala 3 will once more revolutionize the whole programming language landscape. https://softwaremill.com/understanding-capture-checking-in-scala/ https://www.youtube.com/watch?v=p-iWql7fVRg

Every company is for profit. OpenAI was a “non-profit”. “Worse” and “good enough” can be true at the same time. Selfhosting signal is inferior to selfhosting matrix [0]. [0] https://softwaremill.com/can-you-self-host-the-signal-server/

While I get how that's like git, it sounds even closer to unison: https://softwaremill.com/trying-out-unison-part-1-code-as-hashes/

If that's true, you managed to do much better than these folks: https://softwaremill.com/mqperf/ Maybe you should write a letter?

I have looked at elasticmq but not played with it myself. You might also be interested in their benchmarks of all of the existing queues out there: https://softwaremill.com/mqperf/

> Git sucks. Reason? It is entirely imperative. A git repository is a persistent data structure [1]. Operations are typically append-only, requiring --force flags to do any in-place mutation. > And, well, just like in C, there's no safety pillow - if you screw up a call, something simply goes pear-shaped. Git is the safety pillow. When you screw up your code, you restore it with git. Article doesn't mention the reflog, which is the second safety pillow hiding under the first. [1] https://softwaremill.com/persistent-data-structures-in-functional-prog…

It seems that you are mainly interested in how to build preview environments for your app, these posts describe an approach to get there: - https://softwaremill.com/preview-environment-for-code-reviews-part-1-c… - https://softwaremill.com/preview-environment-for-code-reviews-part-2-c… - https://softwaremill.com/preview-environment-for-code-reviews-part-3-r… While the examples use Gitlab, it shouldn't be very hard to port the same idea to a Bitbucket.

Yes indeed. Attaching below a couple of references from my exploration: Transactional outbox pattern: - https://microservices.io/patterns/data/transactional-outbox.html - https://softwaremill.com/microservices-101 - https://brandur.org/job-drain A pattern similar to the one I mention using cache: - https://masstransit-project.com/articles/outbox.html

Up to a few years ago java couldn't see the correct size of memory allocated to the container. https://blog.softwaremill.com/docker-support-in-new-java-8-finally-fd5…

I found this helpful for getting going with OpenJDK 8/Graal on Mac OS X (but the instructions are highly applicable to Linux as well). https://blog.softwaremill.com/graalvm-installation-and-setup-on-macos-…

Cassandra won't try and load a partition into memory. It doesn't work that way. The only way you would get behavior like that is by setting "allow filtering" to on. Allow filtering is a dedicated keyword for "I know I shouldn't do this but I'm going to anyway". If you're trying to run those types of queries, use a different database. If someone is making you use a transactional database without joins for analytical load, get a different job because that's a nightmare. Also, your partitions should never get that large. If you're designing your tables in such away that the partitions grow unbounded, there's an issue. There are lots of ways to ensure that the cardinality of partitions grows as the dataset grows. And you actually control this behavior by managing the partitioning. It's really easy to grok the distribution of data in on disk if you think about how it's keyed. You've basically listed a bunch of examples of what happens when you don't use a wide columnar store correctly. If you're constantly fixing downed nodes, you're probably running the cluster on hardware from Goodwill. This is a pretty good list of what not to do with Cassandra, or any similar database. https://blog.softwaremill.com/7-mistakes-when-using-apache-cassandra-5…