Baron Schwartz

your findings are supported by Baron Schwartz as well (https://www.xaprb.com/blog/2012/09/17/is-automated-failover-the-root-o… and others)

If your nail looks like smaller size important data, CDC / immutable datastores seem like a great hammer. For all other stuff, the answer is: it depends. Some thoughs on the limitations of this approach: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-col…

Immutability is hardly a cure-all, see the discussion here for why RethinkDB moved away from it: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-col… The reality is shared, mutable state is the most efficient way of working with memory-sized data. People can rant and rave all they want about the benefits of immutability vs mutability, but at the end of the day, if performance is important to you, you'd be best to ignore them. Actually, to be more honest, reality is more complicated still. MVCC that many databases use to get ACID semantics over a shared mutable dataset is really a combination of mutable and immutable.

The author also has thoughts on Datomic: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-col… tl;dr; append-only is great as long as you don't use your database much.

I'd like to welcome the github ops/dbas to the club of people who've learned the hard way that automated database failover usually causes more downtime than it prevents. Here's sortof the seminal post on the matter in the mysql community: http://www.xaprb.com/blog/2009/08/30/failure-scenarios-and-solutions-i… Though it turns into an MMM pile-on the tool doesn't matter so much as the scenarios. Automated failover is simply unlikely to make things better, and likely to make things worse, in most scenarios.

This can be flipped into a simple heuristic known as the "square root staffing rule" (sometimes "... law"). The name comes from call centres. Basically, the number of servers you need to serve X amount of demand with Y probability of queuing is not linear with X. It is proportional to the square root of X. The intuition is that a call centre agent is either talking to a customer, or they are not. If they are not talking to a customer, then they can immediately serve a customer. The odds that a given agent is talking to a customer is on a probability distribution, usually assumed to be the normal distribution. In a normal distribution, larger samples lead to more and more appearances of outliers in the sample. More agents means more "outliers", in this case, more idle agents. And that curve is not linear, it follows the shape of the normal distribution. Where it sucks for call centres is that demand shows seasonality and shift planning has relatively inflexible lead times. But in a software scenario we can typically acquire additional capacity quickly, so precision in forecasts is less of a problem. A decent explanation: https://www.networkpages.nl/the-golden-rule-of-staffing-in-contact-cen… A short scenario: https://www.xaprb.com/blog/square-root-staffing-law/ A simple calculator linked from the short scenario: https://www.desmos.com/calculator/8lazp6txab

It might help to think about time-series databases from the requirements they're addressing. "Time-Series Database Requirements" [0] is a good summary of the problem space. [0] https://www.xaprb.com/blog/2014/06/08/time-series-database-requirement…

> We'll never be a spacefaring species, the universe is way too big (we can't even explore our solar system in any meaningful way) and energy requirements for faster travel are simply too high. I can state flatly that heavier than air flying machines are impossible. — Lord Kelvin, 1895 https://www.xaprb.com/blog/flight-is-impossible/

> It is difficult to imagine anything resembling a "general" intelligence that isn't attached to a life-form[...] How is that argument any stronger than these ones that (non-animal) heavier than air flight is impossible? https://www.xaprb.com/blog/flight-is-impossible/ I think these are actually stronger arguments, because I can think of at least three very least significant material-science advantages that birds have even over modern technology that would, from a physics point of view, let me entertain the idea that a 19th century physicist could believe you'd at least need to be made of flesh to fly. I can't think of any comparable case for intelligence being restricted to biology.

I have no comment on the QC claims, but regarding the flight analogy, it seems you're being disingenuous: https://www.xaprb.com/blog/flight-is-impossible/

Although I wouldn’t work for a company which blindly made hiring decisions based on a profile, the approach in this blog post would make the company more attractive: https://www.xaprb.com/blog/personality-assessments-hiring/ Excerpts: > The assessments are never really correct, so a vital step is reviewing the report with the candidate … > How the candidate reacts to the report; which things they think are true or false and why; which things they engage with and which they skip over; those are all revealing.

Related: Baron Schwartz article on Time-series Database Requirements is quite a read: http://www.xaprb.com/blog/2014/06/08/time-series-database-requirements… HN discussion: https://news.ycombinator.com/item?id=9166495

MVCC is not versioning. MVCC is concurrency control. I would have to be convinced that the DB is an appropriate place to handle versioning of the kind you are speaking of. For instance, see this article: http://www.xaprb.com/blog/2013/12/28/immutability-mvcc-and-garbage-col… Upshot is that RethinkDB started out append-only with rewindability etc. and then ran into real-world problems. Every system I've ever seen that did versioning does it at the software level and stores multiple records in the database. Rails gem "paper trail" is an example.

MySQL sort-merge joins, it's right in that documentation, but it calls everything out as optimizations on nested-loop. http://www.xaprb.com/blog/2013/10/01/mysql-isnt-limited-to-nested-loop… It doesn't do hash joins which is the biggest limitation but MySQL is pretty good a joins and certainly was never bad enough to warrant it's reputation. It's toolbox is smaller than other RDBMS products but it's not empty.

> Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because. Maybe this article will shed some light: http://www.xaprb.com/blog/2010/05/10/the-manager-programmer-face-off-o…

TokuMX. They ripped out the native MongoDB storage engine and replaced it with the TokuDB engine. http://www.xaprb.com/blog/2013/04/29/what-tokudb-might-mean-for-mongod…

http://www.xaprb.com/blog/2006/05/02/how-to-write-efficient-archiving-… http://www.maatkit.org/doc/mk-archiver.html

The biggest problem is the query language. Without a query language, a company must hire and keep on-staff a developer who can help express problems to the database. Time after time people told me that they ended up with giant databases and needed to run lots of ad-hoc queries or generate reports from them. Once a time-series database becomes the system of record for important information, it needs to be queried for everything from invoicing to analytics to troubleshooting. In practice, larger companies end up with at least two full time developers who write applications to produce answers from the database. These are answers that the business/marketing/ops/whoever should be able to get themselves by just running queries. As time passes, this is both a huge cost and a bottleneck.[0] Link [1] goes to the archive mirror. [0]: https://www.xaprb.com/blog/2014/03/02/time-series-databases-influxdb/ [1]: https://web.archive.org/web/20170917033402/https://www.xaprb.com/blog/…