VoltDB Blog

This is essentially the idea of VoltDB [1]: All tables are partitioned and each partition is assigned to a single thread. As long as each query only touches a single partition, you get linear horizontal scalability and serializable isolation. The drawback is that if a query touches multiple partitions, only that one query can be executed on the entire cluster, so this architecture only works for very specific OLTP workloads. [1] https://www.voltdb.com/

An unfortunate choice of name. See Volt DB, an in-memory database that's been around for 10 years and raised $41M. Registered trademark. https://www.voltdb.com/ https://www.crunchbase.com/organization/voltdb

I think this is a good way to look at things, and there are vendors who do this! VoltDB, for instance, changed their defaults to be strict serializable even though it imposed a performance hit, following their Jepsen analysis. https://www.voltdb.com/blog/2016/07/voltdb-6-4-passes-official-jepsen-…

One of my favorite things about VoltDB, as one of the people who built it, is that its serializable isolation is actually a fundimental part of the architecture. Many of these systems get slower as you turn up the isolation. VoltDB is faster because of isolation. More here: http://voltdb.com/products/featuresbenefits/reasons-behind-voltdb-arch…

The first time I ran into the NewSQL term was several years ago with VoltDB [0]. [0] http://voltdb.com/

I think durability should not be defined in terms of actual backing hardware. VoltDBs transactions commit and stay committed. If you restore from a backup, just like any other database, you'll have data loss until the backup point. What if my hard drive crashes when it loses power? At that point I'm in the same position as when I lose RAM. VoltDB is often compared to other in-memory NoSQL systems, although I'm not aware of any that actually offer cross-server transactions. VoltDB is best when you have a high volume OLTP workload and need to really scale beyond what traditional RDBMS can do. Edit: Here's a benchmark using VoltDB as a key-value store: http://voltdb.com/company/blog/key-value-benchmarking

Not a lot of info on their website. But it strikes me as similar to VoltDB (http://voltdb.com/). Especially the emphasis on OLTP queries. Is it based on the same paradigm of single-threaded serial access? What differentiates it from VoltDB? Is it the MySQL compatibility and ability to use ad-hoc queries instead of stored procedures? (If so, how do you make that bit fast?)

Isn't that what VoltDB is for? http://voltdb.com/ Edit: Also, I see no mention of whether Druid is going to be open sourced or not.

They're just gauging interest and making sure to get emails they can push adoption to later, it's a classic sales technique. Don't hate. Regardless, the links they send are open: VoltDB product overview: http://www.voltdb.com/_pdf/VoltDBOverview.pdf VoltDB Technical Architecture white paper: http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf

Thank you, it was FoundationDB I was thinking of, post-mortem analysis from the VoltDB guys here[1]. I'll contend they failed though because Apple bought them as an acqui-hire and killed the product. They were acquiring good engineers with a track record of executing well at a premium. It's a common recruiting strategy for tech companies. It doesn't mean that FoundationDB was a successful product - certainly it means it was not successful enough for the founders to risk turning down a lucrative acqui-hire deal. > his is a rigid and narrow view. There's lots of other design possibilities such as fractured mirrors, column decommission only within pages, etc. Yes definitely I over-simplified to make my point, but I'll stand by it. OLAP and OLTP are different enough workloads and the engineering tradeoffs for being good at one of the other make it hard to excel at both. VoltDB is a row store, best suited for OLTP. Vertica is a column store best suited for OLAP. VoltDB can also make a decent OLAP database. Vertica does not make a good OLTP database. Both databases come out of Michael Stonebraker's work (although given the number of people required to make complex products like VoltDB and Vertica happen, it seems unfair to credit him unduly.) But especially in the case of a sharded, distributed database where communication overhead is likely to dominate many types of queries (VoltDB, TiDB), one might not notice the inefficiency of using a poorly-suited design, like a row store…

I noticed that the question on performance was ignored, but I think it's worth noting that FoundationDB died on exactly the same beach. [1] I'm not extremely familiar with the CockroachDB architecture (other than "it's kinda like the F1/Spanner/Bigtable stack"), but I suspect that there are some commonalities. For example, CockroachDB's Bigtable-style two-level range indexing means you could run into exactly this problem: > FDB-SQL stores metadata in the KV store itself. This is great from a distributed correctness point-of-view. It’s also great from a simplicity point-of-view. If I trust the underlying system to be safe and consistent, then my metadata is also safe and consistent. But now I need to do a ton of reads before I run my SQL to know how to run my SQL. Where’s my data, for example? [1] https://www.voltdb.com/blog/2015/04/01/foundationdbs-lesson-fast-key-v…

Yes, this is the argument that VoltDB made as well: https://www.voltdb.com/blog/foundationdbs-lesson-fast-key-value-store-… My feelings on this topic are mixed. On the one hand, I think many of the specific examples chosen in that post are false (and have told John as much in person). On the other hand, the general point that you can squeeze out constant factor performance improvements by violating abstraction boundaries is obviously usually true. Nevertheless, I still think this is a bad argument. While it's true that abstractions are rarely costless, they can often be made so cheap that the low-hanging performance fruit is elsewhere. And in particular, cheap enough that they're worth it when you consider all the other benefits that they bring. When I built a query language and optimizer on top of FoundationDB, my inability to push type information down into the storage engine was about the last thing on my mind. Perhaps someday when I'd made everything else perfect it would've become a big pain (and perhaps someday we would've provided more mechanisms for piercing the various abstractions and providing workload hints), but in the meantime partitioning off all state in the system into a dedicated component that handled it extremely well made the combined distributed systems problem massively more tractable. The increased developer velocity and reduced bugginess in turn meant that I (as a user of the key-value store) could spend scarce engineering resources on…

Sounds like event sourcing [1] applied to databases. The server-side transactions reminded me a lot of VoltDB [2], which also has server-side transactions and turns out was a previous system Abadi was involved in. My naive impression is that VoltDB is more about being in-memory, single-threaded, and lock-free, where as this Calvin approach is more about deterministic ordering of events. Sounds really cool, but there is a reason people don't like writing stored procedures. Besides (historically) meaning you're forced into a non-general programming language, it means you can't do anything else while executing the logic (call out to a 3rd party, read a file from somewhere else, etc.). While SQL has its flaws, the semantics of writing your application logic freely over multiple queries ("make a query, do some work, make a query, do some work, ... commit!") and having the database just make that magically work for you is pretty darn nice and not something I'd easily give up. Maybe a heavy dose of optimistic locking and a pleasant/mostly automated process of coding/integrating/deploying the stored procedures would make it possible to get all the performance benefits of their approach. It does sound pretty smart. [1]: http://martinfowler.com/eaaDev/EventSourcing.html [2]: http://voltdb.com/

From their whitepaper: "Conventional databases experience disk and user stalls within transactions. Rather than let the CPU be idle during the stalls, those DBMSs interleave SQL execution from multiple transactions during the waits so the CPU is always busy. This is what requires much of the complex latching and locking overhead. VoltDB doesn’t experience user stalls (since transactions happen within stored procedures) or disk stalls (because VoltDB processes data in main memory). Therefore, it is able to eliminate the overhead associated with multi-threading (latching) and locking. Each VoltDB execution engine is single-threaded and contains a queue of transaction requests, which it executes sequentially—and exclusively—against its data. Elimination of stalls and the need for locking and latching overhead and allows typical VoltDB SQL operations to complete in microseconds. For single-partition transactions, each VoltDB engine operates autonomously. For multi-partition transactions, one engine distributes and coordinates work plans for the other engines. VoltDB assumes that an application designer can construct a partitioning/cloning scheme and a transaction design that makes a large majority of the transactions local to a single virtual node. Many common applications such as order-fulfillment, software as a service (SaaS), Web 2.0 and trading systems have this property." http://www.voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pdf

There was an SQL layer but performance was sub-par IIRC. There was also a blog post somewhere explaining why it's probably not a good idea to build an SQL layer on top of a KV store, devil in details, etc. Edit: found it https://www.voltdb.com/blog/2015/04/foundationdbs-lesson-fast-key-valu… Not sure what to think of it, I'm not a DB expert by any means but the post sounds plausible enough and the SQL layer is discontinued AFAIK. I guess with each new abstraction layer you leave some perf on the table.

I am not a fan of this buzzword (and indeed buzzwords in general, the comment above was intending to make fun of HSBC IT for even failing at the buzzword game), but the notion of NewSQL was roughly to keep the relational model and consistency guarantees of traditional SQL databases, but also have horizontal scalability in line with NoSQL databases. DBs like VoltDB and Spanner come to mind as examples -- here are two relevant blog posts from those teams: VoltDB: https://www.voltdb.com/blog/2016/09/nosql-vs-newsql-whats-difference/ Spanner: https://cloud.google.com/blog/products/gcp/from-nosql-to-new-sql-how-s…

I think the point is that you might be able to build something else than a standard relational db and make it faster. Something like VoltDB but there are other solutions. There used to be some blogs about their architecture and how they could avoid a couple of steps a normal RMDB needs that takes a lot of time. https://www.voltdb.com/why-voltdb/

> monolithic databases like CockroachDB (SQL) or FaunaDB (NoSQL) trade off extensibility The CockroachDB documentation says their SQL implementation is layered on top of their K/V interface: https://www.cockroachlabs.com/docs/stable/architecture/overview.html#l… This would make it similar to TiDB/TiKV and FoundationDB. Here's and old article from VoltDB insisting that layering SQL on pure K/V forfeits too much performance: https://www.voltdb.com/blog/2015/04/01/foundationdbs-lesson-fast-key-v… Edit: And a response: https://news.ycombinator.com/item?id=15505194

Since becoming open source and the system's architecture is in the open, I wonder how this criticism of FoundationDB from a VoltDB architect https://www.voltdb.com/blog/2015/04/01/foundationdbs-lesson-fast-key-v… fairs against the knowledge that has now become available. (To summarize, the author argues that building an SQL layer on top of an ordered key-value store is suboptimal)

CockroachDB is also built on a key/value layer using a Spanner type design. MemSQL is OLAP that does OLTP well, and Citus can make Postgres handle OLAP pretty well too (as used at Heap). The article you're referring to is about Foundation DB written by John Hugg of VoltDB: https://www.voltdb.com/blog/2015/04/01/foundationdbs-lesson-fast-key-v… The database world is rapidly changing and we're seeing new technologies that are enabling a lot more than we could do before. Just looking at some of the SQL on X engines alone shows massive improvements.

This is a blog post about FoundationDB SQL layer mapping from the VoltDB guys: https://voltdb.com/blog/foundationdbs-lesson-fast-key-value-store-not-…

Yes, the slides are deceptive and they don't motivate me to take time looking the video despite the well-known and rewarded speaker. This page from VoltDB explains far better "the reasons behind the VoltDB architecture" (http://voltdb.com/products/featuresbenefits/reasons-behind-voltdb-arch…

http://voltdb.com/john-hugg-work-volt is this you at voltdb ?

(I work at VoltDB) VoltDB is easy to try. There is a free 30-day trial available on voltdb.com, and a docker-based demo (http://voltdb.com/voltdb-demo).

Looks interesting, but a bit of FUD on their pricing page ( http://www.voltdb.com/pricing ): Open-Source Community Project: "...This edition is unsupported and most likely to be used in non-mission critical applications."

FaunaDB uses Calvin, a transaction protocol developed by Daniel Abadi. Their blog post explains it nicely, after a bit of a slow start: https://fauna.com/blog/consistency-without-clocks-faunadb-transaction-… But in summary: 1. A 'transaction' is a self-contained blob of code which reads input, does deterministic logic, and writes output (so not like a traditional RDBMS transaction, where the application opens a transaction and then interleaves its own logic between reads and writes) 2. When a transaction arrives, the receiving node runs it, and captures the inputs it read, and the outputs it wrote 3. The transaction, with its captured inputs and outputs, is written to a global stream of transactions - this is the only point of synchronisation between the nodes 4. Each node reads the global stream, and writes each transaction into its persistent state; to do that, it repeats all the reads that the transaction did, and checks that they match the captured input - if so, the outputs are committed, and it not, the transaction is aborted, and retried The key idea is that because the process is deterministic, the nodes can write transactions to disk independently without drifting out of sync. It's pretty neat. And it's exactly what Abadi wrote about a couple of months ago: http://dbmsmusings.blogspot.com/2019/01/its-time-to-move-on-from-two-p… This is also what VoltDB does (which Abadi worked on, along with Michael Stonebraker): As an operational store,…

Quick things 1) I expected that thread to look like a Cultural Revolution struggle session". Thankfully it wasn't. 2) As I am sure many others have already said it, durability has very little to do with CAP. CAP is about the A and I in in ACID, D is an orthogonal concern. 3) Durability doesn't necessarily mean losing high performance. Most databases let the user choose how much data they're willing to lose and for what latency decreases -- the standard approach (used even in main-memory databases like RAMCloud[1] and recent versions of VoltDB[2]) is to keep a separate write-ahead log (WAL) and let the end-user choose how frequently to fsync() it to disk as well as how frequently to flush a snapshot of main memory data structures to disk. There are many papers (e.g., http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.174.6205) that talk about various challenges of building WALs, but fundamentally users who want strongest possible single-machine durability can choose to fsync() on every write (and usually use battery backed raid controllers or separate log devices like SSDs with supercapacitors or even NVRAM if the writes are going to be larger than what fits into the RAID controller's write-back cache). Others can choose to live with possibility of losing some writes, but use replication[3] to protect against power supply failure and crashes -- idea being that machines in a datacenter are connected to a UPS, replicas don't all live on the same rack (to protect…

Sure, just a few blog posts and some other things really, but here you go. https://cacm.acm.org/blogs/blog-cacm/50678-the-nosql-discussion-has-no… http://www.labouseur.com/courses/db/Stonebraker-SQL-vs-NoSQL-2010.pdf http://www.redbook.io/ https://www.barrons.com/articles/michael-stonebraker-describes-oracles… https://blog.jooq.org/2013/08/24/mit-prof-michael-stonebraker-the-trad… http://wp.sigmod.org/?p=1629 https://www.voltdb.com/blog/ http://www.paradigm4.com/category/blog/

Could somebody clarify the license? I was excited this was Open Source, but then some people say you are only allowed to run it on MacOS? How can that be? Also, how does FoundationDB compare (in general, and in performance) to: - CockroachDB (competitor) https://www.cockroachlabs.com/blog/2-dot-0-perf-strides/ - Redis (competitor) https://redis.io/topics/benchmarks - GUN (ours) https://github.com/amark/gun/wiki/100000-ops-sec-in-IE6-on-2GB-Atom-CP… - Fauna (competitor) https://fauna.com/enterprise#Scalability - VoltDB (competitor) https://www.voltdb.com/blog/2017/10/02/comparing-fast-data-performance… - RocksDB (competitor) https://github.com/facebook/rocksdb/wiki/performance-benchmarks Is the the big boom in FoundationDB just that it is "now Open Source", or that Apple vetted it (why didn't they keep it closed source?), or is there something distinctly unique about FoundationDB that it surpasses a bunch of other tried and true systems?