MotherDuck Blog

Not affiliated, just been following the product. I've been using this for a proof-of-concept internally at $DAYJOB and liked that it had out of the box support for querying data from MySQL and Postgres (we have separate services that use either one of these depending on the team). I've been working on a pipeline to pull data from both databases, and used GlareDB to do that alongside some transforms. Also looking at DuckDB as well. I saw that MotherDuck[0] opened up signups today and figured I post about GlareDB since there's some feature overlap (like the "hybrid execution"[1]) and I've been following both for a bit. [0]: https://motherduck.com/ [1]: https://glaredb.com/blog/hybrid-execution

I wonder how this relates to Mother Duck (https://motherduck.com/)? They do „DuckDB-powered data warehousing“ but predate this substantially.

Motherduck has an excellent article about this: https://motherduck.com/blog/the-simple-joys-of-scaling-up/

Did you check the cost to run it on Motherduck (https://motherduck.com/)?

Shameless plug - MotherDuck[0] is a serverless managed DuckDB data warehouse with some interesting properties: - Managed storage with zero-copy clone (and upcoming time travel) - Secure Sharing - Hybrid Mode that allows folks to combine client (WASM, UI, CLI, etc) with cloud data - Approved and supported ecosystem of third party vendors - Since we're running DuckDB in production, we're working very closely with the DuckDB team to improve both our service and open source DDB in terms of reliability, semantics, and capabilities [0]https://motherduck.com/ (co-founder at MotherDuck)

You might want to consider seeing how far vertical scaling can bring you. https://motherduck.com/blog/big-data-is-dead/ Is a good read

For those that don't know, this is referring to MotherDuck which was recently announced: https://motherduck.com/

https://motherduck.com/ Is it this thing? So it’s a non existent service that has 47m in funding? Or am I missing something.

Can you comment more on how you use persistent tries with unicity? And what do you mean by "the other way around"? I can't speak to latency fully as of yet. But in my own incremental implementation of a Prolly Tree, it appears to have the same Big-O bounds as a Btree (as you said). But I can see there are places that it has to do a bit more work. - Given the nature of merkle tree nodes identified by their hash, you have to go do look ups, rather than just following pointers. - The tree has to touch a swath of nodes from the bottom up, due to potential rebalancing. Unlike a Btree, where you just change a single node seam from root to leaf, even with splits and merges. - Prolly Trees are constructed from the bottom up, where you have to traverse across a layer. To find the next sibling, you sometimes have to travel up to the parents and come back down--hence subject to the occasional height traversal in the worse case. This doesn't happen often, especially if the cascading effect on insertion I talked about doesn't happen. But like you said, maybe there are other factors that dominate the latency equation. It's hard for me to say at this point. Relevant post: https://motherduck.com/blog/perf-is-not-enough/

MotherDuck | NYC, Amsterdam, San Francisco, or Seattle (Onsite preferred) | Software Engineers in AI, Database, and Ecosystems | Full-Time | $140K – $275K + Equity | https://motherduck.com/ Don’t let the quirky name fool you! We’re on a mission to radically transform the data analyst user experience. We think we can empower and delight analysts using a novel cloud architecture built on top of DuckDB, an open-source foundation with a vibrant community. Our team is a diverse mix of passionate, collaborative, and empathetic data industry veterans from Google, Amazon, Meta, Snowflake, SingleStore, Databricks, and the like. We're expanding our focus on AI, and are looking for Python/C++ engineers to help us bring the power of large language models into the hands of database users. Check out the open role here: https://jobs.ashbyhq.com/MotherDuck/b485b0d6-2cbe-4083-b253-f38bdca144… We also have open positions in our Database and our Ecosystems teams. Come work with us and disrupt the future of data analytics. https://motherduck.com/careers/

I was wrong. Apologies. > They decided to work together, and DuckLabs would become a co-founder of the new endeavor. https://motherduck.com/about-us/

There's been some good implicit/explicit discussion about SoA on this post [1]. For anyone curious, there are a few different terms that refer to basically the same thing: - Struct of Arrays (SoA) vs Array of Structs (AoS); - Row-major order vs column-major order; and - Row-based vs column based / columnar (in databases) Most code has arrays of structs (or "lists of objects", the effect is the same), and most databases are row based (same thing). But many game engines use SoA and keep heterogeneous elements together. Some databases like DuckDB do this too, this is an article about the pros and cons of columnar storage in DBs [2]. 1. https://news.ycombinator.com/item?id=49012056 2. https://motherduck.com/learn/columnar-storage-guide/

Multiple projects are coming to the same point it seems. Motherduck has been marketing "dives" since the beginning of the year (https://motherduck.com/blog/duck-dive-and-answer/) and in the Louie.ai team, we have been iterating on different patterns for similar needs. I'm getting the feeling that the answer to SaaS apps as fixed UIs over databases being dead because of coding agents means just the fixed dashboard pattern is dead, not SaaS, and BYO UI is part of the new table stakes. I'm curious where the pattern will go. My sense is there is a split between cathedrals vs bazaar for approach here, where cathedrals are quite rigid app builders, think framer/wix, while bazaars focus a layer below for more flexibility but less integrated.

I think it’s relevant to first read [1] to see why they’re doing this. It’s basically done as a meme. [1] https://motherduck.com/blog/big-data-is-dead/

This is not a game of, “mine is bigger than yours”. Many many workloads in the wild are smaller than this. Motherduck have a few posts about how few people have “big data”. https://motherduck.com/blog/redshift-files-hunt-for-big-data/

Indeed the "no-moat" (weight exfiltration of any openly accessible model) is something that makes me optimistic. That, and the fact that most tasks have "capacity thresholds" which on-device models will increasingly be able to saturate. One example is SQL query generation from text (example: duckdb-text2sql https://motherduck.com/blog/duckdb-text2sql-llm/).

Sure - https://motherduck.com/blog/15-companies-duckdb-in-prod/

> or are there still good reasons for using it in a new project in 2025 it's not clear there ever was. Most of the big users I'm aware of, like Stripe, don't seem to have needed it and regretted the decision. Big data didn't become a thing in the way people expected[0]. If you really did need the scalability of Mongo you'd choose a NewSql database like TiDB[1]. [0] https://motherduck.com/blog/big-data-is-dead/ [1] https://docs.pingcap.com/

I really like the columns explorer, https://motherduck.com/blog/introducing-column-explorer/. Just a few days ago I have been looking for existing column explorers that look like from Kaggle Dataset, but I was not able to find anything. And this one by DuckDB is better!

The docs say that the extension's server is configured here: https://duckdb.org/docs/stable/extensions/ui#remote-url But yeah, I can't find docs nor source for the UI. And the extension docs refer to MotherDuck's own UI: https://motherduck.com/docs/getting-started/motherduck-quick-tour/ So, a bit confusing way this is set up.

(I designed and built the Column Explorer feature) Observable's column summary feature is very nice! But I do think there's a very common lineage around these kinds of diagnostics which motivated both Observables and ours. See Jeff Heer's profiler paper[1] for more. I'm very passionate about this area because I think "first mile problems" are underserved by most tools, but they take the longest to work out. We had to do some gnarly things[2] to make this feature work well; and there's a lot of room to make it scale nicely and cover all DuckDB data types. [1] http://vis.stanford.edu/papers/profiler [2] https://motherduck.com/blog/introducing-column-explorer/

Probably not exactly what you mean or asking for, but the work Motherduck is doing looks promising. https://motherduck.com/blog/differential-storage-building-block-for-da… Hopefully it finds its way into duckdb's repo some day.

> > This query took 17 seconds to load the dataset into RAM and then aggregating by type was almost instant. It is absolutely incredible to me that I can load every HN post and comment ever into RAM in a few seconds on my (admittedly beefy) dev laptop, and analyze them at will. What an age of abundance! https://motherduck.com/blog/big-data-is-dead/

I am working on https://geobase.app/ which is a platform for geospatial full-stack developers. We have created workflows that a specific to the geospatial, mapping and GIS industry use cases. This is currently in private beta but going live in a few weeks. It is built on top of supabase's self-hosted stack. We were recently also featured on motherduck's blog https://motherduck.com/blog/pushing-geo-boundaries-with-motherduck-geo…

> small dataset (100GB) What counts as large or small definitely varies a lot depending on the context of the conversation/analysis. MotherDuck's "Big Data is Dead" post [0] sticks in mind: > The general feedback we got talking to folks in the industry was that 100 GB was the right order of magnitude for a data warehouse. This is where we focused a lot of our efforts in benchmarking. Another point of reference is [1] > [...] Umbra achieves unprecedentedly low query latencies. On small data sets, it is even faster than interpreter engines like DuckDB > TPC-H Small Dataset = 866k tuples, sf 0.1 [0] https://motherduck.com/blog/big-data-is-dead/ [1] https://db.in.tum.de/~kersten/Tidy%20Tuples%20and%20Flying%20Start%20F…

This is a quite good allegory for the way AI is currently discussed (perhaps the outcome will be different this time round). Particularly the scary slide[1] with the up-and-to-the-right graph, which is used in a near identical fashion today to show an apparently inevitable march of progress in the AI space due to scaling laws. [2]https://motherduck.com/_next/image/?url=https%3A%2F%2Fweb-assets-prod.…

My understanding is that data science implies big data. As Jordan Tigani observes, there are not that many really big data situations[1]. Thus data science proficiency might be in support of business initiatives in a particular application domain and not a means to an end in itself. [1]:https://motherduck.com/blog/big-data-is-dead/

This blog post offers a nice summary: https://motherduck.com/blog/six-reasons-duckdb-slaps/

Curious about your post, and DuckDB, since have seen many previous post here on HN about it...I just took 10 min to do some quick tests, and experiment with DuckDB for the first time. :-) While I am a Linux user, tried this on a available Windows 10 machine (I know...Yuck!) 1) Setup and install no prob. Tracked the duckdb process memory usage with PowerShell, like this: Get-Process -Name duckdb | Select-Object Name, @{Name="Memory (MB)";Expression={[math]::round($_.WorkingSet64/1MB,2)}} 2) Used this simulated netflix table dataset available from an S3 bucket, as used in this example blog. Installed the aws extension not the one mentioned in the blog: https://motherduck.com/blog/duckdb-tutorial-for-beginners/ Section in the blog: "FIRST ANALYTICS PROJECT" Table has 7100 rows as seen like this: SELECT COUNT(*) AS RowCount FROM netflix; 3) Did 8 different queries using DISTINCT ON / GROUP BY The one below, being one example: SELECT DISTINCT ON (Title) Title, "As of", Rank FROM netflix ORDER BY Title, "As of" DESC; I am not seeing any out of memory or memory leak from these quick tests. I also tested, with the parquet file from the thread mentioned below by mgt19937. It is 89475 rows. Did some complex DISTINCT ON / GROUP BY on it, without seeing neither explosive memory use or something similar to a memory leak. Do you have a more specific example?