SQLSync

Great work here! A bit of a silly question - but can I ask you what tool you used to build the beautiful diagrams on the page (eg: https://sqlsync.dev/_astro/pull_changes.DjOYfgBf_2biXxv.webp)

Have you taken a peek at sqlsync? It's a cool take where you write your entire application logic to operate as transactions and queries over a sqlite database but you do it in wasm so that the same transaction logic can run everywhere. The end result is nifty. That's at least my recollection of the thing. https://sqlsync.dev/

I believe that streaming changes from SQLite is what https://sqlsync.dev/ is

If you're looking for a very real use case of SQLite in the browser, there's SQLSync[1]. It was recently discussed on HN[2]. [1]: https://sqlsync.dev/posts/stop-building-databases/, https://sqlsync.dev/ [2]: https://news.ycombinator.com/item?id=38489307

This is a really interesting project, and a great read. I learned a lot. I'm falling down the rabbit hole pretty hard reading about the "Leap" algorithm (https://www.usenix.org/system/files/atc20-maruf.pdf) it uses to predict remote memory prefetches. It's easy to focus on libgraft's SQLite integration (comparing to turso, etc), but I appreciate that the author approached this as a more general and lower-level distributed storage problem. If it proves robust in practice, I could see this being used for a lot more than just sqlite. At the same time, I think "low level general solutions" are often unhinged when they're not guided by concrete experience. The author's experience with sqlsync, and applying graft to sqlite on day one, feels like it gives them standing to take a stab at a general solution. I like the approach they came up with, particularly shifting responsibility for reconciliation to the application/client layer. Because reconciliation lives heavily in tradeoff space, it feels right to require the application to think closely about how they want to do it. A lot of the questions here are requesting comparison's to existing SQLite replication systems, the article actually has a great section on this topic at the bottom: https://sqlsync.dev/posts/stop-syncing-everything/#comparison-with-oth…

This is a write up that may be helpful for you: https://sqlsync.dev/posts/stop-syncing-everything/

Seems like you have lots of real world mobx experience. Have you ever read this article, basically saying that every front end state (including mobx) basically ends up being a worse version of a standard database? https://sqlsync.dev/posts/stop-building-databases/ I ended up finding that article after running into lots of the challenges with mobx State tree. I ended up trying to use watermelondb, a sqlite wrapped for react native, but gave up entirely on offline due to bugs and project abandonment

I think a large benefit to "DB in browser" is about the whole "local-first" software movement. A good overview is written here https://sqlsync.dev/posts/stop-building-databases/ The essence of the approach is that a large majority of FE apps have constructed quite complex caching layers to improve performance over "querying backend data" - take a look at things like Next.js or React Query <- as the post above mentions, they're essentially rebuilding databases. So instead this approach just moves the db to the browser, along with a powerful syncing layer. I think it's an approach that deserves more attention, especially to improve DX where we end up writing a whole lot of database-related logic on clients. Mind as well then just use a database on the client as well

As for your DB remarks, I've recently read an interesting blog post [1] about using SQLite as a cache in the context of web frontend. Apparently it works Well Enough™. [1] https://sqlsync.dev/posts/stop-building-databases/

You're right - it's a bit confusing! I took a crack at explaining it in the blog post under the Consistency section: https://sqlsync.dev/posts/stop-syncing-everything/#consistency-sync-sa… The key idea is that if your system supports offline writes, then by definition the client making those writes can't have general purpose strict serializability. They have to exist under the assumption that when their transactions eventually sync, they are no longer valid. Graft attempts to provide a strong foundation (server side commits are strictly serialized), however let's the client choose how to handle local writes. A client may choose any of these options: 1. If offline, reject all local writes - wait until we are online to commit 2. Rebase local writes on the latest snapshot when you come back online, resulting in the client experiencing "optimistic snapshot isolation" 3. Merge local changes with remote changes - this probably depends heavily on the datastructure you are storing in Graft. For example, storing a Conflict-Free Replicated Datatype (CRDT) would work nicely 4. Fork the Volume entirely and let the user figure out how to manually merge the branches back together 5. Throw away all local changes (probably not what you want, but it works!) My goal is to build a building block on top of which edge native systems can be built. But I'm not too opinionated about what local write semantic you're application needs. :) (edit: added newlines between list items)