Valentin Waeselynck

Probably this post: https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-th… > If you're new to Datomic, you probably have the same misconceptions as I did regarding the use of Datomic's historical features. > bad news: you've probably over-estimated the usefulness of these features for implementing your own specific time travel. Unless you really know what you're doing, I recommend you don't use db.asOf(), db.history(), and :db/txInstant in your business logic code. > good news: you've probably under-estimated the usefulness of these features for managing your entire system as a programmer. >I believe the key to getting past this confusion is the distinction between event time (when things happened) and recording time (when your system learns they happened).

Just the temporal properties alone make it very useful for anything where it matters like billing, finance, inventory. Else you are in views/schema/indexing hell to do it on top of SQL. There is some SQL temporal support but it's not great and varies a lot. Also since it's not native to the storage it has a lot of complexity issues under the rug making it not great. Many financial systems use Event Sourcing (OOP + ORM). I had to suffer this at a previous employer. See https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing…

Datomic is an event-sourced db, and it makes it hard to introduce retroactive corrections to the data when your program's semantic already rely on using datomic's time travelling abilities: at one point you'll need to to distinguish between event time and recording time as explained in this excellent blog post: https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing… This is why I' rather use XTDB [1], a database similar to datomic in spirit, but with bitemporality baked in. [1] https://www.xtdb.com

This is probably the best blog post I've seen on the topic: https://vvvvalvalval.github.io/posts/what-makes-a-good-repl.html > If you're not familiar with Clojure, you may be surprised that I describe the REPL as Clojure's most differentiating feature: after all, most industrial programming languages come with REPLs or 'shells' these days (including Python, Ruby, Javascript, PHP, Scala, Haskell, ...). However, I've never managed to reproduced the productive REPL workflow I had in Clojure with those languages; the truth is that not all REPLs are created equal. Basically, the "REPL" you usually call a REPL is in fact a shell, and not a REPL as normally used in lisp languages.

Simple core languages that are syntactically extensible with libraries have the best of both worlds: https://vvvvalvalval.github.io/posts/2018-01-06-so-yeah-about-clojures…

This is often called "event sourcing". The other commenter mentions Datomic, which probably is the 'latest and greatest' form of event sourcing (I've never used it). https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing… If you want something built on postgres, I don't have any specific recommendations, but you can build a simple event sourcing system yourself. I worked somewhere that used event sourcing on postgres, and the core log was basically just a table with aggregate IDs, event sequence numbers, and a JSONB column for the event payload. I recommend starting with just one part of your application if you're going to adopt event sourcing. You will quickly find that there are a lot of new considerations and pitfalls that you don't have with traditional RDBMS usage. Overall, event sourcing is hard to get right, so you should consider the trade-offs carefully. There are easier ways to get audit logs, for example: https://www.pgaudit.org/ I will also recommend this as a way to start to understand the tricky aspects of event sourcing: https://leanpub.com/esversioning

For me, the real game changer is the REPL-driven workflow you get with both Clojure and ClojureScript, but it's really hard to describe the details of how it's different than hot-reloading in the React ecosystem. I've tried many times, but my explanations seems to mostly confuse people rather than enlighten them. So let's use the words of others instead, here is the best explanation I know about why the Clojure REPL rocks: https://vvvvalvalval.github.io/posts/what-makes-a-good-repl.html Short version is: you can send snippets of code to your REPL from your editor, and because of the common structure of Clojure apps, you can usually evaluate content in the middle of a function without having to change any code. Basically unlimited introspection into a running program. I used to program in PHP, C#, Golang, Ruby, NodeJS/JavaScript (both backend/frontend) and some others, and no language comes close to the dev experience of Clojure. I'm sure other lisps are similar as well, but haven't used anything else than Clojure substantially.

The claim that there's declining interest is absurd, and that's what I'm arguing. Google trends are simply not a measure of anything interesting in practice. It's about as useful as throwing around TIOBE rankings. >Tooling is a big problem, not because the once you learn emacs and cider it is insufficient, but because it is a pain to setup and learn emacs and cider. Not because Atom and proto-repl are bad tools, but because to get it set up you need 10 plugins and 20 config setting and it ends up being fragile - complected in a major way. It's quite clear you haven't used Clojure tooling if that's your impression of it. Again, Clojure has some of the best tooling I've used in any language and I've been doing development for around 2 decades now. Intellij and Cursive are a fantastic combination that takes minutes to install. Calva is another great option for VS Code and it's also a one click install. Both these projects are actually being funded. Cursive is a commercial product and Calva is funded via Clojurists Together. Both Cursive and Calva show just how much the community has grown, since people and companies are willing to pay for tooling nowadays. That's a real measure of growth. >Another part of the problem is that Clojure advocates vastly overvalue the what unique benefits Clojure adds things like hot reloading and the REPL. Clojure and ClojureScript might have better or more pure implementations, but everyone has some form of these features now. Again, this is…

> The Events here give up on the naming, granularity, and semantics problem: they're extremely low level fine-grained changes to fields in a database. You say "fields", I say "facts". Which of a set of nominal events or a graph of facts is higher-level will ultimately depend on the domain, but using Datomic over several years now has led me to think that it's much more often the latter than conventional Event Sourcing has trained us to believe. In fact, it may well be that Event Sourcing has traditionally been relegated to those (few) use cases where the "set of nominal Event Types" approach is better, being the only situations where conventional Event Sourcing is practical. > Events themselves are no longer interesting or semantically meaningful, because they're a single atomic change to a database field. A change in the way state is represented means a different set of events is produced. Subscribing to meaningful occurrences in this model is difficult, Datoms are not 'single' or 'atomic' - they're packed coherently together in a Transaction, and are immediately relatable to entire database values. Subscribing is just pattern matching, and is not hard, as the following section of the article tried to show: https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing… > [...] and probably will eventually result in the creation of a "meta-event" for each action that contains the semantic intent of the…

Link to top of page, https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing…

Event time vs. recording time - I think this was the link you were looking to provide: https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-th…

A very interesting post on this topic: https://vvvvalvalval.github.io/posts/2018-07-23-datascript-as-a-lingua…

Could be an issue. For that reason, the two immutable databases with which I'm familiar (Crux and Datomic[1]) provide operations for removing bits of data from history. [1]on-prem, not cloud However even if the DB doesn't offer that operation, you could use "crypto shredding" where you encrypt the data before putting it in the immutable db and then store the key in some kind of mutable store. (Then delete the key when you want to delete its corresponding data). There's also this post which describes some other methods of making a Datomic cloud system gdpr compliant: https://vvvvalvalval.github.io/posts/2018-05-01-making-a-datomic-syste…

You just discovered repl driven development that lisp users have been advocating for years. Check it out in Clojure: https://vvvvalvalval.github.io/posts/what-makes-a-good-repl.html

Datomic: Event Sourcing without the hassle https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing…

Some contrarian opinion: https://vvvvalvalval.github.io/posts/2018-11-12-datomic-event-sourcing… Basic idea is to make the event log dumb. Instead of capturing business logic, make them as dumb as SQL commit logs: this row was inserted at this time, that row was deleted at that time. Sort of like a glorified WAL file. Benefits of this scheme? Querying old database states and auditing.

There is so much to dig into for one short article. So much. Too much for an HN comment, I think. A few bullet points: - It's not an event sourced system, it's a relational system with an absolutely terrible audit log. - The audit log is (was?) serialised and blocking for every transaction, okay for low volume and low latency transactions. - 30 minute transactions!? That is NOT low latency. That needs engineering to manage contention. - The last updated event information is updated in a transaction and definitionally is always the most recent event… what's it actually for? - The solution is exactly equivalent to a boolean "is_applied" column on the event table… and is still always set "true" inside a transaction. > That sounds like a recipe for disaster when your replay code and update code are not actually in sync? It reads like their replay code is their update code, but they have absolutely no confidence that their event history is complete and consistent ("A lot of things can go wrong with a large history, foreign keys, exceptions, serialization mismatches, events out of order etc.") so it's unlikely they'll ever risk replaying events. The bit that I think has the most interest to consider, though, is the granularity of events. The article refers to database transactions as a monoid, which is to be expected since the application's architecture is built around database transactions and not events, but events can also be treated as a monoid. Making events extremely…

To me, being able to make changes to your code while keeping the current state in the REPL is key to interactive development. My workflow is generally to build up state, and then experiment with functions on that state until I get the correct output. This workflow is very natural in Clojure, Common Lisp and even Python (with IPython and autoreload). However, in Racket you have to restart everything on every change. This works ok for smaller applications, but if for example, your state is a large dataset that you pull from a remote database, it becomes a little more difficult. There are possibly workarounds, and I'm not saying that Racket is bad because of this. There are definite advantages to this approach, mainly for keeping everything simple and predictable. However, this was a roadblock for me, and the main reason why I didn't spend more time working on it. I also miss the ability to make changes to code, refresh the browser and instantly see the changes (instead of having to restart the server after each change). Here's a good discussion (in my opinion) of what makes a good REPL for interactive development: http://vvvvalvalval.github.io/posts/what-makes-a-good-repl.html