Ayende Rahien

Another thing worth mentioning is it's very similar to the structure of columnar formats like Arrow and Parquet. Anyone with familiarity with these formats could build a decoder in a couple of days. If they don't use FastPFOR. I really wish they hadn't used FastPFOR. It's a research library and has an incredibly opaque algorithm: https://ayende.com/blog/199523-C/integer-compression-understanding-fas…

It's also worth while reading the rebuttal from ravendb: https://ayende.com/blog/196161-C/re-are-you-sure-you-want-to-use-mmap-… Also worth mentioning LMDB, probably the fastest embedded K/V db in terms of read perf uses mmap.

I see this as a good follow-up after this thread on the use of mmap to implement a database[1]. Indeed, there is a first implementation using mmap [2] to be compared with a full-fledged implementation [3]. [1] https://news.ycombinator.com/item?id=29936104 [2] https://ayende.com/blog/195588-B/implementing-a-file-pager-in-zig-usin… [3] https://ayende.com/blog/195713-B/implementing-a-file-pager-in-zig-over…

He posted a follow up about error handling in Zig that I thought was interesting: https://ayende.com/blog/194466-A/looking-into-odin-and-zig-my-notes?ke… I’ve seen the “clean up errors yourself” argument before, but, like the author, I don’t think it holds water. Often the correct response to errors is to panic() or pass it up the stack so the caller can deal with it or—more likely—panic() itself.

He was probably mid thirties. I watchrd him grow as a rock solid engineer. He consistently blogs, and owns a database startup among other things. http://ayende.com is him. Have infinite respect for him.

Not a book, but Ayende's blog series on Database Building 101 is a great starting point covering a lot of the issues and opinion from an active storage engine designer. https://ayende.com/blog/posts/series/175041/database-building-101 In other series he also covers Voron, his alternative to Microsoft Esent engine. https://ayende.com/blog/posts/series/175074/voron-internals

This is a very long and very good blog series that I highly recommend. (One note, it's from 2013.) The funniest part is this post, which complains about code duplication: https://ayende.com/blog/162853/reviewing-lightning-memory-mapped-datab… In the last two screenshots, there is a subtle difference with the indentation, which breaks MDB_PREV_DUP. It wasn't noticed until over a year later: https://github.com/LMDB/lmdb/commit/bfe297da47beb31debee48f016222b63d9… I'm still a big fan of LMDB. It's been hugely inspirational for me, and I think it's the best low-level key-value store out there. There is still lots of room for improvement though!

It would be interesting to get Oren Eini's opinion on this, as the author of RavenDb. (http://ayende.com/)

I was reading this 10 minutes ago :) The best bug reports are pull requests: http://ayende.com/Blog/archive/2011/01/27/the-best-bug-reports-are-pul…

> .NET developers love using the standard libs I may not be a typical .Net developer, but my personal feeling is that this is too general and somewhat glib. If you're inexperienced you will (and should!) choose the default option, if there's one available, and the most commonly used option if there isn't a default. So, if you wanted an ORM, then before EF there was NHibernate. But after a while you become able (from painfully gained experience) to determine what you want from a tool and what trade-offs you want to make, so you might use something like Dapper or no ORM at all. Personally, I have found some of the MS provided implementations - shall we say - less than optimal. The Unity Framework. EF (which I dearly wish I have never had to suffer using). MSTest. Enterprise Library (oh god ugh). So I make other - informed - choices about what to use. Some things I keep using because they are 'good enough' and I have a library of utilities and a mental map of how they work - log4net, NUnit - and some things I find I just don't need any more (mocking libraries, for one). MS still support their provided implementations - as they should - but even their own projects can and do use third-party frameworks rather than the MS provided implementation (for example, Bot Framework used AutoFac (back when I was using it, anyway) [1]) - because their developers have been released from the requirement to exclusively use MS provided frameworks, and are making their own choices about what…

I think AGPL is a fairly good choice for anything that's potentially used as a hosted service. All the big enterprises I've worked with so far have a policy that forbids AGPL'd software and rather purchase commercial licenses. SMEs & Startups may make different choices but depending on your product they may not represent the relevant majority of the market anyway. Ayende from RavenDB (which is a great no-nonsense document DB) has written an interesting blog series [0] on how they approach OSS while still making money from it and keeping their interests aligned with users and customers. TL;DR they license the full server and all modules as AGPL, offer free community (commercial) license that don't skimp on core features (clustering, authn/z), treat support as a cost-center. Appears to work well for them and I personally find their mode of operation sympathetic. [0] https://ayende.com/blog/posts/series/186113-A/making-money-from-open-s…

Yes (it helped) and speed, i.e. querying and indexing performance, for sure is only an USP if you also have relevance. I'm confident the relevance is as good as or better than Lucene. I especially like my phrase queries and how they seem more relevant compared to that of a Lucene phrase query. The scoring is a half-way implementation of word2vec (in a lot of ways similar to the scoring mechanics of Lucene's tf-idf scheme). I'm aiming for full word2vec implementation in vNext. I have only my own benchmark tests to tell me I'm faster than Lucene. Which is why I'm contemplating writing a formal proof both of ResinDB's performance and of it's relevance. My test data has been the English verison of Wikipedia plus Project Gutenberg. I suppose I could publish those indices to the world, as a demo search engine. I don't think a soul would care about a proper searchable Project Gutenberg though. Looking into common crawl now. I'm a part-time father of two, employed doing tedious unmotivating work, focusing completely on my spare time project. I need some advise as to what the next step should be, if I wanted to make this into a business that I could spend all of my time with, not only nights and weekends. Formal proof? Demo? Side note: one of the most approachable people in the database building community is Oren Eini, creator of RavenDB. He's reviewing ResinDB on his blog. I've read a preview of the entire series of posts, implemented solutions for the best parts of the…

I'm not familiar with PG but I am planning on using it in my current project, which is why I'm interested in understanding whether these are really DBMS issues. In the .NET world, for all the database adapters I've used (MSSQL, SQLCE, Oracle, MDB), individual statements are implicitly wrapped in transactions when no explicit transaction has been specified - not sessions/connections. I agree the "implicit transaction per session/connection" design is confusing and likely to result in bugs. While the standard .NET approach limits bugs, the downside is poorer performance when an unaware developer uses numerous implicit transactions for multiple statements where a single explicit transaction would suffice. Experts in the area appear to recommend always using explicitly managing transactions (e.g. http://ayende.com/blog/3775/nh-prof-alerts-use-of-implicit-transaction… ), and this is the approach I follow in my own work now.

There's a series of blog posts by Oren Eini that shows that C# can be sufficiently fast even when you don't really optimize anything. Beating Redis in every benchmark is a whole another level, of course. https://ayende.com/blog/197412-B/high-performance-net-building-a-redis…

You will like this https://ayende.com/blog/posts/series/195587-B/implementing-a-file-page… and https://www.reddit.com/r/databasedevelopment/ where it talks about this kind of stuff.

EDIT: Paste this hacky snippet into your browser's console to save all of the links into your clipboard: (() => { let result = [...document.querySelectorAll('td a')] .map((link) => link && !link.href.includes('ycombinator') && !link.href.includes('javascript:void') ? link.href : '' ) .filter(Boolean); result = [...new Set(result)]; result.sort(); copy(result.join('\n\n')); })(); Just to help, here's a compiled-list of all of the links shared here so…

As an aside, the code is also wrong and will fail in some rare cases because it assumes that the `read` syscall will only return partial information if you are at the end of the file. This is false, however, and will lead to data loss if happen. Full post with details here: https://ayende.com/blog/196289-C/dont-assume-the-result-of-read?key=65…

RavenDB's response to this paper: https://ayende.com/blog/196161-C/re-are-you-sure-you-want-to-use-mmap-…

I love reading about articles on Databases, especially about their internal workings. Some of the blogs I follow: 1. Oren Eini - Creator and CTO of Raven DB - https://ayende.com/blog 2. Tyler Neely - Creator of Sled DB - https://medium.com/@tylerneely 3. Philip O'Toole - Creator of rqlite - https://www.philipotoole.com/ 4. Martin Kleppmann - Author of Designing Data-Intensive Applications - https://martin.kleppmann.com/archive.html 5. Glauber Costa - worked on glommio, scylla DB - https://glaubercosta-11125.medium.com/ Do recommend me if you know more!

To give you a glimpse of reality: https://ayende.com/blog/Images/Windows-Live-Writer/Pair-Programming-at… EDIT: Wow i clicked on #Coding and it is like i taught about it...pure waste of time with Rainbowhairs and talking needle slim guy's with more grease on top than the hole movie 'Grease'

I think that it is universal advice for libraries: do not do multithreading inside, let the client code do it and/or pass necessary constructs. Simple example for Java: do not create executorservice instances inside of a library, but allow client code to pass them. Ayende wrote much better than me long time ago https://ayende.com/blog/159201/multi-threaded-design-guidelines-for-li…

Yes, C# does have goto. Ayende made a blog post [1] about some interesting uses of goto for performance reasons. Also likely for performance reasons, async/await state machines in .NET essentially use goto (br.s in CIL). VB.NET has GoTo as well. 1: https://ayende.com/blog/183553-A/using-goto-in-c

The codebase in question may be well beyond the line where any sane person would touch it, seriously. Sure. If you ever find this [1] a rewrite does look like the only sane decision. [1] https://ayende.com/blog/4612/it-really-happened-legacy-programmers-tal…

Coming from nhibernate now working with sqlalchemy, more so than async the one feature i miss when working with sqlalchemy is futures with multiple result queries, like this https://ayende.com/blog/3979/nhibernate-futures

Ayende Rahien made an illustrative code review series of LMDB in 2013, particularly good due to its author Howard Chu's responses: https://ayende.com/blog/posts/series/162754/reviewing-lightning-memory…

Also, he has published a lot of review of database code. In particular, he has 18 blog entries on the leveldb C++ code base: https://ayende.com/blog/161410/reviewing-leveldb-part-i-what-is-this-a…

This is a reply to Sarah Mei's post from Ayende Rahien http://ayende.com/blog/164483/re-why-you-should-never-use-mongodb

Part II： http://ayende.com/blog/163105/stories-from-the-interview-room-part-ii About the interview room series： http://ayende.com/blog/163234/about-the-interview-room-series

I'd have a scan of Oren Eini's multi-part review of this app before getting too excited about that "enterprisey" app. Here's parts 1 and 2: http://ayende.com/blog/19457/review-microsoft-n-layer-app-sample-part-… http://ayende.com/blog/27649/review-microsoft-n-layer-app-sample-part-… The remainder of the reviews are in July 2011's blog posts. I think the conclusion is that's over-engineered junk.

http://ayende.com/blog/153026/embracing-ravendb The points in the blog post apply to most non-relational databases.