Aphyr
aphyr.com · Personal blog · on HN since 2010
Kyle Kingsbury writes on distributed systems correctness, including Jepsen database tests, and satirical technical interview pieces.
What HN saysCommenters treat Aphyr's Jepsen analyses as the standard for database correctness, and share his "Hexing the technical interview" post as a favorite.
551HN people shared it
427linked it in comments
204stories submitted
12Kstory points
3 months agolast shared
2010people per year2026
Most-shared posts
- 1Typing the technical interview55 people · 578 points · first shared 2017 · 80 comments on HN
- 2Hexing the technical interview (2017)50 people · 1.0K points · first shared 2017 · 192 comments on HN
- 3Jepsen33 people · first shared 2013
- 4Call me maybe: MongoDB (2013)28 people · 125 points · first shared 2013 · 72 comments on HN
- 5Reversing the technical interview (2017)27 people · 214 points · first shared 2017 · 35 comments on HN
- 6Rewriting the Technical Interview (2020)22 people · 155 points · first shared 2020 · 26 comments on HN
- 7Call me maybe: Elasticsearch19 people · 392 points · first shared 2014 · 89 comments on HN
- 8Call me maybe: Postgres17 people · 7 points · first shared 2013 · discussion on HN
- 9Jepsen15 people · first shared 2013
- 10Call Me Maybe: MongoDB Stale Reads14 people · 618 points · first shared 2015 · 144 comments on HN
- 11Call Me Maybe: Redis14 people · 1 points · first shared 2013 · discussion on HN
- 12Call me maybe: etcd and Consul13 people · 177 points · first shared 2014 · 24 comments on HN
- 13Jepsen: RethinkDB 2.1.512 people · 379 points · first shared 2016 · 27 comments on HN
- 14Clojure from the ground up12 people · 248 points · first shared 2014 · 21 comments on HN
- 15Strong Consistency Models12 people · 93 points · first shared 2014 · 9 comments on HN
What HN people wrote when they linked it
chubot · 2024-03-10
> Filesystem software, especially databases, can't be ported to Amazon S3
This seems mistaken. Porting databases that run on local disk to S3 seems like a good way to get a lashing from https://aphyr.com/
Can any databases do it correctly?
If so, I doubt they work with the model of partial overwrites. They probably have to do something very custom, and either sacrifice a lot of tail latency, or their uptime is capped by the uptime of a single AWS availability zone. Doesn't seem like a great design.
(copy of lobste.rs comment)
theptip · 2023-09-30
Look at the Raft protocol. In general you’d rather not integrate at the protocol layer; it’s standard to use a consistent store like etcd that implements Raft for the coordination/data that needs to be serializable.
(Kubernetes uses etcd so it scales pretty well for a strongly-consistent k/v store.)
You said “multiple datastores” so I’m assuming you have heterogenous data and something like CockroachDB isn’t an option.
> I'm a beginner in this topic and I find this topic interesting.
Not trying to gatekeepe but rolling your own is dangerous. See https://aphyr.com/ for the gold standard in testing (great educational material). You can use Jepsen to test your distributed systems. But better to just use datastores that Kyle has shown are solid.
marknadal · 2014-11-11
CRDTs are probably the best thing out there, and the most understandable. They focus on data that does not require complicated leader election algorithms - and this is coming from Facebook and the Cassandra team.
Why? Because these systems are very weak and prone toward failure, just check out some of http://aphyr.com/ 's stuff, he tears a lot of popular databases apart. Even if the theories are "proven right" the implementations are often wrong.
ashton314 · 2026-08-06
Ooo I love anything miniKanren related.
I did a close reading of the implementation in the original miniKanren paper; you can see my write-up and implementation here: https://codeberg.org/ashton314/microKanren
miniKanren paper: http://webyrd.net/scheme-2013/papers/HemannMuKanren2013.pdf
Also relevant—a practical use-case for miniKanren: https://aphyr.com/posts/354-unifying-the-technical-interview
mplanchard · 2026-06-15
Thanks for the links, added! The bit at the end with the assembly put me in mind of another favorite post, aphyr’s hexing the tech interview: https://aphyr.com/posts/341-hexing-the-technical-interview
sillysaurusx · 2026-04-16
Question: The author makes this article available on the web as a series of posts, a PDF (https://aphyr.com/data/posts/411/the-future-of-everything-is-lies.pdf) and an EPUB. What do you suppose their tooling looks like?
Is there a single "document containing all the words," and it updates the website, pdf, and epub whenever you change it?
What struck me was that the presentation is beautiful. It seems worth emulating. But that raises the question of what format you'd write your original words in. Do you suppose they just use Markdown files, or something more elaborate?
mpalmer · 2026-04-15
LLMs do not have special or unique insight into how best to prompt them. Not in the slightest.
https://aphyr.com/posts/411-the-future-of-everything-is-lies-i-guess#u…
balamatom · 2026-04-15
I've been reading it for a couple years already, where have yall been looking? At marketing materials?
See e.g. https://aphyr.com/tags/the-future-of-everything-is-lies-i-guess
(That whole series is also available as PDF; I recommend printing it, spiral-binding the printout kinda like a pirated college textbook, and surreptitiously leaving it in cafes and the like.)
Sharlin · 2026-03-12
Very tangential, but this hexing business brought to mind the classic https://aphyr.com/posts/341-hexing-the-technical-interview
andersmurphy · 2025-12-09
Highly recommend you check out the interview series they are a lot of fun.
> They will refuse, of course, and ever so ashamed, cite a lack of culture fit. Alight upon your cloud-pine, and exit through the window. This place could never contain you.
https://aphyr.com/posts/340-reversing-the-technical-interview
ntonozzi · 2025-11-21
One of my favorite blog posts of all time: https://aphyr.com/posts/342-typing-the-technical-interview
saghm · 2025-09-13
Reminds me of https://aphyr.com/posts/340-reversing-the-technical-interview (and the follow-ups to it)
foota · 2025-08-29
"What else are lists, but alternatives?"
(credit to https://aphyr.com/posts/340-reversing-the-technical-interview, I always get a kick out of that and the follow up https://aphyr.com/posts/341-hexing-the-technical-interview).
codethief · 2025-06-24
Reminds me of this classic: https://aphyr.com/posts/342-typing-the-technical-interview
lcnPylGDnU4H9OF · 2025-01-25
It reminds me of Typing the Technical Interview (highly recommended if one is unfamiliar) but seemingly more likely to be non-fiction, given the difference in storytelling style. I wish the author luck in their search.
https://aphyr.com/posts/342-typing-the-technical-interview
isaacdl · 2024-12-19
If you enjoy this essay, you might also enjoy another classic, "Typing the technical interview"[0] by Aphyr (a.k.a. Kyle Kingsbury, the person behind Jepsen[1] distributed systems tests.)
[0] https://aphyr.com/posts/342-typing-the-technical-interview
[1] https://jepsen.io/
Sesse__ · 2024-08-18
> For MySQL, for smaller deployments, I've found Galera to really be a handy HA system to get going:
Well, at least if you don't value your data.
(https://aphyr.com/posts/327-jepsen-mariadb-galera-cluster; Galera failed Jepsen testing in 2015 and the bug is still open with a 2022 mention of basically “we have experimental support [for actually providing the data consistency we promise], but it's not clear if it's worth it because it will be very slow”)
syca_f13 · 2024-06-11
Jepsen has written a fantastic article on this issue. I'm not sure if it has been fixed since then. https://aphyr.com/posts/293-call-me-maybe-kafka
inopinatus · 2024-05-02
If you enjoyed this, you'll love https://aphyr.com/tags/interviews.
Shared404 · 2023-12-13
https://aphyr.com/posts/340-reversing-the-technical-interview
You should read this story :)
It's one of my personal favorites.
vishnumenon · 2023-09-08
Love it! Reminds me of the hilarious "Typing the technical interview" [1] and "Typescripting the technical interview" [2], a couple of my favorite blog posts of all time.
[0] https://aphyr.com/posts/342-typing-the-technical-interview
[1] https://www.richard-towers.com/2023/03/11/typescripting-the-technical-…
thatcherc · 2023-07-07
As the author notes, this is very much in the style of the aphyr's Interview series of stories like "Rewriting the Technical Interview" - [0] - which are all great reads as well.
[0] - https://aphyr.com/posts/353-rewriting-the-technical-interview
ruuda · 2023-05-07
I would love to see a Jepsen test of this when it's ready. The Redis Cluster evaluation [1] was a great read.
[1]: https://aphyr.com/posts/283-jepsen-redis
flir · 2023-01-24
You may enjoy this and it's followups: https://aphyr.com/posts/340-reversing-the-technical-interview
btilly · 2022-08-12
I don't trust Redis clustering to actually work. I only trust the single threaded, single server. Potentially with lots of replicas.
See the following for why I don't trust their various attempts to scale to a truly distributed system.
https://aphyr.com/posts/283-jepsen-redis
https://aphyr.com/posts/307-jepsen-redis-redux
https://jepsen.io/analyses/redis-raft-1b3fbf6
lkrubner · 2022-03-31
> Why do you recommend against using a modern distributed database?
Call me maybe. This is a newer technology and the growing pains have been dramatic. I assume you've followed Jespen?
https://aphyr.com/tags/jepsen
I use MongoDB all the time, for a lot of projects. But I'm also keenly aware of how many years they have struggled to pass the Jespen tests. I can also recall several times engineers from MongoDB have said "We have finally resolved all of the issues raised by Jespen" and then another bug was discovered.
Again, I use MongoDB all the time. But I'm aware it is simpler and safer to use Postgres, for many, many use cases.
exdsq · 2022-02-27
Absolutely love Aphyr's interview stories. Here is my all time favourite https://aphyr.com/posts/342-typing-the-technical-interview
zaptheimpaler · 2022-01-03
Good writing is utilitarian. Boring but it serves the purpose of communication.
Great writing is much more. It communicates and entertains. Here are 2 of my favorites relating to computerz:
https://aphyr.com/posts/341-hexing-the-technical-interview
http://scholar.harvard.edu/files/mickens/files/thenightwatch.pdf
Although I do agree the advice does make sense for most of us, who just aren't very good at writing. But if you do have the skill to write well, do it! I love great writing and love people who aspire to it, even if they fail! Everything you do does not have to be packaged up to maximize mass appeal and resume points.
seanp2k2 · 2021-12-18
This. The article seems like "one weird trick that message queue companies HATE" as it's utilizing, as far as I understand, some SQL semantics in a very specific way to cobble together a way of achieving what other software is designed to do out of the box. It seems fine for a toy system, but I wouldn't stake the success of a real company on this approach.
One could also use DNS TXT as an RDBMS with some interesting fault tolerance and distribution schemes. That doesn't mean it's a good idea or the best way to solve a problem.
If you haven't seen them already, the Jepsen analyses are really worth a read:
https://aphyr.com/posts/293-jepsen-kafka
https://aphyr.com/posts/282-jepsen-postgres
https://aphyr.com/tags/jepsen
turminal · 2021-09-09
The best non-conventional introduction:
https://aphyr.com/posts/342-typing-the-technical-interview