M
Michael Nielsen
michaelnielsen.org · Personal blog · on HN since 2007
Michael Nielsen, a physicist and writer, explains topics like Bitcoin, Bloom filters, Lisp, and quantum computing in long essays.
What HN saysHN commenters cite his 'discovery fiction' explanations on Bitcoin and Bloom filters, and recommend his Lisp essay and search engine post.
208HN people shared it
104linked it in comments
154stories submitted
4.1Kstory points
4 months agolast shared
2007people per year2026
Most-shared posts
- 1If correlation doesn’t imply causation, then what does? (2012)26 people · 242 points · first shared 2012 · 69 comments on HN
- 2How the Bitcoin protocol actually works24 people · 615 points · first shared 2013 · 55 comments on HN
- 3Lisp as the Maxwell Equations of Software (2012)23 people · 759 points · first shared 2012 · 122 comments on HN
- 4How to crawl a quarter billion webpages in 40 hours13 people · 761 points · first shared 2012 · 67 comments on HN
- 5Why Bloom filters work the way they do (2012)12 people · 246 points · first shared 2012 · 7 comments on HN
- 6Quantum computing for the determined9 people · 93 points · first shared 2011 · 2 comments on HN
- 7Principles of Effective Research (2004)8 people · 41 points · first shared 2009 · 1 comments on HN
- 8Zhang's Prime Result Being Improved Everyday8 people · 19 points · first shared 2013 · 4 comments on HN
- 9Quantum computing for everyone7 people · 61 points · first shared 2009 · 21 comments on HN
- 10Three myths about scientific peer review7 people · first shared 2011
- 11The Google technology stack (2008)6 people · 101 points · first shared 2009 · 14 comments on HN
- 12Kasparov vs The World (2007)5 people · 8 points · first shared 2010 · discussion on HN
- 13Collected analysis of Vinay Deolalikar's "P=NP" preprint4 people · 1 points · first shared 2010 · discussion on HN
- 14Extreme thinking, by Michael A. Nielsen3 people · 78 points · first shared 2009 · 10 comments on HN
- 15Pregel - distributed graph processing | DDI3 people · 66 points · first shared 2011 · 4 comments on HN
What HN people wrote when they linked it
wonnage · 2022-02-13
I have to admit I'm not a fan of "discovery fiction" as presented in this post, but reading Michael Nielson's linked post and some of his examples (e.g https://michaelnielsen.org/ddi/why-bloom-filters-work-the-way-they-do/… I think the idea is great if executed well.
The goal isn't to embed information in meaningless dialogue, but to walk the learner through an entire thought process rather than just presenting the finished result. Or to use one of tech's favorite expressions, to construct an idea from first principles.
burkaman · 2022-02-13
I also found the style in this article kind of annoying and difficult to follow, but I think "discovery fiction" usually means something different. The two examples Michael Nielsen gives are https://michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-works… and https://michaelnielsen.org/ddi/why-bloom-filters-work-the-way-they-do/… Another classic example is http://blog.sigfpe.com/2006/08/you-could-have-invented-monads-and.html…
You can see there's no dialogue; the idea is to frame an article as "how might you discover/invent this concept on your own", rather than just directly explaining how a thing works. You are given a fictional goal, and you "discover" the subject matter by building iterative solutions.
DavidCole1 · 2021-12-02
Interesting. I had some interests in building a search engine myself (for playing around ofcourse). I had read a blog post by Michael Nielson [1] which had sparked my interest. Do you have any written material about your architecture and stuff like that? Would love to read up.
[1]: https://michaelnielsen.org/ddi/how-to-crawl-a-quarter-billion-webpages…
diego898 · 2021-04-25
Michael Nielsen's "Lisp as the Maxwell’s equations of software" [1] was my first exposure to this in python, and I highly recommend it!
[1]: https://michaelnielsen.org/ddi/lisp-as-the-maxwells-equations-of-softw…
oefrha · 2020-01-13
This is just a weaker version of the twin prime conjecture. You’re asking for primes that differ by 2^k instead of 2. Last I heard the best result was from Polymath 8a.
http://michaelnielsen.org/polymath1/index.php?title=Bounded_gaps_betwe…
Edit: Actually I was wrong, differing by a binary digit is much stronger than a difference of 2^k. But that’s a starting point.
ThePhysicist · 2019-10-24
Michael Nielsen has a nice series of blog articles on the subject:
http://michaelnielsen.org/
(look for "The Quantum World")
avmich · 2019-08-02
And of course, evolution of the code with great explanations - http://www.michaelnielsen.org/ddi/lisp-as-the-maxwells-equations-of-so… .
Particularly good is (canonical?) lisp-in-lisp code at the end.
drallison · 2019-02-23
Reading Judea Pearl is recommended. You might begin with The Book of Why and then do a deep dive. Michael Neilson has a good blog post on the issue: http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causatio… with interesting comments by Judea Pearl at http://www.michaelnielsen.org/ddi/guest-post-judea-pearl-on-correlatio…
at
joaobatalha · 2017-12-09
It truly is a wonderful paper. Lots of information in it.
You don't need that much background knowledge to understand it, but it can take a few reads to wrap your mind around it.
I would suggest also taking a look at the annotated version of the whitepaper on Fermat's Library:
- https://fermatslibrary.com/s/bitcoin
I wrote some of the annotations and tried as much as possible to make it so that this annotated version would provide a motivated reader with all the resources needed to truly understand the bitcoin protocol.
Michael Nielsen's blogpost about Bitcoin (http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo… is also a great read.
return0 · 2016-04-16
I couldn't recommend enough Michael Nielsen's on quantum computation. He has also written many articles:
http://michaelnielsen.org/blog/quantum-computing-for-the-determined/
asafira · 2016-02-11
Michael Nielsen has a nice intro the protocol, for those looking for another source:
http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo…
(he has other articles I would recommend taking a look at, too --- check them out if you're interested! )
ivan_ah · 2015-09-23
Not sure of what model you have in mind, but simultaneous doesn't seem appropriate: there is definitely a notion of sequence of operations.
For example, applying the X operator followed by the Y operator on a quantum state |ѱ> can be written as:
YX|ѱ>
(You don't need to know what X and Y are for the purpose of this example, but think matrix multiplication, X "touching" the state before Y.)
RE books, this is a good one: http://www.michaelnielsen.org/qcqi/QINFO-book-nielsen-and-chuang-toc-a…
jgrodziski · 2014-05-31
A good complement to Norvig's writings (actually inspired by it) is "Lisp as the Maxwell's equations of software" by Michael Nielsen:
http://www.michaelnielsen.org/ddi/lisp-as-the-maxwells-equations-of-so…
Worth reading!
dvanduzer · 2014-03-08
This is Bruce Schneier's favorite explanation of the blockchain: http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo…
Comparing it to a "traditional DHT" is like kind of like comparing SQL and NoSQL. This is a good start on why that doesn't work: http://codahale.com/you-cant-sacrifice-partition-tolerance/
aestra · 2014-02-11
If you would like to read a more detailed, just as simple, but much longer explanation of bitcoin I recommend this:
http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo…
i2pi · 2013-12-16
This is the best explanation I've read: http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo…
dsego · 2013-09-18
For me this was far better explained in the following text (also posted on HN not so long ago): http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-causatio…
da-bacon · 2013-07-12
Also see the follow up with responses from Judea Pearl: http://www.michaelnielsen.org/ddi/guest-post-judea-pearl-on-correlatio…
michael_nielsen · 2013-05-16
Glad you enjoyed the videos (I created them)! Unfortunately, that link has the videos in reverse order. Here they are in the intended order:
http://michaelnielsen.org/blog/quantum-computing-for-the-determined/
The main thing that's needed to follow along is familiarity and comfort with basic linear algebra.
JumpCrisscross · 2012-07-06
Michael Nielsen explains why explaining quantum comouting is inherently difficult (TL;DR our intuition works classically and thus a complete classical model of a quantum computer would imply that a quantum computer could be simulated classically) http://michaelnielsen.org/blog/quantum-computing-for-everyone/
nl · 2011-07-15
Yes, the wisdom of the crowd is often about connecting with the right individual. One can argue that overall wisdom is always a function of speed of thought, and network effects often speed up contacting the right individual.
But.. there are counterpoints too: For example, this is what Kasparov said about his 1999 chess match against "The World":
It is the greatest game in the history of chess. The sheer number of ideas, the complexity, and the contribution it has made to chess make it the most important game ever played.
Many critics agree with him.
The important thing here is that no participant on "The World" team was remotely as highly rated as Kasparov on their own.
http://michaelnielsen.org/blog/kasparov-versus-the-world/
dmboyd · 2011-05-12
TIL about http://michaelnielsen.org/polymath1/index.php?title=Main_Page
I cant help but think that a wiki isnt the best platform for this. It needs something more analogous to pen-and-paper proofs. ie. discrete steps with the possibility to fork any step and combine identities...
ktta · 2017-06-29
To really understand cryptocurrencies, you must understand Bitcoin. Not just its inner workings but where it came from and how it became what it is today.
The rest of the cryptocurrencies are only hoping to gain Bitcoin's popularity (which Ethereum is getting close to) that it maintained for nearly a decade.
Back in the day the following links helped me understand bitcoin:
http://www.michaelnielsen.org/ddi/how-the-bitcoin-protocol-actually-wo…
http://www.imponderablethings.com/2013/07/how-bitcoin-works-under-hood…
Once you properly understand bitcoin you are ready to understand other cryptocurrencies pretty easily. If some design choices don't make sense (secp256k1), then remember that Bitcoin's creation was close to when someone was caught doing something[1]. This is the answer to why many cryptocurrencies choose non-standard algorithms (aka. not NIST)
[1]:https://en.wikipedia.org/wiki/Dual_EC_DRBG
frahs · 2017-06-15
As an arrogant 20-something year old, I disagree :P.
An important distinction to make here is whether you're teaching someone to intuitively reason about something or to logically calculate it precisely.
I think as long as someone grasps the basics of high school math and has a decent working memory, they should be able to learn how to calculate anything precisely given that you break it down into simple steps. If you don't know how to do that, you have no clue what you're doing.
Then there's intuitive reasoning. I think for most things, if you're familiar with a topic, you should be able to teach it in a way that makes sense. Of course there's exceptions -- some people are incredibly brilliant, but lack social awareness. I think that's the exception rather than the average case.
But not all topics are so easily reduced. There's probably some exceptions. The strongest that comes to mind is quantum computing. As many times as you can explain things like superdense coding, there's still a sense of "magic". Where the results of the math seem unnatural. And you go back through each step and try to figure out where things went weird. But each step is a logical progression from the beginning.
On that note, Michael Nielson's "Quantum Computing for the Determined" is one of the most well-taught courses I've found, and does a very good job of stepping the viewer through a very complicated topic. Michael Nielson clearly knows what he's doing…
markeightfold · 2014-01-02
This is fascinating to me. A couple comments and links.
Here is a course on proof theory that uses Coq [1]. It is actually the course that Vladimir Voevodsky took when he was trying to understand proof and type theory [2], [3]. While writing the midterm paper, he discovered homotopy type theory [4]. More details on the bitcoin scripting language can be found in the excellent blog post by Michael Nielsen[5].
If the escrow model is implemented, you can actually imagine the mathematical proof market maker acting like a bank, loaning out or otherwise investing bitcoins while the world waits for a verified proof; say, of the abc conjecture [6a] or the goldbach conjecture [6b]. This might be one small way to disrupt Wall St [7a], [7b]. In particular, such a market, when more highly developed, provides skilled mathematicians and scientists incentives to work on math and ( computer ) science problems in an open source format that benefits everyone - since everyone can look at and learn from their proofs - rather than quantitative trading in a closed format that actually causes long term social harm and political instability [IMHO]. I am thinking here of Goldman Sachs and Renaissance Technology founded by James Simons. Imagine if an incentive structure existed that encouraged all these smart folks to work on what is essentially verifiable open source software [8].
[1] http://www.cs.princeton.edu/courses/archive/fall09/cos441/index.php
[2]…
k_s · 2012-09-11
Unless I misread, I don't believe I saw anything that implied that the poster was claiming that anyone with a modicum of smarts could contribute significantly to the study of elliptic curves as relates to Diophantine equations. That being said, there are a mind-boggling number of extremely specific sub-specialties in mathematics, each of which would demand a significant investment of time from even a very intelligent person to reach a deep understanding of the state of the art. The fact that only a handful of folks are sufficiently versed in a particular sub-specialty to evaluate a long, involved and presumably brilliant proof attempt that draws significantly upon the state of the art of that sub-specialty does not imply that only a handful of folks are sufficiently intelligent to evaluate the proof. This reminds me of the recent attempt on the P/NP problem by Vinay Deolalikar - plenty of extremely talented mathematicians watched from the sidelines because they did not have sufficient specialized knowledge to properly evaluate the proof; though, that did not imply that the proof was particularly brilliant or even correct (http://michaelnielsen.org/polymath1/index.php?title=Deolalikar%27s_P!%…
apu · 2012-02-05
I'm all for new publishing models in academia, but this post doesn't address the fundamental difference between the web in general and academic papers: expert curation.
The web's "curation" works (to whatever degree it does) because there are thousands, if not millions of people who can rate the quality of most articles/essays/websites. But in many scientific fields, there are often only a very small number of people who would be qualified to review a particular paper (usually well under a hundred, sometimes as few as a handful).
And make no mistake, this is a numbers game -- there's the well known 100/10/1 rule for content websites (for every 100 viewers, there are 10 who rate content, and only 1 who creates content). The exact numbers vary quite a bit from website to website, but the point is that it's orders of magnitude difference. That's all fine if your initial pool of people is huge, but if you start with 20 qualified people...you see the issue.
Another related point not addressed is that peer review is fairly demanding work. You're not just saying "accept" or "reject", but you're obliged to write a page or two suggesting the strong and weak points of the paper, things to improve, etc. You need some mechanism for enforcing this (even if it's via social norms), otherwise review quality deteriorates quite severely, which in turn affects paper quality.
In lieu of this article, I'd recommend seeing lectures by (or essays by) Michael Nielsen, who's thought about this…
desireco42 · 2011-12-24
True, however it has been considered not true. There are example, Werner Heizenberg is one,
http://michaelnielsen.org/blog/malcolm-gladwell%E2%80%99s-new-book-out…
that people made genuinely great discoveries without needing 10,000 hours, and there are other examples where people did 10,000 hours of deliberate practice without achieving results. I actually found a critique (can't really find it now) which convinced me that he fudges data to support his claims and sell his books, author examined his references that support MG claims and found many to not be quite supportive and that it is a pattern that is present in most of his books.
His books are entertaining read, well written and all, just not reliable scientific information.
There is discussion on Quora where some of the criticism is voiced as well:
http://www.quora.com/Malcolm-Gladwell-author/What-are-some-claims-by-M…
gajomi · 2011-11-26
I feel compelled to point out (as is done here http://michaelnielsen.org/blog/quantum-computing-for-everyone/) that it is possible that no such explanation exists. My understanding is that most people doing theoretical research in quantum computing don't need to have intuitive explanations for their results, since the calculations work themselves out. This is a depressing but honest observation.
I found quantum mechanics courses very frustrating for exactly this reason, especially in grad school, when the calculations became especially dense and the rationale impenetrable (I have since left physics). Occasionally there would be some little gem of insight into a particular phenomena where the result could be explained in a clear way, but the default approach was to just "do the calculation".
I can't explain exactly how a quantum computer works (because I don't understand it myself) but I will say that the reason the kind of analogy given above might make sense from a quantum mechanical perspective is that the state of the system and the operations that mutate the state have semantics different from classical computers. In particular, the state of the system is given by a bunch of complex numbers, which you can think of as a collection of arrows on the plane. The length and direction that these arrows are pointed in determines the probability for a bit having a particular sign. For example if an arrow is aligned more east-to west the measured classical bit will more likely…
jmatt · 2010-08-10
All these academics that lay low then pop up with problems and call Deolalikar's contribution unfixable infuriate me.
They bring out the worst in the same way that bad QA employees do. They create no wealth and add little value. They slow down and confuse the process. But in the end they are still necessary. There is something to be said about how one reports such problems. Most here know a great QA person that reports relevant problems in a intelligent manner and is there to help find solutions. Versus the QA person that just cranks out an endless stream of wonky insane problems. All make you look bad and all are difficult to reproduce, understand or address.
It's my understanding that his approach is a radical and new way to attempt to solve the P!=NP problem. So it's no surprise that at first glance the vast majority of average math and computer scientists would think it was not a valid approach. And, again no surprise to me, that has ended up to be the case. I don't pretend to understand this flaw or Deolalikar's paper. But I do understand the type of person that posts comments like that...
It looks like the math community has settle on a wiki to unofficially collect news and information on the paper:
http://michaelnielsen.org/polymath1/index.php?title=Deolalikars_P!%3DN…