Allen Downey
allendowney.blogspot.com · Personal blog · on HN since 2011
Allen Downey, a computer science professor and author, writes about statistics, Bayesian methods, and Python.
What HN saysHN links it for the inspection paradox post, which commenters discuss and try to explain, and for his data on declining religious affiliation.
96HN people shared it
32linked it in comments
123stories submitted
1.9Kstory points
4 months agolast shared
2011people per year2026
Most-shared posts
- 1The inspection paradox is everywhere (2015)7 people · 138 points · first shared 2015 · 42 comments on HN
- 2There is only one statistical test6 people · 8 points · first shared 2021 · 1 comments on HN
- 3Probably Overthinking It: There is only one test4 people · 6 points · first shared 2011 · 1 comments on HN
- 4Homepage4 people · first shared 2019
- 5Correlation Is Evidence of Causation3 people · 5 points · first shared 2014 · discussion on HN
- 6Homepage3 people · first shared 2017
- 7Learning to program is getting harder2 people · 478 points · first shared 2018 · 412 comments on HN
- 8All your Bayes are belong to us: fun Bayes's Theorem problems.2 people · 83 points · first shared 2011 · 5 comments on HN
- 9Inspection Paradox (2015)2 people · 70 points · first shared 2018 · 13 comments on HN
- 10In Search Of: Simpson's Paradox2 people · 23 points · first shared 2021 · 3 comments on HN
- 11Learning to Program Is Getting Harder2 people · 20 points · first shared 2018 · 11 comments on HN
- 12Millennials are not getting married2 people · 10 points · first shared 2020 · 13 comments on HN
- 13The fallacy of teaching programming by showing working programs2 people · 6 points · first shared 2011 · discussion on HN
- 14Bayesian Dice2 people · 4 points · first shared 2021 · discussion on HN
- 15Bayesian Survival Analysis of GOT2 people · 4 points · first shared 2015 · discussion on HN
What HN people wrote when they linked it
alexyes · 2014-08-20
A great guy. More interesting readings on his website (research papers, etc.:
http://allendowney.com/
wesammikhail · 2026-04-05
bla bla far right bla bla far right. Everything is far right. My neighbors cat is far right apparently.
How the heck is Twitter far right AND Telegram Moscow far right? Those two things arent even remotely the same. And since when is Telegram a problem when most voting people in the west don't even know wtf it is.
Reminds me of this: https://www.allendowney.com/blog/wp-content/uploads/2024/01/image.png
But you know... maybe I'm crazy, could you perhaps define "far-right" for me? because at this point I have no clue wtf that even means...
And when defining it I'd appreciate it if you could tell me if the following countries are considered far right by your standard: Israel, Saudi Arabia, Turkey, China, Japan.
k thnx!
emmanueloga_ · 2020-09-30
I cannot recommended them because I have them on my back burner, but I would like to start working through the "Think" Stats series soon [1].
I love the premise: "if you know how to program, you can use that skill to learn other topics."
Perhaps someone here can speak to their experience with some of these books?
1: http://www.allendowney.com/wp/books/
raldi · 2018-11-01
This is a great example of the Inspection Paradox, mentioned on HN earlier this week:
https://allendowney.blogspot.com/2015/08/the-inspection-paradox-is-eve…
xyience · 2016-04-22
http://allendowney.blogspot.com/2016/03/the-retreat-from-religion-is.h… This blogger has been covering the phenomenon among college freshmen since 2008. Notably the most recent results segregate none, atheist, and agnostic whereas before it was just 'none', but the effects are the same regardless of which label you want to associate with / avoid.
Personally I think the actual cultural movement towards 'atheism' is simple Leftism. Most people may still believe in the belief of God and thus differ from atheists on the ontological question, but their beliefs about the world in general are vastly different from those of their (for some religions, even still living) ancestors of the same declared denomination and those beliefs are in conflict with classic doctrine.
eaurouge · 2013-03-27
I counted close to ten free books by Allen B. Downey [http://allendowney.com].
tuatoru · 2024-06-04
There are six (maybe more, but I have six so far) factors in low nuptiality (marriage rates)[1] and consequently sub-replacement fertility, under the headings of religion, medicine, technology, geography, sociology, and ethics.
1. Secularism. Secular populations are less fertile than highly religious populations.
2. Infant survival (low infant mortality).
3. Mechanisation particularly of agriculture.
4. Urbanisation.
5. Credentialism, the social requirement for people to spend an increasing fraction of their fertile years in pre-employment education. Simple literacy is also pretty important in causing early fertility decline in developing countries.
6. Universalism together with a tolerance for inequality. Universalism is applying the same rules to everyone, so women can enter the workforce and make their own life decisions just as men do. Tolerance for inequality I hope I don't have to explain. (I am split whether this is one factor or two.)
There is some interlinkage in these, they are not all totally orthogonal, but they are all fairly distinct dimensions of the problem.
Japan (and South Korea, and China) max out on credentialism and tolerance for inequality but also score highly in all of the others.
Finally this is a great example of one of Iain Banks's "Outside Context Problems".
More than a black swan, it's a problem that governments are simply not equipped to understand or in some cases even see, so they can't do anything to solve it.
1.…
canyon289 · 2024-02-10
Thank you for asking. I find this stuff super interesting. I write about it on my blog
https://ravinkumar.com/
and here's a video for the most common question I get about bayes vs frequentist
https://www.youtube.com/watch?v=foSPfzYs4yY
For recommendations here's my suggestions split by persona
* I want a PHD in statistics or to write novel research - Go read BDA3 cover to cover. Gelman etal are amazing, and amazingly brilliant. The book is dense though, even after years of experience I need to now read chapters 2 or 3 times and write out proofs
* I want have a smooth on ramp into Bayesian stats with lots of code and beautiful writing. I also like video lectures and undergraduate college courses - Statistical Rethinking. As noted below Richard does a wonderful job explaining these concepts with metaphors like golems, interspersing it with this experience as an anthropologist, and using his teaching experience to write well structured introduction into Bayesian stats. This book does assume you understand basic statistics and probability theory.
* I want a great comparison of Bayesian vs Frequentist stats- This book covers both topics well, compares them fairly, and has all the proofs to back things up https://www.routledge.com/Understanding-Advanced-Statistical-Methods/W…
* I'm a programmer type person that likes hands on "build from scratch" using code - Allen Downey's think bayes builds up bayes theorem from Numpy arrays. He's also…
saeranv · 2021-09-29
So I've read this article a couple of times and I still don't quite understand it. Can anyone summarize the key argument for a layperson?
Here's my best interpretation thus far:
This particular post is a continuation of his two earlier posts[1], [2] which seems to critique the way statistical significance is simplistically calculated from a gaussian distribution when effect size is small. This leads to an argument that the statistics community is too dependant on generative models for their interpretation, given that these models are typically to simplistic outside of hard sciences:
> "But in biology, medicine, social science, and economics, our models are much less accurate and less grounded in natural laws. Most of the time, models are selected because they are convenient, not because they are plausible, well motivated from phenomenological principles, or even empirically validated. Freedman built a cottage industry around pointing out how poorly motivated many of the common statistical models are."
This part I am a little unclear on, but it seems like this leads him to suggest focusing on the random sampling as a way to get counts that you can then plug into various statistical formulas, without the need to assume a probabalistic model:
> "So what is the remedy here? The thing is, we already know the answer: if we randomized the assignment, we can estimate log odds by counting the number of positive outcomes under treatment and control, and then just plugging these…
Gibbon1 · 2026-06-06
I read those before but will read them again. That narrative influenced my thinking about this. There is confusion I think because peoples attitudes tend to be stubborn over time. But they tend to match the milieu they were raised under.
An example of that is the plots in this essay. Attitudes don't change much plotted by age cohort over time.
https://www.allendowney.com/blog/2026/05/28/sexual-morality/
Summarize that we've thrown a bunch of historical peasants into an industrial society and they're reacting astutely to the new incentives. But the big change comes from those that grew up in it.
Example Bangladesh fertility rate went from 7 in 1975 to 4 in one generation and dropped to 2 one generation later.
svat · 2025-03-05
Look at "Blog archive" in the sidebar; you can click on each year/month to expand; it looks something like this: https://imgur.com/a/Q91LC4k
And you can find more recent posts at his newer blog: https://www.allendowney.com/blog/ where again there are similar links for each month.
orzig · 2024-02-16
The author also blogs his latest Python projects (mostly Bayesian data analysis) on https://www.allendowney.com/blog/ , which I have really enjoyed.
beefman · 2024-01-28
This apparently does not replicate in other surveys of the U.S.
https://www.allendowney.com/blog/2024/01/28/is-the-ideology-gap-growin…
https://twitter.com/janzilinsky/status/1751431162126307727
https://twitter.com/ryanburge/status/1751615375408521623
yantrams · 2019-02-08
Here are some relatively lesser known blogs that I can recommend, apart from the usual suspects (worrydream, John Baez etc.)
https://jeremykun.com
http://colah.github.io
https://mathwithbaddrawings.com
https://www.allendowney.com/blog/
And yeah I finally decided to inject some character into my personal website and spent a couple of days wrangling with css and made this --- http://linkdot.link . Not much content really though and it is mostly probability puzzles that I solve every now and then.
Edit: Formatting
Sniffnoy · 2018-11-01
combatentropy · 2018-10-29
This is my first exposure to it, but yes, I think so. My paraphrase is: The Inspection Paradox is when you ask someone "in the mix" about the mix. You're only going to get an accurate estimation of the mix by standing outside of the mix.
So yes, if you want an accurate count of siblings, you would consult some spreadsheet that just lists how many children each family had. If you go and start asking the families themselves (those "in the mix") then your results will be skewed.
I thought the article that this article linked to was also very good, "The Inspection Paradox Is Everywhere," by Allen Downey, http://allendowney.blogspot.com/2015/08/the-inspection-paradox-is-ever…
netgusto · 2018-09-28
The article posted on G+ is this one I believe: https://allendowney.blogspot.com/2018/09/two-hour-marathon-in-2031-may…
The only thing I could see violating TOS would be a misinterpretation of this proposition:
> Note that this model is based on data from races.
Where "races" would be taken for "Human races" rather than "Sport races" (as intended).
epalmer · 2017-05-01
I saw Allen Downey speak about this concept a year ago in a small forum at the University of Richmond. Several professors in the audience challenged his thinking. I approached one after that and asked what he thought and the answer was basically that Dr. Downey's approach is spot on. This coming from a CS professor.
I have just started reading his Think Stats book http://www.allendowney.com/wp/books/ which is starting to help me better understand statistics. Not far enough into the book yet to make a complete judgement however.
Got to say I like seeing different perspectives on traditional subjects. My youngest is going to Olin next year in Mechanical Engineering and seeing these sorts of articles keeps me excited about her unconventional engineering education at Olin.
kensai · 2017-05-01
Also check out his companion blog post: "Python as a way of thinking". Has some nice slides there.
(http://allendowney.blogspot.com/2017/04/python-as-way-of-thinking.html…
jqgatsby · 2015-12-02
It is related, and is called the inspection paradox. Identified for me by Allen Downey, with some explanation below:
http://allendowney.blogspot.com/2015/11/recidivism-and-single-case-pro…
staz · 2015-08-31
I'm a layman in probability but isn't it the same thing as the Inspection paradox?
http://allendowney.blogspot.be/2015/08/the-inspection-paradox-is-every…
davmre · 2015-03-28
Since this is the Internet, of course this has been done, several times:
http://www.math.canterbury.ac.nz/~r.vale/got_290814.pdf
http://allendowney.blogspot.com/2015/03/bayesian-survival-analysis-for…
anonymfus · 2014-10-15
You can use overlap to estimate total number of bugs:
http://en.wikipedia.org/wiki/Lincoln_index
http://allendowney.blogspot.com/2014/07/last-year-my-occasional-corres…
arcticfox · 2014-10-10
Looking at a different pivot to the question, Allen Downey gives a good theoretical reason for the decline in times and projects 2041 as the year for the 2-hr marathon. Also, his original projection was made in 2011, so there are three years of data to compare against, which is interesting. http://allendowney.blogspot.com/2014/09/two-hour-marathon-in-2041.html
TeMPOraL · 2014-02-23
It's not like that. Correlation does not imply causation only in the strict meaning of "imply", =>, in mathematical logic. But correlation is sure as hell evidence for casual relationships. Just yesterday I found a blog post by author of "Think Bayes" book explaining that nicely.
http://allendowney.blogspot.com/2014/02/correlation-is-evidence-of-cau…
Moreover, with good enough correlation data, you can even determine the direction of casual relationship if you're willing to do a little bit of maths:
http://lesswrong.com/lw/ev3/causal_diagrams_and_causal_models/
Also, http://xkcd.com/552/ ;).
nowarninglabel · 2012-10-10
So, I'm going to counter here and say I don't find this to be a good intro. I started reading and had not heard of the "Girl named Florida" problem and then went to the linked to blog post http://allendowney.blogspot.com/2011/11/girl-named-florida-solutions.h…
The way he explains it I found to be confusing and counter-intuitive. I've taken basic stats in college, and learned some of the associated problems, though not this one, and learned the material though not in this particular way. I have to agree whole-heartedly with the commenter on that post "JeffJo" who stipulates why it's an ineffectual way to present the material. Furthermore, I found the author's dismissal of the valid criticism to be enough to not want me to read further.
cavedave · 2011-11-02
>This was a fairly large change that we, understandably, only wanted to deploy to a small subset of users. This was facilitated by Bengineer Kamen's GAE/Bingo split-testing framework for App Engine.
I think this method of A/B testing has some faults. I blogged about it A/B testing. Is Khan doing it wrong?http://liveatthewitchtrials.blogspot.com/2011/09/ab-testing-is-khan-do…
and Allen Downey ran some simulations at Repeated tests: how bad can it be?http://allendowney.blogspot.com/2011/10/repeated-tests-how-bad-can-it-…
AllenDowney · 2011-08-30
I read the first article more carefully, and it inspired this new blog post: http://allendowney.blogspot.com/2011/08/jimmy-nut-company-problem.html
j_not_j · 2025-06-02
Downey published a (big-endian, SPARC) method in 2007:
https://allendowney.com/research/rand/
And Lemire in 2017 asked the question
https://lemire.me/blog/2017/02/28/how-many-floating-point-numbers-are-…
The basic approach: collect top 60 bits of a 64-bit PRNG. (Assume the LSBs are corrupt or zero or nonrandom). Set exponent zero. If the top mantissa bit is zero, shift left, subtract one from exponent. Repeat. When you run out of bits, collect another PRNG from your generator and resume until you have 56 bits of mantissa. When done, your floating-point PRNG is 2^exponent * mantissa.
My short explanation: Suppose you want a random distance. Multiplying a PR integer is the same as selecting a random tile in the distance, and measuring the distance to the (far) edge of the tile. This is the multiply method.
But if you want a real-valued distance even for very near distances, you need to scale your random number and ensure you have random bits throughout the mantissa. So reduce the exponent for every leading zero in your PR integer and shift. Add more bits if needed.
Test: the histogram of exponents for a large-enough set of samples is linear, give or take. Very small floating numbers (less than say 1/32768) are 1/16 as likely as numbers in [0.5,1).
red_admiral · 2024-09-12
Back in the days of slatestarcodex, the comment policy [1] was you can comment if your post is at least two of these three things: true, necessary, or kind.
This post is all three: what they're describing is true (these dynamics in meetings do exist, very often), it is kind (in the sense they're giving you a skill to help both yourself and others), and I'll give it necessary in the sense it's used in the original definition (if you want to get ahead in an organization with a nontrivial amount of internal politics - which is most places - you need to have at least some of this skill).
And yet, something about this post gives me "weird vibes".
Basically, with a bit of sarcasm you could sum it up as "DON'T BE AUTISTIC", and if you are then at least get therapy until you can act normal.
When the author says "We [humans] are wired to spidey sense this [vibes] stuff", it turns out some humans are above and some below the mean in this skill distribution. [2]
And sometimes, in a meeting to decide about how you're going to set up your database sharding, it helps the business' bottom line if you pay more attention to the database specialist than the soft-eye-contact specialist.
(Don't you want to hire people who are good at both? Yes, but unless you're really, really lucky, you're going to hit Berkson's paradox [3]. And then if you want your databases to run smoothly, you're going to have to compromise.)
[1]…