Daniël Lakens

I don't think it's an issue of old versus new methods. Daniel Lakens has an great string of blog posts / publications on how a lot of the same things can go wrong with Bayesian anaylses [1], and how p-values can be viewed as a "poor man's Bayesian updating function" [2]. [1]: http://daniellakens.blogspot.com/2017/03/no-p-values-are-not-to-blame-… [2]: https://daniellakens.blogspot.com/2015/11/the-relation-between-p-value…

1. The study you link is the famous one about judges. It doesn't mention juries, nor are juries involved in the study. 2. Whether or not people have had lunch is not the biggest factor in outcomes. The most important thing you can read about this study is Daniel Lakens' commentary: > I appreciate that people have tried to think about which mechanism could cause this effect, and if you are interested, highly recommend reading the commentaries (and perhaps even the response by the authors). > But I want to take a different approach in this blog. I think we should dismiss this finding, simply because it is impossible. When we interpret how impossibly large the effect size is, anyone with even a modest understanding of psychology should be able to conclude that it is impossible that this data pattern is caused by a psychological mechanism. As psychologists, we shouldn’t teach or cite this finding, nor use it in policy decisions as an example of psychological bias in decision making. > If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would…

Per https://www.gleech.org/psych: - Stanford Prison Experiment; Not an experiment, abuse was scripted, experimenter constantly intervened, reactions of participants were faked, and there was not even a scientific hypothesis they were testing. Definitely read the paper[0] debunking it. - Milgram Experiment: (the one where people were ordered to shock actors) No good evidence for it. Researchers did not follow script, implausible levels of agreement between different experiments. Killer line is, “only half of the people who undertook the experiment fully believed it was real and of those, 66% disobeyed the experimenter.” - Robber's Cave: (the one where two groups of kids immediately formed tribal hatred between one another) The conflict was orchestrated by experimenters and the experiment was actually repeated because the first time the kids absolutely refused to turn on one another. More information at [1]. - At best, weak evidence for implicit bias testing and stereotype threat. - Weak evidence of "facial feedback" (smiling causes a good mood and frowning causes a bad mood) - Good evidence against "ego depletion" (the idea that willpower is limited in a muscle-like fashion) - Mixed evidence for Dunning-Kruger effect - Questionable evidence for "hungry judge" effect (the idea that judicial sentences are massively more merciful in the morning and after a lunch recess due to "ego depletion" - this is also thoroughly debunked here [2]) - The 10,000 hours of practice…

There was another article related to this study that hit the HN front page. It talked about the size of effect, and argued that the effect size was ridiculously big, and if true, we should see giant spikes in car crashes around lunch, big enough for common sense to ban driving just before noon. I think you'll enjoy the read. > If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. https://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.htm…

The hungry judge effect hasn't replicated well. For example, https://www.tse-fr.eu/sites/default/files/TSE/documents/doc/wp/2022/wp… , and a few years ago it was pointed out that the actual effect in the original paper seems too large and precise to be true: https://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.htm…

This guy doesn't think it's real https://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.htm…

The thing is, just because you can prove there’s a correlation, doesn’t mean you’ve figured out the actual cause. Like the Flying Spaghetti Monster doctrine of piracy and global warming. And this one they discussed elsewhere in the thread: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html http://sparrowism.soc.srcf.net/home/graph.png

Also relevant: http://daniellakens.blogspot.com.au/2017/07/impossibly-hungry-judges.h… which applies the same logic, but to the famous study showing that judges hand out harsher sentences before lunch than after lunch

Here is one: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html

Yes: https://www.pnas.org/content/108/42/E833 https://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.htm…

Excessively large effect sizes are a warning sign in and of themselves. A good article on this is here: [0] -- simply if the effect is too huge, it becomes incumbent on the researcher to explain why no-one has noticed this before. Methane is a gas with a distinct and noticeable smell. Seaweed is plentiful in many places cows are kept. If the effect was really as dramatic as claimed you'd expect that it'd be well known that letting cows eat seaweed made them less smelly, and the authors of the original study make no attempt to explain why this might be the case. 0: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html

Look at Bem's studies. They highlight this phenomenon clearly. Even meta studies can't prove things. http://daniellakens.blogspot.com/2015/04/why-meta-analysis-of-90-preco…

I have always found this response to the hungry judges study much more compelling than the study itself: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html… > […] I want to take a different approach in this blog. I think we should dismiss this finding, simply because it is impossible. When we interpret how impossibly large the effect size is, anyone with even a modest understanding of psychology should be able to conclude that it is impossible that this data pattern is caused by a psychological mechanism. > If hunger had an effect on our mental resources of this magnitude, our society would fall into minor chaos every day at 11:45. Or at the very least, our society would have organized itself around this incredibly strong effect of mental depletion. Just like manufacturers take size differences between men and women into account when producing items such as golf clubs or watches, we would stop teaching in the time before lunch, doctors would not schedule surgery, and driving before lunch would be illegal.