Count Bayesie

Worth a read https://www.countbayesie.com/blog/2015/2/18/hans-solo-and-bayesian-pri…

I remember the frequentist approach taught in introductory stats classes never making sense to me. I didn't want to shove "stats" into the back of my brain and just focus on graduating. I genuinely wanted to understand the world a little better. I began to research alternative approaches to modeling and conducting inference a few years ago. Discovering Bayesian Inference has had a large impact on the way I think and conduct research. There's a lot of hype and uncertainty about what "Bayesian" actually means. Here's a compact definition that I hope will attract some interest: Bayesian Inference allows you to explicitly quantify your prior beliefs and get a more complete picture of uncertainty when modeling something. If you'd like to learn more, the links below should be helpful. Introduction to Bayes' Theorem (short): https://www.countbayesie.com/blog/2015/2/18/bayes-theorem-with-lego Bayesian A/B testing example (short): https://www.countbayesie.com/blog/2015/4/25/bayesian-ab-testing If you're interested in spending some time learning about applied Bayesian Inference, I highly recommend Statistical Rethinking. The book doesn't assume a strong mathematical background and its filled with practical examples. https://xcelab.net/rm/statistical-rethinking/ McElreath is currently working on a second edition of that textbook, due around 2020: http://elevanth.org/blog/2018/07/14/statistical-rethinking-edition-2-e…

A much simpler exploration of this topic I always liked is the "Linear Diffusion" [0] example which implements a basic Diffusion model using only linear components. Given it's simplicity it gets surprisingly good results at generating digits. 0. https://www.countbayesie.com/blog/2023/4/21/linear-diffusion

As a big fan of entropy (see this post: https://www.reddit.com/r/MachineLearning/comments/8im9eb/d_crossentrop… and links there) I like this explanation: https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence…

Write a good LSTM to generate new chapters in it, would be a good way to start, like this one: https://www.countbayesie.com/blog/2015/5/24/writing-finnegans-wake-wit…

Or C3PO is just a stodgy old frequentist: https://www.countbayesie.com/blog/2015/2/18/hans-solo-and-bayesian-pri…

I'm still quite fond of the Lego example: https://www.countbayesie.com/blog/2015/2/18/bayes-theorem-with-lego

I have read the selection process that was described in Lost Moon (the book of appolo 13). The NASA were literally recruiting superhumans back then. The smartest, healthiest, toughest and on on ... So I am not at all surprised that he managed to pull it off. https://www.countbayesie.com/blog/2015/2/18/hans-solo-and-bayesian-pri…

Original source: http://www.countbayesie.com/blog/2015/2/18/hans-solo-and-bayesian-prio…

This statement is a misunderstanding of the Bayesian approach: "Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement." Bayes Factor, the Bayesian alternative to a NHST, is quite a bit different than simply creating the Bayesian equivalent of a t-test. Bayes factor asks "How many times better is my Hypothesis at explaining the data than an alternate Hypothesis". So the Bayesian statement would first pit one model of the Earth's orbit against another. The Bayesian statement of the question of the Earth's orbit would be: "How much more likely is the astronomical data we've observed to have happened given that the Earth revolves around the sun than it is if the sun revolved around the Earth." For a more concrete example let's suppose that we have a coin. I think the coin has only heads and you think it is a fair coin, with a 50/50 chance of getting heads or tails. We observe three heads in a row. My hypothesis says that the probability of getting 3 heads in a row given a trick coin is 1. Your hypothesis says that the probability of getting 3 heads in a row given a fair coin is 0.5 x 0.5 x 0.5 = 0.125. My hypothesis explains the data 1/0.125 = 8 times better than your hypothesis. Now suppose the next flip is a tail. The probability of HHHT in my model is 0 and yours is 0.5 x 0.5 x 0.5 x 0.5 = 0.0625. You're hypothesis explains the model infinitely better than mine! Now we…

I've long argued that the biggest problem with orthodox NHST for A/B testing is that you actually don't care about 'significance of effect' as much as you do 'magnitude of effect'. Furthermore, p-values tell you nothing about the range of possible improvements (or lack thereof) you're facing. Maybe you are willing to risk potential losses for potentially huge gains, or maybe you can't afford to lose a single customer and would rather exchange time for certainty. My favored approach I've outlined here[0]. Where the problem is basically considered one of Bayesian parameter estimation. Benefits include: 1. Output is a range of possible improvements so you can reason about risk/reward for calling a test early. 2. Allows the use of prior information to prevent very early stopping, and provide better estimates early on. 3. Every piece of the testing setup is, imho, easy to understand (ignore this benefit if you can comfortably derive Student's T-distribution from first principles) [0] https://www.countbayesie.com/blog/2015/4/25/bayesian-ab-testing