Ander Source

SEEKING WORK | ML & DS | Remote Have a nonstandard optimization problem? Could use an interactive visualization to explore complex data? I'd love to hear about it! [email protected] https://andersource.dev

>Arguably the hypothesis that matches the data from the DK paper best is: "Everyone thinks they're average regardless of skill level" No, if you look at the graph[0] everyone thinks they are above average (over 50). The worst think they are a little above average and everyone else thinks they are better and better but increasing by less than the real difference. At any rate, the issue seems to be with how people imagine everyone performs - they seem to think there are a lot of people who are really bad for a start, and seemingly a bit more people who are really good than there are (at least if we assume the results are accurate). 0. https://andersource.dev/assets/dk-autocorrelation/dk.webp

A similar argument was made a couple of years ago, here's the rebuttal of the argument from back then [2022]: https://andersource.dev/2022/04/19/dk-autocorrelation.html

Check out Townscaper. https://andersource.dev/2020/11/06/organic-grid.html

This take is a perfect example of Dunning-Kruger itself, ironically. https://andersource.dev/2022/04/19/dk-autocorrelation.html

Yes but some claim to have debunked the debunking also. [1] This paper (2023) claims "the magnitude of the effect was minimal; bringing its meaningfulness into question." [2] [1] https://andersource.dev/2022/04/19/dk-autocorrelation.html [2] https://www.sciencedirect.com/science/article/abs/pii/S016028962200098…

Another interesting system is the messy organic grid from Townscaper. https://andersource.dev/2020/11/06/organic-grid.html

I think he is talking about this article https://andersource.dev/2022/04/19/dk-autocorrelation.html

Pretty clever. Somewhat related, if you want to match the colours of an entire image to an other, you can use optimal transport [1] Unrelated, the author has a very nice take on generating the organic grid from Oskar Stålberg’s Townscaper game [2]. [1]: https://codimd.math.cnrs.fr/s/2eRBqV9zl [2]: https://andersource.dev/2020/11/06/organic-grid.html

thinking about this more (i'm replying to myself!) -- i guess what the experiments for D/K show is exactly that performance on a test is uncorrelated with your idea of the performance on a test. yes, it's kind of surprising that, having dropped the "real" results in a vat of acid, our hapless researcher replaces the missing data with random numbers and gets the same result -- but that's only because we didn't expect random numbers to model the outcome. instead, we would have expected that, towards the bottom of the distribution of test-takers, those folks would rate themselves lower, while towards the top they would rate themselves higher. at the extreme of perfect self-awareness, the line for subjective results would exactly match the line for objectively-scored results. this is the exact argument that is made in the post linked in the top comment: > by using random data to argue that the Dunning-Kruger effect is not real, the author is arguing to default to the base assumption. But which base assumption do they make? One even more radical than what’s proposed by Dunning-Kruger. In the author’s world, the Dunning-Kruger study should be interpreted in the reverse direction, claiming that there is at least some self-awareness in the way people self-assess. source: https://andersource.dev/2022/04/19/dk-autocorrelation.html

I don't buy this take, and this rebuttal does a better job than I could of explaining why: https://andersource.dev/2022/04/19/dk-autocorrelation.html Basically, this autocorrelation take shows that if performance and evaluation of performance were random and independent, you would get a graph like the D-K one, and therefore it states that the effect is just autocorrelation. But in reality, it would be very surprising if performance and evaluation of performance were independent. We expect people to be able to accurately rate their own ability. And D-K did indeed show a correlation between the two, just not as strong of one as we would expect. Rather, they showed a consistent bias. That's the interesting result. They then posit reasons for this. One could certainly debate those reasons. But to say the whole effect is just a statistical artifact because random, independent variables would act in a similar way ignores the fact that these variables aren't expected to be independent.