C
Cennydd Bowles
cennydd.co.uk · Personal blog · on HN since 2008
Cennydd Bowles, a UX designer and author, writes about design practice, A/B testing, and the ethics and direction of technology.
What HN saysHN commenters cite his posts on A/B test pitfalls and on redesign backlash, which one quotes to illustrate user resistance to change.
15HN people shared it
2linked it in comments
16stories submitted
79story points
13 years agolast shared
2008people per year2026
Most-shared posts
- 1Statistical significance & other A/B test pitfalls5 people · 23 points · first shared 2009 · 15 comments on HN
- 2End hover abuse now2 people · 40 points · first shared 2010 · 24 comments on HN
- 3Fall and Rise of UX2 people · 4 points · first shared 2011 · discussion on HN
- 4Why is technology so dull?1 people · 3 points · first shared 2008 · discussion on HN
- 5Designing with Context1 people · 2 points · first shared 2013 · discussion on HN
- 6The making of Tourdust1 people · 2 points · first shared 2009 · discussion on HN
- 7UX: A Metaphore1 people · 1 points · first shared 2008 · discussion on HN
- 8The gradient chart1 people · 1 points · first shared 2013 · discussion on HN
- 9Slow Swordfighting at the World Chess Championship Candidates Tournament1 people · 1 points · first shared 2013 · discussion on HN
- 10Why don't designers take Android seriously?1 people · 1 points · first shared 2014 · discussion on HN
- 11The site looks a million times better. Why aren’t we converting?1 people · 1 points · first shared 2011 · discussion on HN
- 12Can we avoid redesign backlash1 people · first shared 2008
What HN people wrote when they linked it
cheez80 · 2008-12-28
found it here:
http://www.cennydd.co.uk/2008/can-we-avoid-redesign-backlash/
"In a nutshell, a meaningless background was removed from a seller page. Pandemonium. After strong resistance the background was reinstated, to everyone’s satisfaction. In fact, the rebellious users were so placated that they failed to notice the designers slowly adjusting the background’s hex values over the next few months. The background got lighter and lighter until one day—pop!—it was gone."
dbaupp · 2012-10-11
I don't think that Coding Horror article is very good. It is too hand-wavey and drawing an analogy that is stretched a little too far, without giving it a sufficiently hard background.
I think what he is trying to say is that Phil didn't test the higher order interactions enough, i.e. he didn't check that the effect that some pair of action have when done together (and the effect of triples, etc.). This is very important to note, and a good point. (I think one should preferably run a fully crossed factorial experiment[0], so that every combination is tested.)
But it is just saying "don't do A/B testing wrong", which is obviously true. I think a lot of problems with A/B testing are caused by people who don't have much statistical knowledge missing some of the subtleties that comes with any experimental design.
(On that note, there are many articles[1] about why one needs to be careful using A/B testing, which do have the backing of statistics.)
[0]: https://en.wikipedia.org/wiki/Factorial_experiment
[1]: http://www.evanmiller.org/how-not-to-run-an-ab-test.html, http://www.cennydd.co.uk/2009/statistical-significance-other-ab-test-p…