S
Simon Funk
sifter.org/~simon · Personal blog · on HN since 2008
Simon Funk, known for the Netflix Prize matrix factorization write-up, writes on machine learning, fiction, and DIY projects.
What HN saysCommenters credit his Netflix Prize post with introducing gradient-descent matrix factorization for recommenders, and also link his fiction and his views on automation.
31HN people shared it
16linked it in comments
16stories submitted
381story points
3 months agolast shared
2008people per year2026
Most-shared posts
- 1Netflix Update: Try This at Home (2006)15 people · 21 points · first shared 2008 · discussion on HN
- 2Chaotic Flow, Focus, and Interrupts2 people · 5 points · first shared 2009 · discussion on HN
- 3Pioneers get the arrows, the settlers get the land2 people · 3 points · first shared 2020 · 6 comments on HN
- 4AfterLife2 people · first shared 2010
- 5Index2 people · first shared 2009
- 6$30 Homebrew Automated Blinds Opener (2024)1 people · 329 points · first shared 2025 · 151 comments on HN
- 7Cheap, Deep Learning with Fusion Reflection1 people · 18 points · first shared 2015 · discussion on HN
- 8On Debating1 people · 2 points · first shared 2014 · discussion on HN
- 9Knowledge Representation (2013)1 people · 1 points · first shared 2026 · discussion on HN
- 10Time <b>is</b> Different1 people · 1 points · first shared 2026 · discussion on HN
- 11Behind the Power Curve1 people · 1 points · first shared 2010 · discussion on HN
- 12Journal 201607041 people · first shared 2016
- 13Journal 202404121 people · first shared 2025
What HN people wrote when they linked it
lern_too_spel · 2023-11-27
Matrix factorization was popular in industrial recommender systems for years afterwards, and its ideas are still useful today. https://sifter.org/~simon/journal/20061211.html
samplenoise · 2021-07-07
The author is still underselling the significance of the progress made during the first years IMO. The simple idea that is still behind most practical recommender systems (using gradient descent to do SVD to complete the rating matrix) was first described in 2006 by Simon Funk [1]. Koren, who ended up taking home a big part of the prize, recently wrote another paper about how that basic idea still outperforms most “AI” (deep neural) recommenders today [2].
[1] https://sifter.org/~simon/journal/20061211.html
[2] https://arxiv.org/abs/1905.01395
RobertoG · 2021-02-22
In my opinion, the best fiction book about this subject is 'Permutation City' by Greg Egan.
Also, this one is pretty good:
https://sifter.org/~simon/AfterLife/index.html
And, in a very similar line to "Lena", this one by Vernor Vinge:
https://en.wikipedia.org/wiki/The_Cookie_Monster_(novella)
fossuser · 2021-01-27
I'm an outlier on this one, but I thought Consider Phlebas was bad and disappointing.
The Culture was cool and interesting, and the world had some interesting detail but everything else was pretty weak. I generally agree with this review: https://www.goodreads.com/review/show/48178154?book_show_action=true
While we're on the topic I also really didn't like the three body problem.
I loved Permutation City and most of Ted Chiang (particularly Liking What You See: A Documentary, and The Lifecycle of Software Objects).
I love True Names by Vernor Vinge.
I loved this: https://sifter.org/~simon/AfterLife/
Hopefully someone finds this helpful, sometimes an anti-recommendation can be just as useful for calibration as a recommendation. Here's some of both.
If anyone reading this agrees - I'd love a suggestion from you.
cousin_it · 2016-08-12
Simon Funk [1] makes a great point that in a mixed capitalist/welfare state, automation decreases efficiency.
Imagine a worker, earning the bare minimum to survive, who gets replaced by a robot with slightly lower operating cost. Now the robot is consuming slightly fewer resources that the worker would have, but the worker doesn't stop consuming resources in turn! Unless you're willing to kill the worker, he goes on welfare instead, and the economy as a whole is burning almost 2x more resources. Sure, the employer is spending slightly less, while screwing over the economy via prisoner's dilemma logic.
[1] http://sifter.org/~simon/journal/20160704.html
Nogwater · 2012-08-25
Here's mine if anyone is interested. I wrote it in D and haven't looked at it in years. I'm sure it's not usable as-is, but it might be fun anyway.
https://github.com/nogwater/NetflixPrizeD
The algorithm is based on Simon Funk's blog post here:
http://sifter.org/~simon/journal/20061211.html
For me, the best part was squeezing the data and indexes into memory. :)
mayank · 2011-01-31
On the contrary, within a few days of the Netflix prize challenge being launched, a guy put up a detailed blog post with a simple linear algebra method that could beat Netflix's system at the time (link below). It didn't come anywhere near the 10% improvement threshold, but it beat Netflix's system with a handful of lines of C code, and it ran off his laptop. Sounds like he would have made an excellent hire.
http://sifter.org/~simon/journal/20061211.html
lkozma · 2010-08-01
Good suggestion for dimensionality reduction. Most ready made packages you will find under the name PCA though (principal component analysis). It's almost always good to reduce dimensionality first, depending on the component number you choose, you can remove a lot of the noise from the data.
I'm curious why it didn't work for music recommendation for you. In the netflix challenge for movie recommendation, pretty much everyone ended up using SVD-based methods and variations thereof.
If anyone is interested, this is the most accessible write-up of the idea I know: http://sifter.org/~simon/journal/20061211.html
gaika · 2008-06-30
It is both. Trust and recommendations are mixed together in an SVD like algorithm as described in http://sifter.org/~simon/journal/20061211.html . On top of that jaanix lets you adjust all your personal preferences in real-time.
The biggest effect is that it is discouraging trolls and spammers, as there's no front page they can spam or troll. The downside is that the critical mass required to make the site truly social is a lot higher, and unfortunately we have not figured out how to gain enough users for it to really shine.
scotty79 · 2025-05-19
I think you can read about OP's automation stack here: https://sifter.org/~simon/journal/20240412.html
sytelus · 2017-01-24
Is this a more formal treatment for algorithm that Simon Funk gave?
http://sifter.org/~simon/journal/20061211.html
stillsut · 2015-07-22
There was a parsimonious model that got to 8% improvement. We know this as it was posted publicly by "Simon Funk" [0] after he decided to give up halfway through the competition.
This simple but more accurate model would have been a useful foundation for Netflix to build its own internal models on, and highlight the flaws in the assumption of the Cinermark algo.
[0] http://sifter.org/~simon/journal/20061211.html
benji-york · 2010-10-19
This isn't a series, but a set of fiction that I think fits the bill. Conveniently all of these works are available free online:
After Life: http://sifter.org/~simon/AfterLife
Blindsight: http://www.rifters.com/real/Blindsight.htm
Passages in the Void: http://localroger.com/
The Metamorphosis of Prime Intellect (from localroger.com) is also pretty good in the imagination expanding department but be forewarned: parts of it are very hard to read (quite gruesome).
sprachspiel · 2010-01-21
Unless X has 17,770*480,189 elements like the netflix dataset. Then you can use something like this: http://sifter.org/~simon/journal/20061211.html
The site is currently down, google chache link: http://209.85.129.132/search?q=cache:h4Ljyun3gUcJ:sifter.org/~simon/jo…
donaq · 2009-04-06
Yes they were. And since we're dealing with post-singularity stories, I'll add Simon Funk's novella, "After Life".
http://sifter.org/~simon/AfterLife/index.html
benfrederickson · 2015-04-28
I like matrix factorization models too !
Most of the netflix prize style models will have problems with this sort of data though. The problem here is that you only have positive data, and no negative data: you can assume that artists the user listened are positive, but there is no explicit negative votes for artists the user didn’t like. This means that there is no gradient for Funk style (http://sifter.org/~simon/journal/20061211.html) SGD solvers to descend.
Better matrix factorization models would be something like the weighted ALS (http://labs.yahoo.com/files/HuKorenVolinsky-ICDM08.pdf) or BPR (http://www.ismll.uni-hildesheim.de/pub/pdfs/Rendle_et_al2009-Bayesian_… approaches which can handle this sort of thing directly.
Having said all that - this post isn’t about CF. Its just trying to explain some basic concepts in information retrieval using finding similar artists as a problem.