Berkeley ML Blog
ml.berkeley.edu · Company blog · on HN since 2016
Berkeley's student machine learning group writes explainers and commentary on topics like DALL-E 2, CLIP art, and ML basics.
What HN saysHN links it for explaining image-generation models; commenters cite its DALL-E 2 post on how transformers work and the CLIP art piece.
21HN people shared it
10linked it in comments
12stories submitted
1.6Kstory points
4 years agolast shared
2016people per year2026
Most-shared posts
- 1Alien Dreams: An Emerging Art Scene5 people · 306 points · first shared 2021 · 104 comments on HN
- 2Machine Learning Crash Course: Part 14 people · 337 points · first shared 2016 · 16 comments on HN
- 3Machine Learning Crash Course: The Bias-Variance Dilemma3 people · 550 points · first shared 2017 · 60 comments on HN
- 4Dalle23 people · first shared 2022
- 5Machine Learning Crash Course: Part 22 people · 378 points · first shared 2016 · 36 comments on HN
- 6Machine Learning at Berkeley's Intro to ML Series: Neural Networks and Backprop2 people · 10 points · first shared 2017 · 2 comments on HN
- 7Check out this cool machine learning blog I found1 people · 3 points · first shared 2018 · discussion on HN
- 8Check out this cool machine learning blog I found1 people · 3 points · first shared 2018 · discussion on HN
- 9Check out this cool machine learning blog I found1 people · 2 points · first shared 2018 · discussion on HN
- 10Neural Module Networks for Visual Question Answering1 people · 1 points · first shared 2021 · discussion on HN
- 11Earthquake fit analysis.png1 people · first shared 2017
What HN people wrote when they linked it
dontwearitout · 2023-01-06
Absolutely. These transformer models are trained to autoregressively predict the next token (text, or now audio), and so can be viewed as powerful compression schemes by combining with something like Huffman coding (see https://ml.berkeley.edu/blog/posts/dalle2/ for a lengthy but great read on how language models can be viewed as sophisticated lossless compression algorithms). The log-likelihood loss function used to train these models can quite directly be interpreted as the average bit-per-token needed to compress the dataset.
"Compression is comprehension" - Chaitin
pas · 2022-04-19
https://ml.berkeley.edu/blog/posts/dalle2/
it's "simple" because how it works is "just" brute-fucking-force. of course coming up with the architecture and making it fast (so it scales up well) is the challenge.
and scaling works .. because .. well, no one knows why (but likely because it's just a nice architecture for learning, evolution also converged on it without knowing why)
see also: https://www.gwern.net/Scaling-hypothesis
joshcryer · 2022-04-06
I'm with you there but we still don't know how it works, just that it does. The method though is you take a bunch of images, you plug them into a multi dimensional array (a nice way of saying a tensor), have some kind of tagging system, and when you ask the system for an answer, it will put one out for you. So for example in the astronaut riding the horse, there is, on some level, a picture of a horse with those similar pixels, that exists in the data of some object tagged 'horse.' Likewise with astronaut. What is important is that the data sets are absolutely massive, with billions of parameters.
Here's a more of a 'not 15 year old' explanation: https://ml.berkeley.edu/blog/posts/dalle2/
movomito · 2021-10-25
I had a very different reaction. I had the sense this was an art world person (student or otherwise) who wasn’t technical enough to make it interesting. Many computer generated art projects are much more dynamic and produce much more passable art (https://ml.berkeley.edu/blog/posts/clip-art/). That said, I think I would have to agree with other comments here, I don’t think any deep conclusions could be drawn from these results. For starters the artworks are really repetitive and i would guess it biases respondents quickly toward the negative.
andreyk · 2021-07-12
OP here, also check out the Berkeley ML Blog post on this, really informative - https://ml.berkeley.edu/blog/posts/clip-art/
autokad · 2017-08-29
more likely their model is wrong. it reminds me of fukushima: https://ml.berkeley.edu/blog/2017/07/13/tutorial-4/
blah9874 · 2017-07-17
In my opinion, the real problem in that case was not the overfitting, but that they extrapolated from that data. They didn't have anything above Magnitude 8. (https://ml.berkeley.edu/blog/assets/tutorials/4/earthquake-fit-analysi…
You should never, ever extrapolate. It doesn't matter what your model is, it won't work.
On a side note, it could be that there is a breakpoint at Magnitude 7.25, where the slope of the line really changes, and a segmented linear regression is appropriate (https://en.wikipedia.org/wiki/Segmented_regression). But we would need more data to be sure, anyway.
rdudekul · 2017-07-17
Here are parts 1, 2 & 3:
Introduction, Regression/Classification, Cost Functions, and Gradient Descent:
https://ml.berkeley.edu/blog/2016/11/06/tutorial-1/
Perceptrons, Logistic Regression, and SVMs:
https://ml.berkeley.edu/blog/2016/12/24/tutorial-2/
Neural networks & Backpropagation:
https://ml.berkeley.edu/blog/2017/02/04/tutorial-3/
dccooper · 2017-07-14
Additionally, the Machine Learning Crash Course Series from Berkeley is a good overall explainer of the concepts.
https://ml.berkeley.edu/blog/2016/11/06/tutorial-1/
I've had luck with a mixture of online courses like Datacamp (https://www.datacamp.com/) and finding projects to try on sites like Kaggle.
michaelsbradley · 2016-12-29
Here's Part 1: https://ml.berkeley.edu/blog/2016/11/06/tutorial-1/