B
Beren Millidge
beren.io · Personal blog · on HN since 2023
Beren Millidge writes about AI, brain scale compared to machine learning, LLM tokenization, and alignment.
What HN saysHN commenters cite his posts on integer tokenization problems in GPT-2 and on alignment generalizing further than capabilities, in discussions about AI approaches.
18HN people shared it
5linked it in comments
15stories submitted
129story points
2 months agolast shared
2023people per year2026
Most-shared posts
- 1The Scale of the Brain vs. Machine Learning (2022)2 people · 63 points · first shared 2023 · 36 comments on HN
- 2Integer Tokenization Is Insane (2023)2 people · 35 points · first shared 2024 · 8 comments on HN
- 3How Can LLM RL Work Despite Information-Theoretic Inefficiency2 people · 3 points · first shared 2026 · discussion on HN
- 4LLMs Confabulate Not Hallucinate2 people · 2 points · first shared 2023 · discussion on HN
- 5Right to Left (R2L) Integer Tokenization2 people · 2 points · first shared 2024 · discussion on HN
- 6Scaffolded LLMs as natural language computers1 people · 11 points · first shared 2023 · 1 comments on HN
- 7GPUs vs. Brains. Hardware and Architecture1 people · 2 points · first shared 2024 · discussion on HN
- 8AI Monotheism vs. AI Polytheism1 people · 2 points · first shared 2026 · discussion on HN
- 9Space Warfare Seems Mostly Defense Dominant1 people · 2 points · first shared 2026 · discussion on HN
- 10Intellectual Progress in 20251 people · 2 points · first shared 2026 · discussion on HN
- 11The Singularity as Cognitive Decoupling1 people · 2 points · first shared 2025 · 1 comments on HN
- 12Mitigating Reward Hacking as Institutional Design1 people · 1 points · first shared 2026 · discussion on HN
- 13Reflections on the Bitter Lesson (2020)1 people · 1 points · first shared 2023 · discussion on HN
- 14Capital Ownership Will Not Prevent Human Disempowerment1 people · 1 points · first shared 2026 · discussion on HN
- 152023 04 10 Why GOFAI failed1 people · first shared 2026
What HN people wrote when they linked it
krackers · 2026-03-09
Expert systems are basically decision trees which are "gofai" (good old fashioned ai) as opposed to deep learning. I've never really seen a good definition for what counts as "gofai" (is all statistical learning/regression gofai? What about regression done via gradient descent?). There's some talk in [1]
[1] https://www.beren.io/2023-04-10-Why-GOFAI-failed/
hyperbovine · 2025-08-09
> Sparse models for example show a lot of promise and more closely track real biological systems.
I think sparsity is a consequence of some other fundamental properties of brain function that we've yet to understand. Just sparsifying the models we've got is not going to lead anywhere, IMO. (For example it's estimated that current AI models are already within 1%-10% of a human brain in terms of "number of parameters" (https://www.beren.io/2022-08-06-The-scale-of-the-brain-vs-machine-lear…
kvee · 2024-09-05
On the other hand, it may be that "Alignment likely generalizes further than capabilities." - https://www.beren.io/2024-05-15-Alignment-Likely-Generalizes-Further-T…
mistermann · 2023-10-05
One rarely discussed peculiarity about religion is that it distorts reality for both believers and non-believers (as one can see by observing how anti-religious people discuss what religion "is", what religious people believe, etc).
Possibly relevant:
LLMs confabulate not hallucinate
https://www.beren.io/2023-03-19-LLMs-confabulate-not-hallucinate/
ijk · 2025-06-24
The math tokenization research is probably closest.
GPT-2 tokenization was a demonstratable problem: https://www.beren.io/2023-02-04-Integer-tokenization-is-insane/ (Prior HN discussion: https://news.ycombinator.com/item?id=39728870 )
More recent research:
https://huggingface.co/spaces/huggingface/number-tokenization-blog
Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs: https://arxiv.org/abs/2402.14903
https://www.beren.io/2024-07-07-Right-to-Left-Integer-Tokenization/