Maarten Grootendorst

This is amazing and I think will probably end up being a pretty important development. I was just reading this great breakdown of how diffusion Gemma works: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-diffusi… In reference to the difficulties with applying this to autoregressive LLMs - I wonder if these type of hybrids might be a good candidate for this approach.

A good visual explanation of how text diffusion models like DiffusionGemma work: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-diffusi…

Embedded within that developer page is a good explainer of the encoder free architecture . https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4…

https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4… (in a link from here: https://developers.googleblog.com/gemma-4-12b-the-developer-guide/, which was linked in the text of the post, but not the linkdump at the end).

It's not? There's an mmproj in the GGUFs released by ggml-org: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/main From the visual guide, there's still the 35M parameter embedder, then the linear projector, for vision, and the linear projector for audio, so it does have some parameters used for the multimodal input to project it into the LLM latent space: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4… And the Unsloth quants, which are missing this, don't support multimodal input. (edit: actually, I may have just needed to update my llama.cpp, will check with an updated llama.cpp soon) I'm downloading the ggml-org GGUFs now, I tried Unsloth but got some weird problems, double checking with the bf16 model to see if the issue was just the quant.

I've noticed that sometimes the same Claude model will make logical errors sometimes but not other times. Claude's performance is highly temporal. There's even a graph! https://marginlab.ai/trackers/claude-code/ I haven't seen anyone mention this publicly, but I've noticed that the same model will give wildly different results depending on the quantization. 4-bit is not the same as 8-bit and so on in compute requirements and output quality. https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantiz… I'm aware that frontier models don't work in the same way, but I've often wondered if there's a fidelity dial somewhere that's being used to change the amount of memory / resources each model takes during peak hours v. off hours. Does anyone know if that's the case?

For those who are text oriented: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-a… The paper author has a blog series but I don't think it's for general public https://tridao.me/blog/2024/mamba2-part1-model/

This skips over Bag of Words / N-Gram / TF-IDF and many other things, but paints a reasonable picture of the progression. 1. https://jalammar.github.io/illustrated-word2vec/ 2. https://jalammar.github.io/visualizing-neural-machine-translation-mech… 3. https://jalammar.github.io/illustrated-transformer/ 4. https://jalammar.github.io/illustrated-bert/ 5. https://jalammar.github.io/illustrated-gpt2/ And from there it's mostly work on improving optimization (both at training and inference time), training techniques (many stages), data (quality and modality), and scale. --- There's also state space models, but don't believe they've gone mainstream yet. https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-a… And diffusion models - but I'm struggling to find a good resource so https://ml-gsai.github.io/LLaDA-demo/ --- All this being said- many tasks are solved very well using a linear model and tfidf. And are actually interpretable.