Lilian Weng

I've found Lilian Weng's blog a great resource to get an intuition for the maths behind popular models and concepts. https://lilianweng.github.io/

I don't have any great recommendations and unfortunately my advice may be not what you want to hear. What I tell my students is "You don't need to know math to build good models, but you need to know math to know why your models are wrong." But this is even a contentious statement within the community. (Personally I'm more interested in exploring what we can build and understand rather than focusing on throwing more compute and data at problems. There's a lot of work to be done that does not require significant compute, but it isn't flashy and you'll get little fame. Every famous model you know has some unsung hero(s) who built the foundation before compute was thrown at the problem). I was previously a physicist and we similarly frequently express that you do not know the material unless you can do the math. Physicists are trained in generating analogies as they help communication but this sometimes leads to people convincing themselves that they understand things far more than they actually do. They say the devil is in the details, and boy are there a lot of details. (Of the science communicators, I'm happy those are the ones you mention though!) But do not take this as gatekeeping! These groups are often happy to help with the math and recommend readings. ML is kinda a while west and you can honestly pick a subdomain of math and probably find it useful, but I would start by making sure you have a foundation in multivariate calculus and linear algebra. As to paper…

There are some materials about "how to read scientific paper", like the pdf one from U waterloo [3] with some methodological advice. Some good advice in this old HN thread [1] But I don't see the point of reading a scientific paper unless you're actually curious about a specific topic. They are often hard to read, dense, have so many field-specific jargon that if you're new, you won't be able to read one paper and grasp everything. You would have to read references, or a book/blog that summaries core points. So find a specific field you're interested in, find a good book/blog/homepage/tutorial/video to get your basics going so that when you start reading papers you won't be completely lost. Then find a highly cited survey paper to understand what progress have been made beyond what is now basic. Then you can follow your curiously along that survey, decide a branch of research to read upon. You'll probably then realize that a few labs research/publish a lot in a specific direction. Now you can follow those professors (Twitter, Google scholar email notification) to keep up to date. By reading a lot you'll also start to notice papers that are "published just to get my PhD" and soon enough you can just read abstract + intro/result to judge if it is valuable or not. If ML/LLM is your curiosity probably Lillian Wengs blog [2] is a good start for tutorials / surveys. [1] https://news.ycombinator.com/item?id=24986727 [2] https://lilianweng.github.io/ Edit: direct…

Computation generally is partly moving data around, yes. What transformers do is learn how to move data around in a context-relevant manner. This greatly increases the expressivity of the kinds of computations they can perform over traditional deep nets. https://lilianweng.github.io/posts/2018-06-24-attention/ https://transformer-circuits.pub/2022/in-context-learning-and-inductio… https://transformer-circuits.pub/2021/framework/index.html#residual-co…

Lilian Weng's blog - a goldmine - has an in-depth post on this: https://lilianweng.github.io/posts/2024-07-07-hallucination/. She leads safety and alignment at OpenAI so might be worth checking out :^)

For those that don't know, Lilian Weng wrote one of the best "prompt engineering" howtos on the planet. It is beautiful in its succinct compactness. https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/

Here's a great article that links to a lot of research: https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/

GPT-4 should do significantly better, but still worse than expert humans, on the tasks you mentioned. They would also benefit from good prompting, such as those in - https://lilianweng.github.io/posts/2023-03-15-prompt-engineering - https://help.openai.com/en/articles/6654000-best-practices-for-prompt-… Not a large percentage of humans would be able to do the tasks listed above without significant experience or training either. Average humans are also not great at reasoning.

The "Strategies" section looks valuable. Here are a few more great resources from my notes (including one from Lilian Weng who leads Applied Research at OpenAI): - https://lilianweng.github.io/posts/2023-03-15-prompt-engineering - https://www.promptingguide.ai (check the "Techniques" section for several research-vetted approaches) - https://learnprompting.org/docs/intro

Somewhat amazed, dang, that this topic is not discussed more widely here or elsewhere. There is a lot of HPC and DS expertise out there which lacks understanding of ML system architecture (in the sense of the deployed machinery in toto). Her follow up post [1] is also recommended for those who (like me, are experienced but not in ML) finally had things click because of the OP writeup: Large Transformer Model Inference Optimization (2023) https://lilianweng.github.io/posts/2023-01-10-inference-optimization/ A very cool cite from that article is LLM.int8(): https://arxiv.org/abs/2208.07339

Yes, unfortunately. I speak only for myself though. I found some very good articles there and even then I couldn't help getting turned off when I read them because I associate them with the mountains of subpar material that get published there and the horrible reading experience. A curated personal lightweight website attracts me more: see https://lemire.me/blog/ (Wordpress) https://lilianweng.github.io (powered by https://gohugo.io)

Josh here -- cofounder of Gantry. Depends on the use case. If you're able to gather feedback from your users, then that's a great starting point. If not, then it can be helpful to use a proxy to detect data points that are more likely than average to be failures. For example, a simple proxy is your model's confidence, but this is a deep topic. You can look at the literature on active learning if you want to learn more. This blog post is a great starting point: https://lilianweng.github.io/posts/2022-02-20-active-learning/

I recommend this blog post: https://lilianweng.github.io/posts/2021-07-11-diffusion-models/ Personally, I find the core diffusion papers pretty dense and difficult to follow, so the blog post is where I'd begin. https://arxiv.org/pdf/1503.03585.pdf This paper is a decent starting point on the literature side, but it's a doozy. Both the paper and blog post are pretty math heavy. I have not yet found a really clear intuitive explanation that doesn't get down in the weeds of the math, and it took me a long time to understand what the hell the math is trying to say (and there are some parts I still don't fully understand!)

I follow these three for AI progress and concepts - http://karpathy.github.io/ (best one) - http://bjlkeng.github.io/ - https://lilianweng.github.io/lil-log/

I think the work on domain randomization for visual data is something that can be worth exploring. https://lilianweng.github.io/lil-log/2019/05/05/domain-randomization.h…

While Transformer's Self Attention(SA) is great, there are many applications where SA doesn't apply. For a more comprehensive overview of attention mechanisms, I often find myself coming back to Lilian Weng's post: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.ht…

Beyond those, my favorite resources are: - "An Introduction to Deep Reinforcement Learning" by Vincent François-Lavet et al (https://arxiv.org/pdf/1811.12560.pdf) - "A (Long) Peek into Reinforcement Learning" by Lilian Weng (https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-into-reinf… - "Deep Reinforcement Learning: Pong from Pixels" from Andrej Karpathy (https://karpathy.github.io/2016/05/31/rl/) Those are the basics. Some more resources listed on this post: https://news.ycombinator.com/item?id=18219620

I found this to be a great read on what I think the parent is referring to: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.ht…

In practice, beam search doesn't seem to work well for generative models. Temperature and top_k (two very similar parameters) were both introduced to account for the fact that human text is unpredictable stochastically for each sentence someone might say as such - as shown in this 2021 similar graph/reproduction of an older graph from the 2018/2019 HF documentation: https://lilianweng.github.io/posts/2021-01-02-controllable-text-genera… It could be that beam search with much longer length does turn out to be better or some merging of the techniques works well, but I don't think so. The query-key-value part of transformers is focused on a single total in many ways - in relation to the overall context. The architecture is not meant for longer forms as such - there is no default "two token" system. And with 50k-100k tokens in most GPT models, you would be looking at 50k*50k = A great deal more parameters and then issues with sparsity of data. Just everything about GPT models (e.g. learned positional encodings/embeddings depending on the model iteration) is so focused on bringing the richness of a single token or single token index that the architecture is not designed for beam search like this one could say. Without considering the training complications.

Interesting paper. I have a question about the claim in 6.2 that attention matrices are SPD, if you don't mind my asking. It seems to me that accepting the empirical result that the eigenvalues are positive isn't enough to get a Fourier Transform interpretation. Specifically, I don't understand the assumption that all attention matrices are symmetric. (I'm sure you know that positive eigenvalues are not enough by themselves, but for other folks reading, [[1 1/2] [1/3 1]] is a simple concrete example.) Consider Fig. 17 here: https://lilianweng.github.io/posts/2018-06-24-attention/ (this is Fig.1 in Attention is all you need). I understand that you get symmetric attention matrices for the self-attention matrix in the input stream, as well as the masked attention matrix in the output stream (the first block). But I don't understand how you claim symmetry for the final attention mechanism that combines input and output. And if you don't get symmetry, you don't get the Fourier Transform interpretation and all the nice algebra that follows.

https://yang-song.net/blog/ https://lilianweng.github.io/

I'm fairly certain that the model was not at all trained on language data. It's just that the "neural architecture" (specifically, the Transformer architecture) was first discovered in the NLP realm. If your question is why does an architecture designed for NLP work well for aminoacid sequences, it turns out that the Transformer architecture is surprisingly versatile. It works amazingly well for other sequence-like data (like audio), and even for images (which, to me, is surprising, since images are not a one dimensional sequence like text or audio). The Transformer architecture (Particularly, the Multihead self-attention[0] mechanism within the Transformer) is, in my estimation, once of the key innovations in deep learning in the past 5 years. It's used in pretty much everything in deep learning (GPT-3, AlphaFold, DALLE-2/Stable Diffusion, OpenAI's Whisper, come to mind). [0] https://lilianweng.github.io/posts/2018-06-24-attention/#multi-head-se…

Yes, this is an area of active research. For example https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/

Sounds like you need an agent system, some libs are mentioned here: https://lilianweng.github.io/posts/2023-06-23-agent/

Check out Lilian Weng's blog https://lilianweng.github.io/posts/2023-01-27-the-transformer-family-v…

See contrastive losses and siamese networks, like here (where they use L2 distance): https://lilianweng.github.io/posts/2021-05-31-contrastive/ If documents are similar, you want the two embeddings to be close to each other, if they are dissimilar, you want them to be far apart. Binary relevance judgements of course don't necessarily produce an ordering, but usually over a large enough set of training examples some will be better matches than others. "Learning to Rank" gets you into all kinds of labels and losses if you want to go down that rabbit hoel.

I am curious what additional attacks knowing the last layer of an LLM enables. Eg you go from a black box attack to some sort of white box [1] Does it help with adversarial prompt injection? What % of the network do you need to know to identify whether an item was included in the pretraining data with k% confidence? I assume we will see more of these and possibly complex zero days. Interesting if you can steal any non trivial % of model weights from a production model for relatively little money (compared to pretraining cost) [1] https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/

Good article, but I feel that it misses an important property of diffusion models that they model the score function (derivative of log prob) [1] and that diffusion sampling is akin to Langevin dynamics [2]. IMO these explain why it's easier to train these models than GANs, because of an easier modeling objective. [1] https://yang-song.net/blog/2021/score/ [2] https://lilianweng.github.io/posts/2021-07-11-diffusion-models/

Crash course: LLM: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-… Agent: https://lilianweng.github.io/posts/2023-06-23-agent/