Stanford CRFM Blog

They mostly aren't. The lack of transparency around how many parameters frontier models have and how long they're trained is a big obstacle when it comes to estimating the energy impact of training very large models. A group at Stanford has been benchmarking model providers by transparency here: https://crfm.stanford.edu/fmti/May-2024/index.html I think a great way to create positive change in the world is to pressure OpenAI, Anthropic, Google, XAI, and Meta to all share details about the energy cost of training and inference for their models. If every major provider provided this transparency, it would be less valuable to keep that info secret from a "keep your competitors in the dark" perspective. It would also allow customers to make decisions based on more than just performance and cost.

Ooooh this is very cool. So much amazing AI for music right now like https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.ht… - I wonder if it could be plugged in.

In LLM world, loss or perplexity may not be the best indicator of model performance :-( Perhaps HELM (https://crfm.stanford.edu/helm/latest/) but we didn't take deeper look as we are not the developers of this model

Foundation Models are a well known thing. From the stanford center for research on foundation models [1]: "What is a Foundation Model" In recent years, a new successful paradigm for building AI systems has emerged: Train one model on a huge amount of data and adapt it to many applications. We call such a model a foundation model. From their big paper [2] "A foundation model is any model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al.2019], GPT-3 [Brown et al. 2020], and CLIP [Radford et al. 2021]." [1] https://crfm.stanford.edu/ [2] "On the Opportunities and Risks of Foundation Models", Bommasani/Hudson/et al

Such logs would not be used for training the base model, but rather for fine-tuning the model for instruction following. Instruction tuning requires far less data than is needed for pre-training the foundation model. Stanford Alpaca showed surprisingly strong results from fine-tuning Meta's LLaMA model on just 52k ChatGPT-esque interactions (https://crfm.stanford.edu/2023/03/13/alpaca.html).

In terms of quality, I think BLOOMZ, or mT0, are the best open-source ones. The non-finetuned BLOOM does not appear favorably (in English) compared to GLM or OPT, which both have published weights: https://crfm.stanford.edu/helm/v0.1.0/?group=mmlu and Flan-T5 is above OPT-IML: https://arxiv.org/pdf/2212.12017.pdf > Is the future going to be controlled by big corporations who own the models themselves? On this subject, there is an effort stemming from BigScience to build an open, distributed inference network, so that people that don’t have enough GPUs at home can contribute theirs and get text generation at one word per second: https://github.com/bigscience-workshop/petals#how-does-it-work

I think there's a new approach for “How do you get the data?” that wasn't available when this article was written in 2015. The new text and image generative models can now be used to synthesize training datasets. I was working on an typing autocorrect project and needed a corpus of "text messages". Most of the traditional NLP corpuses like those available through NLTK [0] aren't suitable. But it was easy to script ChatGPT to generate thousands of believable text messages by throwing random topics at it. Similarly, you can synthesize a training dataset by giving GPT the outputs/labels and asking it to generate a variety of inputs. For sentiment analysis... "Give me 1000 negative movie reviews" and "Now give me 1000 positive movie reviews". The Alpaca folks used GPT-3 to generate high-quality instruction-following datasets [1] based on a small set of human samples. Etc. [0] https://www.nltk.org/nltk_data/ [1] https://crfm.stanford.edu/2023/03/13/alpaca.html

Stanford CRFM | Bay Area | Full-Time | On-Site, Hybrid Foundation models like ChatGPT, PaLM, and Stable Diffusion are transforming the world around us. The Stanford Center for Research on Foundation Models (CRFM; https://crfm.stanford.edu/), which is part of Stanford HAI, is an interdisciplinary initiative that aims to make foundation models more reliable, transparent and accessible to the world. We take on ambitious projects that seek to rigorously evaluate existing foundation models and to build new ones. We are currently seeking a research engineer to join our engineering team. This is an unique opportunity to work with seasoned engineers who have spent many years in industry as well as PhD students, post-docs, and faculty at CRFM. You will contribute to cutting-edge research, publish papers, gain access to the latest foundation models, and be immersed in the vibrant CRFM community. You will work on our open source software projects, including: - HELM, a framework for holistic evaluation of large language models - Levanter, a framework for transparent and accessible large-scale language model training For more information or to apply, please go to https://careersearch.stanford.edu/jobs/research-engineer-21381

It's worth pointing out that size isn't everything. From Meta's benchmarking [1] LLaMA 33B outperforms GPT-3 175B, Gopher 280B, Chinchilla 70B and even matches PaLM 540B on a bunch of common evals. Those interested in doing more comparisons can look at https://crfm.stanford.edu/helm/latest/?group=core_scenarios and https://paperswithcode.com/paper/llama-open-and-efficient-foundation-l… to see where it sits (with some GPT 3.5 and 4 numbers here: https://paperswithcode.com/paper/gpt-4-technical-report-1) I'd agree the secret sauce for how great the newest services perform is probably in the fine-tuning. We're seeing almost daily releases of fine-tuning data sets, training methods and models (at lower and lower costs) so I'm personally pretty optimistic that we'll be seeing some big improvement in self-hosted LLM performance pretty quickly. [1] https://ar5iv.labs.arxiv.org/html/2302.13971#:~:text=Table%203%3A,Sens…

It's not a bold assumption. t's the only assumption. We can't control the output of llms completely because we don't know how they generate. Nobody on earth has the faintest clue how all those 175 billion paramters are shaping the response to input. It doesn't matter bout "thinking" or whatever. Any black box system will be uncontrollable in essence. You can not make inviolable rules for a system you don't understand. And saying LLMs hallucinate because they don't understand anything is stupid. And just shows ignorance on your part. Models hallucinate because they're rewarded for plausibly guessing during training when knowledge fails. Plausibly guessing is a much better strategy to reducing loss. And the conclusion is obvious enough. Bugger smarter models hallucinate less because they guess less. That holds true. https://crfm.stanford.edu/helm/latest/?group=core_scenarios All the instruct tuned models on this list follow that trend. From Ada to Babbage to Curie to Claude to Davinci-002/003. Greater size equals Greater truthfulness (evaluated on TruthfulQA)

You could also distill the fabulous Anticipatory Music Transformer from Stanford. https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.ht… Also consider checking their decisions about representation, etc

Perhaps you missed work like https://crfm.stanford.edu/2025/05/28/fast-kernels.html ?

Anthropic has been the only AI company actually caring about AI safety. Here’s a dated benchmark but it’s a trend Ive never seen disputed https://crfm.stanford.edu/helm/air-bench/latest/#/leaderboard

That would be an interesting extension. MedGemma isn't part of the original benchmark either [1]. Since Gemini 2.0 Flash is on 6th place, expectations are for MedGemma to achieve higher than that :) [1]https://crfm.stanford.edu/helm/medhelm/latest/#/leaderboard

I think most of the instruction fine-tuning methods for oss models stem from Alpaca, so it should be included: https://crfm.stanford.edu/2023/03/13/alpaca.html And the one referenced in there on synthetic data generation: https://arxiv.org/abs/2212.10560

This kind of exists, but I doubt there are any commercial solutions based on it yet. https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.ht… Their paper says that they trained it on the Lakh MIDI dataset, and they have a section on potential copyright issues as a result. Assuming you don't care for legal issues, theoretically you could do: raw signal -> something like Spotify Basic Pitch (outputs MIDI) -> Anticipatory (outputs composition) -> Logic Pro/Ableton/etc + Native Instruments plugin suite for full song

Meanwhile there is the 'Foundation model transparency index' from Stanford https://crfm.stanford.edu/fmti/

Leaderboard - https://crfm.stanford.edu/helm/lite/latest/#/leaderboard Scenarios - https://crfm.stanford.edu/helm/lite/latest/#/scenarios Github - https://github.com/stanford-crfm/helm

I think it's about having massive data pipelines and process to clean huge amounts of data, increasing signal noise ratio, and then scale as other are saying having enough gpu power to serve millions of users. When Stanford researchers trained Alpaca[1][2] the hack was to use GPT itself to generate the training data, if I'm not mistaken. But with compromises, as it was like applying loose compression on an already compressed data set. If any other organisation could invest the money in a high quality data pipeline then the results should be as good, at least that my understanding. [1] https://crfm.stanford.edu/2023/03/13/alpaca.html [2] https://newatlas.com/technology/stanford-alpaca-cheap-gpt/

We leverage Flash Decoding (https://crfm.stanford.edu/2023/10/12/flashdecoding.html) in TensorRT-LLM to achieve 100 tokens per second on H100s.

From OpenAI's RLHF paper[1]: "By default, when we train a PPO model on our API distribution, it suffers from an “alignment tax”, as its performance on several public NLP datasets decreases." On the HELM[2] site, you can see accuracy benchmarks for InstructGPT <OpenAI model> vs baseline models. The InstructGPT models perform worse on a lot of benchmarks. 1 - https://arxiv.org/pdf/2203.02155.pdf 2 - https://crfm.stanford.edu/helm/v0.1.0/?group=question_answer...

"We are releasing AlpacaFarm, a simulator enabling everyone to run and study the full RLHF pipeline at a fraction of the time (<24h) and cost (<$200) w/ LLM-simulated annotators. Starting w/ Alpaca, we show RLHF gives big 10+% winrate gains vs davinci003 (http://crfm.stanford.edu/2023/05/22/alpaca-farm.html)"

Yes, the GPT's are literally trained to minimize (cross-)entropy of text. You can see the bits-per-byte of text for lots of models here: https://crfm.stanford.edu/helm/v0.2.0/?group=targeted_evaluations

I am not an expert but as others have said, the InstructGPT and Alpaca models do that https://arxiv.org/abs/2203.02155# https://crfm.stanford.edu/2023/03/13/alpaca.html

People really don't seem to understand just how far ahead OAI is in its reasoning abilities (https://crfm.stanford.edu/helm/v0.2.2/?group=reasoning) Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all

For those wondering why this is interesting: This technique is being used to reproduce[0] the Alpaca results from Stanford[1] with a few hours of training on consumer-grade hardware. I believe there will soon be a cottage industry of providing application-specific fine-tuned models like this, that can run in e.g. AWS very inexpensively. The barrier today seems to be that the base model (here, Meta's LLaMA) is encumbered and can't be used commercially. Someone will soon, I'm confident, release e.g. an MIT-licensed equivalent and we'll all be off to the races. [0] https://github.com/tloen/alpaca-lora [1] https://crfm.stanford.edu/2023/03/13/alpaca.html

"We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003." https://crfm.stanford.edu/2023/03/13/alpaca.html

Yeah, anybody can clone Alpaca and fine tune it. From what I can tell setting things up to train the model is easy. Then you have to generate the training data (can be done using GPT3's API), fine tune Alpaca, and then evaluate it. Haven't done it myself but I believe you can find more info here [1] [1] https://crfm.stanford.edu/2023/03/13/alpaca.html

From the Alpaca report (https://crfm.stanford.edu/2023/03/13/alpaca.html): > we watermark all the model outputs using the method described in Kirchenbauer et al. 2023, so that others can detect (with some probability) whether an output comes from Alpaca 7B https://arxiv.org/abs/2301.10226 > We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient open-source algorithm without access to the language model API or parameters. The watermark works by selecting a randomized set of "green" tokens before a word is generated, and then softly promoting use of green tokens during sampling. We propose a statistical test for detecting the watermark with interpretable p-values

> if these models are trained on essentially raw WWW sourced data, I would personally want raw output. Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html