Meta AI Blog

We are using RISC-V for AI accelerators to great success https://ai.meta.com/blog/meta-mtia-scale-ai-chips-for-billions/ RISC-V was a great choice due to being so customizable and extensible.

LLMs are ideology made code. "Alignment" is the act of making sure the technology reproduces the ruling ideology of the regime that produced it. There is no such thing as "unbiased". It's only ever "following our own biases". When Zuck released LLaMA 4, the announcement post stated that historically LLMs have "leaned left" and that they've been trying to "remove that bias". [0] Translated to the real world, that means that they've been trying to impart a right wing ideology to further the goals of the American ruling elite. LLMs are the perfect propaganda technology. Always consider the motivations of their makers. [0] https://ai.meta.com/blog/llama-4-multimodal-intelligence/

Meta released No Language Left Behind (NLLB) [1], I think in 2022. I wonder why this in not "NLLB 2.0"? These companies love introducing new names to confuse things [1] https://ai.meta.com/research/no-language-left-behind/

I have been looking for the same thing, either from Meta's SAM 3[1] model, either from things like the OP. There has been some research specifically in this area with what appears to be classic ML models [2], but it's unclear to me if it can generalize to dances it has not been trained on. [1] https://ai.meta.com/blog/segment-anything-model-3/ [2] https://arxiv.org/html/2405.19727v1

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any combo of them, imho). [EDIT] added "or communicate externally" to option C. [1] https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/ [2] https://ai.meta.com/blog/practical-ai-agent-security/

Has anyone read up on the recent paper from Meta/FAIR -- CWM: An Open-Weights LLM for Research on Code Generation with World Models Which looks to attempt to give better "coding" understanding to the model instead of mere tokens and positioning and hence improve the coding capabilities of these "brilliant but unpredictable junior engineer" coding agents: - https://ai.meta.com/research/publications/cwm-an-open-weights-llm-for-…

Things like facial recognition come to mind. Also probably poker: https://en.m.wikipedia.org/wiki/Libratus Diplomacy (game) was also pretty notable although best in the world is debatable https://ai.meta.com/research/cicero/diplomacy/ Even before that, computers have been superhuman at arithmetic for a while. Point 3 is satisfied when a machine is better than any human at any cognitive task though.

> You’ll note that nobody is open sourcing their ad recommender Actually... (2019) https://ai.meta.com/blog/dlrm-an-advanced-open-source-deep-learning-re… Source code: https://github.com/facebookresearch/dlrm Paper: https://arxiv.org/abs/1906.00091 Updated 2023 blog post, but solely for content recommendation, but ads recommendation is ~90% the same: https://engineering.fb.com/2023/08/09/ml-applications/scaling-instagra… It's a little out of date, but the internal one is built with the same concepts, just more advanced modeling techniques and data.

Worth mentioning that LlaMa 70b already had pretty high benchmark scores to begin with https://ai.meta.com/blog/meta-llama-3-1/ Still impressive that it can beat top models with fine-tuning, but now I’m mostly impressed by the fact that the 70b model was so good to begin with.

I used FAISS at work in the beginning of this year, and it was fantastic. Company I worked for has massive private medical datasets and will never agree to non-local models or methods. FAISS [0] is wonderful. Give it a try. You can work with FAISS with LangChain, llamaindex and the like. [0]: https://ai.meta.com/tools/faiss

Experimenting applying Meta's V-JEPA [0] architecture for representation learning to chess. One of the challenges is that validating if the model is learning useful dynamics of the game, so I'm using it as an excuse to learn some reinforcement learning by using the representations generated by the JEPA model to approximate useful Q-values [1]. This method currently has no search so I'm planning on comparing with this paper [2] which achieves GM level chess without any search. Honestly, Im unsure if the full pipeline is stable enough to even converge, but it's fun experimenting. I'm bad at chess so I really want to make a bot that challenges the best bots on lichess. [0]: https://ai.meta.com/research/publications/revisiting-feature-predictio… [1]: https://en.wikipedia.org/wiki/Q-learning [2]: https://arxiv.org/abs/2402.04494

Had to track through five links to find out what a "system card" is. It's an instance of a "model card", some broad headings to be filled in.[1] • Model Details. • Intended Use. • Factors. Factors could include demographic or phenotypic groups, environmental conditions, technical attributes, or others • Metrics. • Evaluation Data. • Training Data • Quantitative Analyses • Caveats and Recommendations Meta's PR describes a "system card", but doesn't provide one.[2] It doesn't, interestingly, seem to include the initial built-in prompts used as constraints. [1] https://arxiv.org/pdf/1810.03993 [2] https://ai.meta.com/blog/system-cards-a-new-resource-for-understanding…

Page 9 on the Llama tech report has an interesting graph that predicts task level performance from the cross-entropy loss. The sigmoidal model fits well, and at the steepest part of the S, a .01 change in NLL is worth about 5% task level accuracy. Here is a quick screenshot if you are lazy. https://snipboard.io/C6mipQ.jpg And here is the paper if you want to dig deep into the highest quality published research on LLM frontier. https://ai.meta.com/research/publications/the-llama-3-herd-of-models/

The blog post (https://ai.meta.com/blog/segment-anything-2/) mentions tracking as a use case. Similar objects is known to be challenging and they mention it in the Limitations section. In that video, I only used one frame, but in some other tests even when I prompted in several frames as recommended, it didn't really work, still.

"Meta AI", an AI assistant with a website at http://meta.ai/ , as opposed to "Meta AI", an artificial intelligence laboratory owned by Meta with a website at https://ai.meta.com/ https://web.archive.org/web/20240324073604/https://en.wikipedia.org/wi…

Lots of great details in the blog: https://ai.meta.com/blog/meta-llama-3/ Looks like there's a 400B version coming up that will be much better than GPT-4 and Claude Opus too. Decentralization and OSS for the win!

That wasn’t quite the epitome of generated video a year ago; it was barely trained for temporal coherence. But the best video generators were much worse than Emu Video; there was Make-A-Video[0] from Meta, and Phenaki[1] and Imagen Video[2] from Google. [0]: https://ai.meta.com/blog/generative-ai-text-to-video/ [1]: https://sites.research.google/phenaki/ [2]: https://imagen.research.google/video/

I listened to Call of the Wild and it was excellent. I think some of the intonation and contextual emphasis, tone etc could be improved. I suspect this is definitely the nail in the coffin for artisanal narration. Some of the recent SOTA models (below) do very well in these areas, so I’ll wager in 5 years all ebooks will be narratable via a local or cloud model as part of the reader. https://ai.meta.com/blog/voicebox-generative-ai-model-speech/ https://news.play.ht/post/introducing-playht2-0-the-state-of-the-art-g…

(Disclaimer: I am not an AI expert.) To me, LLMs are a special form "self-supervised learning"[1], which is a special form of unsupervised learning. While supervised learning aims to use input data to predict labels, self-supervised learning uses the context around the input data to predict future data. The key part here is context. This excellent article [2] by Yann Lecun et al. explains how self-supervised learning uses temporal and spatial context to extract patterns, which can then be used to predict future data or generate new data. In fact, prediction and generation go hand-in hand. So, you could definitely apply the techniques of LLMs to arbitrary streams of data with meaningful temporal or spatial patterns. [1] https://en.wikipedia.org/wiki/Self-supervised_learning [2] https://ai.meta.com/blog/self-supervised-learning-the-dark-matter-of-i…

Did anybody try the Llama 2 model with languages other than English? The paper notes that it works best with English and the amount of training data for other languages is only a fraction. Which likely would make it unusable for me.. See table 10 of the whitepaper for the numbers: https://ai.meta.com/research/publications/llama-2-open-foundation-and-… Are there other downloadable models which can be used in a multilingual environment that people here are aware of?

Basically under the actual terms of what open source is there can be no limits to particular users. Meta, in order to protect their competitive market share has said explicitly in Paragraph Two of their licensing terms --- " If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion. -- Doing so, technically, takes the license out of the category of “Open Source.” Sources: Open Source Initiate Argument https://blog.opensource.org/metas-llama-2-license-is-not-open-source/ Llama Terms https://ai.meta.com/resources/models-and-libraries/llama-downloads/

Seems people here on HN were lucky. Signed up ~20 minutes ago and no download link, so surely they're doing some checking, it's not fully automated, unlike what others say here. On Huggingface, the following disclaimer has been put in place: > This is a form to enable access to Llama 2 on Hugging Face after you have been granted access from Meta. Please visit the [Meta website](https://ai.meta.com/resources/models-and-libraries/llama-downloads) and accept our license terms and acceptable use policy before submitting this form. Requests will be processed in 1-2 days https://huggingface.co/meta-llama/Llama-2-70b-chat-hf/tree/main So seems it's a manual flow on their side to "accept" downloads.

AI is the perfect propaganda technology. I don't believe the current race to build AI is actually about any productivity gains (which are questionable at best). I believe the true purpose of the outsized AI investments is to make sure the universal answer machine will give answers that conform to the ideology of the ruling class. You can read hints of that in statements like the Trump AI Action Plan [0], but also things like the Llama 4 announcement. [1] [0] "Ensure that Frontier AI Protects Free Speech and American Values" - https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Acti… [1] "It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." https://ai.meta.com/blog/llama-4-multimodal-intelligence/

This is a bad article. Some of the information is wrong, and it's missing lots of context. For example, it somehow merged Llama 4 Maverick's custom Arena chatbot version with Behemoth, falsely claiming that the former is stopping the latter from being released. It also claims 40B of internet text data is 10B tokens, which seems a little odd. Llama 405B was also trained on more than 15 trillion tokens[1], but the post claims only 3.67 trillion for some reason. It also doesn't mention Mistral large for some reason, even though it's the first good European 100B+ dense model. >The MoE arch. enabled larger models to be trained and used by more people - people without access to thousands of interconnected GPUs You still need thousands of GPUs to train a MoE model of any actual use. This is true for inference in the sense that it's faster I guess, but even that has caveats because MoE models are less powerful than dense models of the same size, though the trade-off has apparently been worth it in many cases. You also didn't need thousands of GPUs to do inference before, even for the largest models. The conclusion is all over the place, and has lots of just weird and incorrect implications. The title is about how big LLMs are, why is there such a focus on token training count? Also no mention of quantized size. This is a bad AI slop article (whoops, turns out the author accidentally said it was AI generated, so it's a bad human slop article). [1]…

> What exactly does the evaluation entail? I believe the US AISI has published less on their specific approach, but they’re largely expected to follow the general approach implemented by the UK AISI [1] and METR [2]. This is mostly focused on evaluating models on potentially dangerous capabilities. Some major areas of work include: - Misuse risks: For example, determining whether models have (dual-use) expert-level knowledge in biology and chemistry, or the capacity to substantially facilitate large scale cyber attacks. A good example of this is the work by Soice et al on bioweapon uplift [5] or Meta's work on CYBERSECEVAL [6], respectively. - Autonomy: Whether models are capable of agent-like behavior, like the kind that would be hard for humans to control. A big sub-area is Autonomous Replication and Adaptation (ARA), like the ability of the model to escape simulated environments and exfiltrate its own weights. A good example is METR's original set of evaluations on ARA capabilities [3]. - Safeguards: How vulnerable these models are to say, prompt injection attacks or jailbreaks, especially if they're also in principle capable of other dangerous capabilities (like the ones above). Good examples here are the UK AISI's work developing in-house attacks on frontier LLMs [4]. Labs like OAI, Anthropic and GDM already perform these internally as they're part of their respective responsible scaling policies, which determine which safety measures they should have implemented…

Lol more like, Perplexity has a terminal diagnosis. Doing RAG using like.. prompt hacking and text embeddings + vector store when you have no access to the underlying model, nor ability to fine tune the generation for RAG, will fail. It will fail in an epic fashion compared to doing RAG the right way. What do I mean by RAG the right way? The RAG term has been overloaded. There's RAG that's just kind of bolted onto the LLM after it's been fine tuned for instruction following, and then there's RAG where the document/fact retrieval is a part of the LLM itself that is differentiated and optimized. Almost everyone is doing the first "hacky" kind of RAG, but Meta published in 2020, the "correct" way to do RAG, where you include a neural retriever in the feedback loop. Almost no one is doing this because it's more expensive (requires fine tuning the model), but will produce much better results than doing "bolted-on" RAG. https://ai.meta.com/blog/retrieval-augmented-generation-streamlining-t… Perplexity does not have access to any GPT model weights. It's unlikely they'll be able to compete on quality. It's game over.

Here's a summary of what's happened the past couple of years and what tools are out there. After ChatGPT released, there was a lot of hype in the space but open source was far behind. Iirc the best open foundation LLM that existed was GPT-2 but it was two generations behind. Awhile later Meta released LLaMA[1], a well trained base foundation model, which brought an explosion to open source. It was soon implemented in the Hugging Face Transformers library[2] and the weights were spread across the Hugging Face website for anyone to use. At first, it was difficult to run locally. Few developers had the system or money to run. It required too much RAM and iirc Meta's original implementation didn't support running on the CPU but developers soon came up with methods to make it smaller via quantization. The biggest project for this was Llama.cpp[3] which probably is still the biggest open source project today for running LLMs locally. Hugging Face Transformers also added quantization support through bitsandbytes[4]. Over the next months there was rapid development in open source. Quantization techniques improved which meant LLaMA was able to run with less and less RAM with greater and greater accuracy on more and more systems. Tools came out that were capable of finetuning LLaMA and there were hundreds of LLaMA finetunes that came out which finetuned LLaMA on instruction following, RLHF, and chat datasets which drastically increased accuracy even further. During this time,…

Thank you very much for the thoughtful and insightful reply! This is obviously speculation/intuition, but it's not terribly surprising to me at least that operating in e.g. pixel-space or a straightforward lifted latent manifold (modern diffusers basically) wouldn't have apparent structure under the fancy t-SNE type things that seem to be the heaviest artillery brought to the party (at least in the open). In pixel space, you get 6-17 fingers on 1-3 hands. The `france - paris + uk === london` thing is real, and it's not surprising because there aren't typically much in the way of nonlinearities in `word2vec`/`fasttext`/`glove` type stuff. But this substantially survives all the leaky relus or whatever in LLMs. They're pretty clearly interpolating in a way that you could get close to with a composition of affine transforms. JEPA (and maybe Sora if..., fuck it) seems a dramatic shift in forcing joint loss into a much higher-level space/manifold with (to me at least) shockingly semantic properties. I mean look at the I-JEPA reconstructions from pre-trained lifted space with some dinky diffuser/VAE-thing eating the hyperplane: https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/ That's not pixel space, and you've got a lot of freedom to make it smoother, I suspect no one says "L1 regularization" anymore, but there's some modern version of that, we know how to do this. AFAIU (and again, I welcome expert correction) TDA at least and really a lot of modern geometry is about…

There is a lot of interesting discussion already, but I think there is also some (perhaps unnecessary) disagreements regarding licensing arising from what I believe are the very different backgrounds in the community. Thus, I will attempt to summarise my own perspective as someone that has been active in the artificial intelligence community for nearly 15 years at this point. Firstly, I will state that I am in full agreement with LeCun on what he states in the video about the need for AI systems such as LLMs to be open [1]. I am not sure if I entirely agree with his very high optimism regarding AI safety, but I have always been far more concerned about current risks and I suspect that in practice we are very much aligned. [1]: https://yewtu.be/watch?v=EGDG3hgPNp8&t=6525s Now, let me move to the issue of licensing that I have already talked about at great length in comments on other, older submissions. So let me do something new here and try to put it into a historical context. Computer science and machine learning in particular has a tradition of openness that goes beyond that of many other fields. What I mean by this is that over the last twenty or so years, there has been a gradual move towards open sharing of research, research outcomes, etc. All under the umbrella of open science and FLOSS. One of the earliest examples I am aware of (there may be others) is how the Journal of Machine Learning Research was established as a direct reaction to the resistance against…

Facebook details the conditions that might terminate the license, and they do not invoke the right to do so at any time or for any reason. Per their license [1], they are not allowed to revoke the license unless you violate the terms of the license. And with respect to complaints they might receive, the only sort I can think of would be with respect to content people find objectionable. There is no content-based provision or restriction in the license except that applicable laws must be followed. Provided you're following the law, the license doesn't seem any more revocable & thereby risky for use than any other open resource made available by a corporation. Facebook is just as bound by this license as they would be if they required commercial users to pay them $1M to use the model. I think this release is less about direct financial gain and more about denying large competitors a moat on the issue of basic access to the model, i.e., elevating the realm of competition to the services built on top of these models. Facebook appears to be betting that it can do better in this area than competitors. [1] https://ai.meta.com/resources/models-and-libraries/llama-downloads/