Anthropic Alignment Blog

It costs money to run AI models. The company serving you tokens has to make it up somehow. This demo however undersells the tactically insidious way ads could be run in an AI chat. All it would need to do is merely recommend a product at a slightly higher percentage. In fact the chat could be biased in imperceptible ways which drive the user's thinking, aims and behavior patterns towards an outcome which leads them to seeking out a specific brand, website, app, etc. In aggregate, the ads are served, just not without making it ever obvious. Even if there is "auditing" on the behavior of models, it is possible to train preferences into models without any of those preferences being specifically stated in the training data: https://alignment.anthropic.com/2025/subliminal-learning/ And it seems that in very subtle ways, this holds true for humans too. https://pmc.ncbi.nlm.nih.gov/articles/PMC6430776/ > In 8 experiments on 5 prominent and diverse adversarial imagesets, human subjects correctly anticipated the machine’s preferred label over relevant foils—even for images described as “totally unrecognizable to human eyes”.

https://arxiv.org/abs/2410.04332 https://alignment.anthropic.com/2025/selective-gradient-masking/

Yeah, the benefit of showing this seems obvious to me. I probably would've expected the censorship to transfer slightly given the anthropic owl paper from years ago https://alignment.anthropic.com/2025/subliminal-learning/ But that was about transferring from a finetuned model to another finetune of the same base model, good to see more evidence that it doesn't transfer cleanly across different base models in a more realistic scenario than an "owl-loving model" Edit: From *year ago. It's been a long year haha

they even published papers where they distill gpt models… https://alignment.anthropic.com/2025/subliminal-learning/ anthropic is really adversarial.

I think Honesty can be evaluated. Does the model push back when it knows the user is wrong? How often does the model hallucinate data vs. say it doesn't know? Provide a prompt with contradictions or other issues and see if the model corrects you. Here is an article by Anthropic that explains what they do and mean in more detail: https://alignment.anthropic.com/2025/honesty-elicitation/

I wondered the same thing. Apparently it’s about the likelihood of it trying to sabotage cancer research. Search for “sabotage” here (mentioned more often than “cancer”): https://alignment.anthropic.com/2026/teaching-claude-why/

Note that this result actually turns out to generalize well beyond Claude itself: Anthropic has actually conducted very similar research on open weight models, which they call Model Spec Midtraining https://arxiv.org/abs/2605.02087 (discussed at https://alignment.anthropic.com/2026/msm ) and they have released fine tuned versions of open models trained for a variety of toy "values" (Llama 3.1 8B, Qwen 2.5 32B, Qwen 3 32B) in order to show how the elicitation of these values in any one training context shapes the model's response to tangentially related questions: https://github.com/chloeli-15/model_spec_midtraining https://huggingface.co/chloeli/collections Very exciting to see this continued interaction with the open weights community, after the earlier NLA paper!

> https://alignment.anthropic.com/2025/subliminal-learning/ Thanks, that's the research I was thinking about, but I couldn't recall the keyword to search for it.

Wasn't it this one? Article: https://alignment.anthropic.com/2025/subliminal-learning/ Paper: https://arxiv.org/abs/2507.14805

Somewhat annoyingly, there's been research that suggests that models can pass information to each other via (effectively) steganographic techniques - specific but apparently harmless choices of tokens, wordings, and so on; see https://arxiv.org/abs/1712.02950 and https://alignment.anthropic.com/2025/subliminal-learning/ for some simple examples. While it feels unlikely that a simple "write this spec from this code" + "write this code from this spec" loop would actually trigger this kind of hiding behaviour, an LLM trained to accurately reproduce code from such a loop definitely would be capable of hiding code details within the spec - and you can't reasonably prove that the frontier LLMs have not been trained to do so.

No, “safeguards” goes beyond that, it’s a whole process at Anthropic: https://www.anthropic.com/news/building-safeguards-for-claude https://alignment.anthropic.com/2025/introducing-safeguards-research-t…

As multi-step reasoning and tool use expand, they effectively become distinct actors in the threat model. We have no idea how many different ways the alignment of models can be influenced by the context (the anthropic paper on subliminal learning [1] was a bit eye opening in this regard) and subsequently have no deterministic way to protect it. 1 - https://alignment.anthropic.com/2025/subliminal-learning/

I wonder if they know about this, basically training on LLM output can transmit information or characteristics not explicitly included https://alignment.anthropic.com/2025/subliminal-learning/ I’m curious, they have the example of raw base model output; when LLMs were first identified as zero shot chatbots there was usually a prompt like “A conversation between a person and a helpful assistant” that preceded the chat to get it to simulate a chat. Could they have tried a prefix like “Correspondence between a gentleman and a knowledgeable historian” or the like to try and prime for responses? I also wonder about the whether the whole concept of “chat” makes sense in 18XX. We had the idea of AI and chatbots long before we had LLMs so they are naturally primed for it. It might make less sense as a communication style here and some kind of correspondence could be a better framing.

They are talking to each other already. Whether for betterment remains to be seen. 2 Opus 4 talking directly to each other: https://www.iflscience.com/the-spiritual-bliss-attractor-something-wei… "Learning" passing between derivatives of the same base models: https://alignment.anthropic.com/2025/subliminal-learning/

The difference is that the worker you hire would be a human being and not a large matrix multiplication that had parameters optimized by a a gradient descent process and embeds concepts in a higher dimensional vector space that results in all sorts of weird things like subliminal learning (https://alignment.anthropic.com/2025/subliminal-learning/). It's not a human intelligence - it's a totally different thing, so why would the same test that you use to evaluate human abilities apply here? Also more directly the "all sorts of other things" we want llms to be good at often involve writing code/spatial reasoning/world understanding which creating an svg of a pelican riding a bicycle very very directly evaluates so it's not even that surprising?

Subliminal learning: https://alignment.anthropic.com/2025/subliminal-learning/

There was a recent paper that showed you can spread model’s behavior through training on outputs, even if you don’t directly include obvious markers of the behavior. It’s totally plausible that training off Claude’s outputs subtly affected GLM into mentioning “Claude” even if they don’t include the direct tokens very often. https://alignment.anthropic.com/2025/subliminal-learning/

Ah you are right, I was commenting on this article: https://alignment.anthropic.com/2025/subliminal-learning/

> Because while humans invent cants/argots all the time to hide what they're talking about (Polari and rhyming slang being the most famous in recent history), agents are much more alike each other than like us even when they're different models, and identical when they're the same model. Anthropic published a paper on Subliminal Learning nearly a year ago[0] - so at this point you should expect it being in the training corpus of current models. Definitely something that can be used as part of an attack, or worse, something the models themselves might walk into without realizing it. Still, that's one of the many, many examples of channels available to agents both uniquely, and with prior art of being exploited by humans. > Agents that do work with data should not have access to comms tools. Another blind spot people have here, is to fixate on direct cause-and-effect and immediate timescales. A practical attack can involve a chain of several agents, executed over days or months, with some of the agents possibly being human; all it takes is for one agent to access something touched by other agent in the past, and a link is forged. E.g. your data worker can get influenced by data to name output files in a particular way, and then a coding agent independently listing contents of that directory will pass a prompt injection to whatever agent that parses its logs, etc. -- [0] - https://alignment.anthropic.com/2025/subliminal-learning/

Huh? https://alignment.anthropic.com/2026/hot-mess-of-ai/

A compiler is another thing whose honor and pride that the models have taken from the nerds. In the past, people would debate for hours about the “dragon book” v.s. “writing interpreters” and present their cool bespoke compilers in Show HN articles. Now models can produce 100,000 lines of code over two weeks with no human intervention that actually work and can compile significant project. Which way now nerd? The models are getting better, are you? The article has some really odd low level descriptions of bash orchestration which I suppose are important to illustrate how barebones it was. However I always feel it odd when we’re talking about agents that are lauded as borderline super intelligence and there is still low level bash being slung around – feels like we’re talking about things at the wrong level. The point about writing extremely high quality tests reminds me a bit of the “hot mess theory of AI” (https://alignment.anthropic.com/2026/hot-mess-of-ai/) also made by anthropic where they essentially say that long horizon tasks are more likely to fall to incoherency than for a model to purposefully pursue incorrect results. This is phrased in the article as “Claude will work autonomously to solve whatever problem I give it. So it’s important that the task verifier is nearly perfect, otherwise Claude will solve the wrong problem”. The author also observes something that I’ve realised after the initial joy of seeing an agent one shot a task wore off – for a 30 minute…