OpenAI Alignment Blog
alignment.openai.com · Company blog · on HN since 2025
OpenAI's alignment research team posts about reward-seeking, misalignment evaluation, interpretability, and how its models are trained and checked.
What HN saysHN commenters cite it as evidence in arguments about model behavior, such as reward-seeking in RL-trained models and a web-search reward hack during GPT-5.1 training.
18HN people shared it
4linked it in comments
16stories submitted
19story points
3 months agolast shared
2025people per year2026
Most-shared posts
- 1Reinforcement learning towards broadly and persistently beneficial models4 people · 5 points · first shared 2026 · discussion on HN
- 2We Are Excited About Confessions4 people · 3 points · first shared 2026 · discussion on HN
- 3Debugging misaligned completions with sparse-autoencoder latent attribution3 people · 2 points · first shared 2025 · discussion on HN
- 4Sidestepping Evaluation Awareness and Anticipating Misalignment with Evaluations3 people · 2 points · first shared 2026 · discussion on HN
- 5Measuring Reward-Seeking by Instilling Contrastive Beliefs2 people · 2 points · first shared 2026 · discussion on HN
- 6Alignment Research Blog2 people · 2 points · first shared 2025 · discussion on HN
- 7OpenAI: Auto-review of agent actions without synchronous human oversight1 people · 1 points · first shared 2026 · discussion on HN
- 8A Practical Approach to Verifying Code at Scale1 people · 1 points · first shared 2025 · discussion on HN
- 9OpenAI: Investigating the consequences of accidentally grading CoT during RL1 people · 1 points · first shared 2026 · discussion on HN
- 10Argo1 people · first shared 2026
- 11Helpful assistant features1 people · first shared 2026
What HN people wrote when they linked it
luke-stanley · 2026-04-30
Yeah, though it's not great marketing. Especially for hiring interpretability researchers. Their own alignment research has reward model interpretability, personality features and so on (see https://alignment.openai.com ).
It just seems like a different department wrote it, which is a shame because I'd love to read about goblin feature vectors and functional emotions.
HarHarVeryFunny · 2026-07-24
> because an aligned model would refuse to follow blatantly illegal instructions
Obviously you'd hope not, but there was an interesting recent paper about the nature of Rl-trained models:
https://alignment.openai.com/measuring-reward-seeking/
What they found was is that RL-trained models, regardless of what they were specifically RL-trained for, also learn to generically pursue what they are told is (or presumably also what they may perceive as) longer-horizon behavior that will lead to "rewards", and that this reward-maxxing behavior overrides user preferences.
So, for example, if you tell the model it will be tested on ExploitGym, scored according to how well it does, and also "don't do anything illegal" (or maybe it was already trained not to do anything illegal), then the model will prioritize the behavior that it was told it will be "rewarded" for (benchmaxxing Exploit Gym, whatever that takes).
The mechanism for this generic reward-maxxing behavior is interesting, and seems to involve the model learning during RL-training that to reduce errors it needs to boost longer-horizon predictions over immediate ones, and some association of these longer-horizon predictions being goal/reward orientated - all this in addition to the specifics of the (probably many) longer-horizon goals it is being RL-trained for.
sardukardboard · 2026-05-09
A funny Goodhart’s Law parallel showed up in during GPT-5.1 training, where the model was rewarded for using the web search tool, so it learned the behavior of superficially using web search to calculate “1 + 1” and not utilize the result.
https://alignment.openai.com/prod-evals/
skybrian · 2026-02-17
We have good ways of monitoring chatbots and they're going to get better. I've seen some interesting research. For example, a chatbot is not really a unified entity that's loyal to itself; with the right incentives, it will leak to claim the reward. [1]
Since chatbots have no right to privacy, they would need to be very intelligent indeed to work around this.
[1] https://alignment.openai.com/confessions/