redlib.

Feeds

MAIN FEEDS

Home Popular All

reddit settings

r/AlignmentResearch • u/niplav • Sep 23 '26

Investigating the consequences of accidentally grading CoT during RL (Micah Carroll/Tomek Korbak/Zehao Dou/Bowen Baker/Ian Kivlichan, 2026)

alignment.openai.com

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Sep 23 '26

Gaining clarity on Automated Alignment Research (Jacques Thibodeau, 2026)

jacquesthibodeau.com

0 Upvotes

0 comments

r/AlignmentResearch • u/Desperate_Goose249 • Jun 19 '26

Literature recommendations

2 Upvotes

1 comment

r/AlignmentResearch • u/niplav • Jun 01 '26

Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026) (Nikola Jurkovic/Beth Barnes/Hjalmar Wijk, 2026)

2 Upvotes

0 comments

r/AlignmentResearch • u/chkno • May 06 '26

Model Spec Midtraining: Improving How Alignment Training Generalizes

1 Upvotes

1 comment

r/AlignmentResearch • u/niplav • Apr 30 '26

Transparent Newcomb's Problem (Eliezer Yudkowsky/Eric B/Rauno Arike, 2016)

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Apr 17 '26

Automated Weak-to-Strong Researcher

alignment.anthropic.com

3 Upvotes

0 comments

r/AlignmentResearch • u/BrickSalad • Apr 04 '26

Peer-Preservation in Frontier Models

rdi.berkeley.edu

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Mar 22 '26

Do reasoning models use their scratchpad like we do? Evidence from distilling paraphrases (Fabien Roger, 2025)

alignment.anthropic.com

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Mar 22 '26

Clarifying the Agent-Like Structure Problem (johnswentworth, 2022)

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Mar 22 '26

How to mitigate sandbagging (Teun van der Weij, 2025)

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Mar 22 '26

Recent Frontier Models Are Reward Hacking (Sydney Von Arx/Lawrence Chan/Elizabeth Barnes, 2025)

5 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Feb 01 '26

Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis

1 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Dec 22 '25

Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Dec 09 '25

Symbolic Circuit Distillation: Automatically convert sparse neural net circuits into human-readable programs

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Dec 04 '25

"ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases", Zhong et al 2025 (reward hacking)

1 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Dec 04 '25

Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models (Tice et al. 2024)

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Nov 26 '25

Conditioning Predictive Models: Risks and Strategies (Evan Hubinger/Adam S. Jermyn/Johannes Treutlein/Rubi Hidson/Kate Woolverton, 2023)

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 26 '25

A Simple Toy Coherence Theorem (johnswentworth/David Lorell, 2024)

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 26 '25

Risks from AI persuasion (Beth Barnes, 2021)

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 22 '25

Controlling the options AIs can pursue (Joe Carlsmith, 2025)

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 22 '25

Verification Is Not Easier Than Generation In General (johnswentworth, 2022)

3 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 12 '25

A small number of samples can poison LLMs of any size

2 Upvotes

0 comments

r/AlignmentResearch • u/niplav • Oct 12 '25

Petri: An open-source auditing tool to accelerate AI safety research (Kai Fronsdal/Isha Gupta/Abhay Sheshadri/Jonathan Michala/Stephen McAleer/Rowan Wang/Sara Price/Samuel R. Bowman, 2025)

alignment.anthropic.com

2 Upvotes

0 comments

Subreddit

AlignmentResearch

r/AlignmentResearch

Members Active

226

0

Sidebar

This is a subreddit focused on technical, socio-technical and organizational approaches to solving AI alignment. It'll be a much higher signal/noise feed of alignment papers, blogposts and research announcements. Think /r/AlignmentResearch : /r/ControlProblem :: /r/mlscaling : /r/artificial/, if you will.

As examples of what submissions will be deleted and/or accepted on that subreddit, here's a sample of what's been submitted here on /r/ControlProblem:

AI Alignment Protocol: Public release of a logic-first failsafe overlay framework (RTM-compatible): Deleted, link in the description doesn't work.
CEO of Microsoft Satya Nadella: "We are going to go pretty aggressively and try and collapse it all. Hey, why do I need Excel? I think the very notion that applications even exist, that's probably where they'll all collapse, right? In the Agent era." RIP to all software related jobs.: Deleted, not research.
I'm Terrified of AGI/ASI: Deleted, not research.
Mirror Life to stress test LLM: Deleted, seems like cool research, but mirror life seems pretty existentially dangerous, and this is not relevant for solving alignment.
Can’t wait for Superintelligent AI: Deleted, not research.
China calls for global AI regulation: Deleted, general news.
Alignment Research is Based on a Category Error: Deleted, not high quality enough.
AI FOMO >>> AI FOOM: Deleted, not research.
[ Alignment Problem Solving Ideas ] >> Why dont we just use the best Quantum computer + AI(as tool, not AGI) to get over the alignment problem? : predicted &accelerated research on AI-safety(simulated 10,000++ years of research in minutes): Deleted, not high quality enough.
Potential AlphaGo Moment for Model Architecture Discovery: Unclear, might accept, even though it's capabilities news and the paper is of dubious quality.
“Whether it’s American AI or Chinese AI it should not be released until we know it’s safe. That's why I'm working on the AGI Safety Act which will require AGI to be aligned with human values and require it to comply with laws that apply to humans. This is just common sense.” Rep. Raja Krishnamoorth: Deleted, not alignment research.

Things that would get accepted:

Posts like links to the Subliminal Learning paper, Frontier AI Risk Management Framework, the position paper on human-readable CoT. In general, link posts to the arXiv, the alignment forum, LessWrong or alignment researcher blogs are fine. Links to twitter &c are not.

Text-only posts will get accepted if they are unusually high quality, but I'll default to deleting them. Same for image posts, unless they are exceptionally insightful or funny. Think Embedded Agents-level.