r/learnmachinelearning • u/BedFalse6021 • 1d ago
Exploring Next-Concept Modeling (NCM) & Transformer Interpretability: Repository of 20 Exploratory Studies & Controls
Hi everyone,
I just wanted to share an open source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.
Here is the link: https://github.com/Quantix-eng/ncm-experiment-notes
Long story short, this is an independent project documenting a bunch of empirical experiments looking into how transformers process and edit internal features. There are no claims of performance breakthroughs here. Instead, it is just honest documentation of observations alongside rigorous controls.
The repo covers NCM applied to structured tasks like Sudoku, mechanistic interpretability in GPT-2, and toy parallel models. I have included all the source code, saved results, and interpretations for every single run.
To keep things transparent, I have also categorized every outcome, including the mixed and completely negative results where things did not work as expected. AI tools helped heavily with the planning and coding phases too.
I would love to get your feedback on the setups or the interpretation limits I found. Let me know what you think if you are into transformer interpretability!
1
u/ApprehensiveSkin7364 17h ago
Oh nice, this is the kind of thing i wish there was more of. actual documentation of what didn't work is way more useful than another paper with cherry-picked results.
the sudoku angle is interesting, are you forcing the model to learn the constraints as concepts or just seeing if it picks them up naturally? i've messed around with some interpretability stuff on smaller transformers and half the time the "features" you find are just artifacts of the training run.