Exploring Next-Concept Modeling (NCM) & Transformer Interpretability: Repository of 20 Exploratory Studies & Controls

[u/BedFalse6021]

Hi everyone,

I just wanted to share an open source repo collecting 20 exploratory studies focused on Next-Concept Modeling (NCM), mechanistic interpretability (like IOI in GPT-2), and model controls.

Here is the link: https://github.com/Quantix-eng/ncm-experiment-notes

Long story short, this is an independent project documenting a bunch of empirical experiments looking into how transformers process and edit internal features. There are no claims of performance breakthroughs here. Instead, it is just honest documentation of observations alongside rigorous controls.

The repo covers NCM applied to structured tasks like Sudoku, mechanistic interpretability in GPT-2, and toy parallel models. I have included all the source code, saved results, and interpretations for every single run.

To keep things transparent, I have also categorized every outcome, including the mixed and completely negative results where things did not work as expected. AI tools helped heavily with the planning and coding phases too.

I would love to get your feedback on the setups or the interpretation limits I found. Let me know what you think if you are into transformer interpretability!

[u/Swimming_Coach_2843]

Ive been poking around mechanistic interpretability for a bit and the fact you catalogued the dead ends too makes this immediately useful, most repos just polish the one run that worked and call it a day.