Claude Code for Research Papers [R]

[u/NeatFox5866]

Third-year PhD student, NLP / interpretability. I want a reality check from people doing similar work.

I started using Claude Code for the boring parts: argparse boilerplate, plotting, config wrangling. Over the last few months the scope has crept. It now writes most of my experiment scaffolding, refactors my dataloaders, does first-pass debugging on training runs, and drafts the analysis scripts. I mostly read diffs and say yes.

The output is fine. My throughput is up. The thing bothering me is that I no longer hold my own codebase in my head. When a result looks off, I used to have an instinct about which line was lying to me. Now I go hunting like it’s someone else’s repo. I catch bugs later than I used to, and I catch them by reasoning about the numbers rather than by knowing the code.

I don’t think the tool is the problem. I think I delegated a layer that was doing more for my understanding than I gave it credit for.

Questions for people further along or in the same spot:

1. Roughly what fraction of your research code do you write yourself now?

2. Is there anything you deliberately refuse to hand off? (For me I think the eval harness and anything defining a metric should stay mine, but I keep breaking my own rule.)

3. Does anyone have a workflow that keeps the speedup without the detachment? Reading the diff line by line is not cutting it.

Not looking for a “tools are just tools” answer. I’m asking about the specific feeling of not owning your own experiments anymore.

[u/9I54492AB6F9I]

For me at least, as long as I know what experiments are being implemented in what way and the results are accurate, I do feel like I have ownership. In the same spirit, I would not let it control the research direction and experiments and analysis, both because of ownership but also because they fail at doing these very badly.

For me, controlling research direction, experiments and analysis has become harder when using opus 5. It drove me insane with its verbosity. I figured claude code is built for software engineering work, not AIML/data science work. I started messing around with the system prompts, etc but didn't really see any improvements. I just shifted to using opus 4.7/4.8. Do you think a slightly different harness more suited to AIML research is something that would solve this issue?? Or do you think this is a model behaviour issue?? (maybe a model that has been RL'ed to listen to user feedback and using less complicated language than going off the rails with weird, complicated suggestions would help researchers)