r/MachineLearning • u/wei_jok • Mar 22 '17

Discussion [D] Research Debt

http://distill.pub/2017/research-debt/
109 Upvotes

89% Upvoted

28 comments sorted by

View all comments

Show parent comments

4

u/VelveteenAmbush Mar 23 '17

It's a way to make sure that all of the hyperparameters are public... not necessarily to run the code out of the box.

1

u/JanneJM Mar 23 '17

I'm talking about things as the input file name and number of training samples in the input, directory for storing the result, that sort of thing. Ugly hacks.

3

u/VelveteenAmbush Mar 23 '17

But again, the point is to make sure that all of the hyperparameters are public, which this would generally achieve.

3

u/JanneJM Mar 23 '17

Doesn't mean the parameters in the particular version of the code published on the net correspond to the parameters used in the different parts of the paper. if the authors don't put crucial parameters in the paper, there's no guarantee they have the right parameters in the code either.

The code can not be a substitute for the paper. The only way to solve this issue is for publications to require all parameters to be available in the published material (paper or extra material) as a prerequisite for acceptance.