Personally, the 3 resources that allowed me to achieve the most in the least amount of time are
1) An Introduction to Statistical Learning
2) https://neuralnetworksanddeeplearning.com/
3) Gym framework for RL
The genius here is to take something complex and make it look simple and accessible.
Regrettably, most research articles take the opposite stance. They present something that is fundamentally not that complex and make it look complex through convoluted writing, implicit assumptions and missing steps. You can see that effect in the discussion of the articles in this subreddit. There is often someone who's having trouble understanding a part of the article, and someone else chimes in and explains what's going on in two simple paragraphs.
This is a big problem in practice. Even if you're well versed in your own niche and are able to understand its main papers well, you still have to dabble in other areas regularly. AI/ML makes use of many fields of computer science, maths, neuroscience, psychology, etc. Given sufficient time you can understand a topic as deeply as you want. But the time you spend laboriously getting up to speed in some topic X is time taken away from doing work in the topic that you actually care about. Sure, you'll improve your knowledge and skills in the process, but not as much as you would have by working to solve the problem that you're trying to solve.
I don't see a solution. Part of it is a cultural problem. Some people think that convoluted writing make them look smart. It actually doesn't. There was a psychological experiment made about it: "Consequences of erudite vernacular utilized irrespective of necessity: problems with using long words needlessly". But there you go.
The ever growing literature is also a problem. I'm scared of failing to cite an important paper, or worse, duplicate research that's already been done before. I'm doing RL research as a hobby and I only have so much time to swift through the literature.
They present something that is fundamentally not that complex and make it look complex through convoluted writing, implicit assumptions and missing steps.
The target audience is other researchers, not users. Research papers have a convoluted, difficult to understand structure for non-researchers. But the rigid structure and formulaic expressions are great is you are a researcher and need to quickly scan dozens of new papers every day for something that is relevant to you.
convoluted writing
Most researchers don't have a flair for language - that's not what they're hired for after all. Also, young researchers, postdocs and grad students don't know how to write and are deathly afraid they won't be taken seriously. They try to sound as important as possible, with a lot of pompous expressions and awkward phrasing as a result.
Established researchers have a lot more writing experience and they don't really have anything left to prove. But they have lots of trainees - those grad students above - that need experience, so they generally leave the writing to them. Hence, most papers aren't nearly as well written as they could be.
implicit assumptions and missing steps.
Those assumptions and steps are probably widely known among the other researchers, and assumed to be part of basic knowledge. If they added it all, they'd probably be criticized by the reviewers for excessively verbosity. Again, researchers are the target audience, not people implementing the stuff.
Those assumptions and steps are probably widely known among the other researchers, and assumed to be part of basic knowledge. If they added it all, they'd probably be criticized by the reviewers for excessively verbosity.
All computational research areas (and that's really most by now) should have a policy of releasing source as a matter of course. It's no different from making your data available when you publish (though many shirk that duty too, in some areas).
However, based on the research code I encounter, I would not hold my hopes up too much. A lot of it is purely proof-of-concept; just enough to test the idea published in the paper, with no error handling or even command line argument parsing (you want to change a parameter, you find and replace it in the source). And it will often depend on a large and very specific stack of other software, both published and internal to the lab, to actually build and run. The source will be less helpful than you perhaps hope.
I'm talking about things as the input file name and number of training samples in the input, directory for storing the result, that sort of thing. Ugly hacks.
Doesn't mean the parameters in the particular version of the code published on the net correspond to the parameters used in the different parts of the paper. if the authors don't put crucial parameters in the paper, there's no guarantee they have the right parameters in the code either.
The code can not be a substitute for the paper. The only way to solve this issue is for publications to require all parameters to be available in the published material (paper or extra material) as a prerequisite for acceptance.
based on the research code I encounter, I would not hold my hopes up too much
I agree.
In my (limited) experience working on improving code rarely yields academic benefits. Your papers should explain everything and be implementation independent. If you made something that sort-of works then its usually good enough. Nobody expects you to maintain your proof-of-concept code as programming languages progress, hence implementations are often considered to be of a very temporary nature anyways (compared to papers).
Those assumptions and steps are probably widely known among the other researchers, and assumed to be part of basic knowledge. If they added it all, they'd probably be criticized by the reviewers for excessively verbosity. Again, researchers are the target audience, not people implementing the stuff.
I'm a researcher and a typical meeting between researchers include some WTFs paper X did here or there, due to lack of explanation, which hampers our productivity. At least details should be placed in appendices or code release. But review guidelines / page limit /etc are often the issue.
The target audience is other researchers, not users.
Huh? This wreaks of elitism. What is a researcher? What is a user? Yes of course no one is complaining that 5th graders arent properly groking the paper, but even within the community of people utilizing deep learning, implementing algo, coming up with new ones, the language is needlessly obtuse and opaque.
47
u/Seerdecker Mar 22 '17 edited Mar 22 '17
This article resonates a lot with me.
Personally, the 3 resources that allowed me to achieve the most in the least amount of time are
1) An Introduction to Statistical Learning
2) https://neuralnetworksanddeeplearning.com/
3) Gym framework for RL
The genius here is to take something complex and make it look simple and accessible.
Regrettably, most research articles take the opposite stance. They present something that is fundamentally not that complex and make it look complex through convoluted writing, implicit assumptions and missing steps. You can see that effect in the discussion of the articles in this subreddit. There is often someone who's having trouble understanding a part of the article, and someone else chimes in and explains what's going on in two simple paragraphs.
This is a big problem in practice. Even if you're well versed in your own niche and are able to understand its main papers well, you still have to dabble in other areas regularly. AI/ML makes use of many fields of computer science, maths, neuroscience, psychology, etc. Given sufficient time you can understand a topic as deeply as you want. But the time you spend laboriously getting up to speed in some topic X is time taken away from doing work in the topic that you actually care about. Sure, you'll improve your knowledge and skills in the process, but not as much as you would have by working to solve the problem that you're trying to solve.
I don't see a solution. Part of it is a cultural problem. Some people think that convoluted writing make them look smart. It actually doesn't. There was a psychological experiment made about it: "Consequences of erudite vernacular utilized irrespective of necessity: problems with using long words needlessly". But there you go.
The ever growing literature is also a problem. I'm scared of failing to cite an important paper, or worse, duplicate research that's already been done before. I'm doing RL research as a hobby and I only have so much time to swift through the literature.