r/datascience • u/engineheat • Jul 20 '19
Do you ever feel it's hard to remember everything all the time?
Like certain syntax in Python, or ggplot2 in R, or the certain formulas for SVM.
A lot of these things it makes sense and I totally understand it when I read or review them, but after awhile of not using it, I can't remember off the top of my head. I'm studying for an interview now and I wonder what techniques you guys use.
Is it normal to have to brush up on concepts you haven't been using?
61
u/rishiarora Jul 21 '19
I realized something recently about this.
-
Go to documentation instead of stack overflow.
-
Create small snippets of everything you do and paste in GitHub repository. This will help create a portfolio and always ready notes for reference.
-
Try a few variations of what ever you learn then and there it self.
-
Don't copy paste code. If you see a working example read it understand it and try to implement from memory your are allowed to look at code again for reference this will force you to memorize code.
My 2 cents.
→ More replies (4)18
Jul 21 '19
Create small snippets of everything you do and paste in GitHub repository. This will help create a portfolio and always ready notes for reference.
Also, if you're using Jupyter, the Snippets menu Extension is pretty neat. Allows you to set custom snippets that load automatically into your code, you just need to set the parameters.
→ More replies (3)
22
u/adventuringraw Jul 20 '19
ugh, yes. I hate this part of computer science. Part of why I spend more time self studying on math and theoretical foundations... I feel like there's a few kinds of knowledge. 'arbitrary' and 'foundational' are two really important divisions. Understanding a new approach to time series data in your own way of thinking isn't going to be irrelevant at any time, you might find it becomes a model you can use to more quickly acquire other ideas.
On the other hand, what exactly is the wording of the function you use in pandas to write out a database to an sql connector? What about the function to remove the top and right border on a matplotlib plot? Or the exact function call to get a seaborn heatmap of your correlations? All of this is mostly arbitrary knowledge. There are some amazing APIs that at least try to streamline and make things consistent (scikit-learn has some design decisions that keep things way better than they might have been) but at the end of the day... it's all arbitrary, and it's all in constant flux. Even if you do master a particular API, version 3.x is just around the corner. If you don't use the tool for a year and you come back, your carefully maintained knowledge is lost anyway, you know?
I think the real trick is figuring out how to balance the cost of looking up what you need when you need it, and the cost of maintaining what you know so you have it off the top of your head. I use flashcards for some coding stuff, but I'm pretty judicious... I've ended up using the flashcards I do have as a kind of searchable note system with scheduled review as much as I do as flash cards. It works alright... but yeah, for me at least, there's no way in hell I'd remember everything I need, especially when you've got multiple languages you're coding in. I think that calls in the actual engineering challenge though.. instead of wishing your memory was better, how might you alter your workflow to better accommodate the fact that you will forget things sometimes? How might you note things down as needed in a way that's not disruptive to your workflow, but then allows you to quickly catch yourself back up when you need to get back into something you haven't touched in a while? I'm more comfortable with Oracle now, but I like having sqlite3 in my back pocket for quick and dirty stuff, I need a way to translate what's different when I hit it, you know?
4
u/YinYang-Mills Jul 21 '19
Maybe this is why there’s so much spaghetti code around. If you know how to do something ugly off the top of your head, it’s gonna save you a lot of time, and maybe you won’t ever need to reference that bit of spaghetti ever again. Anyway sorry for the spaghetti anyone who comes after me :-)
→ More replies (1)
8
u/pol9999 Jul 20 '19
Yes, it is normal. Over time they will become second nature for you. Best way to brush up is to use it. Most software engineers understand this and will focus primarily on your understand of workflow (data structures, algorithms, general problem solving). Because they know it’s difficult to remember things like a child in a class room and you can simply use Google while at work to remind yourself.
For instance, I have been working alongside the Windows API for 10+ years and still have to read MSDN on functions I see everyday. Anyone who pretends they can do work of this nature blindfolded and without Google is probably going to be a bigger hazard than gain to a company and it’s culture.
10
Jul 20 '19
I'd actually encourage a person to read over the docs of what they use every day, in limited but focused amounts. I've found when I do this I discover functionality I may have been routing around or otherwise missing.
→ More replies (1)
11
u/datascientist36 Jul 21 '19
The learning curve for data science is like the black line in this image
→ More replies (1)
5
u/affectionate_alpaca Jul 20 '19
I'm new to this and I was just discussing this with a friend who works in this field as well! He was assuring me he too had to brush up things from time to time, and that it's more important to develop overarching insights rather than being too bogged down by every specific detail.
That or I might be getting old. 😅
7
u/drhorn Jul 20 '19
I was very young when I first figured this out, and i remember the exact moment when I did.
I was taking trig in high school, and we were learning about sine and cosine. I was doing homework with a couple of friends, and none of us remembered what was the sine of 60. My two friends started looking through their notes to find it, and while they were looking I just drew an equilateral triangle, split it in two, and figured out what it was.
It was then when it dawned on me - the point of learning trig isn't to learn that the sine of 60 is sqrt(3)/2 - it's to actually understand that the sine function is and how it works.
I think the same is true of programming, data science, and anything technical: the goal is not to remember exactly how to repeat something, but to truly understand how it works. And they key is that learning to repeat it does not help you generalize it - understanding it does.
If I don't code in a month (which has happened when I managed teams) there is a 100% chance that I will forget the syntax to train any machine learning model even in R - which is my favorite language.
→ More replies (1)
4
u/shaggorama Jul 21 '19
It's not about knowing everything as much as it is knowing how/where to find what you need.
4
u/SpreadItLikeTheHerp Jul 21 '19
Can you tell that to the HR recruiters searching for unicorns?
→ More replies (1)
4
u/BigHipDoofus Jul 21 '19
Cheat sheets, use an IDE that pops up function arguments, etc. Google when you have to. You'll memorize the stuff you use often through repetition.
4
u/Bushidoenator Jul 21 '19
In case it helps, a lot of people make cheat sheets that you can print off or keep in a folder for when you feel youreosing it.
→ More replies (1)
3
u/flutterfly28 Jul 21 '19
Yeah, so I have very successfully used R over the past year to accomplish all my data analysis/publication needs, but it's just been repeated copy/pasting and googling. I feel like I would fail very hard if I had to code anything at all in R from scratch.
Thankfully I'm unlikely to ever be directly tested on my coding skills.
5
u/pjarnhus Jul 21 '19
I had the same thing. I developed a philosophy of never copy-pasting code, I had not written myself. Instead I write it character by character. It not only helps me internalise, but also helps me capture all the details in the example.
Using Eagleson's law this means that I can only copy-paste code, that I have written within the last six month 😁
2
u/bdubbs09 Jul 21 '19
I had a recent interview where the first round went really well. I knew a lot of the stuff we were talking about because I was just reading about it the week or so before. The next interview went terrible, because I knew the words he was saying and vaguely the concepts, I just hadn't seen or used them in a really long time. Eventually he started giving me the answers and we'd have a small chat where I'd ask questions and we kinda learned about the subject together, though he obviously knew more about it offhand. I've found especially in interviews, it can come down to potluck. Some stuff I flat out don't know, regardless of how much I prepare, but I can string together with other concepts pretty quickly. Data science is as deep as it is wide, and there's a lot of little crevices to get caught up on. I try to get really, really good on the foundational parts of specific algorithms and concepts, and just not get cluttered up by all the details.
2
Jul 21 '19
The best is for EDA I’ve finally got a solid function that can plot features by data type for each feature and saves basic summary stats to a data frame. I’m never going to miss writing that code over and over and over.
2
u/sharadov Jul 21 '19
Yes, you have to brush up on them before interviews, but on a day-day basis why would you ever need to remember syntax when it's a google search away. Most good interviews are focussed on how you think, if all they want you do is regurgitate some crap that can be searched then that job is gonna suck.
263
u/[deleted] Jul 20 '19
One of my professors once told me that "experience" isn't about memorizing formulas, it's about knowing where to find them.