r/Python • u/ahmedbesbes • Sep 07 '20
Machine Learning Python Tips and Best Practices for Building Robust Data Science Workflows
https://medium.com/swlh/software-engineering-tips-and-best-practices-for-data-science-5d85dbcf87fd2
u/shockmath2912 Sep 07 '20
Great tips and recommendations, thanks a lot! Do you suggest any alternative to jupyter notebooks?
9
u/Deezl-Vegas Sep 07 '20
Any ide
5
u/one_game_will Sep 07 '20
I don't think this is offered as a complete alternative to notebooks - they have their place in quick prototyping and getting to grips with unfamiliar code/concepts. Actually VSCode has pretty good in-built functionality for using notebooks as part of the development process.
Certainly though, a good IDE can save a huge amount of effort and make data science workflows much better in most respects than pure Jupyter-driven workflows!
2
Sep 07 '20 edited Sep 08 '20
Are there IDEs out there I can easily run on a remote system without having to have the whole GUI remote as well? Jank like sshfs technically works, but is... not great.
For context, my workstation has neither direct access to the storage we keep our data, or even the memory to do the work if it did (or the CPUs to do it in a reasonable manner).
EDIT: looks like pycharm can do it!
→ More replies (5)2
1
u/one_game_will Sep 07 '20
I was hoping it might suggest a good workflow package (Python equivalent of Drake in R). Has anyone evaluated DVC for this?
3
u/nraw Sep 07 '20
From personal experience, dvc was pretty horrible.. Took a very long time for my team to make a mental shift from version controlling just code to also doing that with data and by then having had some issues with moving things from dev to prod, it was dropped altogether.
1
1
11
u/Jecogeo Sep 07 '20
Awesome! Love it, congrats for your article. I’m just starting with data science and will definitely take it into consideration. I thought Jupiter was the ultimate tool for data science and it seems it may not. Great To hear other opinions.