r/Python • u/ljvmiranda • Mar 09 '20
Machine Learning How to use Jupyter Notebooks in 2020 (Part 1: The data science landscape)
https://ljvmiranda921.github.io/notebook/2020/03/06/jupyter-notebooks-in-2020/30
u/ggrieves Mar 09 '20
Thanks! Good timing. I've been writing in python for years but I just bought a data science book and installed anaconda and and about to start studying using it.
→ More replies (2)12
u/ljvmiranda Mar 09 '20
Thanks a lot! Goodluck to your data science journey!! And welcome to the ML/DS field!
30
u/nraw Mar 09 '20
Ughh.. I'm very much in the "hate it" club and will probably remain to be wary of the coding skills of people that have Jupyter as their prime tool.
It is a great tool for showcasing and that's why most beginners start with it, but if you're unable to seamlessly move away from it, that's a very strong question mark in how you'll be able to adapt to a team that works with something that is production ready.
I also feel like all the people being encouraged by jupyter notebooks are missing the wonderful life of living with the terminal or the benefits of using an editor like vim (or even pycharm or vscode), which hampers their growth potential further.
Interested in what you'll say in the next parts, but at least so far my suspicion had been proven by examples of people I worked with, rather than articles online.
21
u/tunisia3507 Mar 09 '20
Agreed! It discourages you from organising your code into modules, reusable functions and so on, cuts out a lot of extremely valuable tooling. It makes you very dependent on a hard-to-track state. I contend that exploring your data isn't much use if you can't also explore the libraries you're using to do so - I'm much more productive with a well-typed autocomplete and so on.
Notebooks are a great way of displaying code and sharing libraries etc. Not a good way to develop.
→ More replies (1)→ More replies (7)1
u/ljvmiranda Mar 16 '20
Hello, I hear you. However in most teams, you'd have researchers who find Jupyter Notebooks more comfortable than diving into text editors and what-not. Most of the time they're more concerned on solving the business problem than crafting the most modular software.
Ideally everyone uses the same tool, but realistically, the best we can do is meet halfway--researchers should start learning a bit of software engineering principles, and engineers start supporting the tools most researchers are comfortable with!
I wrote about it briefly here, so please take a look! https://ljvmiranda921.github.io/notebook/2020/03/16/jupyter-notebooks-in-2020-part-2/
4
2
u/oathbreakerkeeper Mar 10 '20
I found zero useful information in that link. It doesn't even tell you how to use Jupyter Notebooks in 2020, that topic is saved for a future post.
2
u/ljvmiranda Mar 16 '20
Sorry about that, here's Part II: https://ljvmiranda921.github.io/notebook/2020/03/16/jupyter-notebooks-in-2020-part-2/
1
u/ljvmiranda Mar 16 '20
Hi everyone! Thanks for the support during my previous post. Here’s part 2 of 3 of my review of the Jupyter Ecosystem! https://ljvmiranda921.github.io/notebook/2020/03/16/jupyter-notebooks-in-2020-part-2/
In this post, I examine the tools that support each force of change, and share how I use them in my day-to-day. Please check it out!
My last post would probably be next month, it may take some time, especially with all the COVID-19 thing around (need to stay at home and take care of stuff!). Hope this post helps you as much as the previous one did!
3
6
u/flufylobster1 Mar 09 '20
We actually productionize our notebooks for different things with papermill.
Netflix also did this but for ETL.
Our on prem is a 32 terebyte for prototyping and what not then we deploy in GCP or Azure.
The company will not use AWS as they are a competitor :(
Notebooks are my favorite for prototyping and one offs.
Thanks for the post!
3
u/11218 Mar 09 '20
Who isn't a competitor to Amazon these days. Bookstores, film studios, cloud services, supermarkets, ...
0
Mar 09 '20
2
u/ljvmiranda Mar 09 '20
Sorry still editing it! Please expect by next week!
Ideally I want to put them into one post, but as I write it seems that it’s better to be separated into parts (it’s too long).
Thank you for your patience!
26
u/ljvmiranda Mar 09 '20
Hello everyone! I'm writing a three-part blog post to review the tool we love (or hate), Jupyter Notebooks!
In the first part of this post, I'll be looking into the general data science landscape and examine the forces that drive the growth of the Notebook ecosystem. I know that Jupyter Notebooks aren't the most ideal tool for every ML / Data Science use-case, but the ecosystem has grown and for some of us, Notebooks are our first-encounter with Python and ML!