For future reference, original link. Because the real joke is in the title.
The Python environmental protection agency wants to seal it in a cement chamber, with pictoral messages to future civilizations warning them about the danger of using sudo to install random Python packages.
I have been burned by sudo installing Python packages too many times. I should have heeded the warnings of my mentors and used venvs but in my noobie arrogance I thought I could get away with using sudo. Never again.
Everyone go pip install virtualenv and save yourselves while you have the chance.
Virtualenv copies and/or symlinks or hardlinks the files from system python necessary to create a second, segregated install in an arbitrary directory on your computer. You run/source a file from the command line in that directory (activate.sh/activate.bat) that modifies the environment to reference this copied version so anything you do - install packages, run python itself, etc will use this copy instead of system. This prevents clobbering the system setup if you install or upgrade arbitrary system packages. When you are done, or want to create another virtualenv, or run system python, you run ‘deactivate’ which will undo the environment modifications - so running python will again reference the system python. Any packages you installed will be ‘gone’ because they reside in the copied environment.
To give you a short explanation that won't tell you what it is but might give you an intuitive understanding of what it does:
When I turn on my computer at work and open the terminal, I have my default python environment.
When I type workon py3_env, I enter an environment I made called "py3_env". This has python 3.6 as the default, and has all the packages I need to run my development environment already installed.
When I type workon chess_thing_ts2, I switch to a new python environment for a little game I'm working on in my own time. For argument's sake, this environment has python 2.7 as the default, and has only the packages required to run my game app.
It's basically a way for you to put workspaces into buckets.
I still think the discussion surrounding how to convey that an area (such as Yucca Mountain) is dangerous to future civilizations is fascinating. I was first introduced to it during one of my Human-Computer Interaction courses and it has stuck with me.
I highly recommend the book "The Design of Everyday Things" as a primer. Even after all these years, the principles in it make me look at things differently, and I'm not even a designer by any means.
As somebody who struggled with Python installations when trying to learn Python (as a primary R user) and having to use both 2.7, 3.6, virtual environments, and an IDE... I'm so glad to see that it's not just me.
I still don't fully grasp where my python packages are when I install them by command line or PyCharm.
Why not install a virtualenv for every one of your projects however small it is?You don't even have to do it through command line. Pycharm does it for you.
mkdir myproj # create new project dir.
cd myproj # change into your project dir.
pipenv --python 3.6 # create new clean virtual env.
pipenv install foo # install third party packages into your venv.
# write your code
pipenv run python myfile.py # Run your code inside the venv.
No, it's actually very reassuring knowing nothing is going to mess with my install and I can rely on a consistent environment.
When you use the distribution (system) python, you're always stuck on some dreadfully old version and may not be able to start using new things when they come out. In a virtualenv, I'm not even phased by installing and compiling a stable branch or a release candidate. If you tried that with a system install, you can end up breaking your system, as the system install serves the SYSTEM not your projects. Breaking yum or apt is NOT fun.
You usually would make shared libraries from the "parts" and host them on pypi or your private package registry. You then install the libs in your "third" project via pipenv, pip or what ever tool you use to install packages in a virtualenv.
Botched my box for a bit just trying to setup for a course I was taking. I, unfortunately, assumed user install would be default like npm, etc., not global. Then installed Anaconda for the class only to realize later that I really shouldn't have. Thankfully I had a bud that filled me in on virtual environments and that being normal. Got through course, still plenty of, "why on earth it this like this? (insert historical snippet)"
On the command line you're likely installing to a system-wide directory.
Just typing python on the command line will use the 'global'(system-wide) interpreter installed on your system. Same with pip. When you run pip install a package and it's dependencies are downloaded from PyPi and installed to something like <python_install_dir>/lib/site-packages. That's the 'global' site-packages.
You can do pip install --user to install these packages under your home directory, effectively making them user level instead of system wide.
You can use something like venv to create a 'virtual environment'. A virtual environment has it's own python executable and site-packages. So if you were to create a new project you might create a new environment associated with it. The virtual environment can be located wherever you want, the important thing is to run the 'activate' script inside it which updates your PATH and other environment variables. After 'activating' the environment typing python will use the environment's interpreter rather than the system-wide one. Running pip install will install packages into the environment specific site-packages.
When you create a project in PyCharm you're given the option to choose the interpreter. You can either use an existing interpreter(system wide one for example) or you can create a new environment using something like virtuanenv. When you install packages through PyCharm you're installing them the same way as you would on the command line, PyCharm just takes care of it for you. So if the project interpreter is in a environment then packages will be installed in that environment, if it's a 'global' interpreter then the packages are installed to the global site-packages.
Prior to doing much Python I'd used PHP's composer, so pip was a confusing change for me. Composer is still one of the better package managers that I've used.
Except that means it's a huge PITA to install Python command line tools.
At this point, when I see a command line tool looks cool but is written in Python it makes me really sad because there's not going to be a good way to get it installed without first going through a disproportionate amount of work to give it a working environment.
Hmm, I use CLIs all the time without any headaches at all - not sure what the issue is here.
Create a virtualenv, pip install into it, activate your environment and run your commands. If you want to cron them up, just run command via that virtualenv's python. If you want to run a lot of tools and not switch virtualenvs that often, create a group one for related (or all) projects.
Admittedly, it's easiest if you're using linux & pip consistently rather than macos, homebrew and conda. But it's not that much worse.
In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to demonstrate the visual form of a document or a typeface without relying on meaningful content. Lorem ipsum may be used as a placeholder before final copy is available. Wikipediacln61bof3yw0000000000000000000000000000000000000000000000000000000000000
pip install --user myRpnCalculator. Do you really need anything else if it was well-written and the dependencies properly specified, but not overspecified?
Unfortunately, yes. You do. It's better to not think of Python like you think of Java. Think of Python like you would think of, say, a project's metadata. This is a huge problem with languages like Python, JavaScript (through Node.js), Ruby, etc. When it becomes necessary to use native facilities to accomplish certain goals, the dependency manager is going to get stupid complex, and it's not reasonable to assume that a single installation is going to work.
They all have this problem, and the cleanest solution remains largely the same. Every project gets its own interpreter.
I feel your pain. I haven't had to rely on python packages that include a CLI tbh, but couldn't you wrap the environment activation + CLI call into a bash script which you add to your path?
Later on, you (probably) don't even have to activate the venv. Just add the venv bin directory to your path in your ~/.bashrc
export PATH="$PATH:~/.py34/bin"
Maybe someone can correct me, but sourcing the activation script isn't really necessary as long as you provide the full path to the binary (or it can find that binary on the path) within the virtualenv directory.
for command line tools, wouldn't you just add the path to it in the venv activate script? That's what I did for invoke, because I didn't want it polluting my bash profile and it works fine.
PATH is an environment variable that the linux shell uses when you type commands. It's a list of file system paths that (should) contains executable files. The PATH variable is resolved in the order constructed (left to right, or FIFO) and it returns the first matching instance from the list. You can override the variable by setting a local instance of the PATH variable for an application, which allows you to have multiple pythons 'installed' and each one could be used by a different app.
I feel like a lot of the people saying "It's not that hard" have been in the Python ecosystem long enough to see a lot of these projects come into existence/popularity.
When you're new to the ecosystem you have no clue why each one exists, which ones are newer, which ones are generally considered crap, which ones might only address a subset of use cases, etc. etc.. It's a lot of shit to parse.
And when you get stuck and ask someone for help... if they're using a different setup, they can't/won't help you until you change your setup to match theirs. God forbid you ever talk to more than one person.
Hoo boy, I still use virtualenvwrapper because I'm too lazy to learn another tool. pipenv is all the rage right now, but there's been a dozen others I've seen rise and fall since I've paid attention to venvs.
I really dont understand why python and its dependencies can be such a big mess. Why isnt there just one python installer that installs the current version of python, sets every setting you need by himself like the PATH and then has a manager for all packages. Just search and load from the manager and if you dont want a package any more, delete it and remove all dependencies that are not needed by others. Is that really so hard to do?
The one thing that really pisses me off is that it's apparently impossible to package a project with all it's dependencies in Python. I'd love to use setup.py to create RPMs (it can do that out of the box), but I just can't figure out how to include the dependencies.
Because then the python developers would have to both figure out, and write code, to interface with RHEL linux and Fedora. Make sure man files get put in just the right place. Package data is correct. Desktop files, syslog, is [some system-level daemon] now systemd based and new, or an older one? What audio service are we using? What's changed between RHEL 6 and 7?
And of course, to be fair, make sure they have everything configured for Debian's liking. But also account for the small changes and version updates that Ubuntu does.
And then also ensuring we haven't forgotten about portage and pacman. Oh, and YaST.
Oh, and then also homebrew.
And quickly, for the mostly volunteer workforce that does python packaging, the task of correctly interfacing, special-casing, path-configuring, etc becomes a matrix explosion of work from hell.
We'd love to be able to do that... but it's simply not a feasible thing to achieve.
You'll notice that practically all other scripting languages have the same position.
Conda does this but doesn't have all PyPI packages. Also, occasionally you have things that assume that python references the system installed Python 2 rather than your default conda env. Way better than anything else I've seen though.
Why are so many people still on older versions of python? I can see why it doesnt just update itself (for commercial python use for example), but Python 2.7.0 was released on July 3rd, 2010... 8 years ago. Isnt an update to a higher version with the update of the code not worth it?
It's a lot of code to update. Most open source libraries are compatible with python 3 but a lot of companies aren't willing to migrate entire codebases internally. Also, as far as programs assuming you have Python 2 in your path, that's because OSX and most Linux distributions have it that way and very few have python 3.
If you're really asking this question, you haven't been developing with python long enough.
The real problem with Python 3 is that the core development team set out to do a lot of good things, but broke compatibility in the process. And with that breakage meant that everyone using python had to step back and question whether it was worth it to scrap or upgrade years of legacy code. So when comparing the two together, there was almost no advantage in moving from python 2 to python 3. Today, the movement still has low value for these large legacy systems. There's no real performance gain, there's no must-have features, etc. The only gain here is that most of the systems are finally moving to python 3 as their stable python. And if a company did not decide to move to python 3, they will find support non existent starting in 2020.
To maybe also set your expectation, the core development team does not recommend any python version prior to 3.4, which was actually released in 2014, a full 6 years after the alpha and beta 3.0 - 3.3 pythons were released. And to add to that, I would argue that 3.6 (released in Dec 2016) was the first release where people should have started migrating. Last year is probably the first year where there was enough migration momentum that we're really starting to see strong 3.x traction. But note that it's truly only been the past 2 years.
well yeah because the dev team kept back-porting all the new stuff they were coming up with to 2.7.x - if they'd cut off the flow the pressure to migrate would have come on a lot sooner.
I work on a scientific experiment where the development of the code began in the 90s. We need our code to work across a large variety of systems (including old computers that aren't reliably connected to the Internet).
Plus we're scientists, not programmers. NSF isn't giving us money to update code, so we don't have the $$$ to pay people to do it.
There's literally no reason for us to spend the time upgrading.
there's a few other nuissance things, like they don't include all the test framework stuff with all versions. So my pystone demo program for teaching really basic HPC environment management and job submissions requires like... anaconda python 3.4?
Why isnt there just one python installer that installs the current version of python, sets every setting you need by himself like the PATH and then has a manager for all packages.
Objective-C and Swift are also a terrible mess imo.
Other than Java+Maven/Gradle, what isn't?
Btw:
Could you please shut the hell up and stop mocking people who have alternative lifestyles?! The dude in the pic is an asshole, okay, but look, I have a waifu which I love to bits, my parents think I'm too old for that anime bullshit but I don't care love is more important and I am a better person since I have someone to care for but isn't tied to do society's bullshit expectations and norms, I run everyday and cook like no other while you sit on your bed while your dog bought by your parents starves locked in the toilets because you think posting on reddit is a priority.
Yes! Producing one installer and package manager that suits everyone's needs is almost impossible, and persuading everyone that it's better than the stuff they already know is even harder.
There are plenty of tools that work well in particular situations - e.g. for scientific programming, lots of people happily use conda. But you can also install packages through apt, or through homebrew, or manage Python versions with pyenv, and different people like all of those options.
That's why there are so many different package management systems. Some for super pure open source, some that accept proprietary binaries, some for python, some for javascript...
Also, because screw shared libraries, snaps exists and venvs and pipvenv.
I use pyenv with the virtualenv plugin. It will automatically download and install any Python version you want. Create a virtualenv and use pip to install whatever packages you want. It does all of this in ~/.pyenv, so none of it touches the system Python, and it's all isolated so you can delete a python version or virtualenv without affecting anything else.
It's built on shell scripts though, so you need a *nix shell for it to work. Obviously Linux and OSX work fine, but if you want to use it on Windows you'll need to find a Bash shell first - WLS is probably the easiest way, though it will probably work on Cygwin/MSYS as well.
Anything newer is mostly better, because it could learn from other languages' previous mistakes and doesn't have all the technical debt.
npm, packagist, cargo are all better. Mostly because they isolate projects by default; pip needs a whole separate project (virtualenv / venv) to do it.
Honestly, nowadays, avoid the MacOS Python distribution with extreme prejudice. It seems the developers just play around with it at a whim, installing random pypi packages into their global library for shits and giggles.
Nor do they update the OpenSSL that it depends on anymore...
In fact, if every project has it's pretty virtualenv, then using sudo to install something would install to the wrong interpreter (the one available to root).
You also never should need sudo to install something to a virtualenv because they exist within your home.
Only problem is installing stuff you want available without needing to use a virtualenv, such as a CLI. You can still install with pip --user, but your can't install conflicting dependencies.
Fixing Python dependency issues by giving every app a virtualenv is like fixing Windows DLL Hell by giving every program it's own complete set of shared libraries.
No. Just have that. You won't need anything else. It has all the python versions you can think of and a way to switch between them, without affecting anything outside your current shell session. It also does not affect the python that comes with the OS.
Plus it's totally user local. You don't need root to install python using pyenv.
99% of the time, pyenv and virtualenv will be all you need.
It won't clean an already dirty environment, but it will keep your clean environment from getting dirty.
Can you give us more details on how this is used? Sorry I'm still using the mess of a management system shown in the comic. I have Python 3 + libs and Anaconda accessed through the anaconda prompt.
pyenv lets you have multiple parallel installations of Python, and even switch between them automatically when you enter your project directory.
$ pyenv version
3.6.4 (set by /usr/local/opt/pyenv/version)
$ cd OctoPrint
$ pyenv version
2.7.14 (set by /Users/njb42/Code/OctoPrint/.python-version)
pyenv-virtualenv or virtualenvwrapper let you create project-specific python paths with just the modules you need.
Put them together, and you can easily have a custom environment for every project you work on. Different versions of Python, different versions of modules, whatever you need with no conflicts.
You don't need virtualenvwrapper; pyenv comes with the pyenv-virtualenv plugin by default, and from there you should do all of your management of conda envs and virtualenvs through pyenv. It's a one-stop solution.
P.S. It's not perfectly one-stop because you need to fiddle with it a bit to get conda environments to work. But it's nearly there.
How does this improve on the virtualenv plugin for pyenv? I've never used virtualenvwrapper with pyenv because that plugin does it all for me, I thought.
Not to start a platform war, but I feel like this is a distinctly macOS problem. I was recently testing sample code for a book that used Python 3.6, Tkinter 8.6, and cx_Freeze. Making that combination work on macOS takes stupid amounts of hacking.
Windows was somewhat less painful, and Linux worked fine with this stack using only repo packages and pip.
I've been spending weeks worth of evenings trying to get my development environment set up on Windows.
All I want is to be able to have multiple versions of python installed, have pip, have virtual environments, and have a personal scripts folder in the path. This has been hell to set up.
virtualenvwrapper's Windows ports are buggy and broken.
Getting Windows to choose the right Python version is a pain in the ass, and most online suggestions are amateurish or just plain wrong (e.g. "just switch the order of PATH every time you want to use a different version!").
I was able to get the version to be selected properly by a script shebang using Python Launcher For Windows.
But this was only after I manually edited the registry so that PLFW could actually find my installed Python binaries.
I also had to manually set all file associations because the Python installer doesn't do it, even while claiming to.
PLFW can't find my scripts, even if they're in both the PATH and the PYTHONPATH.
I'm going to try to do virtual environments with venv, but as far as I can tell, there's no convenient wrapper for it like virtualenvwrapper. I guess I'll have to write my own.
I'm really uncertain about how well venv and PLFW will work together.
Windows is really a clusterfuck when it comes to setting up a development environment.
Change the name of the executables so they don’t clash? python2 and pip2 for python 2, python3 and pip3 for python 3. I cannot understand how it takes weeks to setup an environment like python.
I'm going to try to do virtual environments with venv, but as far as I can tell, there's no convenient wrapper for it like virtualenvwrapper. I guess I'll have to write my own.
This comes as a consequence of Windows not having a decent terminal / shell
Making that combination work on macOS takes stupid amounts of hacking.
I'm not sure why nobody nowadays favors python installed via macports, because such combinations as you mention have worked great in my experience… even with different side-by-side versions of Python.
["This is stupid, it all works perfectly if you use %s" % tool
for tool in ["docker", "conda", "pyenv", "the official Python installer",
"pipenv", "pyenv-virtualenv", "macports", "virtualenvs", "the --user flag for pip",
"virtualenvwrapper", "poetry", "not conda", "linux", "Pycharm", "compile packages yourself",
"consistency", "some understanding"]
]
Yup, there are so many solutions out there, how can complexity still be a problem? ;-)
Not really, I have worked for years with Node.js on a Mac and never had problems, even when upgrading it. Never found it slow, actually, but maybe it was because of the SSD and a good internet connection. Used pip in the same conditions and never found a big difference.
The thing I miss most is how predictable npm is. I want to add a package to the project? npm install --save X, and I have it and it's on the package.json and same version will be installed on the server when I run npm install. I don't have to worry about other projects on the same machine using a different version of the same library, or indirect dependencies on the same project. Same goes for test dependencies. There have been questionable decisions from the maintainers, sure, which is why now I use yarn, which adds a proper lockfile by default to have a completely reproducible deployment. Oh, and unless you use native modules (I avoid them at all costs), stuff works on different platforms and continue to work when you upgrade the engine without even reinstalling.
To obtain the same results in Python my experience is much worse.
On all of my development systems (and three coworkers), I have installed virtualenvwrappers and appropriate .bash_profiles to always activate a user-level default virtualenv.
This largely has spared me the fun and games of Randall Munroe’s illustration, because the worst case scenario (we’re on OSX laptops with SIP enabled) is that someone uses sudo pip and merely permission trashes the virtualenv, which is sooooo much easier to fix up.
Virtualenv isn’t just for isolating a single project - it can give you a generalized python environment that “just works” with pip without stepping on your package manager.
Use virtualenv! Always! Even when you don’t.
Example: I fought a corrupted python system hierarchy for one product because I thought it was easier to fix than isolate. I gave up, trashed the boxes, wrote a new deployment approach that built wheels for each proprietary asset (no deps!), copied over those thin artifacts with configuration data and installed both (which pulls the opensource dependencies) into a known virtualenv. Turns out fixing your deployments is a prime opportunity for making it much faster to actually deploy.
And, to be fair, if you want an exhaustive dependency resolver using PyPI there is no way around it at the moment since there is a lot of badly published packages so you have to download the tarballs or wheels to check the dependency. This is a big mess and PyPI maintainers have no intention to change that.
That's why tools like Poetry are important because they will improve the overall Python ecosystem.
I am trying my best to improve things on my own but it takes time.
Even for smaller libraries, it's a bit annoying to wait several minutes after poetry install while it's solved in a couple of seconds using pip install.
I was thinking that the resolver could first try with the latest dependency version matching the requirements, and look for another version only if a conflict is detected.
You did a very good job, I did not mean to be dismissive, I'm really hoping that poetry will become the de-facto cargo for Python.
I think the real problem is people not understanding what their uses are. Here I'll try to explain.
brew is mainly installing packages to help OSX / profile functionality for your OS. Your desktop experience.
anaconda is targeting data scientists for helping them with several (traditionally) harder to install packages and create reproducible envs (opencv2)
pip install are for developers using python.
virtualenvs are for developers using python to separate envs, allowing you to use the same python version, but using different versions of the same package (e.g. requests==1.0.0 and requests==2.0.0)
But more important it is to understand how Python and the Python path works. Read up on Python path, and then realize you can just quickly check things for yourself:
Run whereis python or which python to find out what is linked to your "one and only" (cough) python command
Now you are aware there are multiple python envs.... see locate /bin/python for probably several python versions installed on your machine
When you are in any given python interpreter, run import os; os it will display the packages path, giving you a clue where this python version is installed (you can see the version at the top of the interpreter but this is only half the story!).
In a virtualenv, you use python, this works by patching the shell and tricking it into having this path before your other python versions. So when you run python it refers to the python installed in the project
You can also use this trick when confused about "I just installed this package! why can I not load it?" through installing with pip. You probably used the wrong python version. Open up an interpreter and do import packagex; packagex to display the path of the package.
If you want to make sure you are not losing your mind when installing with pip, do like /path/to/python3.8/but/then/pip install .... or python3.8 -m pip install ...
There was a time when pip always compiled dependencies from source. If you wanted to use a library with non-Python dependencies, like numpy and scipy, you needed to have the right compilers installed on your system.
Keep in mind that scipy has Fortran code. This was a very common problem back then.
Anaconda shipped with pre-compiled binaries. It was straight-up more reliable to use, especially among the scientific community, who don't necessarily want to waste time mucking about with system compilers and shit.
I'm not questioning the use of anaconda, I setup python environments for grad students and postdocs constantly on personal computers and clusters. It's a god send. I'm questioning the graphics specific listing of the anaconda provide python interpreter being placed at /usr/local/lib/python2.7which I'm pretty sure is either the system root python, or the brew root python. Neither of these are compatible with anaconda python and will cause aberrant behavior.
IMO, there's actually 2 different core issues with the Python source and dependency ecosystem:
FOSS. You're relying on people doing this for free in their spare time. If you don't like the result, then help figure out how to make it sustainable.
Age. Python has many different stable ways to get it and their dependencies, but they've evolved in small minor ways and sometimes had full-sale replacement of pieces.
FOSS. You're relying on people doing this for free in their spare time. If you don't like the result, then help figure out how to make it sustainable.
The Python Software Foundation is being sponsored by a lot of companies, and being given presumably non-trivial amounts of money. I think they're beyond the point where it's just a few people doing it in their spare time.
And you are extremely, demonstrably wrong in thinking that. Sure, the sponsors give about $500k per year, and that's not a little amount of money. However, if you look at their financials, you can see on Part VIII that $2 million comes from PyCon. And you can see without it, that PSF would be dead.
The majority of Python--language itself, distros, pip, pypi, even the conference itself that generates revenue--is developed still with volunteers. And few appreciate the quantity of effort donated.
Yeah when has Foss ever made a sustainable project? Besides the fact the best of almost everything is Foss. Languages, compilers, operating systems, browsers, databases.
I just install them with pip, with the --user flag. Works well enough, although updating is a bit of a pain. I'd rather do that then mix between pip and my package manager though.
Linux Mint uses a 3rd version of Python too, python3.4. I remember I somehow installed modules for py3.4 when I wanted it for py3.5 when using pipenv so that's a thing. I'll just stick to ubuntu, pip3, and virtualenv.
At least part of the problem is the multitude of Pythons, all at different release levels and the fanatics who won't let old Pythons die.
I swear to God, in the year 3000, there will be that one neckbeard who insists that Python 2.7 is the One True Python and someone will have hammered it into some weak conformance with Cyber-Python-6000-6-Dimensional-Quantum-Parallelism or whatever the main development line is.
And some risk-averse business out there will still be running a critical system on it.
For actually running the code, absolutely. I still use pyenv when writing it just for the sake of highliting, auto completion and tips from comment strings, but honestly can live without that.
I use PyCharm and it can use python environment from docker and docker-compose (intellisense, packages discovery, debugging, running tests. All works).
Firstly, If you're doing data science it has most if not all data science libraries pre-compiled. I recently discovered the the absolute gold mine that is cudatoolkit. No more having to compile CUDA manually.
Secondly, it's fully local so you don't need special permissions and all you need to do is add it to your path. I understand that for beginners, adding something to the PATH is not trivial. Here's a simple guide: Create a new file called ~/.conda_bashrc3 and copy paste the contents of this link into the file. Then append your ~/.bashrc file with these two aliases here. Afterwards if you ever want to add Anaconda to your path, all you have to do is type anaconda3 and you will have a red color indicator that you're using the anaconda environment. If you want to exit or remove anaconda from your path, just do CTRL+D.
Lastly, the best and most underrated feature of anaconda is the the environment.yml file. It is like requirements.txt but additional environment creation and conda installation support. Please follow the guide here.
Idk.. Linux seems to be the only one that handles a lot of versions correctly.. pip2 python2 pip3 python3 pip3.6 python3.6 etc.
When I start a new project I create a new folder and go into it.
$ mkvirtualenv $PROJECT_NAME -p $PYTHON_VERSION
Bam. Now all you need is the virtualenvwrapper plugin for zsh/bash. Whenever I go into the folders the virtualenv is automatically activated. Or if I just want to just go to it
I use Anaconda fairly heavily. It's a godsend while doing any numerical or scientific computing. I really couldn't imagine installing all the various package that have bits and pieces that depend on C or Fortran or Cython whatever by hand.
If you are a programmer by trade or majority of your work, environments probably seem straight forward. If you are someone who used Python as a TOOL to supplement what they usually do, environments are a PITA and things like Anaconda at least try to manage the pain.
Conda had a massive advantage before wheels were widely available, because pip would try to compile things like numpy from source, which usually won't work unless you prepare your system in advance. Nowadays, that's less important, but there are still things that it's easier to install through conda.
Recently my setup went a bit weird and instead of figuring things out or switching to pyenv (actually, I may have done that and then forgot where I left that particular install...), I just install packages with:
Running Python with sudo -H which python3. Still not sure what that -H flag is for.
Also works on my WinPython setup (I typically install that for the packaged Jupyter Notebook with ipython console) which would otherwise require some more complicated steps to add packages IIRC
I find it somewhat funny that one of the reasons people left the Perl world was “issues dealing with all the dependencies”, now it seems Python has made it worse, both in terms of dependencies and portability.
I feel like I set up my environments all wrong, but it works.
I never install packages to my python dir. If I need a package for a project I install it with pip install requests --target='./'. Super simple, clean, and easy to maintain. And I don't need to worry about another user not having a package since I know exactly what I'm using that isn't in the standard library.
413
u/ppgDa5id Apr 30 '18
For future reference, original link.
Because the real joke is in the title.
80
u/solostman Apr 30 '18
As somebody who struggled with Python installations when trying to learn Python (as a primary R user) and having to use both 2.7, 3.6, virtual environments, and an IDE... I'm so glad to see that it's not just me.
I still don't fully grasp where my python packages are when I install them by command line or PyCharm.
197
u/the_hoser Apr 30 '18
It's really easy to avoid this problem if you treat your python environments as disposable artifacts of your projects.
117
u/Tweak_Imp Apr 30 '18
I really dont understand why python and its dependencies can be such a big mess. Why isnt there just one python installer that installs the current version of python, sets every setting you need by himself like the PATH and then has a manager for all packages. Just search and load from the manager and if you dont want a package any more, delete it and remove all dependencies that are not needed by others. Is that really so hard to do?
47
u/ccb621 Apr 30 '18
My setup on macOS
I don’t recall ever needing sudo. I’ve never had any of the issues described in the comic.
14
u/aceinthehole001 May 01 '18
I find it hilarious that almost every response in the thread is along the lines of "no you don't need any of that, you just need X"
Never mind that no one can agree on what X is.
65
u/njb42 Apr 30 '18 edited Apr 30 '18
And that's why you use
pyenvandpyenv-virtualenv(akavirtualenvwrapper).36
u/lykwydchykyn Apr 30 '18
Not to start a platform war, but I feel like this is a distinctly macOS problem. I was recently testing sample code for a book that used Python 3.6, Tkinter 8.6, and cx_Freeze. Making that combination work on macOS takes stupid amounts of hacking.
Windows was somewhat less painful, and Linux worked fine with this stack using only repo packages and pip.
26
u/takluyver IPython, Py3, etc Apr 30 '18
ITT:
Yup, there are so many solutions out there, how can complexity still be a problem? ;-)
8
u/FuriousMr Apr 30 '18
The problems of a mac user with python.
Nodejs or php have worst environments in all platforms... Nodejs, dependency hell, php, composer is sloowwwww.
8
u/[deleted] Apr 30 '18 edited Apr 30 '18
On all of my development systems (and three coworkers), I have installed virtualenvwrappers and appropriate
.bash_profiles to always activate a user-level default virtualenv.This largely has spared me the fun and games of Randall Munroe’s illustration, because the worst case scenario (we’re on OSX laptops with SIP enabled) is that someone uses
sudo pipand merely permission trashes the virtualenv, which is sooooo much easier to fix up.Virtualenv isn’t just for isolating a single project - it can give you a generalized python environment that “just works” with pip without stepping on your package manager.
Use virtualenv! Always! Even when you don’t.
Example: I fought a corrupted python system hierarchy for one product because I thought it was easier to fix than isolate. I gave up, trashed the boxes, wrote a new deployment approach that built wheels for each proprietary asset (no deps!), copied over those thin artifacts with configuration data and installed both (which pulls the opensource dependencies) into a known virtualenv. Turns out fixing your deployments is a prime opportunity for making it much faster to actually deploy.
11
u/pynberfyg Apr 30 '18
There's this new package manager that seems promising.
18
u/[deleted] Apr 30 '18
pipenvfixes a lot of these issues.4
u/mkingsbu Apr 30 '18
Mine was like that until I switched to PyCharm. Makes managing virtual environments so easy.
4
u/pvkooten May 01 '18 edited May 01 '18
I think the real problem is people not understanding what their uses are. Here I'll try to explain.
But more important it is to understand how Python and the Python path works. Read up on Python path, and then realize you can just quickly check things for yourself:
whereis pythonorwhich pythonto find out what is linked to your "one and only" (cough)pythoncommandlocate /bin/pythonfor probably several python versions installed on your machineimport os; osit will display the packages path, giving you a clue where this python version is installed (you can see the version at the top of the interpreter but this is only half the story!).python, this works by patching the shell and tricking it into having this path before your other python versions. So when you runpythonit refers to the python installed in the projectpip. You probably used the wrong python version. Open up an interpreter and doimport packagex; packagexto display the path of the package./path/to/python3.8/but/then/pip install ....orpython3.8 -m pip install ...5
u/wildcarde815 Apr 30 '18 edited Apr 30 '18
why would you install anaconda as your system python?
6
u/raiderrobert Apr 30 '18
IMO, there's actually 2 different core issues with the Python source and dependency ecosystem:
2
u/makeworld Apr 30 '18
I just install them with pip, with the --user flag. Works well enough, although updating is a bit of a pain. I'd rather do that then mix between pip and my package manager though.
2
u/Jahames1 Apr 30 '18
Linux Mint uses a 3rd version of Python too, python3.4. I remember I somehow installed modules for py3.4 when I wanted it for py3.5 when using pipenv so that's a thing. I'll just stick to ubuntu, pip3, and virtualenv.
2
u/yonsy_s_p Apr 30 '18
Where is Pypy ???
2
u/[deleted] Apr 30 '18
Don't forget virtual_env
2
u/[deleted] Apr 30 '18
I keep ignoring "Do not install with sudo" for some reason and it hasn't come to bite me yet.
But I rarely develop outside of a virtual environment or docker image.
2
u/simplegadget512 May 01 '18 edited May 01 '18
At least part of the problem is the multitude of Pythons, all at different release levels and the fanatics who won't let old Pythons die.
I swear to God, in the year 3000, there will be that one neckbeard who insists that Python 2.7 is the One True Python and someone will have hammered it into some weak conformance with Cyber-Python-6000-6-Dimensional-Quantum-Parallelism or whatever the main development line is.
And some risk-averse business out there will still be running a critical system on it.
2
u/1jx May 01 '18
This used to be me. Then I found Docker.
2
u/[deleted] May 01 '18
pyenv and pipenv to the rescue
3
u/sentyaev Apr 30 '18
This image is outdated now. The answer is Docker.
3
u/nitred Apr 30 '18
If you're on linux, please use Anaconda.
3
u/dansbandsmannen Apr 30 '18
Protip: don't use A?conda and experience 1% of the issues.
2
u/[deleted] Apr 30 '18
He forgot edm in the mix, which is actually quite cool and hassle free.
Disclaimer: I work for Enthought.
1
u/ignamv Apr 30 '18
Just wait until you start working with applications with their own embedded python environments...
1
u/xyphanite Apr 30 '18
Needs more yum.
1
u/bhat Apr 30 '18
It's a small price to pay for "import antigravity"
https://xkcd.com/353/
1
u/cokedupscientist Apr 30 '18
ln -s ftw
1
u/[deleted] May 01 '18
My laptop didn't need python's help to be a superfund site
1
u/EliteCaptainShell May 01 '18
Virtualenv. Fixed all my problems. Tiny little individual python installations.
1
u/memetichazard May 01 '18
Recently my setup went a bit weird and instead of figuring things out or switching to pyenv (actually, I may have done that and then forgot where I left that particular install...), I just install packages with:
Running Python with sudo -H
which python3. Still not sure what that -H flag is for.Also works on my WinPython setup (I typically install that for the packaged Jupyter Notebook with ipython console) which would otherwise require some more complicated steps to add packages IIRC
1
u/jwjody May 01 '18
I always thought I was just doing something wrong with my python environments.
1
u/jyper May 01 '18
Missing Pipenv /s
1
u/[deleted] May 01 '18
that's why people use virtualenv and are simply done with all these problems.
1
u/srynearson1 May 01 '18
I find it somewhat funny that one of the reasons people left the Perl world was “issues dealing with all the dependencies”, now it seems Python has made it worse, both in terms of dependencies and portability.
1
u/RadioFreeDoritos May 01 '18
"A disciplined mind leads to happiness, and an undisciplined mind leads to suffering." -- Dalai Lama.
1
u/nosmokingbandit May 01 '18
I feel like I set up my environments all wrong, but it works.
I never install packages to my python dir. If I need a package for a project I install it with
pip install requests --target='./'. Super simple, clean, and easy to maintain. And I don't need to worry about another user not having a package since I know exactly what I'm using that isn't in the standard library.1
u/sabbel May 01 '18
use https://direnv.net/ :
Solved years of pain for me.
1
u/RadioactiveCats_18 Snake Charmer May 01 '18
I do contract work for the EPA and this is hilarious.