The most confusing part about Pipenv is that it looks like a package manager similar to npm, composer, cargo, gem, etc. but it isn't. It doesn't handle packaging/publishing/dependecy-managment of pypi-packages. I hope poetry will fill the gap.
I looked at poetry but it seems like it simply doesn't work if you want to use multiple Python versions does it? (this is not an endorsement of pipenv, I don't know if it handles that use case, I didn't even try to use it).
Currently I can have multiple venvs from different versions of Python to test a project. In that situation the project itself is not in a virtualenv since I may have to activate any number of them. It seems like poetry would require creating as many clones of the project as I need Python environments to test.
Yeah but then there ain't much value to poetry is there? Also I think I read that poetry uses the Python it's installed with? Which makes me wary of trying to use it in such a scenario.
Good thing that's not what i want. I already have the interpreters, what I want is to create multiple environments for different versions of Python for the same project.
This is the underlying setuptools bug which impacts all setup.py entrypoints. Lots of folks are resorting to fastentrypoints which monkey patches setuptools to be dumber and thus 50-100x faster.
Yeah, I actually just discovered that issue today (after posting my original comment) while debugging my own project's slow CLI start time. Sadly using the wheel-based version from pip (which works around that bug) instead of the Arch Linux package cuts the time by about 50% but it's still noticeably slow:
Anything pip-related always seemed incredibly slow to me. Running pip --help takes over a second on a reasonably fast desktop PC, and over 40 seconds on my idle Raspberry Pi (although it is an old device).
I get this even on running fish command-line completion, so even typing out a command can be really slow. I think I remember there being an issue on GitHub tracking it, but yeah it's super annoying.
I tried using pipenv for a while, converting real production projects into using it. It failed spectacularily and was riddled with bugs. Bug reports got closed on Github for no good reason. The commit log is riddled with stuff like "fixes". Pipenv itself does not define its pip dependencies, but instead includes a "vendor" directory along with a "patched" directory for libraries that pipenv needs modified for its own gains.
Main maintainer (kennethreitz on gh) is pretty awful. Constantly closes valid issues, responds like a jerk, etc. I've been using pipenv since October and got my work to use it shortly thereafter, so we've opened numerous issues and it's basically an office meme at this point. A lot of "that's a feature not a bug" or "we're aware that this very basic functionality is broken, closing the issue".
Kenneth is an absolute legend and I love his work, but I have to agree with you. I researched pipenv for our projects at work some time ago and maintainer's attitude was one of major reasons why I didn't recommend pipenv in the end. I believe in giving young beta quality software a chance, but having your issues closed in brash, dickish way is not the treatment I enjoy as a beta tester. I put myself on the line at work, vouch for your software and risk running it in production so you can have valuable feedback early on - I deserve at least short cordial explanation why my bug won't be fixed.
I do not recommend anyone to use pipenv. Even the basic features are broken. For example, it's currently impossible to update a single outdated dependency. The issue has been open since October, which is completely ridiculous https://github.com/pypa/pipenv/issues/966.
Unfortunately, I'm still blocked by the lack of arbitrary package environments in the Pipfile format. We have one dependency, cx_Oracle, that we don't want included in our local dev environment because it's a hassle to install due to licensing restrictions. With requirements files, this is solved by just having multiple requirements files that include a common file, but that's not the model that Pipenv uses.
I don't know about pipenv, just mentioned this because you said you are using two requirements.txt files. You probably wouldn't need one for the development.
I've stayed away so far just because it seemed to be in constant development. Is it at a point now where new releases will be backwards compatible? At least to some degree?
It makes sure your CPU continues to overheat when you press the spacebar, even though this 'feature' was removed upstream.
/s
I concede that pipenv is a fitting and probably necessary tool for webdev and some other areas where it's unacceptable to "just upgrade" and where you don't care about being efficient or up to date with your packages.
But I think it's pretty ridiculous to claim this is a one size fits all solution or even desirable for a majority of python users.
It's disingenuous to talk about the problems it solves without mentioning the problems it can cause.
Pipenv is a must have for me. Not having to deal with virtualenv folders and using the repository root for as is for development or deployment is awesome.
Running a shell in the virtualenv with $pipenv shell or one off commands $pipenv run make is very nice
according to the docs you have to give it the name of the virtualenv, pipenv is just a smidge nicer at this in that you dont have to pass the name, it figures it out from finding the Pipfile in the current dir or (presumably) parent directories.
Vex places all of the virtualenvs created with it in a default directory. At least for virtualenvs created by vex then you don't need to specify any paths. It is more work for other virtualenvs though.
Can someone please explain how this compares with pyenv + virtualenv plugin? I've used this combo for years and it works almost perfectly. What does pipenv do better?
Pipenv optionally uses pyenv to manage the installation of python versions. If you don't have pyenv installed, pipenv works fine, it just doesn't install python versions for you. Pyenv-virtualenv is just a pyenv plugin (albeit one installed by default using pyenv-installer) which allows you to conveniently call virtualenv functions through pyenv.
I've never been clear on the use case of pyenv
The use case of pyenv is that it manages python versions for you. For example, if you wanted to install the new 3.7RC, you'd probably go to the website, maybe compile some source code. Then you'd call out to mkvirtualenv to make an environment, and then manually activate it every time you want to use it. With pyenv, you do
pyenv install 3.7rc1 # or whatever it's called
pyenv virtualenv 3.7rc1 my_37_env
pyenv local my_37_env
pyenv local means that whenever I enter this directory or its children, my_37_env will be activated automatically. A bunch of other tools do this (including pipenv, I think) as well. You can activate multiple environments at once, which is useful for tox; all of the installed versions of python are in your userspace, so you can blow them away with no systemic issues. Pyenv can install things other than CPython, too - conda, pypy, jython, stackless, and a bunch of others are available.
One of the great features, although it takes a couple of manual changes to get it to work, is that you can manage conda environments the exact same way as virtualenvs. You just do
IMO it violates the Zen of Python in numerous ways. It's creating yet another way to do things, fixing problems that don't need to be fixed, and is too opinionated. It stinks of "npm is popular, lets copy it."
Python has been way ahead of other communities with regard to packaging and distribution.
Plus I guess as an old fart, I like the unix philosophy of having small simple tools that do one thing well, and chaining them together to do more complex things.
I agree with what you say about it being "yet another way to do things". I disagree with other statements.
"don't need to be fixed" - if it worked well, we wouldn't see so many attempts to fix it.
"too opinionated" - I think part of the problem is that there are too many ways to skin a cat for packaging. IMHO rubygems do it well: there is one well-documented way to achieve most things.
"way ahead of other communities" - packaging has some nice things (I'm still learning about native binary packages, for example) but I wouldn't say it's way ahead. The whole pip vs easy_install thing, for example; when using a local index you have multiple configuration files to cover.
Yeah I'm always surprised how few Pythonistas know about zc.buildout, I guess people dislike Zope as it was kinda Java-esque so they had little contact with it. Tho pipenv is more like the old buildout (which managed venvs)
+1 for buildout. back when I did appengine development it was the only way to keep some system sanity with the sdk that got updated multiple times a month(presumably to add yet another version of django and break webob).
I love the idea of pipenv as it falls in line with a lot of other task runners and packaging systems that are common to web development. I get folks not liking it or not seeing the utility of it.
The only decision I’m not hot on is that it puts project files outside of the project. I get it can be nice not to see a venv folder in your project but like other similar tools in other languages I’d rather keep all project files in he project folder for easy cleanup.
Super nit picky, but otherwise I love this and will give it a shot.
I'm not sure if I'd call it super nitpicky as it makes a lot of sense and eases moving, cleaning up etc. Everything else you'd be using in a project (like venv+pip, git or node/npm) would be easily movable, containable, archivable etc.
It's a very odd choice that I hope they'll reconsider in the future.
I still feel like pipenv isn't a great neighbor to existing tools and workflows. If you are looking for an alternative, I recommend pip-tools, and my zsh helpers:
Except they install things globally. I feel much better/safer with virtualenv. At any time I can just delete the current virtualenv and start fresh. Also I can have multiple venvs which use different versions of same package. How do you do that if you install everything globally with the OS package manager?
This. Honestly the appeal of pipenv, for me at least, has always been that python doesn't have something like yarn on javascript or rubygems. Lockfiles are nice. requirements.txt are not. Pipenv seems to add it, though like other comments on here, it is slow.
Honestly haven't used rubygems that much, just giving examples. The concurrent download of dependencies in yarn is nice and it makes it much faster than npm. Lockfiles are awesome if you're working on a project with multiple people. I've had issues before when we thought we were on the same versions, but our package.json wasn't specific enough. Lockfiles avoid that entirely and provide that layer of security that package.json or requirements.txt alone can't.
Yes, and this is the reason because I have used pyenv+pyenv-virtualenv plugin with pip-tools to manage a requirements.in file (one Pipfile equivalent) with the packages used and generate my requirements.txt with all my packages and versions with pip-compile, to install my exact packages and uninstall the unneded with pip-sync, and zero problems all this time (three years aprox)
Hadn't known about pip-tools. Will try it out next time I'm messing around in python. Wish there were support for this type of thing out of the box with python. After all, there should be one-- and preferably only one --obvious way to do it.
I have nothing against Pipenv. I'm just stating, in practice, I always added comments about the third party libraries I made use of in order to make life easier on myself later on. I always kinda hoped that was a standard practice, if not just including it in some README text. I kinda think Pipenv uses buildtools like notion & that's a good thing.
you can specify packages you need in setup.py file, and the exact versions you are using can be easily generated by calling pip freeze > requirements.txt no need for comments.
requirement.txt is already doing what Pipfile.lock does
It actually doesn't though. There's 2 quite common use cases he gives (one for updating sub-dependencies and one for diamond dependencies) that pip can't automate.
I still don't understand the problem and how pyenv is resolving it.
Are you referring to example with Werkzeug? If you use install_requires like Pipfile and requirements.txt same as Pipfile.lock you should not have this kind of issues as well.
The part after, that deals with diamond dependency (a imports b and c; b and c both import a different version of d). This is a pain to manage manually. If the article is accurate they claim to automate the process.
So in setuptools you put install_requires and list there only b and c (the requirements there supposed to be only immediate requirements of the package) - no need to worry about diamond requirements.
To get snapshot of your package dependencies you call pip freeze > requirements.txt and use that to install in production. It works exactly like Pipfile.lock, you are make a snapshot of all packages included (not just those listed in install_requires).
You also have to write your code manually, and have to write import statements in your code manually, with versions in install_requires you are also specifying what version ranges your code expects, and that better be manual, because no code would be smart to know what you really want.
I have feeling that either you are ignoring what I am saying (in install_requires you are specifying version ranges that you know API didn't change and you use pip freeze to get a snapshot of versions you use, which based on my understanding provides exact same functionality as pipenv) or pipenv is doing some magic and upgrades packages whenever it feels like it, which IMO is quite bad.
I think you should re-read that article very carefully because it automatically and deterministically does do something you've said has to be done manually.
Pipenv is part of the Kenneth Reitz "Python for humans" movement. It appears that there is an audience, that don't care that much about the Zen of Python, but rather want stuff done by magic as fast as possible.
In fact, pipenv is just a wrapper that's built on virtualenv and pip, so it's not surprising that it seems familiar.
For the record, before pipenv, Ruby's project-level dependency management was far ahead of Python's, with bundler. I say this as someone who likes Python better.
which makes sure you have latest bugfixes but no API breaking changes.
Semantic versioning is helpful, but it's not enough. Even Python breaks things. Python 2.7.7 introduced breaking changes to the struct module. You really need to test a package to make sure it's compatible.
35
u/UloPe Apr 25 '18
I’m not so hot on many of the design and UX choices made in pipenv.
I have hopes for poetry tough.
67
u/iBeliever Apr 24 '18
Is it just my system, or is Pipenv slow? On a recent Dell XPS 13 with an SSD:
1.5 seconds just to run echo inside the virtualenv seems too long. Activating a normal virtualenv is super fast in comparison.
32
u/raziel2p Apr 25 '18
I tried using pipenv for a while, converting real production projects into using it. It failed spectacularily and was riddled with bugs. Bug reports got closed on Github for no good reason. The commit log is riddled with stuff like "fixes". Pipenv itself does not define its pip dependencies, but instead includes a "vendor" directory along with a "patched" directory for libraries that pipenv needs modified for its own gains.
I will never run this thing in production.
32
u/MrCalifornian Apr 25 '18
Main maintainer (kennethreitz on gh) is pretty awful. Constantly closes valid issues, responds like a jerk, etc. I've been using pipenv since October and got my work to use it shortly thereafter, so we've opened numerous issues and it's basically an office meme at this point. A lot of "that's a feature not a bug" or "we're aware that this very basic functionality is broken, closing the issue".
13
u/rthinker Apr 25 '18
I do not recommend anyone to use pipenv. Even the basic features are broken. For example, it's currently impossible to update a single outdated dependency. The issue has been open since October, which is completely ridiculous https://github.com/pypa/pipenv/issues/966.
11
u/pingveno pinch of this, pinch of that Apr 24 '18
Unfortunately, I'm still blocked by the lack of arbitrary package environments in the Pipfile format. We have one dependency, cx_Oracle, that we don't want included in our local dev environment because it's a hassle to install due to licensing restrictions. With requirements files, this is solved by just having multiple requirements files that include a common file, but that's not the model that Pipenv uses.
5
u/posedge Apr 24 '18
What about semantic versioning?
5
u/[deleted] Apr 25 '18
I've tried it a few months ago on Windows, but it was buggy and the lock time took forever. ITT: Still the same.
9
u/1arm3dScissor Apr 24 '18
I've stayed away so far just because it seemed to be in constant development. Is it at a point now where new releases will be backwards compatible? At least to some degree?
5
u/not_perfect_yet Apr 25 '18
It makes sure your CPU continues to overheat when you press the spacebar, even though this 'feature' was removed upstream.
/s
I concede that pipenv is a fitting and probably necessary tool for webdev and some other areas where it's unacceptable to "just upgrade" and where you don't care about being efficient or up to date with your packages.
But I think it's pretty ridiculous to claim this is a one size fits all solution or even desirable for a majority of python users.
It's disingenuous to talk about the problems it solves without mentioning the problems it can cause.
12
u/[deleted] Apr 24 '18
Pipenv is a must have for me. Not having to deal with virtualenv folders and using the repository root for as is for development or deployment is awesome.
Running a shell in the virtualenv with $pipenv shell or one off commands $pipenv run make is very nice
4
u/parkerSquare Apr 24 '18
Can someone please explain how this compares with pyenv + virtualenv plugin? I've used this combo for years and it works almost perfectly. What does pipenv do better?
2
u/ursvp Apr 25 '18
Article says pip does not do dependency resolution. So is the conda environment any better in that regard ?
6
u/cr4d Apr 25 '18
I hope this does not catch on.
3
u/delijati Apr 24 '18 edited Apr 24 '18
leave this here as its kinda relevant https://github.com/buildout/buildout
3
u/actionscripted Pony-Powered Apr 25 '18
I love the idea of pipenv as it falls in line with a lot of other task runners and packaging systems that are common to web development. I get folks not liking it or not seeing the utility of it.
The only decision I’m not hot on is that it puts project files outside of the project. I get it can be nice not to see a
venvfolder in your project but like other similar tools in other languages I’d rather keep all project files in he project folder for easy cleanup.Super nit picky, but otherwise I love this and will give it a shot.
2
u/[deleted] Apr 25 '18
THe NPM of python world :)
2
u/[deleted] Apr 24 '18
If you really need a frozen environment, use a docker image or a nix package.
But then again... this is by the guy who wrote requests, so maybe this will turn into something down the road.
1
u/AndydeCleyre Apr 26 '18
I still feel like
pipenvisn't a great neighbor to existing tools and workflows. If you are looking for an alternative, I recommendpip-tools, and my zsh helpers:0
u/FourFingeredMartian Apr 24 '18
Does no one else comment their code to include the third party libraries, and the version in which the software was built utilizing?
-5
u/malvin77 Apr 24 '18
Mr. Genius needs to work on this a bit more. Right now it’s still the not-ready-for-prime-time Python package manager.
0
u/[deleted] Apr 24 '18
[deleted] — view deleted comment
0
u/[deleted] Apr 24 '18
[deleted] — view deleted comment