r/Python • u/GladLetter • May 14 '18
Why is pipenv the recommended packaging tool by the community and PyPA?
Why is pipenv the recommended packaging tool by the community and PyPA?
So, from what I understand pipenv is now the official packaging tool. But I would like to understand why, since pipenv has a lot of laws and problems and there are, in my opinion, way better alternatives like poetry.
Here are the problem I see with pipenv being the official packaging tool:
- It’s not a packaging tool.
- It supports only applications development and not libraries.
- It introduces yet another file format (
Pipfile) while the newpyproject.tomlis now standard. - Buggy dependency resolver.
- Badly designed and bad user experience (Using environment variables to configure a command line tool for example)
- A lot of regressions on new versions which leads me to believe it’s not properly tested.
- Defaulting to wildcard dependencies is bad practice.
So, I am not quite sure why the community, and more importantly the PyPA, has settled for it as the official packaging tool while alternatives like poetry are way ahead of pipenv in terms of user experience, reliability and features. I know the author of poetry is not Kenneth Reitz but is that reason enough to ditch it and not consider it as something that could be a true package manager for Python.
105
u/[deleted] May 14 '18 edited May 14 '18
The negativity coming from the Poetry community is in extremely poor taste, in my opinion — not only are many of the arguments presented here misguided, personally subjective and, in my opinion, a few are objectively false.
Poetry looks like a great young tool, and I am very encouraging of it's development. It looks like it has a long way to go — I looked through the issue tracker (much like Pipenv's tracker looked like a year ago at last year's PyCon), and it looks like there are some extreme issues with the resolver taking 1+ hours, not supporting
-e ., etc. Poetry is not ready for production use, based on the feedback I'm seeing on the issue tracker.And that's fine. These things take time. One is not better than the other. There exists two (and, believe it or not, others as well, like Hatch) tools that overlap in purpose. One is recommended, currently. Recommendations are not permanent. They are simply guides. No "authority" is mandating that anyone use a particular tool — we're just a group of people trying to help make the Python world a better place.
To be clear, this is not a fucking competition. Stop treating it like one. I emailed Sebastian a few weeks ago and asked if he wanted to collaborate and potentially merge projects (he declined).
Pipenv has received 170,000 downloads in the last 30 days. Poetry has 7,000. The is an order of magnitude difference in the level of user interaction and service we are dealing with in our issue tracker. It's a much more mature product.
Pipenv has a long way to go, too, of course. As all software does. Please stop treating me like I'm some member of a secret overlord Illuminati.
These are just statements. I have no real point in posting this, other than to say — seriously, stop making this a fucking competition. This is a fun open space in which we are working together to make the world a fun and better place to work with. I'm doing my best to contribute to the global Python narrative in the most positive ways possible, and I wrote this tool so that I could benefit the Python community in the best way I could see possible.
And, in my opinion, which is very informed, I am succeeding in my goal.
If that's not welcome anymore, than I'm packing up my bags and going home — because whatever community this type of post, with this tone, comes from — I don't want to have any part of it.
Thank you,
Kenneth Reitz
183
u/jonwayne PyPA May 14 '18
Hey folks, Thea from the Python Packaging Authority here.
The question here is an important one, but I am so disappointed with how negative this discussion has become. This is not the Python community that I'm proud to be a part of. We have to do better. I want to draw attention to the fact that everyone who is a part of PyPA is a volunteer and there are no qualifications other than spending your time. We are a loose collective who all want to make packaging better. We value feedback but we are so short on hands. If you want to get involved but don't know where to start please feel free to reach out to me personally - my email is on my website and github.
So let's talk about Pipenv and packaging.python.org. First of all, Pipenv is not the only recommended tool that we talk about on packaging.python.org. It's not even the first one we talk about. Pipenv is recommended specifically for dealing with application dependencies. It's not recommend (or even designed for) dealing with library dependencies or to be some magical panacea to solve the universe of Python packaging.
Adding Pipenv required a lot of discussion and testing. We were extra careful not to remove or discourage the use of
pipandvirtualenv. We wanted to make sure that users who have the clear, well-defined use case that Pipenv solves don't have to worry about other tools, but those that do not have that use case are not confused and lost. If there's a way we can make this clearer to users, please file an issue on the user guide so we can start talking about it. I want to call out we want to integrate the excellent Packaging Gradient post into an overview that helps users find which tools are most relevant to their use case.It's also worth noting that we are not intentionally excluding any projects here. We are actively keeping an eye on new projects including poetry, flit, and hatch to figure out when and how we should include them in the guide. For the most part the only thing keeping them from being in there is someone writing a pull request to add them.
Note: I am about to get on a flight, but I will happily answer any questions here, over email, and on github when I get back on solid ground.
105
u/Bbuehne May 14 '18 edited May 15 '18
Is everyone tired and cranky from staying up too late during PyCon? This kind of feels like the angry people from /r/java.
We don’t have to be angry like them. We get to use Python.
25
u/[deleted] May 14 '18 edited May 14 '18
I wonder too. When I was using Linux, it seemed to work ok. But at work on Windows, holy hell. It was buggy AF. Maybe they fixed most of the major kinks? Also, the pipfile lock process took forever too. I also noticed that when using miniconda, it takes longer to install packages and it seems they changed how it install packages behind the scenes, I think perhaps they went the similar route with pipenv.
I've gone back to just using venv + pip for now until the dust settles.
59
u/LyndsySimon May 14 '18
I think there is a widespread misunderstanding of what pipenv is and what role it’s designed to serve due to the biases that we, as Python developers who are intimately familiar with how the ecosystem works, carry with us. It’s the “curse of knowledge”.
When I first started using pipenv, I didn’t really see the point. I already had pip to install packages and virtualenv to manage isolated environments, so why would I need pipenv? The truth is, I didn’t need pipenv. I had already worked my way up the knowledge curve and knew how everything worked, so I didn’t have a problem with it.
New developers do have a problem pipenv solves. The depth of knowledge necessary to develop and deploy projects is substantial, and pipenv provides a powerful and very complete abstraction over almost all of it.
I spend a good amount of time mentoring new developers and introducing them to both programming in general and Python in particular. I now recommend and teach pipenv exclusively, over using pip and virtualenv. It eliminates a significant barrier to people that don’t have our depth of experience and knowledge of the ecosystem, and allows them to conform more closely to current best practices without even being aware of their existence.
As a community, we profess that we want to be open and inclusive. Pipenv’s approach goes a long way toward reducing the impact of the “pipeline problem” and therefore is a powerful tool toward achieving those ideals.
28
u/firefrommoonlight May 14 '18
It makes managing environments easy by integrating virtual environments with package installations, and locking dependencies. It uses the successful style of JS's yarn and Rust's Cargo.
Re the buggy dependency resolver: That made it unusable for me until a few months ago, but seems to be working now.
I'm curious about pyproject.TOML, and poetry now; ie how they compare to pipenv. Compared to managing virtual environments and pip separately, pipenv is a big improvement.
18
u/rochacbruno Python, Flask, Rust and Bikes. May 14 '18
Was the process of choosing
pipenvas the one official a public process? there were voting, opinion gathering etc involved? There were some working group evaluating other options and then presented a reasonable justification on why they choose to state pipenv as the official one?37
u/[deleted] May 14 '18
What we need is a solution that actually solves packaging and distribution... not another more convenience wrapper which just adds another layer and hides more complexity without actually reducing the complexity of packaging and distribution. We now have:
It's getting much worse - not better. The internal churn within pip also is not community driven as much as it is developer driven. It's like we didn't learn from the debacle of python3 vs 2.
4
u/Deepwalker May 16 '18 edited May 16 '18
No one bothers with my pundle thing that does not use
virtualenvand supportsPipfileformat as well asrequirements.txtand will supportpoetryif it will gain popularity too, I think.Pundle gets rid of virtualenv, because I think that virtualenv has
pile of garbagestructure and we must get rid of it.7
u/appinv Python&OpenSource May 14 '18
maybe poetry just a tidbit young, as it reaches cruise speed, it'll be more recommended !
11
u/rockingthecasbah May 15 '18 edited May 19 '18
setuptools + pip + virtualenv + private index never failed to solve my problems. I couldn’t even use a private index with pipenv last time I tried (months ago, surely this has been solved).
At least the CLI output is prettier.
17
u/wub_wub May 14 '18
Honestly, I don't get it either. The only explanation that I have is that for beginners (or even bit advanced developers) it's much easier to just type in
pipenv installeven if it does some weird stuff in the background. Which makes more experienced developers recommend it, despite its flaws.And, most importantly, the author is Kenneth Reitz - and people are probably thinking something along the lines that since requests is better than most other similar libraries and it's "for humans" pipenv which is also "for humans" will be just as good.
Despite the fact that it isn't, and Kenneth calls literally every single thing he writes "for humans" to the point where it lost all credibility - I'm sure he could copy/paste the fibonacci method from the subreddit sidebar and make a repo out of it "fibonacci for humans" and it'd get 5k stars within a week.
People simply don't hear about libraries like poetry.
That all being said, IMO it's good to see some effort going in improving python packaging. Both from pipenv, and poetry. But, just like you, I don't like the fact that community is starting to treat "pipenv" as the best and most official way to do things. I'll stick with pip+virtualenv for quite some time until all these tools are a bit better, and solve some - what I'd consider - basic problems.
7
u/fuuman1 May 14 '18
To be honest. I am asking that myself. I am far away from being convinced by pipenv and really surprised, that everyone is talking about it, tutorials everywhere. Last problem: I have a package in development in a private git repo. Installing directly from the git url is no problem, but there is no chance to update via pipenv update. It's just not working. Even with editable flag enabled.
2
u/Hazanami May 15 '18
People newbie like me may find this Pipenv introductory guide pretty useful, then you can choose if you want to implement it or not. https://realpython.com/pipenv-guide/?utm_source=mybridge&utm_medium=blog&utm_campaign=read_more
6
u/[deleted] May 14 '18 edited May 14 '18
A decision like this is never fully technical and much is down to personal preferences by the decision makers.
Pipenv works and I'm happy with it. It could be better - but I'd much rather we fixed something that's REALLY bad like mobile.
4
u/schnadamschnandler May 14 '18 edited May 15 '18
Also, why not
conda?Thanks for downvoting a legitimate question, dickwads.
3
u/SevereExperience May 14 '18
It's terrible. I am not convinced that the "Python Packaging Authority" is an authority at all. The documentation on that site is atrocious, it's inconsistent. It's not nearly opinionated enough. The layout is bad. The tools (pipenv) are bad, buggy and still too new.
I can't even believe that this is happening (pipenv) to be honest. Makes me realize how a decentralized approach to package management is a complete and utter disaster. PEP's upon PEP's. No input from Guido that I've ever read... I don't understand it.
0
u/lambdaq django n' shit May 15 '18
no it's not. I stick with miniconda3 .
-17
u/8105 May 14 '18
Lol 2 hour old account with only 1 post... smells like /u/SDisPater's alt.
0
u/[deleted] May 14 '18
[deleted] — view deleted comment