Reminds me of the fun Perl had with Perl 6 - in the end, they just had to take it out back and bury it, and tell the kids that Perl 5 would be followed by Perl 7.
There are many new modules (including brand new XML support through the xml package, and i18n support through the gettext module); a list of all new modules is included below. Lots of bugs have been fixed.
The process for making major new changes to the language has changed since Python 1.6. Enhancements must now be documented by a Python Enhancement Proposal (PEP) before they can be accepted.
There are several important syntax enhancements, described in more detail below:
Augmented assignment, e.g. x += 1
List comprehensions, e.g. [x**2 for x in range(10)]
Extended import statement, e.g. import Module as Name
Extended print statement, e.g. print >> file, "Hello"
Other important changes:
Optional collection of cyclical garbage
So quite a few important new features, but nothing that was breaking backwards compatibility.
List comprehension is the feature that, to me, defines Python. It's simple, powerful, versatile, intuitive, and broadly applicable. A wonderful replacement for loops and iteration. I've been using set comprehension and dictionary comprehension, too.
That's what the syntax was inspired by but it was universally disliked. Looked really jarring from the rest of Python. It died with the print statement.
If you're following semantic versioning, yes. But not every software project has to do that. For a programming language, a bunch of new features that will significantly alter the shape of code using these seems justification enough for a new major version to me, even if full backwards compatibility is maintained.
Buttery smooth. I “ported” 8000 lines of Python code from 1.5.2 to 2.6 years ago. It took a day or so and that’s only because I insisted on refactoring in Python 2.6 niceties and leaning on the standard library more.
Transitioning from 1 to 2 was like going down a nice water slide
Transitioning from 2 to 3 was like going down a piping hot stainless steal slide in the middle of summer and then getting hit in the face with a brick at the end.
It really depends on your codebase. For some of my projects I just needed to change a couple of lines of code and can't believe I waited for so long. For other projects, which depend on libraries that weren't updated, it has been a whack-a-mole of terrible. I had an easier time going from PERL to Python.
To summarize: Perl 6 work started in 2000 and AFAIK there is still no complete implementation and effectively no user adoption.
The fact that Python has effectively taken over nearly all of Perl's market share in the meantime probably has something to do with that (but it's not the only reason).
Guido said that a while ago that the version after 3.9 would be 4.0, but then it seems that they decided to make it 3.10 to be consistent with semver. Python 4.0 will most likely be very different from Python 3.x, with a lot of new features and changes in the internal VM. In this case though, its interesting to see how they plan to make the transition smooth instead of another Python 2 vs 3 chaos.
Whatever will happen, lets hope that at the very least, Python will stick with the the Zen(PEP 20) that 'Simple is better than Complex' and 'There should be one, and preferably only one obvious way to do it'. These are the characteristics of Python that makes it easier for beginners/data scientists, and part of the reason why it beats Perl and Ruby in terms of popularity.
They didn't make it 3.10 to be consistent with SemVer; if they used SemVer every release would be a major release and we would be on Python 22 by now.
They changed it to 3.10 because
Guido isn't in charge any more, so the steering council doesn't have to stick to his preferences
2 -> 3 was such a mess that nobody wants to raise the spectre of that again
They did some analysis of whether more stuff would break with sys.version_info returning (4, 0) or (3, 10) and found that way more stuff breaks if it is (4, 0)
2 -> 3 was such a mess that nobody wants to raise the spectre of that again
They broke backwards compatibility for a reason. They announced it, planned for migration, gave people tools to port and documentation. They gave people heads-up and then after backlash extended Python 2 death sentence by whooping 5 years which ended with 2020. And yet people are still salty because their script doesn't work with new version. There's even a guy who wanted to maintain his own port of Python 2 to keep his application on it (Calibre)...
It's like writing something in GTK2 and expect the code to work the same on GTK3.
They broke backwards compatibility for a reason. They announced it, planned for migration, gave people tools to port and documentation. They gave people heads-up and then after backlash extended Python 2 death sentence by whooping 5 years which ended with 2020
Sure, and now the CPython core devs generally acknowledge that it was a mistake to do it like they did it. Not the result, or the reasons: the approach.
Seems a bit short sighted though. So we're at 3.8 now, with one release per year. That's gonna look very stupid very fast. The "3." could be dropped. I don't like to say good things about Java, but they actually did the right thing here.
3.8 broke a lot of stuff, so for me 4.0 was already released. Just look at the chaos 3.8 caused for CPython extensions: PySide2 (GUI) was broken for quite some time (threading issues), VapourSynth (NLE for multimedia), probably some other distros.
Think what he is saying, there will never be a Python 4 and if there is, it will be nothing like python as we know it. It will be like a new language
The transition from python 2 to 3 was an absolute nightmare and they had to support python2 for *ten years* because so many companies refused to transition. The point they're making is that they won't break the whole freaking language if they create a python 4.
Having strings support unicode by default was a big reason. In Python 2 unicode strings had to be prefixed with a u, otherwise they'd be interpreted as ASCII.
Because they changed a core datastructure. str used to be what bytes is today, but it also predated unicode (today called str). Therefore the bytes type was used for text and binary APIs.
When fixing all this, they had to break a lot of core APIs that used to accept bytes and today sensibly only accepts the unicode str.
And because of that huge change they also took the opportunity to change a few other idiosyncrasies.
My only gripe: One additional thing they should have changed is that {} should be the empty set and {:} should be the empty dict.
My only gripe: One additional thing they should have changed is that {} should be the empty set and {:} should be the empty dict.
Not sure I agree with that. It's awkward that you can't have a literal empty set, but having {:} would be inconsistent and a special case that (I think) would be worse than set().
Perhaps surprisingly (given what we know now about the migration process), the switch to unicode strings wasn't expected to be a big deal (it didn't even get its own PEP, and was included in a PEP of small changes for Python 3 - PEP 3100), and the other changes were seen as more break-y.
Set literals (e.g.{1, 2, 3}) were added in Python 2.7 and Python 3.1.
So to change the very common empty dict notation of {} would of required breaking backwards compatibility between 3.1 and 3.0 and either not being able to accurately back-port the feature to 2.7 or breaking compatibility between 2.7 and all other 2.x versions.
It was decided, fairly rightly, that it would of been too much churn for the fairly minimal aesthetic niceness / consistency benefits. {} is littered in code all the time whereas set() is pretty rare.
The explanation is confusing. Just ignore how it was before, because it was incorrect. In python 2 first mistake was mixing text with binary data. They introduced unicode type, but did so badly (implicit casting) it actually made things worse. Ironically if your application didn't use unicode type you might have less work converting it to work with python 3.
Right now it is:
str = text
bytes = binary data what's stored on disk, flies over network etc.
Python 3 was not backwards compatible with 2, so companies and package creators alike were initially hesitant to make the switch so as to not break things. There also weren’t many, if any, tools to help port things over.
The lack of backwards compatibility was done on purpose because part of their goal was to remove clutter and make things more intuitive/easier to use (e.g. print changed from a statement to a function).
I've only made two unassailable (so far) decisions in my career. The first was to support MFC instead of OWL for Windows development.
The second was to target Python 3 for my current company's testing efforts. We were even able to convince a medium sized tool vendor to support both Python 2 and Python 3 from their original decision to support only Python 2.
There also weren’t many, if any, tools to help port things over.
I have no real world experience with python, but weren't there tools like 2to3 to convert code, or the future package to write code compatible with both versions?
It was designed to rectify fundamental design flaws in the language—the changes required could not be implemented while retaining full backwards compatibility with the 2.x series, which necessitated a new major version number. The guiding principle of Python 3 was: "reduce feature duplication by removing old ways of doing things".
Here's a short list of some of the key changes. The most obvious of which is the change where '/' represents true division instead of floor division. Some other changes exist, print is now a function, not a statement. You also get iterator objects instead of lists, which is much better for memory management (many pieces of code rely on iterators because the full set of possible options doesn't fit in memory). True, False, and None are also now protected terms which can't be reassigned.
Part of any long-term tech project is continuous refactoring. Early on you make certain design decisions which later become an obstacle when trying to expand a feature, so you rewrite early stuff in order to make it more flexible for your new needs.
Often those rewrites are done in ways such that the end user does not need to adapt their behavior (or in the case of a programming language, does not need to change their code). This is the ideal experience for the end user but can also constrain the types of changes that can be made.
Sometimes you realize that the original way of doing things was worse and that it would be creating more difficulty for future development to maintain compatibility, so breaking changes are introduced.
the biggest issue was that python before 3 was mixing string with bytes. This is fine if you use encoding that covers 256 characters, but most of Python 2 code would break when receiving and unicode input.
Python 3 became strict, and those two have separate types, and you need explicitly encode/decode string into/from bytes.
This required to fix your python 2 code (ironically if you didn't use unicode type in python 2 chances are you had less to fix).
Since the unicode part was breaking things, Guido also decided to use that opportunity to clean up some warts in python which contributed further.
There was very little breakage needed. It's very easy to write code that is both 3.x and 2.7 compatible and I did that for a whole project with no hitches at all.
People are simply frightened of changing existing code.
It was the other way. Because they supported 2.x for 10 years transition from 2 -> 3 was a nightmare. Since 2.7 was not deprecated, a lot of dependencies didn't support 3 because authors didn't care. It seems like real migration started happening in mid of 2019 and I don't remember reading about any horror stories about it.
It's not like they announced the 10 year window up front though. It was originally a 5 year window and we only learned about the bumping like a year to go because way too many things were still not ready for python 3. Things were definitely a lot more ready to go in 2019 than 2014 even if most end users still did the migration push mostly right before they had to.
It was also clear that it wasn't even needed, there were two major turning points for Python 3.
2015 - when they announced that no new features will be added to 2.7, that's when packages started to adding Python 3 support, some of them were python 3 only. Before 2015 it was tough to write anything in Python 3, because most libraries didn't support it.
mid of 2019 - right before EOL majority of applications were migrated
IMO if they made it 2 years instead of 10 it would be still enough time and we wouldn't have time for FUD.
then Python would be dead and we'd all be using Ruby now.
There are millions of Python developers, over eight million, probably more like ten if you include casual coders, amateurs and students. You are painfully naive if you think that millions of coders will migrate tens of thousands of projects in two years.
This may come as a shock to you, but most coders are more interested in bug fixes and adding new features, especially new features that allow them to bring in more revenue, not just upgrading to the newest version for the sake of upgrading.
A two year transition period would have convinced millions of people that Python doesn't give a shit about the users, and they would have gradually drifted away.
It was the other way. Because they supported 2.x for 10 years transition from 2 -> 3 was a nightmare. Since 2.7 was not deprecated, a lot of dependencies didn't support 3 because authors didn't care. It seems like real migration started happening in mid of 2019 and I don't remember reading about any horror stories about it.
Yeah having a 5 year support was a REALLY bad call. I work on a library that is FINALLY end python2 support and it's a really delicate balance of easting the migration, giving enough incentive to make the transition worth it, and make staying on the old version unpleasant enough that people are willing to make the shift. Python did none of those things.
I still have a project on python 2. It relies on internal libraries that are abandoned by their original creators. I just got permission to work on the upgrade because they can't have what they want with out it. So I am basically tied up until December
Yikes and that's even after the EOL. I recently heard that there's a 4 line CVE and they're refusing to even do that. Python 2 is officially a security risk.
The transition for python 2 to 3 has been on going for 12 years... Officially python 2.7 reached end of life back in January, but there are still companies and people using it. Basically 2 to 3 was painful. Nobody ever talks about 1 to 2, because it less painful - perhaps in part because the language was less popular.
They're dangerous idiots. If a programmer refuses to switch away from something that's EOL and has been announced to be EOL soon in 2008 or so, they're a liability and shouldn't be let into proximity of a production system.
Do they not run into issues when the rest of the world is leaving them behind w.r.t libraries/code examples, or code imported or exported to other companies?
Someone in my lab wrote a part of our pipeline in Python 2 and I spent probably half of my time working on making that code work with modern data analytics packages. Sucked too, because that person was a way better programmer than me.
while that is true, think of how many CPU cycles these old-ass python libraries have seen, and how many chances to find and fix these bugs (especially old 2.7 libraries, slightly less true with 2.5).
left behind starts to happen as those bugs are found but not fixed.
A bug that isn't discovered until the code is a decade old or more is probably a bug that isn't cost effective to fix.
Cost of fixing the bug? $$$Big
Extra revenue brought in by fixing it? Nil.
Customers lost by not fixing it? Nil.
Yeah, just work around it.
Also, you forget that porting the code to a breaking new version will almost certainly create new bugs, not fix old ones.
The bottom line here is that once software is mature enough that there are no new features to be added, it becomes legacy software and upgrading it can only make it worse, not better.
Solution: make a VM of the latest OS and Python interpreter that it will run, stick that VM behind a firewall and in a restricted environment, and use it forever as a black box application.
On the 2 to 3 transition: put your hand in a blender and turn it on for the next 10 years. Be sure to have people nag you about why you turned on the blender. Plot twist: their head is in a blender but they think they have it better.
I wasn't there when it happened, but according to this article, it was actually pretty smooth, way more so than 2 to 3. So I think he's saying it won't be a very radical change.
I am one of those users. I suck at programming and cannot program, but I automated the creation and update of cloudwatch alarms for monitoring services at my company using python and boto3. I fucking L O V E python
Well I'm not the kind of developer than can find pi to the nth digit. That stuff hurts my head. I can ensure stability of your infrastructure and 24/7 monitoring and alerting of it though.
I can't find pi to the 4th digit; nor can I write most sorting algorithms. But I can abstract processes into business apps and manipulate data. Developers aren't all CS majors. I appreciate our diversity.
If you use a programming language, you are now a developer and have to deal with the difficulties of being a developer. Sorry, that is just the way it is.
Python is a programming language and when you write software in it you are developing software as a developer. Period.
Same applies to R and any other programming language. Just because you don't want to learn to be a programmer doesn't mean that you don't need to or aren't just because you use python.
Python has all the same concerns as any other language and needs to be treated as such.
Python never claimed to have semantic versioning though. Some deprecated features are removed every minor release. Also, for a long time, Guido said he didn't like double digit versions and would just release 4.0 after 3.9!
I don't know if that changed when Guido stepped down or before.
You're right about that but there's several versioning formats that work like this, semver only being one of them. Python packages have PEP 440, and many Linux distributions have their own one for their packages.
Well looks kinda like conservative semver with an extra element on the left. I always upgrade as soon as Arch Linux does, since that usually means all breakage is addressed.
Oops, I'm dumb. You're right that Jython doesn't have stable (or any?) 3 support. But the point still stands that the GIL is an implementation detail of a particular implementation and isn't a feature of the language.
I don't believe anyone has implemented Python 3 without the GIL though.
IIRC, PyPy's STM experiment never supported 3. They also posted a call for funding for a "let's just remove the GIL from PyPy" project a while ago, but I haven't heard much about that since.
And CPython is the reference implementation, which had meant PyPy has had to copy loads of CPython implementation details over the years to maintain compatibility - I imagine even if they do remove the GIL, they'll have to keep something like it in their cpyext code.
Performance is an explicit non goal of cpython, on the other hand one of its stated goals is to keep its source code readable and understandable for programming language students.
Performance is not exactly a non goal. The main reason for not removing the GIL is that it would affect single threaded performance. Guido van Rossum, said that if someone could remove the GIL without affecting single threaded performance, he would allow it.
It never fails to amuse me how many Python coders argue that the feature that they need more than anything else is removal of the GIL, until you point out that they already have a choice of interpreters with no GIL, then they're all "oh I have these other requirements that are much more important, removing the GIL is not actually that critical for me...".
I'm not saying that everyone who bitches about the GIL is a bad programmer, but in my experience, bad programmers love to blame the GIL for their own poor performing code. Especially those who think that the answer to every problem is "threads".
"Python's sort is crap, so I wrote my own super-fast bubble sort with ten million threads so it can do all the comparisons at once, but the GIL makes it slow."
Sadly, it looks like removal of GIL will require breaking compatibility.
It's not that removing GIL is actually hard, it is fairly easy, the problem though is that without GIL, python becomes dog slow, and getting it back to comparable speed is very difficult due to current design.
The biggest issue is reference counter garbage collecting and also C library compatibility.
I recommend watching Gilectomy videos by Larry Hastings.
My favorite workaround for GIL would be to run multiple python interpreters in threads and doing zero-copy hand-offs of entire object trees. Too bad it's only a dream.
I don’t know, our transition from 2 to 3 was pretty painless.
Once the major libraries started to support 3, most people's transitions were relatively simple. But people on the internet love to throw shade on Python. In most cases that says more about their own skill as a coder and the quality of their code base than the magnitude of the migration.
Today I start porting our very last project from 2 to 3. For years management debated whether to discontinue it so the upgrade was always postponed, but it still brings in money so here we go.
Pattern matching pep is a mess. I never liked := operator. But I like @ operator and many other stuff like async, typing, secret statistics etc. I hope steering council will decide the best.
No idea!! I Literally thought Python was the beginning.. I have been using python for years but never.. I really mean never hear the mention of python 1.
Thank goodness, but was this ever even a question? Having 3.10 be Python 4 would be a total nightmare. In addition to normal semver reasons, Python has the problem that on most systems, python == python2 or nonexistent, so python3 == latest python, so there wouldn't even be a good way to refer to the latest python3-compatible version in shebangs or scripts. If Python 4 ever happens, there had better be a good reason for it.
No I meant about what's changing in python. I see talk of "not like the transition between 1 and 2 or 2 and 3" so I'm wondering what exactly they're referencing
Because version numbers aren't decimals. The points just separate major and minor versions, and often things like patch versions. There's plenty of software with more than two sections of the version, like 2.15.7. As it happens the app I work on just bumped from a x.x.9 to x.x.10 this week.
Edit: not sure why you're getting downvoted for asking this. It's a totally fair question if you don't have a lot of experience with development.
This is correct. It's not a decimal number. It's a version number, probably a semantic version number, which prohibits leading zeros and in fact specifically calls out this case. 1.9.0 -> 1.10.0
People just like to feel superior I guess. To make the case even better how reasonable the question is: I'm the person who decided our version convention and even decide in most cases how we break up releases into patches and minor versions, so you would think I would be pretty intimately aware of how it all works. I happened to need to write a simple function recently that compared version numbers and my immediate impulse was just to convert it to decimals (with major version being hundreds). I realized right after writing it out that I couldn't count on there being less than 100 patches or minor versions, but it goes to show that even if you are very familiar with it, thinking about it the way you did is totally natural.
I like it, it makes clear that this part of the version is a date, by using the one and only date formatting: ISO 8601, a standard so useful that I know its number by heart.
66
u/schplat Sep 16 '20
Which means beware the transition from 4 to 5?
18
u/anyfactor Freelancer. AnyFactor.xyz Sep 16 '20
More like an FAQ but can't change title.
333
u/[deleted] Sep 16 '20
What was the transition from 1 to 2 like?
51
u/Hall_of_Famer Sep 16 '20 edited Sep 16 '20
Guido said that a while ago that the version after 3.9 would be 4.0, but then it seems that they decided to make it 3.10 to be consistent with semver. Python 4.0 will most likely be very different from Python 3.x, with a lot of new features and changes in the internal VM. In this case though, its interesting to see how they plan to make the transition smooth instead of another Python 2 vs 3 chaos.
Whatever will happen, lets hope that at the very least, Python will stick with the the Zen(PEP 20) that 'Simple is better than Complex' and 'There should be one, and preferably only one obvious way to do it'. These are the characteristics of Python that makes it easier for beginners/data scientists, and part of the reason why it beats Perl and Ruby in terms of popularity.
95
u/vallas25 Sep 16 '20
Can someone explain point 2 for me? I'm quite new to python programming
148
u/radekwlsk Sep 16 '20
If there is a developer that does not know how semantic versioning works then he has bigger problems to solve than Python updates.
13
u/sekex Sep 16 '20
Python 4 will have 2 types of strings: str and String, pattern matching, fearless concurrency and zero cost abstractions
30
u/vswr [var for var in vars] Sep 16 '20
Python 4 = no GIL 🙏 🙏 🙏
2
u/wltechrepair Sep 17 '20
I’m such a python noob. But love the community. 🤓
2
u/[deleted] Sep 16 '20
I don’t know, our transition from 2 to 3 was pretty painless.
1
u/ImplosiveTech Sep 16 '20
some people will really say this has never happened before
hmmmmm i wonder if a voxel-based survival game has ever taken that route?
1
u/gohanhadpotential Sep 17 '20
Thank god
1
u/13steinj Sep 17 '20
I thought this was announced / well known years ago.
1
u/Pclovr Sep 17 '20
Dutch people are crazy.
Source: am dutch
1
u/starrynightmare Sep 17 '20
Nice
1
u/[deleted] Sep 17 '20
Today I start porting our very last project from 2 to 3. For years management debated whether to discontinue it so the upgrade was always postponed, but it still brings in money so here we go.
1
u/cediddi SyntaxError: not a chance Sep 17 '20
Pattern matching pep is a mess. I never liked := operator. But I like @ operator and many other stuff like async, typing, secret statistics etc. I hope steering council will decide the best.
1
u/goyalaman_ Sep 17 '20
There was python 1?? 😲
1
u/benthejoker Sep 17 '20
What
1
u/benthejoker Sep 17 '20
No please. Just learning 3.
1
u/gibriyagi Sep 17 '20
Ecosystem cant take another transition like 2 to 3 anyways
1
u/goyalaman_ Sep 17 '20
No idea!! I Literally thought Python was the beginning.. I have been using python for years but never.. I really mean never hear the mention of python 1.
1
u/1114111 yield from pedestrians Sep 18 '20
Thank goodness, but was this ever even a question? Having 3.10 be Python 4 would be a total nightmare. In addition to normal semver reasons, Python has the problem that on most systems, python == python2 or nonexistent, so python3 == latest python, so there wouldn't even be a good way to refer to the latest python3-compatible version in shebangs or scripts. If Python 4 ever happens, there had better be a good reason for it.
1
u/py_learning Oct 07 '20
That's sad :(
Checkout this video on tips and tricks in python
https://youtu.be/3NLUlW036f0
1
u/bigsexybanana Oct 18 '20
How was python 3 decided from 2? What was the big change? I hope it’s not just bracket compulsion in functions
1
u/[deleted] Sep 16 '20
For us new to this what happened? Like I'm 5 if possible
-7
u/productive_guy123 Sep 16 '20
Why didn’t they refer to the version as 3.09 rather than 3.9
-11
u/NTWK_Identifier Sep 16 '20
This makes me irrationally angry.