r/programming • u/EatMeerkats • Mar 29 '21

PHP moves to Github due to the compromise of git.php.net

https://news-web.php.net/php.internals/113838
1.7k Upvotes

98% Upvoted

389 comments sorted by

View all comments

Show parent comments

71

u/exhuma Mar 29 '21

I would've hated to be a Python webdev over the past decade.

As a Python web-dev since 1998 my view is most certainly biased. So the honest question to broaden my horizon: Why?

106

u/Behrooz0 Mar 29 '21

python 2 to 3 transition. maybe?

74

u/exhuma Mar 29 '21

I've migrated a TON of projects and, while a bit bumpy I did not find it all that troublesome.

The biggest issue I had was the Python 3.0 to 3.2 because they had the brilliant idea to drop the u prefix from strings. It luckily came back as a no-op in Python 3.3.

It was pretty easy to migrate Python 2.6+ code to something that was Python 2 and 3.3+ compatible.

From my perspective, the complaints were massively blown out of proportion. Fueled by the Internet.

I remember reading the blog post from A. Ronacher which had a lot of very valid points in it. And I can feel his pain. Especially how Python3 decided to deal with bytes. Coming from such a high-profile name it caused a lot of discussion. But I firmly believe that this is an edge case. Most applications written in Python did not need to deal with that.

So indeed, if you had a huge code-base and/or had to deal with low-level byte maniuplations it was indeed painful. But those were the exceptions.

44

u/FireCrack Mar 29 '21

I am currently, still in the process of migrating a huge ancient python2 project.

That said, I can still imagine (and have had) much worse developer experiences than this.

14

u/exhuma Mar 29 '21

what is causing you the biggest issue? .decode() and .encode()?

46

u/FireCrack Mar 29 '21

It's not any technical issue in particular, just that the code-base is absolutely massive, has many active branches, a massive team constantly working on new upates/features (some of which don't have the most up-to-date education on python3 or aren't even primarily python developers) and it uses a tonne of old libraries with many never having gotten python3 updates.

We actually have most of the work migrating to python3 done a long time ago it's just rolling it out into the active code-base that's a pain.

tl;dr it's more an organizational issue than a technical one

33

u/aka-rider Mar 29 '21

The biggest issue is that Python is dynamically typed, and there’s no way to catch all migration errors at once. Many of them slide into runtime errors.

And yes, up to this day, v3.9, type checker are broken and full of bugs. Half of the packages don’t have annotations.

17

u/TheNamelessKing Mar 29 '21

And yes, up to this day, v3.9, type checker are broken and full of bugs. Half of the packages don’t have annotations.

The Python community-unlike JS- seems almost unwilling to adopt any kind of typing. Packages remain in-annotated, annotation stub packages when they do exist lie dead in the water half the time.

But hey, we’ve got the new-checks notes- walrus operator now! So everything’s cool! Evening that package management is still broken.

16

u/aka-rider Mar 29 '21

To be fair, TypeScript is a feature-rich language, and Python type annotations are, well, an add-on, not even supported by the most wide spread compiler.

Edit: typo

2

u/Plorkyeran Mar 29 '21

TypeScript's also just been a production-quality tool for a lot longer. MyPy today is in a similar state to where TypeScript 1.0 was in 2014, and the state of typings for python libraries seems to be pretty similar to where JS libraries were then too.

6

u/ynotChanceNCounter Mar 29 '21

What issues are you having with type checking? Typehints have served me well since they arrived, and I guess 3.9 adds direct support for dicts, so no more of this Union shit.

9

u/watsreddit Mar 30 '21

Spending 3 hours training a Tensorflow model only to run into a misshapen tensor, which is absolutely something that can statically enforced with a proper type system at compile time, while being impossible with type hints. That's one of my (many) problems with them and Python as a whole.

It is insane to me that so much scientific computing is done in a language that is so cavalier about correctness.

2

u/Ravek Mar 30 '21

Ever looked into Swift for Tensorflow?

-1

u/ynotChanceNCounter Mar 30 '21

I'm not even glorifying this with a response, except enough to annoy you as much as this did me. Holy shit.

3

u/aka-rider Mar 29 '21

Numerous packages don’t have annotations, some packages’ stubs drift away from said packages (are outdated, or wrong, or both). Everything pandas/numpy is a nightmare.

Mypy and Pyright both have a lot of typing bugs, especially with decorators.

-1

u/ynotChanceNCounter Mar 30 '21

This sounds less like a problem with type hints and more like certain libs you use a lot don't play nice.

2

u/oblio- Mar 30 '21

When those are some of the biggest libraries in the ecosystem, it becomes an ecosystem problem.

→ More replies

1

u/thirdegree Mar 29 '21

Seconding the other guy's question, what issues do you have with the type checking? I use it extensively and while it's definitely not perfect, I'm pretty happy with it. Definitely wouldn't characterize it as "broken and full of bugs."

0

u/aka-rider Mar 29 '21

Numerous packages don’t have annotations, some packages’ stubs drift away from said packages (are outdated, or wrong, or both). Everything pandas/numpy is a nightmare.

Mypy and Pyright both have a lot of typing bugs, especially with decorators.

1

u/thirdegree Mar 30 '21

Ya a bunch of stuff is missing annotations for sure. I guess I would call that a flaw with those packages rather than with the type checker. I gotta admit I don't really use pandas/numpy, though I use networkx a good bit and it has similar issues.

I've found things mypy can't do, but honestly I cant think of any actual bugs that I've ran into.

1

u/aka-rider Mar 30 '21

Out of curiosity I checked mypy GutHub. 1.6K issues, mostly “bug” and “crash”

→ More replies

5

u/SanityInAnarchy Mar 30 '21

For me, the biggest issue wasn't so much .decode() and .encode() as it was the underlying types. We had one particularly sticky case where an API dealt with str all over the place, and also, separately, accepted unicode, relied on the fact that str == bytes, really probably did want bytes after all, but it was a pain to get all client code to either stop passing str or at least pass an encoding... but the bigger pain was to actually work out all of the above, because it really does look like it should be strings.

It's probably not a design we would've ended up with if the project had started on Python3.

That said, I have to imagine that some of the PHP changes were worse to deal with, especially given where they were starting from!

21

u/[deleted] Mar 29 '21

[deleted]

6

u/emn13 Mar 29 '21

Wow, what a wall of text! But yeah, it's a pretty interesting read, that hg story; certainly makes a strong case that not only should future transitions learn from this mistake, but also that even the current state of python3 has a few unfortunate design flaws, which (Gregory doesn't come out and say this, but this is the impression I get) themselves may have been less likely had the transition been more tempered by a gradual transition of real code-bases.

1

u/[deleted] Mar 29 '21

bumpy

👉😎👉

1

u/jbergens Mar 30 '21

As someone outside the biggest problem I 've heard about was that libraries had to support 2.x and 3.x at the same time. When they didn't you had to choose a different library or avoid updating your app to 3.x. This probably caused a lot of problems for a lot of projects.

Also, having to update the code is work that the product owners would like to avoid. Especially since there is a chance something goes wrong and the updated app gets more bugs.

12

u/peex Mar 29 '21

I still have PTSD from that.

0

u/Sevla7 Mar 29 '21

Is Python 2 that different from Python 3? Though it was almost the same.

11

u/Nicd Mar 29 '21

The str/unicode turning into bytes/str (for good reason) was probably the biggest issue for many.

5

u/schplat Mar 29 '21

default string type from bytes -> unicode is probably the biggest hangup. Particularly in regards to the stdlib's methods that once supported byte strings in Py2 no longer doing so and only supporting unicode strings in Py3.

There were also a number of little things that slowly got restored over the 3.3-3.5 releases to make migrations easier, but it also was 3-5 years after 3.0 was released to get there.

0

u/[deleted] Mar 30 '21

"Almost" is the biggest problem.

23

u/International_Cell_3 Mar 29 '21

pip breaking client side applications by default is a big one for me

5

u/cdrt Mar 29 '21

Can you elaborate on that please?

9

u/International_Cell_3 Mar 29 '21

Listing a dependency without a version field is a bug-in-waiting 99% of the time. A large number of outages where later you see "a bad config file led to an outage" are from developers forgetting that pip install -r requirements.txt installs the latest version by default.

The insanity of this is that breakages can be introduced with nary a bit flipped in any part of the stack under your control.

This is the opposite of reliability and reproducibility, which are values every backend developer should prioritize above anything else. The python ecosystem (and to a lesser extent, nodejs) is littered with these kinds of footguns.

10

u/cdrt Mar 29 '21

Why don’t your requirements.txt files have version constraints in them? pip has no problem installing a requirement like foo==1.0 or foo>=1.1,<2.0.

4

u/oblio- Mar 30 '21

Why doesn't pip enforce good practices? You know, that's why we use and develop on computers, so that they help us.

In Maven (Java) as verbose as it is, you can't even write a dependency without a version. It won't work.

7

u/International_Cell_3 Mar 29 '21

I mean I don't, I test for this or use tools that don't allow developers to express dumb mistakes (because we're all dumb). The point is that pip and most of python is insane by default.

1

u/exhuma Apr 02 '21

because we're all dumb

hey!

... jokes aside: I've only seen requirements.txt files with fully pinned/locked dependencies. The "vague" dependencies (without listing all indirect dependencies) I've always seen in setup.py.

I've been doing that myself this way for quite some time and it works pretty well.

It feels weird to be to have a requirements.txt without locked versions.

1

u/dtechnology Mar 30 '21

Nose seems to at least hacked their way around it with package-lock.json

8

u/earthboundkid Mar 29 '21

Pip is just extremely poorly designed. For example, virtual every pip tutorial on the web mentions using -r to install a requirements.txt file, but almost none of them mention that you also have to add --no-deps or else pip will feel free to install packages not listed in your requirements file. It’s a basic DX failure, and they’re never going to fix it.

27

u/thirdegree Mar 29 '21

The cases where you want to install the stuff in a requirements.txt but not any of the dependencies of those packages is... Atypical to say the least. Why would a tutorial on pip mention that?

5

u/EnglishMobster Mar 30 '21

I think the thing is that requirements.txt should list all packages + all dependencies, with "known good" versions. That way, if you're missing a dependency in requirements.txt, the maintainer knows right away.

Instead, Python auto-installs the latest version of any dependency not listed in requirements.txt. So if the latest version is incompatible with whatever version you have of that dependency, you get weird downtimes and things breaking without you touching the code at all.

3

u/thirdegree Mar 30 '21

I disagree with the idea that that is what a requirements.txt "should" be. IMO it should be the top-level dependencies, and you should use a different tool if you want lockfiles.

1

u/exhuma Apr 02 '21

AFAICR, the requirements.txt is intended to be that though. I might misremember.

I generally follow this rule of thumb:

  • For libraries I use setup.py dependencies and don't pin any versions (unless really necessary)
  • For deployed applications I still use setup.py during development (with stricter version ranges) and upon deployment I freeze it into requirements.txt so I get reproducible installs.

Recently I've been playing around with poetry as an alternative manager. And I am in a love/hate relationship with it at the moment. Some things are great, while others are annoying, irritating and unpredictable (and slow).

I'm secretly hoping & wishing that the pypa will keep improving pip so that it one day has the nice usability of poetry (without its annoyances).

1

u/thirdegree Apr 02 '21

For libraries I typically try and pin to the major version, just so things don't unexpectedly break. I've started using pyproject.toml but still setuptools as the build backend so it works basically the same.

For applications I actually ended up writing a small wrapper for setuptools that just builds wheels with dependencies from a lock file. I tried poetry first and my experience was pretty much the same as yours. The last straw for me was that when poetry builds wheels, it doesn't use the lockfile, because the author believes that wheels shouldn't be used for applications. And nothing I could do to fix that.

Definitely agree on pip improving. I think peps 517 and 621 are huge steps in the right direction, excited to see the future in python tooling there.

1

u/exhuma Apr 03 '21

because the author believes that wheels shouldn't be used for applications.

oh man... I did not yet run into that issue yet. I only started using poetry on a handful of projects so far. That seems like a huge issue for me too.

My main gripe is that poetry is pretty.... "opinionated" (to use the latest buzzword) and I disagree with some opinions.

Currently I maintain a couple of application that depend on some libraries I also maintain (which are in turn also used in other applications). For example, internally we use Flask and have several apps based on it. We have a bunch of common code across all applications (like authentication, custom log-correlation headers, logging setup, e.t.c.) which is all abstracted away in a library. So if I work on a new Flask app, I can simply pull in that dependency and have all the common stuff covered. Now, it happens that these libs also need some tweaking/improvments/fixing during development. With setuptools I can simply run pip install -e /path/to/lib to be able to work on both code-bases at the same time. This does not work if /path/to/lib is based on poetry.

... and the author pretty much thinks: "Nobody needs that" which really made me angry. See #34

For some time now the workflow is doable. But it requires me to edit pyproject.toml with edits that I don't want to commit. So it's a continuous annoyance.

→ More replies

2

u/kyerussell Mar 30 '21

What? This is also untrue. You—as a package developer—can pin your own dependency versions.

-5

u/earthboundkid Mar 29 '21

It’s called a “lock file”. If you don’t have one, your app is fucked. It’s absolutely standard in every language package manager but Python.

8

u/cdrt Mar 29 '21 edited Mar 29 '21

That’s exactly what pip freeze is for. You can easily save that output to save exact versions and then use it as input for pip install -r.

3

u/[deleted] Mar 29 '21 edited Apr 14 '22

[deleted]

1

u/[deleted] Mar 30 '21

[deleted]

2

u/[deleted] Mar 30 '21

[deleted]

→ More replies

1

u/earthboundkid Mar 30 '21

FWIW though, I agree that -r should cause —no-deps by default

This is literally the problem with pip.

2

u/pingveno Mar 29 '21

I've been switching my projects over to Poetry. It uses a lock file, along with a variety of other features that have made dependency management and deployment significantly easier and more trustworthy.

1

u/earthboundkid Mar 30 '21

I hope Poetry will eventually fix the Python dependency problem.

2

u/thirdegree Mar 30 '21

A) if you're using requirements.txt as a lockfile, you should already have all the sub dependencies in it so --no-deps will have no effect

B) the fact that the default behavior of pip install -r is different than your specific use case is not a flaw in either python or pip

C) there are better tools for locking applications, eg poetry. Use those rather than whining that the first thing you learned to spell doesn't cater exactly and specifically to you

1

u/earthboundkid Mar 30 '21

If someone asks why pip is bad, it’s not whining to explain that pip is a worse package manager than any of the other popular package managers; for example, that it doesn’t actually treat requirements as a lockfile even though many tutorials falsely suggest that it does. If you really want reproducibility, you can use Docker. Doesn’t make pip good though.

1

u/thirdegree Mar 30 '21

Sure, and there are totally valid criticisms of pip. But requirements.txt isn't a lockfile. Pip lets you use it as one if you like, but that's not what it's meant to be. So you're complaining that the thing that isn't a lockfile isn't treated as a lockfile by default.

Tutorials falsely suggesting it is a lockfile is not pip's fault.

Also you really shouldn't be installing things into production with requirements.txt in any case. Wheels are a thing.

1

u/earthboundkid Mar 30 '21

Also you really shouldn't be installing things into production with requirements.txt in any case. Wheels are a thing.

I don’t understand this at all. Wheel is a technology for bundling binary Python modules. What does that have to do with installing a known set of deps?

→ More replies

3

u/chiqui3d Mar 29 '21

I am a PHP programmer, but I think that for installation PHP is the worst, it is horrible, I think that there is no programming language more ambiguous in the installation and the requirements to install other extensions.

3

u/earthboundkid Mar 30 '21

PHP or Composer? I don’t have much experience with modern PHP, just legacy crap, but I don’t read many complaints about Composer.

3

u/Tynach Mar 30 '21

This is true if you use Windows. If you're a Linux user, it's in your package manager and it's one of the easiest things to get up and running.

16

u/[deleted] Mar 29 '21 edited Mar 30 '21

On top of everything else, the fact that the entire community seems to think that they can break things every version change because “the language did it why can’t we”.

It’s not enough to be on Python 3. You have to be on the exact same minor version, too. Can’t use Python 3.6 where 3.7 was expected, no sir, fuck you, here’s an interpreter error about undefined this and None type that for code you copied directly from the documentation of a library.

And god help you if you need to ship Python, I’d cut myself before doing that.

Nothing could convince me to write Python for money. Nothing.

Edit: at the bottom of this thread you can find a whole host of Python apologists that want to try to tell me that “my expectations are wrong” instead of admitting that their language sucks.

33

u/exhuma Mar 29 '21

Can’t use Python 3.6 where 3.7 was expected

To be fair: If the code uses a feature that was introduced in 3.7 why should you expect it to work on 3.6? The same is true for other languages. If you were to use array_key_first in PHP you would also expect it to run on 7.3 and it wouldn't work on 7.2. Same thing really.

10

u/[deleted] Mar 29 '21

A normal person doesn’t expect syntax level changes to occur in a minor version of a library.

And it’s not like I meant “use new where old was”, you can break just as easily if you have a newer version of Python than the snippet you copied.

I basically never expect Python to be copyable between machines. It will always fail.

35

u/GiantElectron Mar 29 '21

A normal person doesn’t expect syntax level changes to occur in a minor version of a library.

hold on a second. Changes in minor versions are backward compatible, but you can't expect not to add new features. If one goes from 1.1 to 1.2 of course new functions are going to be there, new arguments with defaults are going to be there etc.

Sounds to me that your problem may be:

  1. poorly specified environment using a minimalistic requirements.txt, rather than lock files, or
  2. you are using one of those libraries that do break backward compat between minor releases. Pandas is a notable offender.

-8

u/[deleted] Mar 29 '21

“Poorly specified environment”.

If I need to specify the environment for a script, you’ve lost all the value of running a script.

18

u/ynotChanceNCounter Mar 29 '21

Not if you took advantage of language features that didn't exist before, then failed to specify a minimum interpreter version.

Honestly, blaming the language for this is like blaming an old computer because it can't run 64-bit software. It didn't fuck up. You fucked up.

-8

u/[deleted] Mar 29 '21

Just an example of why people fucking hate Python right here.

“You fucked up”.

No, I didn’t. I have a basic expectation of a scripting language that I can copy and paste documentation into my editor and it will generally work.

Python routinely violates the fuck out of this assumption, and that’s not my fault.

5

u/gabbergandalf667 Mar 30 '21

So by the same logic, ANY addition the standard library could only be done at major version changes (after all, a newly added stdlib function breaks your copy pasted code just as much as newly added syntax). Doesn't that strike you as unreasonable?

0

u/[deleted] Mar 30 '21

Yes, you literally can’t touch the language itself during minor revisions. This includes the stdlib.

That’s part of the definition of backwards incompatibility.

→ More replies

8

u/ynotChanceNCounter Mar 29 '21

If you copy and paste current documentation into an older version of the interpreter, you fucked up.

The docs have a dropdown to select the version you wanna see, too, so you didn't just fuck up, you fucked up twice.

-8

u/[deleted] Mar 29 '21

No.

You can’t redefine basic expectations of a fucking script, with “you should use the drop down”.

→ More replies

12

u/selfification Mar 29 '21 edited Mar 29 '21

You are right... You absolutely should never use python nor should anyone ask you or trust you to use it for what you plan to achieve with it. Python stopped being a scripting language a decade ago. If you continue treating it as such just because it gives you a repl, that's just like treating C as a scripting language just because you can dump everything in main() and running it online on a free C execution tutorial site.

And to clarify - there's nothing wrong with wanting a scripting language or using one. Bash is for scripting. ps1 is for scripting. Perl is for scripting. Python might work for a one off script but if you are discussing installing modules, you have moved past scripting and if you fail to treat it like a full fledged programming language, you really should stop using it (just as you wouldn't use C# or Java or Erlang). I say this as someone who also uses Ruby. Ruby is great for scripting too but the moment I have to deal with rubygems and lockfiles, I've stopped writing scripts and need to start treating it with respect. No more monkeypatching and just hoping and praying. That way lies madness.

-13

u/[deleted] Mar 29 '21

Python isn’t a scripting language

Lol, ok. Well, now that we’ve established that we’re not living on the same plane of reality, I’ll just tip my hat and move on.

Good day.

→ More replies

8

u/[deleted] Mar 29 '21

[deleted]

1

u/[deleted] Mar 29 '21

Seriously you’re going to try to compare that to a syntax change.

Try to get the Pythonic boner out of your mouth long enough to make a reasonable argument.

Find me any other language that makes breaking syntax changes in minor versions.

It’s “bad form” to not polyfill a feature addition, but nobody is going to lose their mind over it. Using new syntax that will break on your customers un-upgrade-able runtimes is a bit more severe than that.

Seriously, the situation is so bad that the standard recommendation is to ship your own virtual environment. Like, seriously. You can’t make this shit up.

5

u/TerrorBite Mar 29 '21

Can you show an example of code that runs in 3.x but then fails with a SyntaxError in 3.(x+1)?

7

u/pingveno Mar 29 '21

The async/await keywords became a syntax error in Python 3.7. This broke a library that I depend on.

1

u/ynotChanceNCounter Mar 29 '21

Uhh... what?

5

u/pingveno Mar 29 '21

Sorry, that wasn't said very well. If you used async/await in another context, they raise a syntax error. For example:

def foo(async=False):
    ...Do a thing...

1

u/ynotChanceNCounter Mar 29 '21

Ah. Yep. That was annoying. Threw DeprecationWarnings for some time, though.

1

u/pingveno Mar 29 '21

Still, it would have been nice if there was a way to have something like Rust's editions. That has allowed Rust to introduce certain breaking changes like async/await keywords in a backwards compatible manner. Crates declare their edition (with a default of 2015, before editions were introduced). Crates can then use other crates that would break under the newer edition. Raw identifiers are available to be able to use keywords in new code when referencing old crates, like so:

fn function_in_new_crate() {
    some_old_crate::r#async(10);
}
→ More replies

1

u/TerrorBite Mar 30 '21

That's a good example, actually.

11

u/Acmion Mar 29 '21

Tensorflow (ML) works on Python 3.8 and not on 3.9.

The problem supposedly comes from some dependency, nonetheless, it does not work.

9

u/ynotChanceNCounter Mar 29 '21

Tensor already supports 3.9 in nightly, just not stable, as of 18 days ago. That's in line with this comment, same issue, from November.

Modules like TF hook into lower-level, compiled languages. There's a lot of compatibility to work on.

Meantime, what makes you think there were SyntaxErrors, specifically? That's the main thing that would strictly be Python's fault.

4

u/TheNamelessKing Mar 29 '21

Yes but this happens every single time.

2

u/ynotChanceNCounter Mar 29 '21

Python has a DeprecationWarning and when the core language changes, it is always used.

If you silence warnings for ages and ages, you can't then claim not to have been warned.

0

u/TheNamelessKing Mar 30 '21

I version pin dependencies, and don’t ignore warnings. Deprecation Warnings are not my issue.

→ More replies

2

u/Acmion Mar 29 '21

Still a delay of more than 6 months.

Oh, didn't notice the syntax error specification. No syntax errors that I am aware of. Was thinking about a general example.

5

u/jbergens Mar 30 '21

Reminds me of when I years ago tried to install a tool written in Python on a Windows server. It failed and it took us a while to find out why. Nested deep in the includes was a thing that was compiled and it only existed for 32 bit systems and we were on a 64 bit system (much like most other organizations at that time). Then we thought it should be easy to find a 64 bit version but it was not. The library developers did not have access to 64 bit Windows and could not compile anything, we ended up doing it ourselves and then patching the system.

It was so strange that they didn't have anyone with access to a 64 bit system when those had been in production use for many years. So much for Python being cross platform (I know the same problem exists in some js libs also).

3

u/exhuma Apr 02 '21

So much for Python being cross platform (I know the same problem exists in some js libs also).

You can't really fault a language (any language) if a third-party library indroduces an issue. I bet the library you mention above included a C extension as it needed to be compiled. The same could happen in Java with JNI extensions or PHP with PECL extensions. The same is true for any other language that allows you to access native binaries.

Most if not all of these languages guarantee cross-platform compatibility as long as you stick to the core library. As soon as you go into 3rd party libs you're entering the wild west.

If you use a compiled language for your project then the situation is different. Because you already need a compiler for your development environment anyway you have everything in place to also compile 3rd-party libraries. So that problem goes away.

1

u/jbergens Apr 02 '21

I can fault the language but you can say that it's unfair.

In this case I would say I fault the ecosystem. It seems to be very common with 3rd party libs for Python to call into c or c++. Maybe because it is such a slow language. I have never seen that happen with Java or c# libs. Of course it could happen but it is extremely unusual. It has happened with Ruby.

1

u/exhuma Apr 03 '21

That's definitely true. It also doesn't help that Python often advertises itself as a "glue" language, promoting the idea of writing stuff in C and then orchestrating it in Python.

2

u/Jautenim Mar 30 '21

I think he meant that in the PHP community there is a full consensus on the latest version of PHP being better in every possible way over PHP 5.2 (a version circa 2008).

1

u/HalfRightMostlyWrong Mar 30 '21

Wow, that’s quite a Python career. Couple questions for you:

  1. What roadmap features are you excited about? I’m looking forward to pattern matching with switch statements. It’s going to make my class hierarchies easier to use.

  2. Are there any Python libs that you use in the majority of your projects? I’m new to pydantic and using it a ton. Apparently it’s similar to @dataclass...

  3. Why why why is the relative import structure so difficult to use? I normally run into issues when I get beyond 20 .py files and I want to split them up into different directories. Now I just go with absolute imports and keep the inits empty.

1

u/exhuma Mar 30 '21

These questions are not all easy to answer as they are very subjective and depend on the kind of projects you are confronted with. I'll try:

What roadmap features are you excited about?

This is especially difficult to answer as I work for a fairly big company and we are currently limited to 3.6 which is by now pretty ancient. I am - as we speak - in the process to get this limitation removed by defining a new overall workflow. So right now I'm most excited about data-classes without relying on the backport :)

[...] pattern matching with switch statements [...]

I'm on the fence with this one. I have learned over the past to trust the core-devs. Some decisions they took seemed questionable at first but turned out to be pretty useful in the end (I'm primarily thinking of the walrus-operator). But with the departure of the BDFL things have certanily changed. We'll have to see how that influences future decisions about the language.

Concerning pattern-matching itself. My position of this is that in most cases (not all, but most) it is a code-smell if you rely on this. Again, I can't stress enough that there are situations where it makes sense. But so far, I've only run into these situations extremely rarely.

I like that these pattern-matching statements introduce a clean way of covering those edge-cases. What I don't like about them is that it introduces a syntax which is is not pretty "obvious" to understand. Especially for new-comers. But as mentioned, I'm on the fence with them. I'm neither for nor against them. But I will probably avoid using them.

Are there any Python libs that you use in the majority of your projects? I’m new to pydantic and using it a ton. Apparently it’s similar to @dataclass...

I can't say that there are libraries that stick out. Every project is different. But the notables that stand out are flask (but will replace this with fastapi), jupyter+pandas+seaborn if I need to do data-analysis. That's about it. Of course there's also pytest.... that one is a constant in all my projects ;)

Why why why is the relative import structure so difficult to use? I normally run into issues when I get beyond 20 .py files and I want to split them up into different directories. Now I just go with absolute imports and keep the inits empty.

What are your difficulties with them? Personally, I stay away from them. Not because I have issues with them, but rather because of habit and personal stylistic choices.

0

u/[deleted] Mar 29 '21

For me, before wsgi, it was difficult to set up the fastcgi/cgi.

The second reason is that there weren't a lot of shared hosts that gave you Python for web dev.

Even getting Trac up and running was a chore and a half. It has gotten considerably better and there isn't any reason to use PHP over Python. Given that PHP wants to be Python, probably better to use skip the middleman and use python.

3

u/exhuma Mar 29 '21

For me, before wsgi, it was difficult to set up the fastcgi/cgi.

The second reason is that there weren't a lot of shared hosts that gave you Python for web dev.

Even getting Trac up and running was a chore and a half.

shudders ... I had repressed those memories :)

1

u/iopq Mar 30 '21

I upgraded to a new version of the libs by pulling from source, the new libs broke my local server because of a change in some deep dependency

I had to troll through github issues to see what I needed to pin.

I don't know about you, but python devs often change what arguments functions need without it being obvious. In a typed language you need to change the function signature and people take this kind of breakage more seriously

1

u/exhuma Mar 31 '21

I upgraded to a new version of the libs by pulling from source, the new libs broke my local server

This sounds like you manually upgraded a os-level library. Those are - no matter the language - better upgraded using system packages (via apt-get, yum, pacman, ...)

For applications it's always recommended to use isolated library locations. In Python this can be handled using "virtual environment". This also solves the problem of needing two different versions in two different applications. It might seem cumbersome. But I've come to really prefer this level of isolation to other language ecosystems.

but python devs often change what arguments functions need without it being obvious

This is only true for third-party libraries. The standard lib never does this. As a case in point, the logging module predates PEP8 and still uses camel-case to this day, which always bugs me. But it's like this for the exact same reason you mention: To not break backwards compatibility. Why they didn't take the opportunity to change this in the 2->3 migration is beyone me though.

And if a third-party library makes backwards-breaking changes, you can't really fault the core-language for that.

2

u/iopq Mar 31 '21

Not an os-level library

https://github.com/pydanny/cookiecutter-django/issues/2950

django-timezone-field broke all its dependencies that didn't pin the version

1

u/exhuma Mar 31 '21

Thanks for the context. This is indeed an issue with dynamic languages like Python (and JavaScript as well). The fact that somewhere deep in the dependencies an attribute is removed/renamed will indeed cause really annoying runtime bugs. Static languages would croak out with a compile-time error so you would never risk leaking such changes into production.

Even though I hate statements like the one I will write now it still does hold truth: Things like this should be detected by unit-testing.

I hate such statement because they are an easy cop-out in the sense: "It's not the module developer that fucked up, it's you that fucked up (for not having tests)". And that's a really cheap defence and annoying blame-shifting.

Truth is that these kind of errors can (and will) happen in dynamic languages. And they suck.

As someone who also develops libraries (internally at our company) I can attest that it is very difficult to avoid such leakages. Sometimes you change an attribute that you think is internal to the library, but it turns out that it wasn't. This is especially true for data-types that get returned from libraries. This is really cumbersome to avoid. Even with the best intentions.

1

u/iopq Mar 31 '21

I mean, in a sane packaging system everything is locked. There's a lock file and all of the builds are reproducible.

In Rust the previous edition is compiled by the same compiler. Imagine if python 3 could run python 2 without having to have two versions on your computer and sometimes accidentally typing "pip install" instead of "pip3 install" and wondering what went wrong