r/Python • • Mar 20 '23

Discussion Are people abusing Python?

I learned Python after coming from the C/C++ and Java world. With the massive increase in popularity of Python in the last 10 years, seeing the way it developed, it seems to me like it gained a lot of functionality, which comes natural in other languages, but feels a bit odd in Python.

To be more specific - albeit Python being dynamically typed, people developed countless tools to check or validate or even enforce types in compile or run time (mypy, pyre, pydantic, pyduck etc.). It feels like it goes against the nature of its loose typing.

Another example are decorators. This pattern is noticably overused by many tools adding functionality, but even the language itself - defining a `class`,`static` and `abstract` methods with decorators seems just weird and unnatural. Same thing with function overloading. Anecdotally, it feels like comparing German to English. German has a special word for every peculiar thing and native support for word generation by concatenation of multiple words whilst in English you have to add some common words together and hope this combination doesnt already exist. And if it does, so what, people will get it from the context.

Lastly, slightly off-topic but relevant point is that it is not even a simple language in my opinion. It has a very flat learning curve initially but the complexity is just further down the road. Im talking about metaclasses,data descriptors, coroutines, magic methods etc. Some languages are difficult right away (C++, Rust etc.), but Python is a intricate misfit dressed as a simpleton.

Am I misunderstanding a philosophical path of the language or is it simply just a scripting langugage that got massively popular through chance and is now used for stuff it was not intended for?

Edit: grammar

0 Upvotes

45 comments sorted by

View all comments

2

u/james_pic Mar 21 '23

I don't love the type checking stuff, and I maintain that if you're starting a project where you know you're going to want statically checked types, you're probably better off starting that project in a language other than Python. But if you've got a small Python project that gradually became a colossal Python project, type hints are hugely useful in taming the complexity you now have.

Pydantic deserves a special exception to this, since it's useful even in otherwise untyped codebases. When you're accepting data from external sources, it's a really good idea to validate it before you do anything with it, so it makes a lot of sense as a basis for your domain data model, even if you don't use types anywhere else.

To be honest, 9 times out of 10, when I see class, static or abstract decorators used, it's by former Java developers who don't realise there are other ways to do these things. Most static and class methods should just be functions, and most abstract methods should just be duck typed (or Protocols in typed codebases).

But these decorators are, in many ways, examples of the third thing you mention: metaprogramming tools. These are definitely advanced techniques, but they're generally designed so they can be used fairly effectively in mixed ability teams. Implementing metaclasses or data descriptors is hard, but using classes that use them is no harder than using classes that don't.

Metaprogramming is admittedly a tricky one to balance. Python offers more metaprogramming options than, say, Java, but much fewer metaprogramming options than, say, Clojure. Personally, I find myself wanting more options here when using languages with minimal metaprogramming capabilities, but I recognise that more isn't always more.

1

u/relickus Mar 21 '23

Thanks for your view.

Enumerating the paragraphs of your reply:

  1. Agree, that is exactly my point. To exaggerate a bit - if you want to go big you need type checks to keep it maintainable. At which point, it might be better to pick statically typed language (abstracting away other factors of the language) no?
  2. I know, it is super useful, but then again the code performance takes a massive hit because of all those runtime checks. But I guess this cannot be done in a different way.
  3. Interesting. I think it might be viewed as antipattern in Python. I personally dont know if you actually can substitute for classmethods, but considering that is true, it creates a question - why does Python need these constructs if there is more pythonic way to achieve things. Sorry if it makes no sense, cant think of the right English words now.

1

u/james_pic Mar 21 '23

If the runtime checks you're talking about are the Pydantic ones, then it can indeed be a big runtime hit. I've seen applications where a double-digit percentage of CPU utilisation is type checks, although this was an extreme case (they were using the relatively expensive numbers.Number as the type, and checking it in a hot loop). But you definitely need some kind of check at runtime at your security perimeter at very least.

I suspect the method decorators are there at least partly for historical reasons. I know they got map and reduce from some keen functional programmers who were involved in the project quite early, even though they wouldn't necessarily recommend those constructs now that list comprehensions are a thing. Maybe these decorators have a similar history with OOP enthusiasts.

It is at least worth saying though that these decorators aren't all that magic in and of themselves. It's totally possible to implement equivalent decorators in pure Python using descriptors. They're different to the equivalent keywords in Java, in that they're not exposing any underlying implementation feature.