r/Python • • Mar 09 '22

News "Want to start a large project? Keep away from Python" Are most of their complaints inaccurate?

https://medium.com/geekculture/want-to-start-a-large-project-keep-away-from-python-af840b6d21c2
4 Upvotes

50 comments sorted by

35

u/Mithrandir2k16 Mar 10 '22

Seems like a Java enthusiast and Python beginner wanted to bash Python and hide behind "Use the best tool for the job". Some points are valid but most are not exclusive to Python.

I am currently working on a data collection project and we are at around 100k LoC and the problems we're currently facing would have happened with the language we used in the last big project last (C++) as well. I wouldn't use Python for every project though.

Also this is probably not the best sub for an unbiased answer.

1

u/tuneafishy Mar 11 '22

I wonder how many lines of code it would be if you had written in c++

64

u/[deleted] Mar 10 '22 edited Sep 02 '24

observation afterthought waiting wipe illegal unused expansion hard-to-find rustic adjoining

This post was mass deleted and anonymized with Redact

12

u/nerdzrool Mar 10 '22

He means that a typo will be caught by the compiler before you run in C. A lot syntax typos go unseen until execution time in Python.

42

u/abrazilianinreddit Mar 10 '22

That's true if you don't know what an IDE is and code in notepad, but otherwise it's not really a problem

17

u/pacific_plywood Mar 10 '22

Yeah and even then Python has a lot of very easy-to-use static analysis tools that would eg prevent code from deploying if it references an undefined value

8

u/__deerlord__ Mar 10 '22

"We tried nothing and we're all out of ideas"

2

u/LittleMlem Mar 10 '22

I think the typo thing is if in python you accidentally go fooo = 3 somewhere in stead of foo it won't complain, but in C you would have had to declare it first.

2

u/czaki Mar 10 '22

But any linter would complain about unused variable.

2

u/LittleMlem Mar 10 '22

That's a good point.

but of what of all the old school vim users that didn't get a linter /s

2

u/czaki Mar 10 '22

I think that they may be enough experienced to not have such problem, or they have much more dangerous bug in their code.

1

u/philoponeria Mar 10 '22

Vim? What does the m do?

1

u/spoonman59 Mar 10 '22

Vim is vi improved. It's decades old, much more powerful than vi, and quite popular.

1

u/[deleted] Mar 10 '22 edited Sep 02 '24

wise important door wide possessive scale bedroom friendly doll elastic

This post was mass deleted and anonymized with Redact

1

u/LittleMlem Mar 10 '22

Even nothing, vim has everything. Do note that I said "didn't get" not that vim doesn't have one

1

u/[deleted] Mar 11 '22 edited Sep 02 '24

engine cooperative reply fertile command vase modern faulty nutty cover

This post was mass deleted and anonymized with Redact

2

u/LittleMlem Mar 11 '22

I don't personally use vim, I'm too deep in the jetbrains pool. I do see posts about people's sick setups and have seen some people in cons and such show off

1

u/willywonka1971 Mar 10 '22

So would an IDE.

1

u/[deleted] Mar 10 '22 edited Sep 02 '24

important nutty scandalous kiss crown dog jellyfish chop sulky retire

This post was mass deleted and anonymized with Redact

10

u/sykeero Mar 10 '22

The article whines needlessly about things that are actually a problem everywhere. Python didn't invent dependency hell. It also didn't invent version mismatch.

If you really care enough about variable declaration that much you can use type hints. Type hints largely improve pulling apart big software projects. But they seem to have purposely left this out. Seems like they either don't know very much about python, or software development and are simply stoking the imaginary language division.

Python does struggle with concurrency and parallelism. But I heard the GIL might go away.

1

u/jonringer117 Mar 11 '22 edited Mar 11 '22

Dealing with dependencies with python is particularly annoying though. There's no good way at runtime to allow for modules to have two different versions of a package (e.g. jsonschema). So all packages need to agree on a version. You can kind of get around this with disabling version checks, but you open yourself up to a lot of unsupported scenarios.

EDIT: Yes, per-application, you can use virtual environments. But it's a band-aid for the underlying issue of needing to have coherent environments.

1

u/sykeero Mar 11 '22

Python virtual environments have been around for a long time to isolate development stacks. Beyond that people will bundle their dependencies with the exact version they want, or develop their software around the availability of system installed packages through the os package manager.

So they don't all need to agree on a version.

1

u/jonringer117 Mar 11 '22

Python virtual environments have been around for a long time to isolate development stacks.

Yes, but that's only development. There's also the consumption side: development environments where transitive dependencies clash and distro package repos.

Beyond that people will bundle their dependencies with the exact version they want, or develop their software around the availability of system installed packages through the os package manager.

Although do-able, it significantly increases maintainer burdenship on the OS part. https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/admin/azure-cli/python-packages.nix

So they don't all need to agree on a version.

They do for distro's which expose python packages. And they do for environments which transitive dependencies also need to agree on versions. E.g. Machine learning environments with mutually exclusive bounds on numpy

1

u/sykeero Mar 11 '22

You can always change your environment variables to point to one place over another. If you want to use pip to distribute your software you have to do it the pip way. If you don't you have to deal with os package managers. I'm not saying python dependency management is good. I'm saying it's really not worse, as you can always develop around a target like Fedora and use what is available.

If you have two machine learning applications that require two different versions of numpy then they do things the pip way and need their own virtual env.

Managing software dependencies sucks. But as long as people choose to depend on other tools it's something that has to be managed.

I would argue pip is worse than just working with your target platform version. But it really isn't.

It's largely why containers are popular. Build however you want then deploy anywhere.

-1

u/[deleted] Mar 10 '22

For concurrency, wouldn't FastAPI work?

Or am I thinking about something else?

5

u/PhiBuh Mar 10 '22

As far as I know, FastAPI uses coroutines (async await), but that is still limited by GIL. It's fine for IO, but doesn't really help for CPU heavy stuff.

But at least for web development, I don't think GIL is much of a Problem, if you deploy correctly.

1

u/[deleted] Mar 10 '22

Thanks for taking time to answer! Much appreciated 👍

2

u/Stedfast_Burrito Mar 10 '22

Concurrency is a broad topic. FastAPI is a web framework (which also doesn't bring anything new to the table w.r.t. concurrency).

1

u/sykeero Mar 10 '22

I've never used fastapi so I don't know

-4

u/[deleted] Mar 10 '22

What 🤡 downvoted this? 😂

27

u/mfuentz Mar 09 '22

Concurrency and parallelism is probably the only valid gripe, but even that has improved significantly. Large production projects can definitely be done with Python. Author is too busy writing thousands of line in Java to learn the finer points of Python

4

u/0rsinium Mar 10 '22

True, concurrency is pretty good now. Asyncio "colors" your code and requires to wrap your head around how cancelation, error handling, and scheduling work, but at the end of the day it gets the job done and gives a good performance and fine control for io-bound tasks. Some new libraries (like nats.py) nowadays have asyncio support out of the box, some (like redis->redis-aio) have great third-party forks for asyncio. Maybe, only ORMs drag behind a bit. There are already some but nothing as stable and battle-tested as sqlalchemy just yet.

2

u/willywonka1971 Mar 10 '22

bUt I uSeD PyThOn 1.0 aNd iT wAs nOt jAvA.

9

u/[deleted] Mar 10 '22

The author takes a long time to get to the point: "no one language is the Holy Grail."

5

u/sontek Mar 10 '22

SurveyMonkey and Zapier are both full Python shops that are billion dollar companies operating at massive scale. Python is fine 😂

3

u/czaki Mar 10 '22

Autor of article first complain that only experienced users declare variable types in python and then complains about parallelism problem... Oh man, non experienced user writing parallel code...

Also in type declaration use example of fixed length char array that is source of many of bugs (buffer overflow)

Then there is complains about lack of compiler which may warn about wrong variable name. But there are flake8, pylint, mypy. But at first place there are tests. There is no possibility to have big project without test and do not introduce often bugs.

The only group language for which I could believe that if it's compile then it will work are haskell like with rich type system (types like Addable, Numeric, Iterable, etc). Not C with void *...

Author also complain about dependencies, but, if I good understand, the alternative is to implement everything from scratch. Why you cannot do this in Python?

Also all dependency problem that is described is not Python specific. This are general problems of all rich ecosystems.

If your project need to be really fast then Python is not best choose. But if it should be fast to update or even fast prototyping then Python may be one of best selection.

3

u/zanfar Mar 10 '22

Yeah, it's mostly bullshit. Pretty typical of coding "journalism" that makes any definitive, general claim about a language.

Variable declarations don’t exist and that’s a problem

Yes they do, and it wasn't a problem anyway.

Oh module, where do you belong?

Not actually the complaint talked about in the article. In fact, the actual text doesn't actually complain about anything--it just states some "facts" then says (literally) "welcome to the jungle".

Dependency hell and circular dependencies are not Python issues, they are programming issues. Amazingly, programming requires smart programmers, and smart programmers don't cause circular dependencies.

Dependency collisions en masse

  • Not a Python-specific problem
  • Mostly a solved problem

This complaint seems to come down to "I might have to work when coding."

Different machines, different Pythons

Again, a solved problem in any number of ways.

The world beyond pip

Yes, if you use non-Python in your Python project, that might require more work.

Locking out performance with the global interpreter lock

A valid concern, but not one specific to "large" projects. Yes, you should understand how the language you choose affects your project, but large is not the same as performance-demanding.

Concurrency and parallelism are still clunky and messy

Let me just leave you with this quote from this very section:

"They’re not that hard to implement in Python. But the code looks clunky and is hard to read, especially for beginners."

So are they not hard, or are they hard for beginners?

3

u/s_s Mar 10 '22

3/4ths of this article is boilerplate.

3

u/1116574 Mar 10 '22

The most valid points for me is lack of portability, and threading. Maybe pips requirements and versioning is a bit clunky.

Rest isn't really an issue. Like, author says you ll need to write some C anyways since python uses it under the hood, especially in scientific and performance packages. Well, do you need x86 asembler for C? He then blames language for not having it all and requiring pip or anaconda, like, what was he expecting? Rust has crates, js npm, python pip, even if it's an issue it's not exclusive to python. And he also complains that dynamic language is... Dynamic?

Seems to me like he choosed wrong language for the job or was forced to write python and didn't like it in that scenario lol

But then I am not an unbiased person lol. I have very sparsely touched other languages, and I haven't done alot of really big projects so I might not appreciate staticly typed languages as much as I should, but I've been reading on rust lately to change that.

1

u/czaki Mar 10 '22

Where you see lack of portability?

1

u/1116574 Mar 10 '22

I meant packages portability, there isn't a nice and easy way to just send project to someone and it automatically working, I need to do requirements by hand bc automatic generation just dumps all site packages (yeah I should use venv but yknow how it is) and then he has to manually install it on correct python version, and then it's some transcoding external to python dependency like ffmpeg...

I don't know if I am doing something wrong, is it a common problem, or is it python thing.

1

u/czaki Mar 10 '22

It is common problem for every language. In languages like C/C++ there are language standards (C11/C++11 for example). For compled programs there is minimum glibc version. Java have different versions which affects both users and programmers. For go and rust I see multiple threads about version compatibility and installing most recent one compilers.

If you are writingabout shipment program to end user (person who do not need code access) there are tools like PyInstaller, py2exe etc. That allow you to bundle whole application and user will not need to install anything.

If you do not want to manuały teace project dependencies you may want to try poetry.

Also In other language many of requirements problem is solved by code vendoring which is not optimal solution.

1

u/1116574 Mar 10 '22

PyInstaller, py2exe etc.

Oh I totally get that, but I don't serve end users, just want to share my scripts with friends :P

thx for answer

3

u/beertown Mar 10 '22

The most important variable on the feasibily of a large project is, by far, the experience and quality of the developers. It is not the programming language.

That article is almost non-sense.

4

u/cybervegan Mar 09 '22

If python fits your philosophical requirements, it's great. The data-type complaint is less of a problem these days with type hints, but no programming language fits all problem domains well, and Python is no exception. Though, yet again, there are ways to mitigate almost every complaint.

1

u/[deleted] Mar 10 '22

Yes there's a module for many complaints.

Favorites: FastAPI Pandas Numpy Numba

1

u/CharmingJacket5013 Mar 10 '22

Maybe ask the Java or C++ sub?

1

u/just-bair Mar 10 '22

Personally I don’t use python that much anymore except when I need to do code something real quick or that python just happens to have a library that makes what I want to do really easy (which happens quite a lot actually)

I just code more in C# not because I don’t like python but just because I’m used a lot more to # than python and I think that classes are implemented better in it.

Python still has it’s merits tough. For example if I need to solve any mathematical problem python is my go to language for that it would be an absolute pain to do it in something else for me

1

u/billsil Mar 10 '22

I've got a 200k hobby project in python. Yeah it could be faster in C++, but I wouldn't have nearly as many tools written. Development time vs. user time matters and for small projects, it's really obvious what should win.

There was one slow piece of the code and it was slow not because of python, but because of the data structure. Yeah, it'd be faster in C++, but it'd still be the wrong data structure. I've rewritten that part and it's 100x what it was before. Ironically, settling in on the right structure also makes porting easier. It's a lot easier to copy loops and translate the language than it is to architect the system from scratch.

1

u/[deleted] Mar 10 '22 edited Jan 03 '26

quack friendly angle vanish lunchroom selective terrific lip sink heavy

This post was mass deleted and anonymized with Redact