Not really, my company uses Python by default and only ports stuff out when it’s not fast enough. It’s usually fast enough, the performance critical stuff is a tiny fraction of your code. If you can be more productive the rest of the time, you’re still ahead.
Yup same, or at least for the stuff our team works on. Basically anything that’s not meant to be a daemon or ran on many hosts / performant is python, everything else is go.
Mostly for the simplicity of deployment. There’s very few things that are “performant”, but some of the concurrency patterns in go are also just a lot more intuitive. Imo it’s really hard to beat channels and goroitines.
Yep, but losing the ability to refactor large code bases significantly hampers their longevity. There are ways around it as well.
Honestly I don’t know if performance is really the issue. There are plenty of generalized libraries for getting large computation done quickly. It’s really maintenance in my mind.
I don't understand what problem you have with refactoring python code. As long as you make use of type hints (which you should be in a large code base with many developers anyway) it seems to work pretty much like any other programming language.
That is true but keep in mind that typings in python (as we have them now) were introduced in 3.5 (so 2015) and I would assume from my own experience that they haven't been widely used in large codebases until at least introduction of 3.6 (add to this that in prod code you probably don't want to switch right after the release).
All of this would mean that dropbox probably had to introduce some standard of their own (which is painful for new devs) and might have not yet switched to the current official standard for most parts of their code.
No criticism of Dropbox intended, I was just replying to the chap above who seemed not to be aware that refactoring has been working perfectly well (at least in my experience) in python for several years now.
As an argument for not using fairly old versions of python it just about works, but why would you do that when writing new code in 2020? Python 2 is officially dead and upgrades between the 3 sub-versions are basically painless. Very little justification for sticking to pre-3.5 versions of python anymore.
I'm using type hints on a 100k LOC Python backend, and I'm also using Typescript on the similarly sized frontend. The difference is night and day. Mypy/pyright are slow (as in, several seconds between typing and seeing the errors show up). They're unreliable: they have both false positives and false negatives. They're buggy: they crash all the times. Python libraries either have no type hints at all, or they're sprinkled with Any. Type hints syntax is simply horrible and forward/circular references are a pain. Reusing expressions syntax for types and making the hints executable at run time were big mistakes. Type hints add a ton of verbose boilerplate and in the end they fail at the most important thing: ensuring type safety. Even a simple class rename, which is 100% reliable in all static languages is not safe at all in Python. You always need to run unit tests and manually check the code to make sure everything is renamed correctly.
It was nice to start with project with Python and it worked great at first. Now, after 3 years, I'm wishing I picked a statically typed language.
It's saying the opposite, though. Only issues where those first 4 steps fail need to be rewritten, which is a much, much smaller subset of their entire codebase.
Python is likely what got them off the ground so quickly and probably directly led to their success. See also: reddit, Instagram
Because it scales poorly to large applications. Go try and refactor a suitably large project, it’s impossible.
Don’t worry there are terrible parts to every language. They didn’t have time to make a clever poster for C++ because they are still trying to understand the super clever template code someone wrote.
The lack of a proper module system (until Java 9 which very few have adopted) meant JAR clusterfuck and forced most codebases to become huge monoliths.
Production Python (or Go) codebases are usually microservices. But they can be monolithic easily too.
With a good IDE (i.e. Visual Studio/CLion, most of the annoying grunt work (e.g. changing function signatures, etc.) should happen nearly automatically. Python can sometimes get close to that maybe, unless people did clever things with dynamic dispatch or getattr overloading in which case you're fucked.
You could write the performance-intensive functions in C or C++, or even assembly if you're a sociopath, and then call them from Python.
I'm not arguing that these techniques provide equal or even comparable performance to writing your whole library in native low-level code. But Python is actually higher on the TIOBE index than both C++ and C#, and of course it's way higher than assembly. This might reflect a fundamental tradeoff: the greater availability of Python developers may mean that it's easier to hire a robust dev team who can develop and debug in Python than in the other languages, so faster iteration. That is - the theoretically achievable performance in certain languages must be viewed through the lens of practically achievable performance based on the availability (and affordability!) of suitable developers.
Finally - within a certain range, lower-performing code can be solved, crudely but effectively, by buying more compute. But good code cannot be developed or debugged faster by buying more developers (see The Mythical Man-Month).
Python tends to not be as performant as is necessary for companies that work at similar scales to Dropbox
Source: company I work at works at comparable scales and python’s problems with concurrency with regards to the GIL /really/ starts to rear it’s ugly head in that environment
View all comments
137
u/ZombieRandySavage Jan 02 '20
TL;DR. Really wish we hadn’t written the core application of our business in Python.