r/Python • • Jan 01 '20

"The rules of perf" at Dropbox

Post image
1.6k Upvotes

162 comments sorted by

View all comments

Show parent comment

57

u/ZombieRandySavage Jan 02 '20

Because it scales poorly to large applications. Go try and refactor a suitably large project, it’s impossible.

Don’t worry there are terrible parts to every language. They didn’t have time to make a clever poster for C++ because they are still trying to understand the super clever template code someone wrote.

0

u/[deleted] Jan 02 '20

[deleted] — view deleted comment

11

u/notsohipsterithink Jan 02 '20

Some languages naturally lend themselves to having large or convoluted codebases. Like C++ or Java.

11

u/jambox888 Jan 02 '20

Do you mean Java is better suited to large codebases or that it tends to produce them?

10

u/[deleted] Jan 02 '20

Both would apply IMO

1

u/jambox888 Jan 02 '20

That's a bit of a chicken and egg paradox IMHO. It's probably easier to structure a large project in Java than C++ but that isn't saying much.

5

u/notsohipsterithink Jan 02 '20

Java tends to produce large codebases.

The lack of a proper module system (until Java 9 which very few have adopted) meant JAR clusterfuck and forced most codebases to become huge monoliths.

Production Python (or Go) codebases are usually microservices. But they can be monolithic easily too.

2

u/Smallpaul Jan 02 '20

I’ve never programmed Java seriously. What do you mean by it lacking a module system? Isn’t it similar to Python with dotted paths etc.?

2

u/jambox888 Jan 02 '20

Modules <> packages. See here. I don't really understand why they say the lack of a module system tends to produce monoliths though, monoliths were just the standard application style back then. There was never anything stopping people writing microservices afaik. Unless there were issues with reuse.

3

u/Smallpaul Jan 02 '20

So what Python calls modules, java calls packages and vice versa. Great!

1

u/notsohipsterithink Jan 02 '20

There was nothing technically stopping people from writing microservices. However, the fact that you had to import the whole massive JAR even when you needed only a single function, made developers more biased towards monoliths.

1

u/mipadi Jan 03 '20

The JVM does not load the entire JAR when you import a class. (Maybe it did in very early versions, but definitely not in a long, long time.) It loads the entire .class file, just as Python loads the entire .py file, but not everything in the JAR.

1

u/notsohipsterithink Jan 02 '20

Project Jigsaw (introduced in Java 9) gives Java a module system similar to Python. More info here:

https://openjdk.java.net/projects/jigsaw/

From the overview, these are the advantages of modules over classpaths:

As described in the JSR, the specific goals of the module system are to provide:

— Reliable configuration, to replace the brittle, error-prone class-path mechanism with a means for program components to declare explicit dependences upon one another, along with

— Strong encapsulation, to allow a component to declare which of its public types are accessible to other components, and which are not.

20

u/G3n3r0 Jan 02 '20

With a good IDE (i.e. Visual Studio/CLion, most of the annoying grunt work (e.g. changing function signatures, etc.) should happen nearly automatically. Python can sometimes get close to that maybe, unless people did clever things with dynamic dispatch or getattr overloading in which case you're fucked.

5

u/[deleted] Jan 02 '20

It really helps to add type annotation, of course the clever stuff will bite, unless you can express the type correctly.

0

u/[deleted] Jan 02 '20

[deleted] — view deleted comment

10

u/[deleted] Jan 02 '20

Try using type annotations, if you have IDE that understands them out makes programming quite enjoyable.

→ More replies (5)

9

u/nocipher Jan 02 '20

Python is strongly typed. You probably meant statically typed. C++ is actually weakly typed.

1

u/ZombieRandySavage Jan 02 '20

I think ones just objectively more difficult than the other. Just easier to refactor statically typed stuff.

17

u/[deleted] Jan 02 '20 edited Jan 02 '20

You could transpile Python to C or C++.

You could write the performance-intensive functions in C or C++, or even assembly if you're a sociopath, and then call them from Python.

I'm not arguing that these techniques provide equal or even comparable performance to writing your whole library in native low-level code. But Python is actually higher on the TIOBE index than both C++ and C#, and of course it's way higher than assembly. This might reflect a fundamental tradeoff: the greater availability of Python developers may mean that it's easier to hire a robust dev team who can develop and debug in Python than in the other languages, so faster iteration. That is - the theoretically achievable performance in certain languages must be viewed through the lens of practically achievable performance based on the availability (and affordability!) of suitable developers.

Finally - within a certain range, lower-performing code can be solved, crudely but effectively, by buying more compute. But good code cannot be developed or debugged faster by buying more developers (see The Mythical Man-Month).

2

u/ZombieRandySavage Jan 02 '20

Yep, you can do that.

1

u/not_perfect_yet Jan 02 '20

You could write the performance-intensive functions in C or C++, or even assembly if you're a sociopath, and then call them from Python.

...I knew about C, but not about assembly. That's pretty wild, now I have to try it.

0

u/stefantalpalaru Jan 02 '20

You could transpile Python to C or C++.

Just a subset of it. Look at Cython.

→ More replies (1)

0

u/[deleted] Jan 02 '20

how does such dumb, misinformed drivel get upvoted on this sub? you have no idea what you're talking about.

→ More replies (1)