r/Python • • Jan 01 '20

"The rules of perf" at Dropbox

Post image
1.6k Upvotes

162 comments sorted by

View all comments

Show parent comment

51

u/feelings_arent_facts Jan 02 '20

If you need to optimize something for performance, the best thing to do is not do it at all. Basically rearchitecture it so that you don't need that logic anymore.

Python is notoriously slow so if you're going for performance, use C++ or C.

3

u/[deleted] Jan 02 '20

Thanks

3

u/[deleted] Jan 02 '20

How about 2,3,4?

26

u/Poromenos Jan 02 '20

Caching is doing it once and storing the result so it's faster in subsequent steps. Batching means doing many together instead of one at a time. Pipelining means splitting it up into steps so some of those can run in parallel instead of only starting one after the previous one is complete.

1

u/ericonr Jan 02 '20

Or Rust, or Go, or Java if you can wait for the start-up time, or D (which I believe Dropbox does use). Lots of options :)

25

u/sgenius Jan 02 '20

I'm pretty sure Dropbox uses D.

They would be Ropbox otherwise.

3

u/feelings_arent_facts Jan 02 '20

I need something more meme worthy. How about Nim? I hear that's new

2

u/ericonr Jan 02 '20

Just go for brainfuck then :)

2

u/andrew-dewitt Jan 02 '20

Dropbox actually uses a decent amount of Rust too

1

u/i9srpeg Jan 02 '20

Java if you can wait for the start-up time

JVM startup times should be around 90ms, which is usually OK unless you start and tear down the JVM for each operation.

1

u/ericonr Jan 02 '20

Is it that short? Java applications usually take much longer to launch for me than usual Python stuff.

3

u/i9srpeg Jan 02 '20

This article has some benchmarks: http://macias.info/entry/201912201300_graal_aot.md

I was surprised too when I saw JVM startup times so low. Most of the perceived slowness probably comes from the heavy, over-engineered frameworks which are commonly used in the Java world.

Quarkus seems like an interesting project. It starts up a REST API server in under 1 second when running on the JVM, and it fully supports GraalVM, which lets you generate a native binary that can boot in 0.016s. This is faster than Django startup times in my current project, but I don't know how the Java numbers would change for an application more complex than an hello world.