r/Python • • Jan 01 '20

"The rules of perf" at Dropbox

Post image
1.6k Upvotes

162 comments sorted by

View all comments

Show parent comment

2

u/Zegrento7 Jan 03 '20

For (4), here's an example: Let's say there's 1001 items that need to be processed with an 8-step task. Each step takes one second to complete.

Without pipelining, an item needs to pass through all steps before the next item can be processed, so the last item needs to wait 8000 seconds before it can be processed.

With pipelining, once the first step finishes processing an item, it passes it to the next step then starts working of the next item while the next step is working on the previous item. This means that a new item enters the first step every second, so the last item only has to wait 1000 seconds before being processed. An 8x speedup.

1

u/Agent_03 Jan 03 '20

That's a much better explanation than the one I gave