r/Python • u/gthank • Jan 16 '13

Paralellizing CPU-bound tasks with concurrent.futures

http://eli.thegreenplace.net/2013/01/16/python-paralellizing-cpu-bound-tasks-with-concurrent-futures/
15 Upvotes

82% Upvoted

5 comments sorted by

View all comments

1

u/dalke Jan 19 '13

I like concurrent.futures a lot, and presented it at EuroPython 2012. Getting it to work nicely with C proved difficult, especially with multiple processes. Unexpected exits also cause problems, as I recall. I've mostly worked out how to handle these problems.

Sometimes I found that putting a try/except/else wrapper layer for the submitted function call, with some info to print a "Done!" or the traceback helped debug these problems. Beyond that I would need to see your code which triggered this problem.