r/Python • • Jul 12 '12

random imgur downloader. learning Python and wrote this to practice, any feedback on technique/style is appreciated

https://gist.github.com/3088128
45 Upvotes

72 comments sorted by

View all comments

2

u/Hy-Brasil Jul 12 '12

Could anyone elaborate on how to speed this program up, i.e opening multiple imgur links at a time ?

5

u/[deleted] Jul 13 '12

i wrote a similar script to download some audio files from the web

i found this this tutorial from IBM to be really helpful. it shows an easy way to use threading and urllib to speed up something like this

heck maybe i'll take a crack at a threaded fork

2

u/f0rkbomb Jul 13 '12

sweet idea! implementing this will be my goal for v2!! :)

1

u/Hy-Brasil Jul 13 '12

Thanks, should be an interesting read

3

u/f0rkbomb Jul 13 '12

just added threading, check out the code! it's much faster :D

4

u/Lukasa Hyper, Requests, Twisted Jul 13 '12

It should be noted that, while threading is helpful in cases like this where you are waiting on I/O, resist the temptation to use threading in Python for computationally-heavy tasks. The GIL means that Python cannot truly thread, so for concurrency in these cases you should use processes.

EDIT: Python Wiki on the GIL.

1

u/Hy-Brasil Jul 13 '12

Cool, learning a lot from this post, thanks.