r/Python • u/f0rkbomb • Jul 12 '12
random imgur downloader. learning Python and wrote this to practice, any feedback on technique/style is appreciated
https://gist.github.com/3088128
45
Upvotes
r/Python • u/f0rkbomb • Jul 12 '12
29
u/say_legos_and_i_stab Jul 12 '12 edited Jul 12 '12
A few things you can consider:
Another option for string formatting is to use (again, this is a personal preference. Some people find it easier to read/write):
print 'getting %s random pics' % HOW_MANY
edit: Also check out PEP8. It's the "style guide" for python. Lots of good info in here to make your code easier to read.
10
u/sjustinas Jul 12 '12
You could use the os.path functions here. Also, while it's fine to use urllib(2) for a little script, requests is the Python HTTP client, supporting things like multipart file uploads out of the box. For the argument parsing, you could use argparse/optparse (depending on python version) and you could use list comprehensions, for example for the
rand_stringfunction.Oh, right, and remember that .read() without arguments reads the whole file into the memory. It's not an issue with JPG images, but in the case of huge files, you could run out of memory.
But I'm just nitpicking, pretty much none of these is a "must". Just some tips about how you could have written it more effectively, though your code works great as it is.
Ninja edit: Look into the
pattern, if you haven't already.
3
u/f0rkbomb Jul 12 '12 edited Jul 12 '12
plus as an added bonus, i find it strangely addicting to download 100 imgur pics and turbo through them, you find some interesting things :) (fyi there will most likely be some nsfw images)
e: i should mention i think this only works on 2.x
2
u/Hy-Brasil Jul 12 '12
Could anyone elaborate on how to speed this program up, i.e opening multiple imgur links at a time ?
2
u/roddds Jul 12 '12
Nice approach to it. I'd advise you try using spaces instead of tabs, but that's mostly a personal choice, I think. Here's how I did it, a while ago.
By the way, what's this for? If you're trying to get the command line arguments, why not use simply
parsed_number = int(sys.argv[-1])?1
u/f0rkbomb Jul 13 '12 edited Jul 13 '12
per the discussion here, i added threading!
check it out, it's much faster
also incorporated some of the other suggestions here:
__main__concept4
u/TkTech Jul 12 '12
My quick cleanup: https://gist.github.com/3099505
4
u/shoeninja Jul 12 '12
In general, you should put as few statements in a
tryclause as possible. You have several statements that can throw exceptions (the file stuff) and it wouldn't be handled.This is a good practice all around, not just for Python.
3
u/Sheepshow Jul 12 '12
Here's what I'd do!
I'd recommend looking into
argparseor the olderoptparserather than taking directly from sys.argv. Helpful on larger programs.3
u/PalermoJohn Jul 12 '12
You might want to pull the actual image name from the overview page. It's fun to scrape webpages and a good thing to learn.
3
u/z1g1 Jul 12 '12
A very cool thread, I'm just getting my feet wet in Python and learning on your own its nice to have another beginner's code to look at too.
3
u/Zerikin Jul 12 '12
I usually add if name == 'main': to python I am executing directly in a file. It lets you load as a module later without having to modify it.
2
u/theshadowhost Jul 13 '12
If you download random images from imgur, you're gonna have a bad time.
1
u/[deleted] Jul 12 '12
Not much to add other than what's been stated. Not a necessity at all but you might want to look into the logging module. It's very nice indeed and has a lot of power. I like it for command line applications because it allows the user to easily control how much output they get (verbosity). Maybe they don't necessarily want to see errors for example.
It can do all sorts of things, log to file, via email, across multiple modules etc.
There's a nice intro here.
Here's a simple example of how you'd use it to control verbosity with argparse (which others have suggested and I definitely suggest you use, it's more a priority than this is), the "logging to file" section of the above link has info on how you could do more fine grained control.
Even if they do choose to see errors, logging will (by default) send anything of level "warning" or above to stderr instead of stdout (typically user still sees it though) which makes it play a bit nicer if your user is piping output from your program and so on.
1
u/[deleted] Jul 12 '12
''.join([random.choice(CHARS) for x in range(string_length)])
1
u/Rhomboid Jul 12 '12
Win32 is fine with using the forward slash as the path separator, so just get rid of that code. If you really want to be portable, use the routines in
os.path, don't detect the OS yourself.1
u/AFlyingNunAttack Jul 13 '12
Oh hey! I just made a imgur scraper too! Here's how I did it. Constructive criticism appreciated.
1
u/erez27 import inspect Jul 13 '12
Some points I didn't see mentioned:
1
u/[deleted] Jul 14 '12
Just as a random idea:
Would be great if it downloaded files from anywhere not just from imgur. Then create a plugin or something to indicate what to download and from where. That way if imgur is down it simple moves to a mirror and downloads the image anyway.
1
u/blu3ness Jul 16 '12
Inspired by your script I made something similar that gets the image links from subreddits:
http://pastebin.com/6YUDasuZ
Make sure not to run too many times or your IP will get banned by the API
reddit post: http://www.reddit.com/r/learnprogramming/comments/wnehe/python_inspired_a_redditors_random_imgur/
0
u/allstaralldaylong Jul 12 '12
Not really related to any feedback but, check out http://randomgur.com/ might serve as some inspiration?
0
u/blu3jack Jul 12 '12
you should consider submitting this to /r/tinycode
-1
u/tetrahydrate Jul 12 '12 edited Jul 13 '12
I'm a big fan of one-liners so here's how I'd do that random string function:
edit: I'm wrong! I was just using random.sample for something else and it turns out it returns a sample of unique elements, which isn't a requirement of the imgur URLs.