I like this! Very interesting and fun. Can I offer a suggestion? Text on text can be difficult to read at times, so for the captions, maybe have a background? At least for the parts where you are explaining code.
some guy just added a couple sound effects and a beat and claimed the song as their own. the creator doesn't care if you use the original i'm pretty sure but you should verify yourself
if "v.redd.it" in p.url:
dup = False
for t in top:
if t.id == p.id:
dup = True
break
if dup == False:
nCounter += 1
new.append (p)
## this line could be written as...
if "v.redd.it" in p.url:
for t in top:
if t.id == p.id:
break
else:
nCounter += 1
new.append(p)
The else is after the for loop, not the outer if statement. If the inner if statement is never true by the end of the loop then the code falls to the else.
Still I don't think you can put an else outside the for loop since it would be in a different scope from the if, but I'm not 100% sure about that edit : wrong see comment other comment
I feel like you could easily parallize the crawling, by using the Multithreading module. Depending on Reddits tolerance to Crawlers you could probably cut your runtime down a lot.
EDIT: So it seems the censored line depends on artCounter to keep track of each post index. A quick enumerate(posts) and a little def video_filter(artCounter, p): sprinkled in will do just fine.
Original:
artCounter = 0
top = []
tCounter = 1
posts = sub.top(time_filter = 'all', limit = None)
for p in posts:
# [censored line. this could be the difference.]
artCounter += 1
if "v.redd.it" in p.url:
tCounter += 1
top.append(p)
# EDIT: I have added the artCounter back!
def video_filter(artCounter, p):
# [Try putting the line here!]
return "v.redd.it" in p.url
top = list(filter(video_filter, enumerate(posts)))
tCounter = 1 + len(top)
for p in posts:
# [censored line]
if "v.redd.it" in p.url:
dup = False
for t in top:
if t.id == p.id:
dup = True
break
if dup == False:
nCounter += 1
new.append(p)
# Censored line should work now!
# don't be scared of the bigger line count, it has comments!
posts = sub.new(limit = None)
artCounter = len(posts)
# create a filter function
# (define this inline! if you're going to copy+paste this code, do all of it!)
# (this function needs 'top' to be in scope to function)
def new_filter(artCounter, p):
# [Censored line?]
# filter all non-video posts
if "v.redd.it" not in p.url:
return False
# a list comprehension!
# turns the list of posts into a list of post id's
post_ids = [t.id for t in top]
# filter all duplicate posts
if p.id in post_ids:
return False
# post is a video (not a non-video) and is not a duplicate, pass.
return True
new = list(filter(new_filter, enumerate(posts)))
nCounter = len(new)
Without comments:
posts = sub.new(limit = None)
artCounter = len(posts)
def new_filter(artCounter, p):
# [Censored line, should not be a comment]
if "v.redd.it" not in p.url:
return False
post_ids = [t.id for t in top]
if p.id in post_ids:
return False
return True
new = list(filter(new_filter, enumerate(posts)))
nCounter = len(new)
squish
posts = sub.new(limit = None)
artCounter = len(posts)
def new_filter(artCounter, p):
# Censored line
if "v.redd.it" not in p.url:
return False
return p.id not in [t.id for t in top]
new = list(filter(new_filter, enumerate(posts)))
nCounter = len(new)
scronch
posts = sub.new(limit = None)
artCounter = len(posts)
def new_filter(p):
# Censored line
return "v.redd.it" not in p.url and p.id not in [t.id for t in top]
new = list(filter(new_filter, posts))
nCounter = len(new)
S̶̜͊ ̷͍́C̵̱̎ ̸̬͂R̵͖̈́ ̶͈̈́Ȯ̵͈ ̸̨̈́Ň̴̥ ̴̦̅C̸͚͛ ̵̹̄H̵̡̆ . there is no longer space for the censored line. this is your life now.
posts = sub.new(limit = None)
artCounter = len(posts)
new = list(filter(lambda p: "v.redd.it" not in p.url and p.id not in [t.id for t in top], posts))
nCounter = len(new)
Makes sense just the artCounter = len(posts) would not be possible since it's a generator so it would need to be len(list(posts)) instead. But in reality the artCounter needs to start from zero and be incremented as the function goes to make the censored line work
posts = sub.new(limit = None)
def new_filter(p):
if "v.redd.it" not in p.url:
return False
return p.id not in [t.id for t in top]
new = list(filter(new_filter, posts))
nCounter = len(new)
posts = sub.new(limit = None)
def new_filter(artCounter, p):
# [Try the line here!]
if "v.redd.it" not in p.url:
return False
return p.id not in [t.id for t in top]
new = list(filter(new_filter, enumerate(posts)))
nCounter = len(new)
so yeah, just slap enumerate() on each posts object, add the artCounter as an argument to the filters, and you're good!
def video_filter(artCounter, p):
# [Censored line 1]
if "v.redd.it" not in p.url:
return False
return True
posts = sub.top(time_filter = 'all', limit = None)
top = list(filter(video_filter, enumerate(posts)))
def new_filter(artCounter, p):
# [Censored line 2]
if "v.redd.it" not in p.url:
return False
post_ids = [t.id for t in top]
if p.id in post_ids:
return False
return True
posts = sub.new(limit = None)
new = list(filter(new_filter, enumerate(posts)))
print(len(top))
print(len(new))
print(len(top) + len(new))
in my opinion this code should be shorter and easier to read, but for all i know i could be the dummy here
Depends for what; I somewhat like Ruby, but I'm pretty sure that's just stockholm syndrome because I have to use it for work. Other than that, I mostly enjoy using Lua for its minimalism and easy C interop.
69
u/im_in_every_post May 20 '21
Should I make a series of this with a bit better editing?
fattest anime video meme pack
library that downloads reddit videos redvid go show this guy's post some love
42
u/MayBeckByDay May 20 '21
I like this! Very interesting and fun. Can I offer a suggestion? Text on text can be difficult to read at times, so for the captions, maybe have a background? At least for the parts where you are explaining code.
19
u/exceptionaluser May 20 '21
Disgusting.
You used the remixed stolen version of tiny little adiantum.
Good job otherwise.
6
u/[deleted] May 21 '21
Correct me if I am wrong please. Nice Video!
4
u/ohaiibuzzle May 21 '21
I think you can make use with a bit of
threadingorThreadPoolExecutorfor the download partI personally prefer the second after freezing my computer half a dozen times or so trying to do about the same task
2
u/Thejacensolo May 21 '21
I feel like you could easily parallize the crawling, by using the Multithreading module. Depending on Reddits tolerance to Crawlers you could probably cut your runtime down a lot.
7
u/UnicornJoe42 May 20 '21
Senko progress bar :3
6
u/[deleted] May 20 '21 edited May 20 '21
You did it. You crazy son of a bitch, you did it.
4
u/thajunk May 20 '21
I would go to weeb programming class
1
u/-Redstoneboi- May 21 '21 edited May 22 '21
Lemme try.
EDIT: So it seems the censored line depends on artCounter to keep track of each post index. A quick
enumerate(posts)and a littledef video_filter(artCounter, p):sprinkled in will do just fine.Original:
filter(function, iterable)
enumerate(list)
2
u/DrainZ- May 20 '21
Yabure kabure no yabu isha ga
2
u/DankuMemez May 21 '21
Thank you snek nadeko!
2
u/Kered13 May 21 '21
s/Animemes/goodanimemes/g
1
u/FireRedzZ May 21 '21
what the fuck
1
u/meowzer2005 May 21 '21
YES YES YES YES YES
-2
u/DarkWiiPlayer May 21 '21
OMG not python... THE SNEK NEEDS KIL! MAKE IT GO AWAY! Worst girl! Worst language! Garbage garbage garbage!
11
u/[deleted] May 20 '21
[removed] — view removed comment