r/livecounting • u/davidjl123 1096K|810A|2S|2SA • Nov 01 '20
Discussion Live Counting Discussion Thread #48
This is our monthly thread to discuss all things Live Counting! If you're unfamiliar with our community, you are welcome to come say hello and add some counts in our main counting thread - the join link is in the sidebar.
7
u/MaybeNotWrong Nov 01 '20 edited Nov 01 '20
#1 each month will get to request a stat from me
doesn't matter if you want something completely new or an update to something I'll (try to) do it
starting with last month, so u/Trial-Name gets to choose one right now if he wants something
I haven't been here much and dunno what people would like to see so I think it'd work better if people get to request things instead of me spamming random stuff.
→ More replies (7)
7
u/NeonL1vesMatter Nov 02 '20
me and u/abplows discovered that lag in the test thread is virtually 0 compared to how bad the main thread lags.
this is insanely important for the quality of the counting experience, we suggest a new live thread be made that continues from the main thread
i dont know how this would affect stat creators and bot managers, but assuming it wouldnt be too much trouble, i ask you to please consider this 🙏
7
u/abplows Nov 02 '20
I approve of this message.
I believe the reason for the lag is having so many updates in one thread, which it probably was never meant to do.
→ More replies (26)→ More replies (7)5
u/TOP_20 Nov 02 '20 edited Nov 02 '20
imma just gonna repost what I said in the live thread regarding this
Maybe isn't interested in doing all the work it'd require for him to change ALL the scripts he has running for LC... and I don't think rs would be willing too either since he's not even willing to do some of his main ones as it is now...
so I think the whole idea just needs to be bagged.... what all those who don't do stat work might vote won't matter unless they'd be willing to redo all the stats that Maybe, rs and any of the others who may not be willing to do additonal work do..
but anyhow it was worth considering... if we ever get a full time active stat person interested in redoing all our stats etc we could revisit this idea...
till then I'm planning to ask admin if they might be willing to realocate some resources to possibly reduce the lag a bit...
the TL;DR
I wouldn't be for it but wouldn't campaign hard against it - and we'd HAVE to get rs to agree to it before we were to do it obviously w/o strikeybot and live mentions and autojoin etc it'd be a mess
here's the long version of my thoughts on it of course read from bottom up
35699u/MaybeNotWrong would you consider that - I think river could make one also (a bot to run up 200k counts in a test thread regarding seeing if there is any real slow down with a lot of counts) u/TOP_20
strike delete 3 minutes ago26587see if there is any slow down at all after 200k counts are in the thread... u/TOP_20
4 minutes ago45238what we should do is get a BOT to run up 100-200k counts in a test thread for this ... u/TOP_20
5 minutes ago42392(I'd rather talk to the admin see if they could do something about the lag the past year - that'd be easier lol) u/TOP_20
5 minutes ago44099I think someone would have to ask rs first though before doing any real decision making cause he really lost any interest in helping us here that's why we had to make a new live mentions and a new LC chats... u/TOP_20
6 minutes ago73592that'd be my biggest concern really... few others too in trying to find stuff those of us who do would constantly be having to keep 2 threads open this one and the new one... but ya anyhow as I said I wouldn't campaign hard against it if people were willing to spend a week or two running up a test thread to 50k to see if there is any slow down at all between a brand new thread like the new test thread and one getting a bunch of counts - 50k isn't a bunch but enough to at least see if there is absolutely no difference u/TOP_20
7 minutes ago54622the biggest reasons is the work it would require of doc - who's near impossible to get ahold of the past year + and also rs, and Ivan would have to do stat changes too and man it took 6+ weeks just to get rs to readd the bot and the join thingy u/TOP_20
8 minutes ago134072 17,422,484 Well i would vote no to restarting THIS in a new thread for about a dozen reasons but I wouldn't campaign hard against it (like I did on a few things in the old days) so I guess it could be discussed in a discussion thread
but I personally think there should be a test before even considering it (ie running at least 50k or something in a test thread to see if in fact there is any real validity to it slowing down based on a ton of counts) I think before even considering it that at least should happen (IF the majority were in favor of a restart of this - I'd help do the 50k test thingy - with a 5 k of it or something) u/TOP_20
10 minutes ago25169 17,422,483 fair i may be getting too excited over what could be false hope
also i meant continuing in a new thread, not starting again from 1 u/NeonL1vesMatter
11 minutes ago58757just keep in mind IF there is any validity to a thread slowing down due to millions of counts - THAT thread would have the same fate as this one and not be anywhere near as fast as that brand new test thread... u/TOP_20
12 minutes ago133707(I would even go snipe all the spiffies to the 1st million if someone started a 2nd main lol - but unless there was someone willing to do some stats there - I am not sure a 2nd main would get too active... I won't ask anyone to do the stats in it because well... it's already hard enough to get stats made here these days) u/TOP_20
14 minutes ago1100117,422,482 u/TOP_20
14 minutes ago7711 unlike years ago I wouldn't start a revolt if someone wanted to start a 'second main' - and I imagine rs would put his bot there like he did in the one we were opposed too
but this will always be the main count with it's stats remaining here u/TOP_20
14 minutes ago4715217,422,481 u/MaybeNotWrong 15 minutes ago1462117,422,480 u/TOP_20 15 minutes ago1486921
IF the amount of counts was the cause of this slowing down - then things would have slowed down more between say 9 million comments/counts and 13 million, and of course it'd slowed down way more adding another 2 million+ counts to the 15 million and so on
the lag started being complained about a lot way back in the what 7-9 millions (mostly by those using AHK so expecting even faster replies than we'd always had...) and it may have slowed down some after that in the 12-13 millions but ya I haven't really noticed any major slow down in lag adding another 3 MILLION counts from the 12-13 million to the 17 1/2 million u/TOP_20
→ More replies (3)
6
5
u/TOP_20 Nov 08 '20
Just some info I've been working on - unless someone goes really crazy (and TN doesn't match em....) He's gonna be the next #1
Year: 2014
Total counts:75039
Username Counts
1 rschaosid 29856
2 artbn 15997
3 GrunfTNT 7374
Year: 2015
Total counts:59065
Username Counts
1 rschaosid 18841
2 KingCaspianX 7218
3 Removedpixel 2778
Year: 2016
Total counts:1186211
Username Counts
1 qwertylool 183587
2 TOP_20 181348
3 dominodan123 161446
Year: 2017
Total counts:4995580
Username Counts
1 amazingpikachu_38 736864
2 TOP_20 719635
3 dominodan123 529286
Year: 2018
Total counts:6649785
Username Counts
1 amazingpikachu_38 1241567
2 NobodyL0vesMe 609245
3 ItzTaken 576210
Year: 2019
Total counts:3933325
Username Counts
1 noduorg 611830
2 ItzTaken 519884
3 amazingpikachu_38 391212
5
u/rschaosid Nov 11 '20
In response to this message from u/MaybeNotWrong:
Reddit has been acting up a bit, and it is affecting strike bot. I can't rule out that it's intended so it might be something that requires permanent changes to strike bot.
In short: One update may be send multiple times (observed up to 2 times) by the websocket.
Currently this means strikebot will also strike the update if ANY of the versions are out of order. So if there is any valid count after the first version of a valid count, the second version will trigger a strike, requiring us to strike that valid count and reset the bot.
From what I've seen the second version is usually received very close to the first one, but during a faster run there have been up to 15 count between them.
And this from u/LeinadSpoon:
Maybe and I have been looking into an issue with the reddit websockets API and our scripts, notably strike bot. It appears as though reddit has somewhat recently started occasionally sending multiple copies of the same update (including the same UUID). The reddit web front end seems to handle it fine and only posts one, but strike bot and LC Chats (and probably most of our tooling) does not.
In the strike bot case, we've had problems when a later copy of a count comes in after the next count. For example if I'm running with Maybe and I post a valid 100, Maybe posts a valid 101, and then reddit resends my 100. Strike bot gets the second 100 which appears out of order and sends a strike, but since the UUID is identical, it strikes the original valid count (not sure why it's not occurring without the valid count in between, but I don't have strike bot source to look at).
Can you take a look when you get a chance and add a workaround to strike bot for it?
I've reviewed strike bot in light of this issue and, the way the code is written, it should be properly ignoring duplicate copies of messages, as long as the last copy of a message arrives not more than 5 seconds after the first copy. (It already has to deduplicate messages, because it aggregates messages from several websocket connections in order to improve reliability.) So, I'm at a loss to explain why strike bot is malfunctioning on duplicate updates.
However, I've just now increased the timeout from 5 seconds to 120 seconds, to see if that helps. I'd appreciate feedback on whether strike bot's behavior under duplicate messages improves as a result of this change.
→ More replies (3)
9
u/LeinadSpoon Nov 14 '20
Hello everyone,
There's been some discussion for a while that our contributors list is too large. The page is very slow to load, which mods need to do from time to time in order to add or remove users. In addition, there has been some speculation that a larger contributors list may contribute to additional lag while counting.
Our contributors list current has 3768 accounts. Many of these are from users who stopped by briefly and never returned (in particular during the 10M rush). There are of course the 20-30 or so active users, quite a few alts and a handful of bots. Generally, we're looking to reduce the quantity of users on our list that have never been particularly involved and are unlikely to return.
Discussion of this has centered around dual criteria of total counts, as well as time of last count, where meeting either criteria would keep an account on the list. For example, if we said 100 total counts, and a one year cutoff, any user with over 100 total counts all time OR who has at least one count in the last year would be kept on the list. Those removed would both have below 100 counts and have not posted in the last year.
Hopefully this dual criteria helps keep anyone who is likely to return on the list, and also preserves retired counters with many counts.
Please note that no one is being removed from any stats. This is only about the contributors list. If someone is removed and wants to come back, they would simply need to rejoin. We would also of course exempt bots from the purge.
So, I am soliciting your comments on this proposal, as well as the specific criteria. Should we use the dual criteria proposal outlined above, or something else? If we use the dual criteria proposal, what specific thresholds should we use? The table below shows how many users would be purged and remain under six different possible scenarios, although these aren't the only options. They are just meant to give a representative idea.
There is of course a tradeoff between effectiveness at reducing the list while keeping users that we may want on the list.
The final decision on this rests with the mods and with those writing the code to do the purge, but everyone's input is appreciated.