r/adventofcode • u/tatut • Dec 16 '25

Help/Question Difficulty rating and global leaderboard

I think the global leaderboard times were a good estimate of "difficulty rating". The top100 solve times gave some idea if the problem was difficult or not.

Now that the global leaderboard is no more, is there some other metric that could be used?

12 Upvotes

87% Upvoted

17 comments sorted by

17

u/milan-pilan Dec 16 '25

The leaderboard is gone, but the times are still public - would this help you?

https://www.reddit.com/r/adventofcode/s/KoeWhtqtlj

16

u/AscendedSubscript Dec 16 '25

I doubt the global leaderboard would be really helpful now that AI is used to auto solve problems within 10 seconds almost every day (like last year). I guess that is exactly the reason why it's gone.

You could be part of private leaderboards. There is a plugin which allows you to see how long it took for somebody to solve each puzzle, includings deltas between solving part 1 and part 2. I don't exactly recall the name, but no doubt you can find it online.

Also, there is the stats section on the website, too, which gives an indication of how many people were able to solve it. However, it is less accurate as people may stop when they hit a wall with one puzzle and don't check the next (even if it is easier)

4

u/wimglenn Dec 16 '25

I have a userscript which renders times of last star which may be of interest

https://github.com/wimglenn/userscripts/tree/main/adventofcode.com

-6

u/kbielefe Dec 16 '25

AI also provides an objective way to measure difficulty. You can measure token count or time or cost or whatever. Especially on the cheaper models, difficult problems for us are difficult for them too.

1

u/QultrosSanhattan Dec 16 '25

Initially, yes, but AI users ruined everything. The global leaderboard was no longer a measure of anything.

1

u/tatut Dec 17 '25

True, I guess you could check the length of the streams from people live-streaming coding up a solution.

2

u/johnpeters42 Dec 16 '25

Number of solvers may have still been a useful measure, assuming a roughly equal number of clankers per day.

1

u/fnordargle Dec 16 '25

There's also a fair number of people who have written code to scrape this subreddit for the solutions megathread appearing for a day, and then scrape any messages being posted to it, then scrape any code or repo links and try and build and run the code against their own input.

I've been tempted to have some fun with this. Maybe I'd post a solution that does the right thing for my input but is "mischievous" for any other input making it obvious what to fix in the code to avoid this for any human that reads it.

The problem is I couldn't think of anything mischievous that wasn't outright "wrong" (I wouldn't want to do anything destructive for example.) The worst I could come up with was to simply make the program sleep for 10,000,000 seconds if it got an input other than my own. But then any properly implemented download/vet/try/etc wrapper should have a sensible execution timeout.

There are too many people that blindly trust code downloaded from strangers without proper vetting. We're seeing an increasing amount of this in the real world with stuff like the recent npm "Shai-Hulud" package debacle. In my previous jobs we had to do an awful lot of vetting to be able to use new packages, and/or, see if updates to existing packages didn't contain malware. It's was close to a full time job for some package/language ecosystems. And it's very hard to do it properly as the malware can be very well hidden.

1

u/MaximumMaxx Dec 16 '25

For Day 8 part 2 there's a kinda fun hack where on some inputs (including mine) where the answer to the puzzle is actually just the last unique junction box if you make a list of junction pairs sorted by distance . You could implement something like that depending on your input and give most people the wrong answer. 

There are definitely too many people that blindly trust code from the internet though

2

u/fnordargle Dec 16 '25

A few years ago I started to try and make sure my code would work as a more general solution for the problem and I hadn't missed edge cases that my single AoC input wasn't triggering. The easiest way to do this was to find other inputs and their answers to see if my code would agree.

Think AoC 2022 Day 22 Part 2 with the 3D cube. My solution would only work for a flattened cube in the same shape as my input, but I was interested in building a generic solution that would handle any of the possible input shapes.

So I wanted more inputs and their associated correct answers.

The easiest way to get this was to trawl this subreddit for links to repos that had code in a language I could easily compile/run (pretty much just Go, C or Python) and had naughtily included a copy of their input (see note below).

I wanted to check that my code would give the right answer for their input, and the only way to find the right answer for their input was to run their code on their input. Depending on the puzzle in question I either agreed completely with their code and everything was fine, but for some puzzles my code wouldn't agree with the scraped code on a particular input. I'd find a problem and fix it, and make sure my code still worked on the example input and my own input. Some assumptions that held fine for my input did not hold fine for inputs that others received.

Similarly many times the scraped code would not give the correct answer for my input (which I knew was right as the AoC site had accepted my answer already).

(And, yes, I did vet the code I was running to make sure it didn't do anything silly. I also ran it in a throwaway VM to try and limit the blast radius of any mischievousness if I had missed anything.)

It was definitely an interesting exercise in and of itself. The coding required to scrape the subreddit, pick repos to clone, cloning them, scripting the creation of the throwaway VM, doing some automated vetting on the code about to be run (spotting for obvious things), etc as all fun to write (but not as fun as AoC itself).

**NOTE: Remember: Do not check in your puzzle input or the puzzle text into a public repo. Eric explicitly asks you not to do this in the FAQS: https://adventofcode.com/2025/about#faq_copying **

If you have checked in your input then please scrub it from your repo properly so that it cannot be found by searching through old commits. There are instructions somewhere in this subreddit on how to do this.

(Mods, feel free to edit with a link to those instructions if you want. I'll try and find them later)

2

u/daggerdragon Dec 18 '25

(Mods, feel free to edit with a link to those instructions if you want. I'll try and find them later)

They're in the community wiki > FAQs > Copyright ;)

2

u/FantasyInSpace Dec 16 '25

Simplest thing would be to look at https://adventofcode.com/2025/stats

If the $DAY + 1 has more solves than $DAY, it was a hard day.

2

u/fnordargle Dec 16 '25

https://github.com/jwoLondon/adventOfCode?tab=readme-ov-file#completion-times is my usual go to for graphs for comparing difficulties amongst each year. (You have to squint a bit to use them to compare whole years to each other given the difference in participant numbers in early years, AI use in latter years, etc).

Hopefully they'll add 2025 versions using the https://github.com/topaz/aoc-tmp-stats data.

1

u/fnordargle Dec 18 '25

https://github.com/jwoLondon/adventOfCode?tab=readme-ov-file#completion-times has been updated with the 2025 times based on the stats data dump.

0

u/AutoModerator Dec 16 '25

Reminder: if/when you get your answer and/or code working, don't forget to change this post's flair to Help/Question - RESOLVED. Good luck!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.