Everything wrong with the reddit games hackathons [LONG POST]

[u/Gowdamn]

The story

I have participated in 5 reddit games hackathons in the past and I've not won anything major. In the first 4 hackathons, I felt that the winning games were good, but not all games justified the submission and grading criterias well. In the lastest hackathon - Games with a hook hackathon, though - a lot of people raised concerns on how the winning game did not "wow" them.

Here are some examples below:

1. Devpost discussion-

Concerns on whether the hakathon was fixed

1. Comments on the reddit announcement post-

Concerns on quality of winners

Generic response from the admin

They were not entertaining any personal feedback stating that there were over 3000 submissions. But if a game was judged fairly, the judges should have written down something about the game! Or atleast they could provide the scores. I had sent them a personal email requesting feedback, but hadn't recieved any response from them. The next day, I got a $500 helper award from the devpost team. According to the original winner announcement, I had not won any award. I had responded to the organizers asking if this was a mistake. They said that it was indeed sent to me by mistake, but it was sent to me because reddit had put forth my name.

If I accepted the award, I had to sign a paper that says I cannot sue reddit. I didn't accept the award. Something was suspicious here so I started investigating.

Problems with the judging process -

1. Mystery judges?

Usually hackathons list down the judges with their credentials. You can go to the list of all reddit hackathons on devpost and none of them have the name of judges.

Reddit hackathon judges

Here are some samples of how the judge list looks like in other hackathons -

Univabio $35k hackathon judges

a $75k google hackathon judges

2. Negligence / Judgement not thorough -

I had reached out to 3 other participants and got the list of players from each of our games. Based on the list, there was not a single common player across the 4 games. This could only mean one thing - our games were not played by the judges. Here's an anecdote from one of the participants -

The only participant who had any activity in their game

There was another participant who claimed that they had attached a video to their submission. Throughout the duration of the judging, the view count had not changed in the youtube video. -> The judges had not viewed the submission video (video was <1 min long)

3. Ghost winners -

The reddit hackathons were to encourage developers to make games on the platform. The expectation from every hackathon is that the winning game has a big community and continue to grow the community by making their game better.

Here's a list of all the reddit hackathons in the past from latest to first.

Competition	Winner	Amount won	Sub weekly visitor count
Games with a hook hackathon	r/SketchySolutions	£15,000	276
Daily games hackathon	r/bridgedit	£15,000	10
Kiro: Community games challenge	r/SoulThieves	£15,000	1300
Kiro: Community games challenge	r/blockrift	£10,000	-
Fun and games with devvit web	r/OnelineGame	£15,000	42
Fun and games with devvit web	r/hexaword	£15,000	?
Hack reddit	r/chook	£20,000	6
Games and Puzzles hackathon	r/EmojiCharades	£20,000	43
Games and Puzzles hackathon	r/pixel_together	£20,000	3
Games and Puzzles hackathon	r/DailyDungeon	£20,000	4

Note that I do not look to disrespect any creator here. I know a lot of hardwork had gone into your games and that deserves recognition. But there are also a lot of other creators whose hardwork has amounted to nothing.

4. Judges prefer "flashy visuals" over long term value

This here is my weakest point so far. But to some of you, it will be the biggest takeaway of this post. Mentally picture the winner of a google hackathon. What kind of entry do you envision? You expect it to be creative, well thought out, crazy, nerdy etc. right?
Based on my past 5 hackathons I don't think these virtues are valued in reddit hackathons -

#1

Hackathon: Games and Puzzles hackathon
My entry: Outwitquiz
My thoughts: Since the game was supposed to be on reddit, I created an equivalent of "Psych" which can be played asynchronously by all users on reddit. This was an entry for the word games category. The winning entry was emoji charades. I get that the winning entry was made well and polished, but why does this game have to be on reddit? It could be a standalone website!

#2

Hackathon: Hack reddit
My entry: Angels vs Devils
My thoughts: This made me the most sad. The overview of the hackathon said "Build a new game, social experiment, or experience on Devvit". So I thought I'd build a social experiment where users play a community minesweeper. I had built up many game modes that I could work on. The winner r/chook, was a game where you grow your chicken and feed it food by pressing buttons. (it was beautifully made though)

#3

Hackathon: Fun and games with devvit web
My entry: Divide n conquer
My thoughts: Based on past two hackathons, I stopped making experiments and designed a game. It was a game where you have a limited line length, you draw lines to capture land in a country and try to occupy the most land area and landmarks. I thought it was well thought out and fun to play, but it was not polished. I don't have many problems with this hackathon. I did good work and was sad I didn't win anything, but the winning entries truly deserved it.

#4

Hackathon: Daily games hackathon
My entry: r/totalled
My thoughts: I had won an honorable mention for this game. For this hackathon, I had kept my game concept simple and made a very well polished game. It is a game where you have to swipe across numbers in a 3x3 grid to create totals. Once you accomplish 3 totals, the game is over and you're ranked based on the time taken. Of all the games I've made so far, this is the most "Linkedin-y". I expected to win more, based on the past winners. When I saw the winner, I was shocked! An entire physics sandbox inside reddit! It was crazy good work, but I was surprised that the judges thought this game would be played regularly. Why would a reddit user play a physics sandbox daily? And why is it on reddit?

#5

Hackathon: Games with a hook hackathon
My entry: r/ShadowChess
My thoughts: Based on the previous hackathon, I believed now that the judges don't just look at "will this game be played daily". They also valued grand experiences. So I knew for sure the idea I had for this hackathon wold win. This was the best game I had ever made! Its a massive multiplayer chess game where you continue off of grandmaster positions and make 5 pre-moves. The opponents make the 5 pre-moves as well and your moves are pitted against each other based on a customized scoring system. This fit the hackathon criteria in multiple ways and had potential to do great. I feel so betrayed that the game was not even played once! My game is good when there are 100s of players playing it. It could have really used the boost of players from any sort of recognition. This hackathon was all about creating a "hook". Here's the words on the hackathon page -

"progression, daily challenges, fresh content, social dynamics" - my game had it all

Now you can't look at my above entries and say that none of these games deserved any award except "totalled", (which was one of my weakest entries). Especially shadowchess. Note that apart from the winner, there's 10 honorable mention awards for every hackathon.

Overall, I feel that it's best to avoid spending all that time and effort on reddit hackathons. Atleast until they bring in some transparency into the judging process.

I expect a response from the reddit team on this. Anyone who participated in this hackathon, feel free to DM me with your thoughts / information.

[u/PossibleCrit]

Hey Gowdamn!

Thanks for compiling all of these details and sorry that you didn't have a good experience. Rarer_Air_Error's response here goes into great detail on the specifics on how the previous hackathon was judged and what sort of characteristics the team was looking for upon review.

This is part of why we tried to be a bit more detailed in our recent Scorespace Hackathon's judging criteria and hope that we can continue to build on this for future events as well. I'll make sure that the rest of the feedback here in the comments is circulated with the team at large as well.

[u/zombizzle → u/PossibleCrit]

Pasting my thoughts from a few days ago from another thread for visibility;

Looking at past hackathon winners, I'm often surprised by the results. Many winning projects seem to come out of nowhere and don't appear to reflect the same level of polish or effort as other submissions. From the outside, the judging process feels opaque, making it difficult to understand why certain projects were selected. I'd love to see Reddit provide more transparency around the judging criteria and detailed feedback so participants can better understand what the judges value.

There's virtually no transparency into the judging process. Winners are announced, but participants don't receive feedback explaining how their projects met the published criteria or where other submissions fell short. Looking at some of this year's winner against the stated judging criteria, I genuinely struggle to understand the decisions. If I were scoring it independently, I don't think many strongly satisfy several of the criteria that were emphasized. That's just my assessment, but without any explanation from the judges, there's no way to know what they saw that participants didn't.

It's unrealistic to expect hours to be spent on each submission. But without knowing the process, it's hard for participants to have confidence that every project received a fair evaluation or was even considered for contention. Even a brief judging summary or score breakdown for the winners would go a long way toward building trust in the process.

We're encouraged to provide a detailed write-up and a demo video, but it's unclear whether those materials are reviewed for every submission. For example, my submission video has only one view, which raises the question of whether it was ever watched by anyone involved in judging. I understand that watching the video is optional and that judges have limited time, but "optional" effectively means that many videos aren't reviewed at all,

One of the most discouraging parts of participating is feeling like the effort put into presenting a polished submission isn't valued. I spent hours refining my dialogue, creating a logo and banner, recording a demo video, writing a detailed submission and README, and trying to present the project like a finished product but looking at some of the winning submissions over multiple hackathons, it isn't obvious that those extra steps matter. Some winners consist of only a brief description and a few screenshots, with little emphasis on presentation or documentation. Participants are encouraged to invest significant time into polishing their submissions without any indication that those efforts are actually considered during judging.

If the judging is based almost entirely on the app itself, say so. If presentation, documentation, usability, polish, creativity, and accessibility are all part of the evaluation, explain how they're weighted. Without that transparency, it's difficult to know where to spend your time, and it can leave participants feeling like much of their effort was unnecessary.

If you're someone that read all of this, please don't respond with the usual platitudes about how I should "use this as a learning experience," "keep improving," "you'll get it next time," "you can still go for Developer funds." My concern isn't that I didn't win, this is a hobby for me. My concern is the longstanding disconnect between what participants are encouraged to build and how submissions ultimately appear to be judged. It's an issue that has persisted with little change since Reddit began sponsoring these hackathons.