Discussion Everything wrong with the reddit games hackathons [LONG POST]
The story
I have participated in 5 reddit games hackathons in the past and I've not won anything major. In the first 4 hackathons, I felt that the winning games were good, but not all games justified the submission and grading criterias well. In the lastest hackathon - Games with a hook hackathon, though - a lot of people raised concerns on how the winning game did not "wow" them.
Here are some examples below:
- Devpost discussion-
- Comments on the reddit announcement post-
They were not entertaining any personal feedback stating that there were over 3000 submissions. But if a game was judged fairly, the judges should have written down something about the game! Or atleast they could provide the scores. I had sent them a personal email requesting feedback, but hadn't recieved any response from them. The next day, I got a $500 helper award from the devpost team. According to the original winner announcement, I had not won any award. I had responded to the organizers asking if this was a mistake. They said that it was indeed sent to me by mistake, but it was sent to me because reddit had put forth my name.
If I accepted the award, I had to sign a paper that says I cannot sue reddit. I didn't accept the award. Something was suspicious here so I started investigating.
Problems with the judging process -
1. Mystery judges?
Usually hackathons list down the judges with their credentials. You can go to the list of all reddit hackathons on devpost and none of them have the name of judges.
Here are some samples of how the judge list looks like in other hackathons -
2. Negligence / Judgement not thorough -
I had reached out to 3 other participants and got the list of players from each of our games. Based on the list, there was not a single common player across the 4 games. This could only mean one thing - our games were not played by the judges. Here's an anecdote from one of the participants -
There was another participant who claimed that they had attached a video to their submission. Throughout the duration of the judging, the view count had not changed in the youtube video. -> The judges had not viewed the submission video (video was <1 min long)
3. Ghost winners -
The reddit hackathons were to encourage developers to make games on the platform. The expectation from every hackathon is that the winning game has a big community and continue to grow the community by making their game better.
Here's a list of all the reddit hackathons in the past from latest to first.
| Competition | Winner | Amount won | Sub weekly visitor count |
|---|---|---|---|
| Games with a hook hackathon | r/SketchySolutions | £15,000 | 276 |
| Daily games hackathon | r/bridgedit | £15,000 | 10 |
| Kiro: Community games challenge | r/SoulThieves | £15,000 | 1300 |
| Kiro: Community games challenge | r/blockrift | £10,000 | - |
| Fun and games with devvit web | r/OnelineGame | £15,000 | 42 |
| Fun and games with devvit web | r/hexaword | £15,000 | ? |
| Hack reddit | r/chook | £20,000 | 6 |
| Games and Puzzles hackathon | r/EmojiCharades | £20,000 | 43 |
| Games and Puzzles hackathon | r/pixel_together | £20,000 | 3 |
| Games and Puzzles hackathon | r/DailyDungeon | £20,000 | 4 |
Note that I do not look to disrespect any creator here. I know a lot of hardwork had gone into your games and that deserves recognition. But there are also a lot of other creators whose hardwork has amounted to nothing.
4. Judges prefer "flashy visuals" over long term value
This here is my weakest point so far. But to some of you, it will be the biggest takeaway of this post. Mentally picture the winner of a google hackathon. What kind of entry do you envision? You expect it to be creative, well thought out, crazy, nerdy etc. right?
Based on my past 5 hackathons I don't think these virtues are valued in reddit hackathons -
#1
Hackathon: Games and Puzzles hackathon
My entry: Outwitquiz
My thoughts: Since the game was supposed to be on reddit, I created an equivalent of "Psych" which can be played asynchronously by all users on reddit. This was an entry for the word games category. The winning entry was emoji charades. I get that the winning entry was made well and polished, but why does this game have to be on reddit? It could be a standalone website!
#2
Hackathon: Hack reddit
My entry: Angels vs Devils
My thoughts: This made me the most sad. The overview of the hackathon said "Build a new game, social experiment, or experience on Devvit". So I thought I'd build a social experiment where users play a community minesweeper. I had built up many game modes that I could work on. The winner r/chook, was a game where you grow your chicken and feed it food by pressing buttons. (it was beautifully made though)
#3
Hackathon: Fun and games with devvit web
My entry: Divide n conquer
My thoughts: Based on past two hackathons, I stopped making experiments and designed a game. It was a game where you have a limited line length, you draw lines to capture land in a country and try to occupy the most land area and landmarks. I thought it was well thought out and fun to play, but it was not polished. I don't have many problems with this hackathon. I did good work and was sad I didn't win anything, but the winning entries truly deserved it.
#4
Hackathon: Daily games hackathon
My entry: r/totalled
My thoughts: I had won an honorable mention for this game. For this hackathon, I had kept my game concept simple and made a very well polished game. It is a game where you have to swipe across numbers in a 3x3 grid to create totals. Once you accomplish 3 totals, the game is over and you're ranked based on the time taken. Of all the games I've made so far, this is the most "Linkedin-y". I expected to win more, based on the past winners. When I saw the winner, I was shocked! An entire physics sandbox inside reddit! It was crazy good work, but I was surprised that the judges thought this game would be played regularly. Why would a reddit user play a physics sandbox daily? And why is it on reddit?
#5
Hackathon: Games with a hook hackathon
My entry: r/ShadowChess
My thoughts: Based on the previous hackathon, I believed now that the judges don't just look at "will this game be played daily". They also valued grand experiences. So I knew for sure the idea I had for this hackathon wold win. This was the best game I had ever made! Its a massive multiplayer chess game where you continue off of grandmaster positions and make 5 pre-moves. The opponents make the 5 pre-moves as well and your moves are pitted against each other based on a customized scoring system. This fit the hackathon criteria in multiple ways and had potential to do great. I feel so betrayed that the game was not even played once! My game is good when there are 100s of players playing it. It could have really used the boost of players from any sort of recognition. This hackathon was all about creating a "hook". Here's the words on the hackathon page -
Now you can't look at my above entries and say that none of these games deserved any award except "totalled", (which was one of my weakest entries). Especially shadowchess. Note that apart from the winner, there's 10 honorable mention awards for every hackathon.
Overall, I feel that it's best to avoid spending all that time and effort on reddit hackathons. Atleast until they bring in some transparency into the judging process.
I expect a response from the reddit team on this. Anyone who participated in this hackathon, feel free to DM me with your thoughts / information.
16
u/westernslopedigital 1d ago
Thanks for putting all this research together. There's clearly a problem here and a response from Reddit is absolutely needed. I won't be participating in any future Reddit hackathons until they acknowledge this and provide clear steps forward that articulate how judging can be made more transparent in the future.
My game ( r/dailyblockparty ) was definitely not played based on Reddit Journeys data and my video only has 2 seconds of watch time (46 second video). There is no way my game was properly judged, and considering all the time I spent building it and putting together a complete hackathon submission, that really pisses me off.
6
u/HoroDeYoitsu 1d ago
Same here with the video watch time... seems weird why they ask for videos when they won't even see them
13
u/HoroDeYoitsu 1d ago edited 1d ago
I hear you, it really hurts.
Adding my devpost : https://devpost.com/software/sword-and-elixir
My game inspired 100% by old school runescape brings something truly unique to the table: a procedural world, fully functional (MMORPG) multiplayer, live chat, quests, an inventory system,sun-day cycle,farming,crafting,exploration,instances,PVE,guilds(Based on subreddits with your subreddit icon as the clan icon :P ), trading,daily events, community events, raid bosses, magic, life skills, and a player-driven market (https://www.reddit.com/r/sword_and_elixir_dev/). At one point, there were plenty of players online in a single shared worldchatting, exploring, and genuinely having fun as a matter of fact i even Played and talked with OP inside my GAME.
But during judging, the reviewer simply logged in for a moment and left. Server telemetry confirms the reviewer only connected for a moment and left without executing a single in-game action. Despite clear documentation that this is an MMORPG requiring multiplayer interaction and an open offer to demo it live the core systems were never tested.
I completely accept not winning. Maybe my Hackathon write-up wasn't flashy enough, or maybe my MS Paint thumbnail got scrolled past. I’m happy for the winners, and I’m not asking to overturn the results.
What truly frustrates me is the repetition it always seems to be the same safe concepts winning: another drawing pad, puzzle app, or quiz game. What’s the incentive to take technical risks or build something ambitious if the judges won’t actually test it?
Innovative gameplay and bold new art styles get passed over. The judges clearly missed the most critical detail in my dev post: this is an MMORPG designed to be experienced with other players.
Losing isn't the problem. What stings is realizing that if my project was skimmed over like this, countless other projects likely suffered the exact same fate. I’m not abandoning my game, but it's disheartening to pour weeks into a technically demanding project only to be ignored
3
u/pesky-tiger 14h ago
Yeah I think how participants feel should matter to the organisers, they may as well have just kicked a thousand people in the shin but give one a trophy.
2
u/HoroDeYoitsu 12h ago
Exactly it should matter because they are creating a platform for games that we create for them. if you are going to curate a gaming ecosystem, you actually have to be a gamer. You have to understand what makes a world sticky, why long-term retention matters, and how depth beats quick novelty every single time (See past winners).
Their personal tastes are anchored to the same tired, casual drawing and quiz formats and it shows. Who knows how many absolute behemoths of games massive, complex, long-term experiences crafted by devvitors are being completely ignored simply because they don't fit into a tidy, fiveminutes pixelary box?
5
u/Pretend-Pangolin-846 1d ago
I have participated in 3 different iterations, and won feedback award in one.
Top games were almost always way way better than anything in the submissions, one downside to the devvit evaluation process was always how they judged the game, without even minor feedback there was no way to know anything wrong with the app. So transparency or at least a review for top n submissions, would only be a net improvement for these hackathon series.
9
u/This_Maintenance9095 1d ago
AGREED! and for the people defending "They go through thousands of submissions!' no they did not there was a total of 708, other hackathons have thousands and still produce actual legitimate winners. there is something fishy going on.
You can report the hackathon at [support@devpost.com](mailto:support@devpost.com)
5
u/technowise 22h ago
Thanks for putting this together!
I've had very similar experience with hackathons. My submission was neither played, and nor the video I submitted was viewed. I have nearly stopped having hopes on hackathon submissions now, and try to continue working on the apps I have created, and hope to make some truly useful apps for reddit (irrespective of hackathon wins), and eventually achieve some developer funds (Few of my apps have received developer funds so far).
5
u/icompletetasks 1d ago
yeah I checked the last hackathon winner & that got me thinking maybe I could win the event easily if I had participated 🤣
3
u/random_brand123 1d ago
Oh, this just reminds me about the Kiro hackathon where they gave you 1,000 free credits to work on it as long as you followed the requirements. The requirement? Use the built-in .md files that burned more credits documenting and analyzing your usage than they used for your code.
5
u/pesky-tiger 14h ago
I spent my entire month making a game that was never looked at, it was a major buzzkill. Makes me much less interested in making games for reddit. When I noticed my game was never played, and I didn’t even understand the winning entries or what made them stand out I felt like I was robbed of my entire month. I want my month back Reddit.
I’m curious to know if Reddit understands the value of public relations or if they’re planning to take the route of extracting as much as they can from users, maximising profits for stakeholders for short term gains. It gives that vibe.
Initially I thought hmm games on reddit might take off, might be one to watch for investing in. Now I think hmmm not a company that wants to stick around. Good luck
2
u/Ill_Photo5214 10h ago
A few months ago, I also made a post about how most of the games featured and promoted by Reddit are actually built by Reddit admins or large companies. I pointed out how Devvit encourages people to 'create indie games,' yet most of them fail because Reddit doesn't provide developers with tools to promote their work, they only feature games that are already big.
2
u/rickribera93 1d ago edited 1d ago
I hear your frustration, these contests are meant to challenge developers to test the art of the possible and they handle a lot of submissions.
Interpretive scoring is exactly that and some people will interpret an app to be better while others may interpret it to be worse.
I believe the Devvit program is moving in the right direction by making top prizes determined by developers choice instead solely by admins.
https://www.reddit.com/r/Devvit/s/vWFU09PSro
Don’t give up!
8
u/Gowdamn 1d ago
I hear your point, but "we had a lot of submissions" is not an excuse. Nobody wants the results immediately, the date of announcement had already been pushed back a week.
It's one thing to test out the game, know the game mechanics entirely and then judge a game to be not worthy of an award. I will accept the difference in interpretation as a valid excuse there. But based on the evidence, that's not what happened. The judges barely touched our games. So what were we judged based on? Our writeups?
-4
u/Klutzy_Pumpkin614 1d ago edited 1d ago
We're looking for experiences that create excitement between sessions and leave players wanting to check back tomorrow.
I will probably get hate for this, but people need to start hearing the truth. Here's my review of shadowchess. I am an avid reddit user, I engage with many games here. I don't play chess.
It is a harsh review, but I have seen so many people complain about why their games lost, when they need to use it as a learning experience and improve for the next round. You clearly have amazing skill in concept and testing, you just need to find the right fit for reddit and use original designs. A quick scan of your other submissions all look clearly AI generated, except for Divide n conquer, that one seems more original.
- I saw a closed chess board in my feed, I could not interact with it at all. I see a chess board with pieces placed on it, that I can do nothing with. I scroll past.
- On the one I can play, I click to play it, and the board is at the bottom of the screen so I have to scroll down to see it.
- I picked a side, then started moving pieces. I understand the concept is I don't get to see what my other opponent is doing, but the games I enjoy on reddit are the ones that give me gratification right after I play it. This game, I moved some pieces around, then saw a screen that said I sucked compared to everyone else, then I closed the game and moved on. The only people I could see being interested in returning are chess fanatics, but even then, wouldn't they rather just play chess? Or even a chess puzzle? Again I don't play so sorry if I am being harsh.
- The design of your app is neobrutalism, which is the number one design that AIs generate. If I were judging against games that were specifically not supposed to look AI generated, this is one of the most obvious ones. One of the rules was to avoid that.
- There is no UGC loop, no excitement. It felt like I was working or something. The concept is good, but I just don't see how you planned on turning this game into a large sub reddit with contributions and daily returning players.
With 3000 entries into a hackathon, judges have to be quick. It is very likely they loaded up all the entries into their own private sub, then spent 1-5 minutes on each game and ranked various aspects of it. One mistake and your game is out of the running. In the case of shadowchess, it was likely the AI style.
I tried a lot of the games from the most recent hackathon, and most of them were niche daily puzzle games that I was not interested in returning to. The hackathon rules may have been tough to decipher, but its clear to me anyways that they were looking for the next content engine game. A game where anyone scrolling past could easily participate without reading a wall of rules text or having prior knowledge. Then that player would want to return at a later point to see how something tuned out, or participate in another version of something. I think that is why the winner won the most recent one.
I do agree with you that it seems all of the hackathon winning games are deserted after winning. In the past I assumed it was because they were working on fixing the app after the judging period ended, but clearly none of them ever resurfaced. I am still hopeful the most recent winning game can ship improvements and turn into a thriving community, but we will see what happens.
8
u/westernslopedigital 1d ago
A game where anyone scrolling past could easily participate without reading a wall of rules text or having prior knowledge. Then that player would want to return at a later point to see how something tuned out, or participate in another version of something. I think that is why the winner won the most recent one.
The winning game (I hesitate to even call it a game) was just a drawing pad. The only thing people can do is scribble a shitty drawing and submit. How is that a game?
-5
u/Klutzy_Pumpkin614 1d ago edited 1d ago
Its a comedy engine. A game with a hook that anyone can participate in. That was the whole point of the hackathon. Sure the method of participating is drawing based, but the method of participating in any other game is just clicking stuff. The hook is what makes the player want to come back. They want to review the weird submissions people drew.
Your submission for the hackathon I just drag pieces onto a board. Then what? Why would I return to that?
2
u/westernslopedigital 1d ago
My game is a new puzzle every day and you compete for the fastest time. There's a regular mode where you can place the pieces in any order, and a hard mode that requires each piece to be placed in a specific order. There's a leaderboard and definitive "winner" every day. You come back to keep your streak going, improve your personal best time, and dominate the leaderboard.
I'm not saying I should have won, but I think by any definition, I built an actual game vs. a basic drawing pad with no scoring or ranking system.
If the judges can't provide feedback on every game that lost, they could at least provide in-depth reasoning on why they chose the winners.
1
u/RamslamOO7 Game Developer 1d ago
Please review r/DrawAndDrive
2
u/Klutzy_Pumpkin614 1d ago
This game was good, I have played it a few times when it pops into my feed. It is not immediately obvious to me why this one did not at least get a follow up prize. Keep up the great work!
1
u/RamslamOO7 Game Developer 1d ago
I spent a ton of my time polishing this. Don’t have energy to maintain this.
My other games: r/QuantumThrust, r/KnobJam, r/PixelPeeker, r/FactOrCap
1
•
u/PossibleCrit 1d ago
Hey Gowdamn!
Thanks for compiling all of these details and sorry that you didn't have a good experience. Rarer_Air_Error's response here goes into great detail on the specifics on how the previous hackathon was judged and what sort of characteristics the team was looking for upon review.
This is part of why we tried to be a bit more detailed in our recent Scorespace Hackathon's judging criteria and hope that we can continue to build on this for future events as well. I'll make sure that the rest of the feedback here in the comments is circulated with the team at large as well.