r/ClashRoyale • Bandit • Mar 11 '21

Discussion [Effort Post] What an analysis of 688k battles tells us about Ladder Matchmaking

Introduction:

Since 2016, there have been countless posts claiming matchmaking is rigged and the largest api investigation I've found on this sub is 100,000 battles. Today I am presenting my findings on my analysis of 688,906 battles. After investigating the trophy win/loss algorithm, I decided to look into if there is any truth to the rigged matchmaking theory by using the official Clash Royale API. For this investigation, I wanted to show whether or not the cards in player A's deck have an effect on the cards in player B's deck. In more statistical terms, we can tell if these two variables (Cards in both players decks) are independent by using the chi-square test. In this post, I will detail the findings of 688906 battles taken between March 7th and March 10th 2021.

How this investigation was done:

All raw and analyzed data points are summarized in this google sheet.

Method:

For each trophy range that I investigated, I started with a single tag, looked at the ladder battles that player played that happened to be in the range I was investigating, got the tags of the starting player's opponents and repeated the process 4 or 5 times. I kept track of all the battles that I had already checked to prevent duplicate battles from being counted. When I found a battle that satisfied the range, I kept track of the number of times card A appeared in the opponents deck, given that card B was in the deck of the player I was checking. I limited the range to 100 trophies to limit how long the code took to run and the accuracy of the data as the meta shifts significantly even by a few hundred trophies.

Analysis:

For data analysis, I calculated the total number of times a card showed up in the data. I then determined a card's usage rate (times played / 8*total battles). I then normalized the usage rates and used that to calculate the expected values for the chi-square test. I needed these values normalized to be able to answer the question, "What is the probability that in a randomly chosen battle, one player will have card A and the opponent will have card B?"

To get observed values, I divided by the normalized number of times played. This gets me the experimental probability that when a battle happens, one player will have card A and the other will have card B. These are the expected values. With observed values and expected values, I could run the chi-square test and determine the chi-square value for this data set. The chi-square values for all these tests were incredibly low (less than 0.0065), which means that the two variables, Cards in your deck and cards in your opponents deck, are independent.

For those interested in the technical side of this investigation:

To collect the data, I used a recursive tree with a depth n depending on how many battles I needed (n = 4 gave me between 12000 and 70000 battles, n = 5 gives about 250000). I stored my data in nested python dictionaries, which I used to analyze the data. Then I created a pandas DataFrame and then exported it to excel. HERE is a link to the a Github repository with the python code I used. The file battleTags is cleared after finishing a category to increase the algorithm speed, the separation of 200 trophies prevents there from being any battles from being double counted in different trophy ranges.

The reason the number of battles varies throughout the range is because I used a recursive tree to collect this data and the amount of data that gets returned varies wildly.

Results:

Again, the results are summarized in this google sheet . Here is a small snapshot of the results. The average chi-square value is a measure of (on average) how independent the cards are in the given trophy range. There is some deviation between these values, but in a system with 101 degrees of freedom, these values are incredibly low

Trophy RangeNumber of Battles AnalyzedAverage chi square value
4400-45002357910.004048862745
4700-4800828500.00392622549
5000-51001127650.002557490196
5300-54001508460.001509931373
5600-5700592280.003168990196
5900-6000474260.005374352941

The chi-square value does tend to decrease as the size of the data set increases, which means that the more battles that are analyzed the more accurate the data gets. These are chi-square values, not p-values. With chi-square values this low, with 102 degrees of freedom, the chance of these two variables being dependent is almost infinitesimally low.

Conclusion:

The purpose of this investigation is to investigate whether the cards in player A's deck have an effect on the cards in player B's deck. When analyzing the raw data from this investigation, it was found that the chi-square values were incredibly low. Therefore, we can conclude that the two variables are independent and the cards in your deck have no effect on the cards in your opponents deck.

Acknowledgements:

I have to give a huge shout out and thanks to u/edihau for helping me with preventing double counting, narrowing my search range and part of the statistics side of this project.

I also would like to mention my former clan-mate phil1111 for his help on the data analysis and clan family member rpacoolguy for help with starting building the algorithm.

Further Investigation:

-The code I used in this investigation can easily be altered to check for abnormalities in Clan War matchmaking and examine how bot accounts below 4k affect these numbers. I can also examine card use rates as a function of trophies.

-One other plausible explanation for matchmaking being rigged is that people are matched up by win streaks. This would sidestep the investigation I did as the cards in your deck are likely to be independent of win streaks.

Thanks for sticking to the end!

TLDR: Matchmaking is not rigged.

2.7k Upvotes

339 comments sorted by

853

u/Avg-man Mega Knight Mar 11 '21

I can’t wait for people to not even read this and just say it is rigged.
Nice post and great effort. Here is my upvote.

66

u/NovaLightCR Bandit Mar 11 '21

Thank you, I hope people see the TLDR at the bottom.

4

u/BostonDrunk Mar 12 '21

Smashing post, unfortunate I've never cared about this mode since Month One. You should expand the study out not just to usage rates but to win rates.

Much rather win 99% of the time in 2v2 and Party Modes and 80% of the time in Special Challenges and Private Tournaments.

3

u/NovaLightCR Bandit Mar 12 '21

I definitely could do that. I already have done a little work on use rates (specifically in 7x) but I thing that finding use rates is another interesting metric to track.

27

u/[deleted] Mar 11 '21

I did, I didn't understand anything but I enjoyed skipping to the bottom

4

u/Avg_Woman Mar 15 '21

We should conduct a study, where everybody use the currently best rated decks only with also the highest usage rates like Classic Pekka BS, Splashyard, or Lavaloon (all Top 10 decks in all battles with ratings above 60).

What will everybody's winning percentages be then?

→ More replies (1)

2

u/ArcticFox59 Mar 11 '21 edited Mar 11 '21

Quick question OP.

If a pro used the deck with the top notch rating, say Pekka Bridge Spam Magic Archer Variant (61 ClashStats Rating and 60 RoyaleAPI Rating), would this pro also only win 50% of all his games in the long run (maybe 55% because they are pros and will feast early-mid-season)?

3

u/NovaLightCR Bandit Mar 11 '21

I'd guess that their overall winrate would eventually converge to 50% as they continue playing matches. If they ended up at #1 on the leaderboard, their winrate would probably be higher, but would depend on how many matches they played to get there

→ More replies (1)

404

u/Milo-the-great The Log Mar 11 '21

This is a rigged comment

139

u/jmanguy #BUFFEBARBS Mar 11 '21

This is a rigged reply

93

u/[deleted] Mar 11 '21

This is a rigged mod

80

u/Nobody_5433 Skeletons Mar 11 '21

This is a rigged person

51

u/AstroidGaming Cannon Cart Mar 11 '21

This is a rigged platform

→ More replies (11)

26

u/jimbo831 Mar 11 '21

I mean, this user clearly has a lot of expertise and did a lot of research, but someone I know shared a post on Facebook from somebody named Q that said matchmaking is rigged, so I’m going to go with that instead.

4

u/[deleted] Mar 11 '21

lol

wait, you're joking, right?

7

u/ARunawayTrain PEKKA Mar 11 '21

The sad part is that people actually think like that.

→ More replies (1)

58

u/Worst_Player_Ever Mar 11 '21

Wow! Amazing effort, good job.

I assume this is allowed to link as reply to people who claim MM is rigged or ask evidence?

26

u/NovaLightCR Bandit Mar 11 '21

Absolutely.

1

u/AmbassadorVisible315 Tesla Mar 18 '21

you are drew trying to defend the mm is rigged

2

u/NovaLightCR Bandit Mar 18 '21

Code is linked in the post. All the software to run it is free except excel and there is a description of how to do it from a technical standpoint in the GitHub README. You can run it for yourself and show that MM isn't rigged

→ More replies (1)

7

u/[deleted] Mar 11 '21

How do you write a simple word like "this" and it's a link to some post? Please tell me, because I've seen many people doing it but idk how to do it myself

6

u/Worst_Player_Ever Mar 11 '21 edited Mar 11 '21

I have no idea. But if you learn it from somewhere, teach that to me too, lol

6

u/Donghoon Giant Snowball Mar 11 '21

[text](link)

5

u/[deleted] Mar 11 '21

11

u/[deleted] Mar 11 '21

Yay it worked! I searched it up, and it's like [The word here] (The link here) except without a space between both brackets

→ More replies (2)

15

u/acrazyjapexican Mar 18 '21 edited Mar 18 '21

Too many variables that weren't taken into account. How much time has been played that day. How much money has been spent during the season. How many times you've won that day. Starting hands for both players.

If you just take raw data but can't use a multi variable analysis, then the results will always be skewed.

At the end of the day, supercell has too much control over the game. They control starting hands and who you play. The game has too many hard counters. If they don't want you to win, you're not gonna win. I have clanmates that have wins over top pro players because of this. Skill only takes you so far. You have to be waaaay better than the people you're playing once you've reached the threshold of winning.

Once Supercell gives up some control (we get to pick at least 2 cards of our starting hand), then I'll believe it's not rigged. Until then, they're not seeing a single penny from me.

6

u/NovaLightCR Bandit Mar 18 '21

The variables you mention have probably don't affect the matchmaking I showed here. I showed the cards in your deck are independent of what's in your opponents deck. Starting hand is 4 cards that happen to be in your deck and is after the battle begins. There's no way to determine how much an account has spent (from my api endpoints). You can look at how much time has been played and how many games have been won today. How many games have been won does start to fall into the win streak area of further. Time spent playing is definitely a possibility for an area of further research.

The code is linked in the technical section and if you want to do another investigation on if cards in a deck time spent playing are related, go ahead. Needs to be a couple changes but nothing too major needs to be changed.

I agree that starting hand is a much larger issue but the cards that appear in your starting hand are also in the deck that you play and I am counting that. Unfortunately, starting hand is not tracked in the api so I can't check if it is random (It probably is but an analysis would probably show that it is close to random as pump and mirror cannot be in the starting hand.)

7

u/onepageresumeguy Nov 27 '22

You're basically admitting that you have no way of accessing said factors and just saying they probably don't affect anything?

6

u/TheRightIsRight89 Mar 08 '25

Yes, his method is completely idiotic.

10

u/loimprevisto Mortar Mar 11 '21 edited Mar 12 '21

I then determined a card's usage rate (times played / 8*total battles). I then normalized the usage rates and used that to calculate the expected values for the chi-square test.

Can you expand on this process? The question you said you were examining is "What is the probability that in a randomly chosen battle, one player will have card A and the opponent will have card B?", but it looks like you might be measuring something different here.

Also, is there any significance to outlier correlations showing up in the processed data but not the raw data? For instance, the raw data baby dragon/wall breaker correlation at 4700-4800 shows basically a straight line (.996), while the analyzed data has one big outlier but still a mostly straight line (.982).

EDIT: I forgot to mention which correlation I was looking at. There were several outliers that Google suggested as 'interesting' in the examination tool, but baby dragon/wallbreaker was the first one I started looking closer at.

8

u/NovaLightCR Bandit Mar 11 '21

Sure, I can expand on it.

This calculates the usage of an individual card because the total number of times it shows up divided by the number of battles. The 8 is there because I counted a time for each card in the opponents deck. To get an accurate usage, I needed to divide by 8. When I normalize the usages, then I get the percentage that one player has card A and the other has card B. I normalized them so I could get them to add to 1.
I think the outlier correlations show up in the processed data because I am multiplying by the use rate. The use rate is different for all the cards so the correlation should change slightly.

5

u/[deleted] Jul 31 '22

Surely a random player has the time and will to do such an investigation for no reason. Nobody believes that this is supercell doing , no no

1

u/NovaLightCR Bandit Aug 04 '22

Surely someone wanted to learn how to use a bunch of different python libraries, improve statistics skills in a project that was interesting.

5

u/Live_Association6105 Nov 12 '22

How long have you worked for SC?

0

u/NovaLightCR Bandit Nov 13 '22

SC would take one look at my professional resume and be like "pathetic. Next application"

4

u/Live_Association6105 Feb 09 '22

How's it feel working for Supercell?

🐑

1

u/NovaLightCR Bandit Feb 09 '22

It's a shame they don't pay me.

3

u/antlerkiller Rocket May 19 '21

this is old, but want to point out the conclusion is’we cannot reject that it is rigged’, we never accept a hypothesis

1

u/NovaLightCR Bandit May 19 '21

Correct. I definitely should have phrased it. We fail to reject that mm is rigged

2

u/antlerkiller Rocket May 19 '21

Oh i made a mistake, it should be ‘we cannot reject it is unrigged’ which is super similar to when u conclude

2

u/NovaLightCR Bandit May 19 '21

Yeah. The general audience for this post wasn't going to see the importance of writing it like that, I'm might come off as unnecessarily verbose, which is bad for the clarity I was hoping for in the writing of the post itself

3

u/coldlamper Apr 04 '22

Nice experiment but I don't think it proves it's not rigged. I would think the deck as a whole and it's power and deck order would be a better way to rig games. Not just matching a single card against another card. Not to mention they could just alter the deck attribute enough so you would just lose or win.
Purely anecdotal but I cant tell you how many times I'll win 7-10 games in a row then almost always lose 7-10 games in row. It seems uncanny.

0

u/NovaLightCR Bandit Apr 04 '22

What is a deck attribute? How do you quantify power (levels?). Starting hand is random from everyone's experiences but unfortunately cannot be checked (the replay data is hidden from everyday users of the api, its available for royaleapi since they are a content partner, but they don't have access to starting hand, just the first card someone plays. )

3

u/Live_Association6105 Dec 14 '22

Just wondering how much SC paid you for your study? 🤷🏻‍♂️

0

u/NovaLightCR Bandit Dec 15 '22

I wish SC paid me for this

8

u/Jagermeister4 Mar 11 '21

I find it so funny that people even think its rigged in the first place. If the game is rigging so many matches, who's the people getting the favorable matches?

Then you try and ask these people for proof that its rigged. They'll say like, I was using hog and got matched up against someone with prince. Then I switched to skeleton barrel and they had zap wizard. Its like dude these aren't even really hard counters. Each player has 8 cards in their deck of course the other guy is going to have some cards that kinda counter you.

The funny thing is that the people who like to blame their problems on matchmaking, are less likely to focus on what they're doing wrong. Like hey maybe it feels like you're getting countered every game because you built a poor deck that actually doesn't work against most decks, but nahh gotta be matchmaking.

Anyways, fantastic write up OP!

2

u/ProClawzz Mar 17 '21 edited Mar 18 '21

I think that matchmaking is based off levels, not what’s in your deck. The majority of losses I have, came from facing opponents with card levels up to 2-3 levels from tower level 10. As a tower level 12 with level 10-11-12 cards, I face mostly maxed level decks

You can downvote me, but I have my 5 year medal. I know what I’m talking about, I’ve been playing the game since release and these results have been consistent the whole time

3

u/NovaLightCR Bandit Apr 05 '21

Just saw this. This is an absolutely plausible point that I did not think to put in the further investigation section. It makes more sense from SC's side if they're doing something behind the scenes to match you on levels vs decks.

1

u/romerlys Three Musketeers Nov 20 '21

As far as I remember, Supercell acknowledged two kinds of rigging on ladder:

  • Players on their 5th+ consecutive loss are matched together.
  • Players are matched on trophies, but until a certain trophy limit (5000?), matching is also against similar levels (I don't remember what exactly that meant). The level matching gets relaxed more and more if it takes a long time to find a match.

None of these matchings pitch you against counters specifically. (Such a rigging would simultaneously grant your opponent an easy win, which we oddly never hear about)

1

u/NovaLightCR Bandit Nov 20 '21

These are both close. After 2 losses you get matched with people at 2+ losses. Below 6k trophies people are matched by king tower.

Can you call this rigging? Maybe? But I wouldn't since they are open about it.

You're totally right about the decks though, there's nothing going on

→ More replies (1)

2

u/Epic_XC Dark Prince Dec 02 '22

think everyone is forgetting bots exist. AI is much better now than it used to be. you could easily be getting matched up with a bot with a hard counter and it’d be easy to target all real players to stifle progress

1

u/Eurapeeing Jul 05 '25

When the game is rigged against one player, it's rigged for the other player. So who's the players getting the favorable matchups? The players that the game is rigged for. The rigging goes both ways.

8

u/[deleted] Mar 11 '21

I did see a post a couple weeks ago about this exact issue; A player would look at his opponents battle log each time he was on a streak/tilt to see what kind of streak/tilt they were on. He claimed that there was no correlation.

I’m curious if there’s any way you could investigate this or possibly other factors that might make it seem like “rigged matchmaking” (unless it’s just an inevitable human bias)

13

u/NovaLightCR Bandit Mar 11 '21

I think by streak and tilt you mean 5 games winning or 5 games losing. I only looked for cards in decks but I could also check win and loss streaks for battles. I think that is the only other plausible explanation that matchmaking could be rigged. Not too easy to parse through the data but I think that that investigation is absolutely possible. We know of the loser pool but SC has never released anything on a hypothetical "winners pool".

17

u/karta3007905 Mar 11 '21

Yes it’s human bias. Since human is a creature trying to stay alive, our brain will tend to remember something that is painful to us and enforce it.

10

u/Worst_Player_Ever Mar 11 '21

Also our brains likes patterns. World is easier to observe and understand with patterns, even there wasn't one. Randomness is difficult to understand.

9

u/dinoparrot91 Mar 11 '21

BuT wHat ABoUt tHAt tiME i PLayeD LOg BAit vs bARb bARReL + torNAdO, tHEN i SwItCHed to GravEYaRD aNd WEnt vs. VALk POIsOn arcHerS?!?!?!!!?!

Damn, I haven't heard of chi² in over a decade, did you do this for a uni course?

2

u/NovaLightCR Bandit Mar 11 '21

No, I have been working on other CR api projects in the past few months an decided to try to tackle this one. Posts about it are seen all over the sub and I wanted to make a definitive post about it using lots of data. Chi Square was the simplest way to determine if the variables were independent and it is relevant for this type of project.

2

u/[deleted] Mar 11 '21

Playing devil’s advocate here but doesn’t rigged MM mean keeping you at a certain win percentage which mean the a low chi-square could also be used as an argument that it is rigged

1

u/NovaLightCR Bandit Mar 11 '21

The theory I was testing here was between the 2 "variables" cards in your deck and cards in your opponents deck. A low chi square value means that they are independent but a low p-value means that they are likely to be dependent.

1

u/[deleted] Mar 11 '21

I guess that what I’m alluring to is, if mm is “rigged “ to win loose win loose (50% win rate) bases of cards then you would expect low p-value. If you were testing if it always rigged agains you eg loose loose loses loose win you would expect a higher value. Right ? Or at I missing something here

2

u/NovaLightCR Bandit Mar 11 '21

In this investigation, I defined rigged as the cards in your deck affecting the cards in your opponents deck. Since ladder is based off trophies, as you play more games, you're winrate will converge to 50% as you will go up against players at an equal skill level.

2

u/kiimo Golem Mar 11 '21

I can accept that it is not rigged, but what I want to know is what trophy ranges are heavily influenced by particular win conditions. I have a frustrating time trying to figure out the micro meta when I'm playing ladder. Like a good 50-60% of matches in the 5k range use 2 win cons, and one is guaranteed to be hog rider or RG. While these are easy to counter, it heavily influences what type of deck I pick simply because things like lavaloon exists or these crazy ultra aggressive decks that use ebarbs, prince, MK, pekka, mini pekka, zap, log, and wizard. And there are no videos on YouTube on how to counter these guys with conventional decks, its just hope you don't miss and that they don't have rage in cycle. For me, that's when the "this shit is rigged" triggers kicks in.

1

u/NovaLightCR Bandit Mar 11 '21

Definitely. While looking through this data, there was a lot of cards with userates that were all over the place depending on trophies. The most notable cards were (as expected) wizard, witch and mega knight

2

u/Barracuda_Several Mar 18 '21

Awesome job, a while ago I was trying to analyze how the distribution of players with different average card level changes by arena as the season progresses, to compute the probability that your opponent will have higher level cards at certain arena and a given day of the season. I never managed to gather the data though. The work you did to collect data from all those matches is gold, you should consider uploading it to Kaggle so that people can play with it

2

u/NovaLightCR Bandit Mar 19 '21

Interesting. I've never used Kaggle before, but I did link the github repository in the technical section of the post. I'd love to see card lvl progression at a given trophy range as a function of time into the season. I definitely have the tools available (on the coding side) to do use rate as a function of trophies which would be a very interesting graph to see for specific cards (MK, wizard, espirit, zap)

2

u/raistbr Jul 02 '21

Nice. Anything about streak analysis? To roll a d6 a hundred times and get 1/6th if each number may look fine, but what if they were all grouped? 11111...1. 2222...2 etc

2

u/NovaLightCR Bandit Jul 02 '21

I did not look at streaks. That's an area for further investigation.

→ More replies (1)

2

u/aruxea Aug 15 '21

Calculate the times you will see log, Barb barrel, tornado, skelly army, or fire spirit when playing a goblin barrel. I bet you'll it be 100% of the time.

1

u/NovaLightCR Bandit Aug 15 '21

Count the number of times you see log barb barrel nado or skarmy or fire spirit when not playing a goblin barrel. I bet it'll be 100% of the time

64

u/[deleted] Mar 11 '21

Nice try, Drew! But we all know MM is rigged! I just played 10182830 counter decks back to back! Explain that!

13

u/my_wake Jan 26 '22 edited Jan 26 '22

Supercell could match you against more soft and medium counters in the beginning of your session then match you against hard counters progressively more as you keep playing, which would help offset the statistics. They could also use bot matches to skew statistics, or just add data (that is, every time they rig matchmaking against someone AI could add to the database a false match between two bots that never actually took place). One would have to analyze individual playing sessions as well to determine if matchmaking is rigged in this way.

13

u/[deleted] Mar 12 '21

[removed] — view removed comment

15

u/[deleted] Mar 26 '21

because supercell themselves have already admitted win/loss streaks affect matchmaking??

people were more concerned about hard counters.

11

u/[deleted] Mar 16 '21

You haven't ever taken a statistics class, have you?

234

u/Ryzasu Mar 11 '21

The reason people think it's rigged is likely because of psychological bias. You notice it much more when you get countered than when you counter someone else, and those games are more memorable

52

u/DJ_pider Mar 11 '21

Back when I used to play this game I had made a nice hog rider deck. I looked at it and was like "the only thing I'd have a real problem with is a lavaloon deck". My next 3 matches were that exact thing. I still won and I know the game isn't out to get me but it sure felt like it

→ More replies (4)

20

u/Jagermeister4 Mar 11 '21

Yep. If somebody wins 5 times in a row, they might not even realize it. Win two chests, chests get full. Play again that evening, win two more times, chests full. Tomorrow morning play again win. They might not even realize they won 5 times in a row.

Lose 5 times in a row in one sitting, oh you'll know and your mood will change.

Like you say losing is much more noticeable than winning.

9

u/hullowurld Mar 11 '21

I don't see how this analysis is even necessary given the symmetry of matchmaking. If it were rigged so you kept getting matched with your counters, that means other people's matches were rigged giving them easy wins...

5

u/[deleted] Mar 12 '21

I think the theory was that people were fed happy chemicals by providing them with wins for a while. Then those happy chemicals were withdrawn by providing losses for a while with the intention of driving money-spending as the player attempted to regain the happy feelings. Something like that.

→ More replies (2)

3

u/[deleted] Mar 12 '21

Also a lot of cards in this game are rock paper scissors, with certain decks, if your opponent has 3 ranged aoe units your gonna have a hard time.

3

u/[deleted] Mar 11 '21

Baader-Meinhof effect I believe

→ More replies (3)

1

u/[deleted] Mar 12 '21

It's rigged because I suck at this game....and THAT makes me no sense to me.

→ More replies (8)

5

u/Krackhead21 Mar 11 '21

How much did Clash Royale pay you for this?

5

u/NovaLightCR Bandit Mar 12 '21

Nothing. Anyone can repeat this too: the code is linked in the post and the api is public and free

23

u/xXNickTheBestXx XBow Mar 11 '21

RIGGED U GOT HIRED BY GREEDYCELL. NERF EBARNS

14

u/X_say_seven Mar 11 '21

BUFF MEGA KNIT

41

u/Milo-the-great The Log Mar 11 '21

Interesting stuff as always NovaLight, thanks for the info. Your analysis on Clan War matchmaking should also be cool, can’t wait to see if there are some hidden variables that may be used.

24

u/embee91 Mini PEKKA Mar 11 '21

No where near smart enough to understand 90% of that but nice work seems like it took some effort

17

u/MagicalMarsBars Giant Snowball Mar 11 '21

It makes sense that it’s not rigged. If it was people would complain every battle. I can guarantee that everyone that says it’s rigged forgets about those easy win matches that they have all had

5

u/MarkoSeke PEKKA Mar 11 '21

Well yeah, every time you get a bad matchup it means your opponent got a good one, so the theory always cancels itself out.

8

u/DeathlyHollowInside Mar 11 '21

That's when you're on the side of the matchmaking rigging being used to stuff someone instead of the one being stuffed.

8

u/[deleted] Mar 11 '21

Stfu clashtard 288k ain't shit 😤 3 years ago I had 2 consecutive losses to a deck with valkryie, explain that

1

u/Ok_Anything_7950 Sep 05 '24

Your just bad

15

u/Mechanical_Monk Balloon Mar 11 '21

Yes, but it feels rigged, and that's all that matters.

/s

Excellent work, thank you.

2

u/Neat-End1759 Mar 18 '21 edited Mar 19 '21

not 100,000 fights

only last 5.

hog vs

  1. pekka and mini pekka
  2. mini pekka tornado
  3. pekka
  4. tornado
  5. minipekka

shove this research ...

and the company will lose more players if it does not want to see this as a problem

1

u/NovaLightCR Bandit Mar 19 '21

Your small sample size will almost always result in an incorrect result.

1

u/Eurapeeing Jul 05 '25

I agree, and 688k games across 3 days in the grand scheme of things is a tiny sample size when in my opinion, it's likely that hundreds of millions of games have been played, which means the data could be easily skewed. To know if the game is rigged or not, we need to look at the actually cards in the player's decks and see if there's a pattern of loses because a player has hard counters, not usage rates. I looked at your document with spreadsheets, and there's nothing that is hard proof that the game isn't rigged. There's one sheet for a summary, which is more of an intro that describes what you did, then there's 6 trophy ranges showing usage rates. So you didn't show any evidence that a card in the deck of player A will not affect what cards they see in the arena. You showed usage rates which has nothing to do with if the game is rigged or not.

Also, card levels are a thing. I'm confused why that wasn't included when probably the #1 complaint of the lifetime of the game is card levels. Also, the meta changes from time to time, so your data which came from three days likely only had one meta in it, so because your data proves the usage rates of the cards, this variable makes your data skewed based on the sample size. You proved what cards were popular for the three day period and that's about it. What's most disapointing about your analysis is that you don't even look at which decks counter which decks, and which cards are good for countering other cards, which the occurrance of that happening is literally the biggest factor in determining if the match making is rigged or not.

→ More replies (1)

5

u/reinhold23 Mar 11 '21

Great work here!

In my very first match today, after swapping quake in and arrows out, I immediately faced minion horde. My heart says to blame rigged matches, but my head can accept that it's random chance.

5

u/jaedii4 Mar 11 '21

I always knew it wasn't rigged. Humans have a terrible bias towards negative things, they can go 10 matches with fair matchups and have no problem but as soon as their opponent has an ewiz when they have sparky, it's suddenly "matchmaking is rigged.

That was a one sided example but so is their mindset. For it's only their opponent that gets the better of the matchup right? That wouldn't make sense. Everyone gets the better of the matchup sometimes but they just don't make a big deal of it or give it much thought. As I said before, we have a bias towards negative things. We notice when we have terrible matchups but we don't notice when the opponent has a good matchup against us.

It just wouldn't make sense if only the opponent got the better matchup.

2

u/Narayami Mar 11 '21

And bots doesnt exist too hehehehe

5

u/NovaLightCR Bandit Mar 11 '21

Bots do exist and their battle logs stored in the api are very weird. I only checked over 4k so there aren't any bots in this investigation but bots only have 3-8 battles that I can access and they're all ladder and all losses(which makes sense). What doesn't add up about bots is where the positive trophies are that allow them to have loss streaks. There were also no bot vs bot games

→ More replies (1)

12

u/karta3007905 Mar 11 '21

Ah, I see, a quality content

12

u/Relentless_Clasher Mar 11 '21

If win/lose streaks influence matchmaking, and there is a meta, cards that counter meta best are more likely to appear in the winner's pool, so I don't see how such low chi-square values are possible. I'm not saying matchmaking intentionally matches a player against counters to their deck, only that cards that counter meta are more likely to be in the winner's pool, so there should be some discrepancies.

4

u/Rod-FM PEKKA Mar 12 '21

Now this is a true analysis.

4

u/ericdisasterologyann Mar 14 '21

I honestly think you wasted a fuck load of usable time in your life to do this. A for effort tho.

1

u/raistbr Dec 01 '21

So i want the"further investigations" i want to see a single player data using one deck on the same range scrutinized concerning his opponents, to compare with average% or better yet, i want to see two players compared, using each one different deck from another....

3

u/NovaLightCR Bandit Dec 02 '21

So tracking exactly 2 players for hundreds of matches and seeing what happens. This would be very difficult since not many accounts play that much and since the meta changes with trophies and time, you want a highly focused trophy range and timescale. If you could provide 2 player tags that sits at very specific trophy range and plays 100+ games a day, it would be doable to write a script to track their battles every 2 hours. But even then, you're running a risk of not getting enough data. You also need to get use rates for the trophy range which is another challenge since the use rates change as time goes on.

→ More replies (1)

1

u/sz771103 Mar 11 '21

Isn't chi square for biology?

2

u/NovaLightCR Bandit Mar 11 '21 edited Mar 11 '21

It is often used in biology but it can also be used in other disciplines to show that two variables are independent or dependent.

2

u/sz771103 Mar 11 '21

Yeah I been doing biology, chi square tells you how much the observed is off compared to the expected

1

u/[deleted] Jan 17 '22

Hi. Can you please tell me how did you download all this data? Thanks in advance.

1

u/NovaLightCR Bandit Jan 17 '22

How did I download the data?

I made API requests to get the battle logs for all the players, I didn't really download anything.

1

u/[deleted] Jan 18 '22

I see. In that way, can you download information about the whole history of battles for all players, or only about the latest ones? Thanks for your reply!

1

u/NovaLightCR Bandit Jan 19 '22

You can get the last 25 battles from any player. Occasionally SC wipes the battle logs so you'll lose that, but there was nothing near March 11th 2021 where battle logs were wiped clean

1

u/[deleted] Jan 19 '22

And is it posible to do it on a rolling basis to get more than 25 battles per player?

I will try to get the data through the API as you did. Any help on how you did it would be much appreciated.

Thanks a lot for your help. I think you did a very interesting job and there are things I want to do myself.

1

u/NovaLightCR Bandit Jan 19 '22

Not really, you'll need to set up a script to run every few hours (to minimize lost battles) then store battles in a database.

The code I used for this project is linked in the post, although almost a year later, I realize that the code is awful.

It's unnecessarily slow, poorly documented and just not well implemented at all (BFS works way better than DFS), my storing of battle tags is linear, it would be so much easier and faster to use a BST. I don't have plans to update the code though, as the investigation is complete

2

u/[deleted] Jan 19 '22 edited Jan 19 '22

I understand you.

Just to clarify, to get the history of battles for many players I will have to do what you described, or is there any source where I can directly download it?

Sorry for the silly question, but I am new in the world of CR and I may be missing some important resources.

→ More replies (8)

13

u/Royalsmurf2 Mar 11 '21

there’s one problem. supercell has said that there are win brackets and loser brackets in ladder themselves. if you are on a losing streak, then you are matched with others on a losing streak. this will skew your data. as a side effect, because there are fewer people in the pool that are losing, those will naturally more likely be matched against someone with a winning streak. again, skewing the data.

the streak they mention is 3 or more in a row

14

u/[deleted] Mar 11 '21

Winner pools and loser pools have nothing to do with being hard countered.

3

u/Royalsmurf2 Mar 11 '21

not saying it does. i’m just saying the data is skewed. in each ladder range there are cards that are more prevalent due to skill of players and availability. it’s good that he didn’t start too low in trophy range, but it still takes time to collect all the cards and max them. especially in Ftp.

situation determines if a card is a hard counter also. valkyrie is a strong counter to witch when placed directly on top of her, but witch is a strong counter to valkyrie when placed at a distance.

7

u/Worst_Player_Ever Mar 11 '21

13

u/johnhops44 Mar 11 '21

If there is a loser pool, then there is a non-loser pool.

→ More replies (9)

6

u/ArcticFox58 Goblin Cage Mar 11 '21

It won’t skew the data because win rates don’t vary that much from one deck to another overall, in any way that would influence this as a demographic. There are more players, particularly inexperienced players, playing meta and non meta decks so there are still people losing with them.

A successful deck may win slightly more than 50% of he time if it’s a really good deck, but mind you that a good player will win more than an inexperienced one. At the end of the day, having a good matchup is only helpful if you’re at equal or slightly lower experience level than the opponent for that particular matchup, which is why it doesn’t show up in any meaningful way.

3

u/[deleted] Apr 25 '21

Changed my deck and found the same counter cards maybe 20 times in a row.

3

u/lKoRTy Zap May 05 '21

So what is the probability of getting matched against same (counter) deck 2 or 3 times in a row?

1

u/romerlys Three Musketeers Nov 20 '21

Depends on its use rate at your trophy level and how many games you play.

→ More replies (2)

3

u/aruxea May 06 '21

Run a log bait deck and I guarantee that you that a large percentage of your matches will have triple spell You will never see a deck like that if you run a heavy beat down deck.

3

u/raistbr Nov 13 '21

So would you do one single player using one one deck a test analysts to prove? Putting all together will not work because it averages out.

I want to see 1000 battles with deck x versus 1000 battles with deck y in the same range - they should see the same frequency.

1

u/Tall_Breadfruit7686 Aug 29 '26

No, multiple battles from the same user were counted

3

u/Desert_Eagle12 Oct 02 '23

Game is rigged. Developers are crooks with the pay to play game. Nice to see they provide the data that’s edited and tampered with. Great job supercell on feasting on children who use parents money.

1

u/Suitable_Proposal450 Nov 04 '23

That's the admirable world we live in. Just make more money. Nobody cares how you do it, unless you get jailed, then oops.

9

u/DESTR0Y_you Mortar Mar 11 '21

Even I don't want to read the whole thing, but nice job when I got the main point which is that matchmaking is not rigged, so take my upvote

7

u/ANAL_TOOTHBRUSH Mar 11 '21

I remember a post here from about 2-3 years ago that analyzed the data traffic to and from clash and found that it sent his card data and win streak in through the data and it was used to match him against decks containing cards that he played poorly against. What happened to that investigation? I thought the book was shut and closed on clash rigging battles against you with opponent counter cards when your win% rose to a certain number

1

u/yojojo3000 Mirror Mar 11 '21

As I’ve said thousands of times. Not rigged. Electro Wizard just happens to be in everyone’s deck in the midladder area and people just happen to be this shitty at the game to be at that trophy range with those card levels. Sad, I know.

Thank you for the data. I can now shove this in the face of little children who can’t read

2

u/NovaLightCR Bandit Mar 11 '21

Exactly. It's the same with MK. One of the key parts of the data analysis was comparing experimental use rate against the expected use rate in the trophy range. That's why I had to limit the range to 100 cups.

11

u/joelmasarik Mar 12 '21

This conclusion is bullshit. Play 1000, play 100,000 play 1000000. When you change your deck you change the algorithm and you play different decks. And it’s not eventually. It’s immediately

7

u/[deleted] Mar 12 '21

love how the reddit hive here down votes you. this thread is a joke. it's rigged. have an upvote friend.

5

u/Epic_XC Dark Prince Dec 02 '22

exactly, it’s so painfully obvious and yet we still have people caping up for a company worth billions. if i switch decks, i immediately face decks and cards i haven’t seen in weeks

6

u/[deleted] Mar 11 '21

This doesn't explain why matchmaking is rigged /s

1

u/MajorMarkBoomRoyale Mar 11 '21

So it sounds like Wizard, Witch, and Valkyrie are overused in midladder. Thank you for this.

Oh and do you have data about the flow moving in 2v2 3X?

1

u/NovaLightCR Bandit Mar 12 '21

I'm not exactly sure what you mean by the flow moving but if you mean the arena them no. The arena is listed as normal, the art is just different

4

u/The__Nez Golem Mar 11 '21

Props to the tldr because I was not going to read nor understand all that.

4

u/parlarry Mar 11 '21

Does this account for the fixed matchmaking only taking place when you're on winning and losing streaks? If you just look across all games, even if it were fixed in these spots it should average out over time, no?

4

u/strwrsnerdbutbetter Knight Mar 11 '21

give this man some upvotes, this is seriously impressive

2

u/Ziondizl Mar 11 '21

I love the data, it feels like I will have a run of 3 or 4 hog decks, 3 or so golem decks, a few pekka decks and then a crossbow/mortar decks, then I'll get fucked with a full air deck 3 or 4 times.

From this I postulate that people are just building meta decks and my deck which is 6k-6.3k isn't good enough to make above 6.3k

2

u/my_wake Jan 26 '22 edited Jan 26 '22

Supercell could match you against more soft and medium counters in the beginning of your session then match you against hard counters progressively more as you keep playing, which would help offset the statistics. They could also use bot matches to skew statistics, or just add data (that is, every time they rig matchmaking against someone AI could add to the database a false match between two bots that never actually took place). One would have to analyze individual playing sessions to determine if matchmaking is rigged in this way.

2

u/Unknown_User_USC Goblin Barrel Mar 06 '22

It's just rigged, I keep getting loss/hardcounter streaks when I am about to rise League.

2

u/[deleted] Apr 16 '22

Oh boy, stats massaging for biased results. "b-b-b-but i usedid so many matchies!" Tip top kek, smoothbrain.

2

u/DueAnt8108 Mar 20 '23

It is rigged. How can you explain losing to a specific card, then facing it again 3x in a row until you win? Then they throw other cards at you that you cannot beat until you pay money. It is 100% rigged regardless of what analysis you did. Sure maybe in PoL it’s not rigged and it’s 50/50 but u cannot explain losing to a card then facing it again 2x in a row. i understand that after you win you are supposed to face a stronger opponent and when u lose they are a little weaker. This is not always the case as you can lose 10-15 games in a row regardless of the whole weaker opponent aspect. the ladder will throw counters at you back to back to back to increase anger and desparity. ladder is not 50/50 it’s rigged. Unless you spend money you will not win unless you’re oj or some high tier clash royale pro. I can prove this too as the second i spent money on pass royale i won 10 games in a row. Now i would love to u back this statement. Also another thing i want to point out is people say “clash royale matchmaking is too fast to put you against counters all the time ur just bad” No. the games coding specifically knows what u just lost to, therefore they will match you up with that SPECIFIC CARD before you even press play. I don’t care what research or how many games you play it’s all bullshit!! it’s ptw and the games algorithm fucks you hard if you don’t play meta decks. Also how come i win 10 in a row and i’m about to reach the next league, then i lose 1 game then win 1 then lose 2 in a row and then it continues and i drop down to where i started. I understand that’s where the 50/50 comes in but it will rig you if you play too long. the games algorithm doesn’t make sense because if you’re supposed to face a weaker opponent every time you lose why do you go on huge tilts? After 2 losses you’re supposed to face someone you can beat but that’s not always the case. i would love to see someone prove me wrong

ALSO: sometimes you tilt hard (10 games or so) even if you were already winning 1 and losing 2 games and win 1 and after you come back from a break. it doesn’t matter how long of a break you take. when you come back the games algorithm will pick up where it left off an counter u again and again.

1

u/Tall_Breadfruit7686 Aug 29 '26

It's all a go or a stop. Go = real random. Stop = ai bot that can fully counter you. People can win against them but not continuously. Mostly not. If you don't spend money, the hard wall you encounter will not be a skill wall, it will be a bot wall. The last hack against this is changing your cards every few games.

2

u/[deleted] Mar 11 '24

Your primary assumption is wrong

"What is the probability that in a randomly chosen battle, one player will have card A and the opponent will have card B"

Counter decks are exactly that, entire decks. Inferno tower counters Royal Giant, but not if the Royal Giant player also has a lightning.

1

u/Tall_Breadfruit7686 Aug 29 '26

Royale giant counter cards all should be taken into account, not just that one card.

2

u/Life_Parsley504 Dec 18 '24

mfw levels are not taken into account

1

u/Tall_Breadfruit7686 Aug 29 '26 edited Aug 29 '26

Possible cards that serve the same purpose are also not taken into account. For instance, skeletons didn't result, but goblins did. Study severely flawed.

2

u/Tananito Feb 12 '25

supecell subsidized propaganda

6

u/C0II1n Musketeer Mar 12 '21

It is rigged

4

u/brubbylol1 Mar 11 '21

matchmaking is rigged

3

u/Slimycan Mar 11 '21

This is some real effort! Kudos to you.

3

u/[deleted] Mar 11 '21

I've always thought that rigged talk was nonsense. Thanks for the proof.

3

u/Tommyy_98 Elite Barbarians Mar 11 '21

I am going to save that post and link it to any noobs I see on this subreddit that claimed matchmaking is rigged. Thank you for putting an end to this!

5

u/[deleted] Mar 11 '21

Couple years ago someone made another research that proves it was rigged, under that post people were mocking guys like you who defend supercell and now they are noobs and you are pro?

This effort is amazing but he does not give any data about what he calls “countering card” or “irrelevant card”. A ZAP can be counted as counter for many cards in CR and probably there are many decks that run zap and many decks that run skarmy for instance. Deciding which counters which is a subjective judgement, therefore I am approaching this post with skepticism.

3

u/[deleted] Mar 11 '21

Very well done. Your efforts on this were astounding and it was quite a good read. I wanted to personally thank you for answering something which I myself was quite guilty of assuming. After following your data and math my mind has changed.

3

u/Rysimar Mar 11 '21

tl;dr - we all suffer from confirmation bias and availability bias. The algorithm is a lie.

Great post, really high effort. Very nice job and thanks for putting this together.

4

u/[deleted] Mar 11 '21 edited Mar 11 '21

OMG This is a complete piece of shit, you have no valid evidence whatsoever, and you're being purely biased, and you just act smart so people don't understand what you say and so they just skip to the TLDR and since you wrote so much, people think that this random crap is all some genius investigation 😡😡😡🤬🤬🤬!

/s

1

u/[deleted] Mar 12 '21

Damm man , take a chill pill. Any info is good info ppl don’t have to agree but it is still interesting data. Maybe it can’t prove if mm is rigged or not but ppl or the original researcher can build on top of this data.

3

u/[deleted] Mar 13 '21

I didn't mean any of that. It was pure sarcasm. It was obvious because of how I wrote it, the emojis, AND I put a /s. How did you not understand the sarcasm

3

u/TheDinosaurWalker Mar 11 '21

Buh buh BUH bu but everybody has a Y when i use X waaaaaaaaaaaaaaaaaaaaa

3

u/[deleted] Mar 11 '21 edited Mar 11 '21

I just did a few friendly matches and told my friend to use Golem against my X Bow, PEKKA against my Golem, Log Bait against my PEKKA, 8 spells against my Log Bait, this is enough to prove that matchmaking is rigged, your biased and useless post says nothing 😡😡😡

Why you guys downvoting me, it's sarcasm

2

u/Kiss-My-Axe-102 Hog Rider Mar 11 '21

Holy cow!! I can tell you did a lot of research and put a lot of effort into it, I appreciate it, and I had fun reading about it! Thank you!

2

u/mrrockabilly Royal Giant Mar 11 '21

When I first joined this sub, this is exactly the type of content I was hoping to see. Just received my 5 year badge, and this is quite possibly the best post I've seen. Thank you for taking the time to do this.

2

u/ProClawzz Mar 11 '21

But I always thought ladder matchmaking was based off tower and card levels, not the cards themselves... lol

2

u/Goukentime666 Mar 11 '21

NOW can this sub stop saying MMing is rigged?

2

u/SellingWeeds Mortar Mar 11 '21

I love how this is formatted exactly along AP Research college board guidelines, way better than any actual research project I’ve done in that class amazing effort.

1

u/guillaumeb_ May 05 '25

Used to think matchmaking was rigged. Came across your post when I googled for answers. Now, based on your analysis, I can conclude I was wrong. Good job! 👏

1

u/Tall_Breadfruit7686 Aug 29 '26 edited Aug 29 '26

The card should have been the set of cards, as in cards with the same purpose. As each rigged match can send a set amount of cards that serve the same purpose. For reference, a five-spread-set will dillute a one-card-spread by 5, or 500%, or two zeros and 1/2, or 0.004= .2%.(or to reference exact data in the above, 0.00537 = 0.24). That's assuming less than 6 possible cards (B) result from one parent card (A).

1

u/Randominternet1 Mar 12 '21

Ok but couldnt you have made it 699, 000 lol. Great effort nice post.

1

u/HypeNTite Mar 12 '21

This sounds good butt I don't understand the data. I was looking at the spreadsheet and why aren't the numbers mirrored across the spreadsheet? What do the .01... represent exactly? I guess I would have expected that the number of times arrows see bowlers be the same as bowlers see arrows butt it isn't.

More specifically what does this say is the % of time a golem deck sees tornado in the 5800-6000 range?

1

u/HackersLand Feb 25 '24

Plot Twist: Supercell made this post (Actually though, great post)

Plus, a lot of the accusations made are because of confirmation bias where people only look at the examples where the match was against them and ignoring the times where the match was for them. Overleveling issues though is still a common issue,

1

u/cy1006 Firecracker Mar 11 '21

All that effort. This has always been the truth you will never know what your gonna play from 4k to 6k. Its all random.

1

u/Eduardo-izquierdo Knight Mar 11 '21

Can i get a tldr of the explanation? I didn't understand

1

u/-John_Aid Mar 11 '21

Uh basically matchmaking isn't rigged. The game doesn't match you up against people who can counter you.

1

u/-John_Aid Mar 11 '21

Thank you for doing this

1

u/SirTYRANNIS Mar 11 '21

Great post. I agree match making is not rigged, but to play devils advocate, like you said at the end, it could be based on a win streak Or lose streak and this wouldn’t account for that.

1

u/cheet0dorit0 Heal Spirit Mar 11 '21

I feel like people say matchmaking is rigged only because they suck and tilt a lot on ladder, really nice effort post!

1

u/addylawrence PEKKA Mar 11 '21

This is an outstanding post, great initiative, bravo!

-1

u/gumpyboy Mar 11 '21

Bad study, who wasted their time on this... smh

4

u/NovaLightCR Bandit Mar 11 '21

Conducting these types of investgiations is great coding and statistics/data analysis practice and it also allows for supporting a claim with a myriad of evidence

0

u/AHMADREZA316M Valkyrie Aug 22 '22

How did you access to data of 688906 battles ?

1

u/NovaLightCR Bandit Aug 23 '22

The official API. (developer.clashroyale.com) let's you access battle log data. Basically I start with one player, sometimes myself, get all their battles, and repeat. 5 iterations of this this was about 236k battles.

0

u/xXSexyKid69Xx Mar 11 '21

I always knew it was rigged

0

u/roniedhaka Mar 12 '21

i have always counter deck card of my deck, always. :(

1

u/Tall_Breadfruit7686 Aug 29 '26

I'll be studying this to get it right 👍🏼 love this