What you should know about KQB’s MMR Reset
“After launch, we quickly realized that our initial approach to distributing MMR was far too simple to produce good results given the complexity of the game.” – cwal
Killer Queen Black’s original algorithm was essentially a modified Elo system that compared each player’s rating to the average rating of the other team when distributing points. Had the initial player population been large enough to ensure that player ratings in every game were relatively close together, this system would have been passable. However, in our situation, where teams were often made up of players across a wide range of ratings, it ended up causing large swings and ‘felt’ unfair.
Consider a team with each player rated close to 2200 against a team of 3x players with a rating of 2000 and one player with a rating of 2800. If the latter team lost, the high ranking player would lose a huge amount of points against a rating of 2200 without considering the lower rating of his teammates. Conversely, if that team won, likely due to some amazing effort from the higher ranked player, the 3x players rated at 2000 would get quite a boost to their ratings.
As mentioned earlier, the results often felt unfair even though over time things would have evened out – so as a band-aid, we capped losses to -10 points (players would often lose 20-30 points in a match they clearly were not likely to win). This solved the issue in the short run, but in the long run caused the large drift up in the rating distribution we see today, where most players are rated way above the minimum points to reach obsidian. We even see that most new players are now placing right into into gold, platinum, or even obsidian.
A little help from our friends
When the Liquid Bit team started looking at overhauling the rating algorithm, we realized there were a number of directions we could explore, but the best option would be to get help from experts. We reached out to a friend at P3 Analytics who specializes in game data analysis, and they were eager to help out on such a unique project.
P3 Analytics ran a statistical analysis of thousands of competitive games and were able to determine what variables actually mattered in predicting which team would win a match.
You may be thinking, “obviously it’s the number of ha-ha emotes per player per minute,” but that one was actually too low on the list to be statistically significant. But there were about 6 or 7 factors that were, and so P3 was able to create an algorithm to both accurately predict the outcome of a match based on the ranks of each player, as well as fairly distribute points after the match. We quickly implemented it in the game behind-the-scenes to test it out.
At this time, Liquid Bit will not be releasing the algorithm – know that it is something we will be constantly monitoring and tweaking.
It’s been running for a few months now and we think the results are promising. Just before we launch on Xbox Game Pass, all players will have their ratings reset and go through placements again. Historical data (matches played, wins/losses etc.) will temporarily be hidden while in placements, but will come back once you’ve completed the ten placement matches.
In the new system, there’s no cap on point losses, but distribution should always ‘feel’ fair thanks to the new algorithm. But, the new tier cutoffs will be distributed as such that only the best of the best should make it to obsidian. We’ve also added demotions, so it will be possible to go down a tier if you don’t keep up your record.
We appreciate the community’s feedback as always and will be tuning the algorithm and player experience as needed in future resets. We’re incredibly excited for both our upcoming launch on Xbox Game Pass and the MMR reset and can’t wait to see what happens with the competitive scene. Good luck out there!