The Flawed system of Championship Points: an open letter

[Axew]

For those of you that don't know me, my name is Liam Williams. I am a player from the state of Washington with decent tournament performances, across several years. I have devoted much of my time to growing the game and making it more enjoyable for everyone involved, but I am leaving the game for Magic: The Gathering for several personal reasons, the largest of which being the system Pokemon has adopted (indefinitely) beginning in the 2011-2012 Organized Play Cycle.

1: Championship Points skew the importance of smaller events

Winning a city championship event is currently worth as much as finishing in the Top 16 of a National Championships. If somebody say, won 5 City Championships and 8 Battle roads (The maximum number that points can be earned from), you would have 30 points from city championship level events and 16 from battle roads. How close would this get you to a worlds invite?

40 James Roll Master Florida, US 69 49 1859.11

Well hot damn! You'd be a mere 3 Championship Points away from qualifying for the World Championships! 3 Championship points should be fairly easy, considering a 4-2 record at a 6 round state championships puts you in the top 16 (or even higher!), netting one a full 3 or 4 championship points, depending on where you finish in the top 16. 8 Battle Roads, 5 Cities, and 11 SPT/Regionals finish later, and you lock up a worlds invite. This is ABSURD. One could argue that winning 8 Battle Roads and 5 City Championships would be no easy feat, but with the possibility of 3 Cities Marathons, one could attend upwards of 25 City Championships! (I know several people who actually did attend multiple marathons, including one who went to a full 3)
Point values should be adjusted everywhere to make this less of a problem

2: The system encourages travelling, and lots of it

For the 2012-2013 Organized Play cycle, expect to roll up that odometer! With a full suite of 3 State Championships and 3 Regionals to attend, players will have their hands tied for several weekends chasing that invite. With no incentive not to play, record breaking attendance has happened worldwide, and this really screws players in areas without the current tournament every weekend. Over the past year, I have watched dedicated tournament grinders from British Columbia, Canada come to Washington to play in our cities, crossing the border almost every weekend to try and get precious Championship Points unavailable to them. For players in areas without 10+ cities, enjoy the road trips.

3: The system rewards attendance over consistent play.

I'm going to quote this article, co-written by several professional MtG players for this one:

"Week after week, she’ll go through this grind. If she plays well, she’ll win a higher percentage of her matches, scoring more points. Making day 2 of more GPs keeps her in high multipliers, which is good, but decreases participation points and matches played, which is bad; in many ways the side events are actually the better bet as she gets more participation points, more rounds and easier opponents. If there is punishment here it is light.

At the end of the season, Alice and all her competitors for those 100 slots will compare their performances. Those who went to the maximum number of Grands Prix will have a prohibitive advantage, as will those with three byes. Those who got to start the season with a Pro Tour will also have huge head starts, as no one is kicked out of day two so most will get four or five hundred points; this alone is a huge source of threshold inflation."

http://www.channelfireball.com/articles/an-open-letter-regarding-planeswalker-points/


4: No matter how much you think you can qualify without CPs, you can't

very few players have the ability to top 8 nationals consistently: Kyle Sucevich, Tom Dolezal. That's it.

Even fewer can top 4 worlds consistently: Jeremy Maron did it twice 2 years consecutively, and Jason Klaczynski did it 2 times in 3 years. David Cohen has done it twice in 3, but one of them was in the senior division. Rounding out our multiple top 4 finishers are Tsuguyoshi Yamato(2004 and 2007), Yuta Komatsuda(2006 and 2010), and Ross Cawthon(2005 and 2011)

Since Nintendo's first Pokemon world championships in 2004, no players have made the top 4 of the world championships more than twice. Nobody can "rely" on top 4ing worlds or top 8ing nationals for their invite.

5: World Championships awarding CP

this is a very bad decision on OP's fault, simply because it makes chaining invites easier and gives players at worlds an unfair advantage going into the next year. In 2011, when I qualified for worlds out of relative obscurity, I had done literally nothing of note the year before. I qualified very close to the bubble, and probably would have missed had 4 players from the world championships the year before who barely miss had a small boost in rating, giving them the edge again this year, and the next provided they continue to qualify.

6: Best Finish Limits

With a BFL of 4 at SPT/Regional events, missing one event means you should earn points at every other event of that level you attend, or be far behind on the points race. This is a pretty simple fix, just lowering the Best Finish limit on everything to create less pressure to do well at more events.


7. Records going unrewarded and top cut caps
Battle Roads, an event of significance due to the amount of possible points from them, have a top cut capped at 4. This ordinarily wouldn't be a problem until you go to a state like Oregon and miss top 4 in a 6 round battle roads at a 5-1 record and walk away with nothing; no boosters, no cp. This problem is especially apparent at smaller level events with large attendance. This problem could be solved fairly easily by simply making the kicker requirements smaller(divide everything by 2 should be about right) and extend deeper down into the standings.

Championship Points are bad for the game as a whole, and I will not be returning to competitive pokemon until they are fixed or switched to a new invitational method.


Sincerely,
Liam Williams

 

[Ruiner]

As someone who has played the game through every qualifying structure we have had, and qualified for the first 9 World Championships in a row before retiring from active play this year, let me say that I disagree with you. While I too have stopped playing this game competitively to go play Magic, my motivation to do so has nothing to do with my displeasure with Pokemon, merely that at the moment, I feel Magic has provided me with an overall more enjoyable experience, and it is potentially more profitable. Pokemon has done nothing to alienate me, I just have to choose to focus on one game or the other.

Now, let me say that I don't feel your points are invalid. All of your complaints hold merit. Most of them, though, are null and void if the values of CPs awarded or BFLs are adjusted. I believe Pokemon has already stated that the numbers are up for reconsideration, but they had to start somewhere. After Tom Dolezal started Nats 08 with a 1600 rating and made top 8, he qualified for Worlds not only due to Nats placement, but also by breaking into the top 40 from ONE EVENT! This caused the K value of Nationals to be dropped the following year. There is precedent for numbers to be adjusted. I don't think the numbers are nearly as off as you apparently do, but I'd tweak a few here and there.

Now, still acknowledging that there are flaws ( the stress on attendance for example ) we have to weigh them against the other flaws of prior systems. Originally invites were given out for Worlds t4, Nats top placement ( I think t8 ) t4 at Regionals, and winning a Gym Challenge. The issue then was that all of your eggs were put in one basket ( acknowledging that a top placement at Nats/Worlds/Regionals isn't reliable yourself, we can focus mainly on the GC aspect ) and this lead to extreme variance. As someone who played through those events, it was not a good experience, and this rewarded players in areas with less players, and particularly areas with bad players. At the time, in Ohio, we had myself, Seena Ghaziaskar, Tom Dolezal and Drew Holton all actively attending events, so it made Gym Challenges very difficult compared to states with smaller player bases. That being said, you can make arguments that this system is very similar to the Pro Tour Qualifier events held for Magic, but there are significant differences. One, the # of PTQs vs GCs is much, much higher. In addition, there were other ways to gain the necessary pro points to make it onto Magic's Pro Tour, and if you did make it onto a pro tour, it was possible to chain appearances once you got "on the train" as it is referred to. A Gym Challenge/PTQ structure works better when you have a very active tournament season, and compared to Magic, Pokemon has very very few events. If Pokemon held 3-4 Worlds a year, with Gym Challenges for each, then it would be more comparable, but since its 1 "big event" each year, a very small number of "all in" qualifying events is a terrible structure.

After this we got the rating system. This started the "marathons are beneficial" issue, as more attendance equals more rounds, which is more potential points. To make it worse, the problem compounds as more rounds in more tournaments leads to an oversaturation of points in that region, etc etc etc. This still gave the advantage to players who traveled and had access to lots of events. I fail to see how a CP structure is any different in this regard. Unless you run unreasonably well at a select few local events, and I mean like 8-0ing 5 cities, your still losing the rating race to the people who travel, trust me, I did that rat race, I know better than anyone the ins and outs of playing the rating system game. That brings up the other issue. It encourages players not to play. Pokemon is a high variance game. Due to the low prizes for events besides Nats and Worlds, the upside of playing events vs risking rating to get into Worlds is not worth it. You can give me whatever "spirit/love of the game" arguments you'd like, but players shouldn't be put in a position to make choices like that. Magic shifted away from a rating system to a similar CP system with "Planeswalker Points" as well primarily because they didnt want to force players to sit out smaller events because the risk wasn't worth it. Not only is it a bad choice to make the players make, but it hurts the game overall if players are given incentive not to play. It seems like Pokemon came to the same conclusion, and I am glad for it.

Now, a Ratings system isn't all bad. It did allow you to average out your performances, so losing in the finals of you "gym challenge" wasn't a season ender. It took out a lot of the eggs-in-one-basket issues, but it brought with it a slew of other issues. I was supportive of the rating invite structure when it was released, and I stand by it being an improvement over the GC structure, just how I feel the CP structure is better than Ratings. Is it perfect? No. I could likely come up with some better alternatives myself, and I am sure you could too, but to say it isn't better than any prior structures is pretty inaccurate. At worst, its a lateral change from last years structure, and shouldn't be a catalyst for leaving the game.

The biggest issue we have is that we get 40 invites for all of N.A. Whatever system we use will not negate the fact that this is far too small of cut off. As the player base gets larger and larger and larger, we get more and more good players, and when you only have one big event a year ( unlike Magic's 3 Pro Tours and a Worlds they ran ) only letting 40 players in is going to disenfranchise more and more players. Pokemon isn't even paying to fly a vast majority of Worlds players to the event, so why cap the event at an international 128 per age group? It just doesn't line up with the growth of the game. Magic has a Grand Prix pretty much every weekend, and 3 Pro Tours a year, with attendance far higher than what Pokemon is letting into Worlds. Now, lets do an comparison. Pro Tours get roughly 500 players. Worlds gets 128. Now, Magic clearly has a bigger Organized Play network than Pokemon, but its comparable to look at U.S. Nats, who had 1000+ players for Masters alone, with a Grand Prix, which get 900-2000 players roughly ( usually between 1100 and 1500 ). So while Magic does have more players, there are still a huge amount of Pokemon players trying to enjoy the competitive aspect of this game. Magic lets roughly 1500 players participate in the Pro Tour each year. We get 128. There is no financial downside to making Worlds bigger. By letting more players experience the event, you are only more likely to attract casual and semi competitive players to try and invest more time and money into the game, which from a business perspective is great.

I'd much rather see it where you have a goal to reach. Say, once you hit 40 CPs? You are in Worlds. If 60 people get more than 40? Irrelevent. You got your 40. You made Worlds. This makes it less of a chase, and it doesn't matter if a set number of those with free time and money hit up a ridiculous number of events, you know what you need to attain yourself. This makes the attendance at Worlds more of an unknown prior to the end of Nats, but the # should be pretty similar year to year, and any fluctuations should be minimal.

 

[Moss Factor]

It's better than having literally all of the game's top players simply never finish tournaments all year and then sit out Nats. How could you even call it a "national championships" when most of your nation's highest ranked players sat out?

You still had to travel a ton under the old system and people still got most of their points grinding out smaller tournaments. and awarding points for worlds is rewarding consistent play -- across seasons! This is good for the game.

The fact of the matter is under the old system people were rewarded for not playing Pokemon. They were rewarded for sitting out of top cuts and major tournaments. Someone may have sat out a nationals they were destined to win in order to protect a worlds invite they ended up going 3-3 miss at. The new system fixes that and eliminates all incentives for people who want to play Pokemon to not do so. You can tweak it around the edges, but this is the best change to the game in years.

 

[ShadowGuard]

1: Championship Points skew the importance of smaller events

Awarding success at Battle Roads and City Championships more than before is a good thing. Yes, marathons are inflating CC points. But that's not a reason to lower them for all CCs in all countries. The real problem is not the type of an event, but the actual attendance. I'll go into that below.

2: The system encourages travelling, and lots of it

Well, of course there are differeces in how many tournament a player can attend. Dutch player have a premier event every weekend, while other countries maybe don't. However, it was the same with ELO rating.

3: The system rewards attendance over consistent play.

That is the purpose of the system.
More playing = better.
No punishment for not doing well at a tournament.
Gives the option to try out some new decks etc without losing rating.
btw you can't compare CP with Planeswalker points. Planeswalker Points reward going 3-4 at three events higher than going 7-0 at one event of the same type. CP don't.

4: No matter how much you think you can qualify without CPs, you can't

And where is the problem with this? That was the case with the ELO system as well.

5: World Championships awarding CP

Top 4 players at Worlds get an invite and trips. It's just fair that players who are doing well at Worlds are at least rewarded with CP.

6: Best Finish Limits

There are now 3 Regionals in the US. Best finish limit of 4 seems fine to me.
The only limit I'd lower is for Battle Roads. If they're big enough to be worth their points, you probably won't win 8 anyway.

7. Records going unrewarded and top cut caps

This is absolutely true and should be fixed. Giving out CP to anyone who wins at least 75 % of their matches would be an option.

While pointing out seven things from which most (except 7) aren't really a problem, you forgot the biggest flaw we have in the CP system. It doesn't regard actual attendance on events. In the US the differences might only be that huge at smaller events like BRs and CCs, but in Europe that also applies to Regionals and even Nationals. Winning a 10-masters-Regionals in Austria adds the same CP as winning the European Challenge Cup (just to mention the most ridiculous example of this season).

Bullados posted a decent solution for this in another thread, that uses lower base points and added kicker points for every place based on attendance.

With that one and the bubbling issue fixed, CP will be a good way to determine World invites.

 

[Axew]

The problem with Bullados' suggestion is increasing points for first creates an imposssibly high threshold for some players if they live in an area with little to no op

 

[ShadowGuard]

The problem with Bullados' suggestion is increasing points for first creates an imposssibly high threshold for some players if they live in an area with little to no op

It's also more difficult to win a big event.
If, let's say, 50 players are chasing for a total of 20 CP to distribute, it's just fair when 100 players are chasing for a total of 35 CP (took just random numbers).
And players with less OP at home can travel to the big events, possibly getting more CP out of one tournament, and don't need to go to 2-3 of them.

 

[NoPoke]

It doesn't matter how hard it is to win a big event it is guaranteed that someone will. It is that feature that dooms Bullados's suggestion because if it were ever implemented then it makes playing in sparse OP areas pointless.

---------- Post added 07/18/2012 at 02:17 PM ----------

....
btw you can't compare CP with Planeswalker points. Planeswalker Points reward going 3-4 at three events higher than going 7-0 at one event of the same type. CP don't..

Its even worse than ShadowGuard states.

Go 3-4 at three events and get 39 Planeswalker points.
Go 11-0 at a single event in the same class and get 37 Planeswalker points.

 

[Box of Fail]

I prefer ELO, but I have to say I understand the reason for CP and considering many people had been pushing for it, I don't think they will switch back, and I don't personally mind that all that much.

I think the problem is the things they've done along with CP. They've increased the number of states dates to 3, and the number of regionals dates from 1 to 2 to 3 (and likely 4 next year).

The last thing a CP system needs is more opportunities to attend big-name events, and this is compounded by the fact that they aren't even increasing the number of actual regionals tournaments. To even be competitive now, you have to commit to travel a lot. I don't even know what the next closest regionals is for me after Philly. It seems like it is no longer enough to be good, you also have to be very, very actively devoted to attending events. I don't think that players who are good enough to qualify for worlds should have to play all-year round.

An easy way to at least mitigate this problem, imo, is to mess with the BFLs. Right now, SPTs and Regionals are compounded into the same BFL group, with a total of 4. That means 4 of 5 SPT/ Regs were counted last year! Instead, to ensure people can be comfortable enough after winning regs not to feel compelled to drive 6+ hours to a remote Regionals location, SPTs should be capped at 2, and Regionals at 1 (the amount you went to under ELO). That way, the extra tournaments can only help you, but you aren't compelled to go to keep up with everyone else if you finish well at the initial tournaments. jmho.

 

[SD PokeMom]

could we wait until the details for next season's CPs, BFL and ranking/rating invites are actually announced before you all tear it apart?

after all, they out and out STATED at the beginning of this season and the CP system that things WOULD change in subsequent seasons after they got the results of this one...

jmho.

 

[psychup2034]

could we wait until the details for next season's CPs, BFL and ranking/rating invites are actually announced before you all tear it apart?

after all, they out and out STATED at the beginning of this season and the CP system that things WOULD change in subsequent seasons after they got the results of this one...

jmho.

Could players be allowed to give our input and constructive criticism of the CP system, BFL and ranking/rating invites structure before a tweaked system comes into effect next year?

After all, they STATED at the beginning of the season that they were looking for constructive player feedback on the new system as they work to tweak it...

jmho.

---------- Post added 07/18/2012 at 12:30 PM ----------

Axew, first of all, thanks for your well thought-out post. I basically agree with everything you have said except point #3:

3: The system rewards attendance over consistent play.

I think the system gives players 2 avenues to get an invite. The first is to attend more events, while the second is to have consistent play. Obviously, a less consistent player who attends a lot of events will accumulate Championship Points, but that player still needs to have a certain level of dedication to the game in order to go to all those events. Moreover, because of the best finish limit, a less skilled player has a soft cap on the number of points he/she can accumulate: even if such a player attends 15 City Championship points, he/she may only top cut 5/15 times.

On the other hand, a more consistent player can get all the Championship points he/she needs from playing in fewer events. For example, I only played in 5 Cities and 4 States/Regionals this season, yet I have an invite. The benefit of being super-consistent, especially during the larger tournaments (Cities and States) is that you don't need to attend that many.

There needs to be a balance between rewarding consistency and dedication. I feel like a person who attended 12 Cities and 5 States/Regionals, but performed a little less consistently than I did overall, deserves the invite as much as I do. After all, he's taking the time to attend more events, and he should be rewarded for that. I don't think the current Championship Point system rewards attendance excessively. After all, wouldn't the franchise want to reward its most dedicated players?

The one exception, in my opinion, is Battle Roads. The best finish limit for Battle Roads is a little high (as it's almost impossible to hit), indirectly forcing the "competitive" players to attend all of them. So in that sense, Battle Roads do reward attendance disproportionately over consistency.

 

[Prime]

Marathons are a growing trend. Why not work towards getting one in your area?

 

[Box of Fail]

could we wait until the details for next season's CPs, BFL and ranking/rating invites are actually announced before you all tear it apart?

after all, they out and out STATED at the beginning of this season and the CP system that things WOULD change in subsequent seasons after they got the results of this one...

jmho.

Yeah, that's what I'm saying - lower the BFL. I understand that last year was the first year of implementation. I understand they may make some positive changes.

What irks me is that you just seem peeved that people have been criticizing the system currently in place. If OP lowers the BFL, good for them. I'll be impressed. But there is nothing wrong with our coming out here now, criticizing the system from last year, and explaining how and why we think it could be improved.

We want them to change the CP system. That's why we're posting. Obviously it's better to post before they've already made the changes, if our goal is to offer what little support we can in favor of changes we would like to see next year. Isn't that better than waiting and objecting after the decisions have already been finalized? Just because someone implies that the system has room for improvement as it stands isn't a reason to get defensive and go into "how dare you criticize OP" mode. We're not ragging on OP, we're offering possible fixes. Isn't that what OP wants anyways, to be responsive to the player base? Isn't that why CP are around at all?

 

[NoPoke]

Hint : instead of bashing last year (even if we know it deserved it) try a positive proposal where you say what you'd like to change and how it will improve matters.

You get far fewer rubbish posts clogging the thread about how awful POP are and how they don't care. You will get more constructive analysis from those that really do want to see change and improvement. You may even get POP to join in

Note that a worked out proposal including consequences and desired outcomes and maybe undesired ones takes a lot of time. Just saying reduce the BFLs is too much of an instinctive response to get POP to sit up and notice what you are saying among all the other noise on a fan forum.

[Edit : that isn't specifically directed at BofF , its unfortunate that the thread started off as a negative one as that sets the tone for how posts are read]

 

[DonphanAtoZ]

The last thing a CP system needs is more opportunities to attend big-name events

Boom. This. End of story. CP would be pretty nice if we didn't have this issue.

 

[Axew]

Another issue I would like to discuss is the scooping problem. At Battle Roads Spring, I saw many players intentionally conceding to other players because they knew they were out of the running for a worlds invite. Under the old ELO system, nobody was ever truly out of the rating race, because a near perfect swiss or deep top cut run at nationals could still cross the threshold.

@Fulop: The problem with basing your opinions on how good cp is AFTER a tweaked system is that we have yet to receive details of a new CP system, and as such actually have no idea.

@Moss: Having points nobody else can get count towards more points more points nobody else can get is bad for the game. Period.

@Shadowguard: Playing is good. Winning is better.

@NoPoke: You really don't understand Planeswalker points do you? The system rewards playing in much more difficult and larger tournaments because going even 5-4 at a Grand Prix tournament is much more difficult than 5-2ing a PTG or 5-0ing an FNM, similar how states was weighted to be worth more than cities.


@Kevin: Completely agree

 

[NoPoke]

Axew Magic scrapped the use of elo too. So if that is where you are heading I'd warn you that Planeswalker Points are worse than anything POP have done.

JMHO

 

[Jaeger]

could we wait until the details for next season's CPs, BFL and ranking/rating invites are actually announced before you all tear it apart?

after all, they out and out STATED at the beginning of this season and the CP system that things WOULD change in subsequent seasons after they got the results of this one...

jmho.

Than isn't public discussion healthy and should be encouraged? Sure 1 on 1 emails with P!P are helpful but an overall message from the player base is good as well.

 

[psychup2034]

Another issue I would like to discuss is the scooping problem. At Battle Roads Spring, I saw many players intentionally conceding to other players because they knew they were out of the running for a worlds invite. Under the old ELO system, nobody was ever truly out of the rating race, because a near perfect swiss or deep top cut run at nationals could still cross the threshold.

Scooping has always been a reality in Pokemon. People scooped under the ELO system as well. Under the CP system, a deep Nationals run (top 8) still gets you the Worlds invite.

Put yourself in the position of the player who has to make that decision. Your good friend is a couple Championship Points away from the Worlds invite. You're playing him in the finals of a Battle Roads when you're out of contention already. Is there really a reason why you wouldn't scoop to him? Why would you screw your good friend out of an invite? Not scooping in this situation is a good way to get people in your area to think lowly of you.

I was sitting on an invite during the end of Spring Battle Roads. Had I played anyone who was contending for an invite, I would've scooped to them without being asked. Why would I screw over a local player? There's nothing immoral or wrong about scooping, and it's perfectly within the rules of the game. I wouldn't want to gain a reputation as "that _____ who screwed someone out of an invite."

 

[Jaeger]

Scooping has always been a reality in Pokemon. People scooped under the ELO system as well. Under the CP system, a deep Nationals run (top 8) still gets you the Worlds invite.

Put yourself in the position of the player who has to make that decision. Your good friend is a couple Championship Points away from the Worlds invite. You're playing him in the finals of a Battle Roads when you're out of contention already. Is there really a reason why you wouldn't scoop to him? Why would you screw your good friend out of an invite? Not scooping in this situation is a good way to get people in your area to think lowly of you.

I was sitting on an invite during the end of Spring Battle Roads. Had I played anyone who was contending for an invite, I would've scooped to them without being asked. Why would I screw over a local player? There's nothing immoral or wrong about scooping, and it's perfectly within the rules of the game. I wouldn't want to gain a reputation as "that _____ who screwed someone out of an invite."

Did they really deserve the invite if they can't earn it?

 

[psychup2034]

Did they really deserve the invite if they can't earn it?

That's a really good question Jay. Here's my take on it:

For the effect of scooping to affect someone’s Worlds invite chances, that person would have to have performed well enough throughout the season to be in close contention for Worlds. This season, people generally didn’t scoop to each other until Spring Battle Roads because there’s a financial prize at stake for States/Regionals, and everyone still has a chance at Worlds during Cities.

I think that all people who are above a certain hard-to-define CP range “deserve” the Worlds invite. I can’t say that someone who had 47 or 48 CP this year didn’t “deserve” the invite. These are good players who just had a couple bad breaks throughout the season. If a couple coin flips/topdecks had gone differently, they would have an invite instead of someone else. While the people who bubbled “deserve” invites in my opinion, it’s just unfortunate that they didn’t make it over the threshold.

From my perspective, there are more than 40 people in North American Masters who deserve an invite. I can firmly say that the person who got 20th clearly deserved the invite, and the person who got 60th clearly did not. For those players around the 40th rank, scooping may make a difference on who gets an invite, but for the 2-3 points gained from scooping to matter, that person would have to be close to the invite anyway.

I understand that this explanation may be unsatisfactory for some people, and I think there isn’t a right answer to this issue. If you have a different opinion, please respond to me as I’m very interested to hear.