Re: Pete. (aka. Gaming the system-- who's to blame?)

[JandPDS]

What about having a point-based invite? Give a certain number of points for top cutting in events, and allow players who, say, total to 10 points, to be allowed to play in Worlds. For example, getting top 8 in cities would have 1st place earning 5 points, 2nd place earning 3 points, 3rd-4th earning 2 points, and 5th-8th earning 1 point. The point values could go up based on the event - states/regionals would have 1st place automatically getting 10 points for example. This would always encourage players to play at an event since they can't lose points, only gain them. They could adjust the number of points needed for worlds based on the number of events in the year and how many people they want attending. I'm sure there's some formula that can be used to predict the number of players that qualify based on the # of points given away, the number of events, and their location.

The problem is Point based invites. They system worked better when Points were for fun only and did not really Mean anything. Back then invites to Worlds went to the winners of Sates, the winners of Regional’s and the Winners of Stadium or latter Gym Challenges. SO those players already had their invites to Worlds locked up and they did not loose anything if the played out Nationals had ended up with a Subpar day (face it a bad day at Pokémon happens to everyone at one point or another) Back then we never had players go to Nationals and play one or two rounds and then drop. That was unthinkable. So I would much rather keep the point system in place because it is a fun way to track your performance throughout the year, but do not make it mean anything. All invites to worlds should go back to the winners of Sates and Higher tourneys and instead of giving invites to the Top 40 in North America based on score you could give them to the Top 16 at US Nationals and T8 and Canadian and Mexican Nationals. Then play a true swiss LCQ at worlds where the invites go to the T16. That would be the best way to select the field for Worlds.

I am more concerned with the way they destroyed the LCQ this yesr. The current setup for the Grinder this year is so unfair; they might as well have cancelled it. The players who make it though will be the luckiest ones who managed to avoid playing against an auto loss deck not the best players in the field that day. IF you are playing a super good Zekrom deck but you get paired R2 vs. a Donphan, should that cost you a place in Worlds?

Many, many players have grinded into worlds with one or two losses in the LCQ and then went on to go very deep into the top cut at worlds. The first year the Grinder was in Hawaii, a Japanese player had a very effective Scizor EX deck. He lost to a Washington player in Round 2, did not loose again until Round #8 went 6-2 finished in 6th place and they took the top 7. The next day he went 5-1 at Worlds (It was super small that year) finished as the #2 seed after swiss and went on to place 4th over all at Worlds and received a invite and trip to Orlando the following year. He proved without a doubt that he belonged in the World Championships, but under this years sham of an LCQ he would have been kicked out of the event after the 2nd round (the player that beat him ended up with 3 losses and did not make it)

So losing early in the Grinder should not eliminate a deserving player form Worlds. That is even more unfair then the rating point problem for invites.

 

[Shino Bug Master]

What about having a point-based invite? Give a certain number of points for top cutting in events, and allow players who, say, total to 10 points, to be allowed to play in Worlds. For example, getting top 8 in cities would have 1st place earning 5 points, 2nd place earning 3 points, 3rd-4th earning 2 points, and 5th-8th earning 1 point. The point values could go up based on the event - states/regionals would have 1st place automatically getting 10 points for example. This would always encourage players to play at an event since they can't lose points, only gain them. They could adjust the number of points needed for worlds based on the number of events in the year and how many people they want attending. I'm sure there's some formula that can be used to predict the number of players that qualify based on the # of points given away, the number of events, and their location.

While a decent idea, i can see an arguement in the way places are achieved. If someone goes 5-3 and squeak in at top 32, they already lost 3 games. Lets say they then make it all the way to Top 2 where they play an undefeated player, now at 12-0 to there 9-3, top cut is a total donkfest and they end 10-3 and 12-1. Now the 12-1 player recieves less points in a new system despite making a safer and better deck play as proved throughout the day. It may just be a burden of change, but the one thing i like about the current system is that every match really matters, rather than how you can dish out amongst top cut. Its just the darn cards the print that allow for 3 second games that ruin it.

While it does discourage dropping which is a plus, it also restrics players to staying sometimes later than they can to have any gain whatsoever.

 

[JimboPro]

Is there a ratings system that only adds points for wins, but doesn't take them away for losses?

Would such a system solve the problem, or only create a new problem to be gamed differently?

I'm a huge fan of Pokemon, don't complain, or tread intemperately into needlessly controversial threads; but I do see a problem, it has existed for a while, and it isn't good for the game.

I had a good player, convinced he had enough points for a World's invite, drop after swiss at Regionals. Players good enough to top cut a Regionals should not be dropping because dropping is the smartest thing they can do.

Players who win States or Regionals should not have to be put in the position where dropping before Nats starts, or after just one or two games won - or lost, is the smart choice. Yoshi described well feeling somewhat raw for not having dropped after top cutting Nats.

I completely understand and support Pete's, TPCi's, desire to see people with travel support show up to play their tournaments. Heck, if I were them, I would expect it...except the system doesn't reward, but can actually punish our top players that do play in the Nats they have won travel support to attend.

Again, I would like to see a system that awards, but doesn't subtract points, so drops are eliminated. I am probably missing something obvious, or else this would already be in place, because it seems to me that not losing points would mean players play all the rounds they could.

Anyway, I hope you all figure this out to the satisfaction of both the tournament travel support givers and the players who travel to those events.

I think the problem that most are missing with the only gaining points and not being able to lose points is that not all of us can make as many events as others. I would have no chance making worlds, actually neither would the Washington State Champion this year if you could only gain points and not lose any... I hope this is clear cause it doesn't seem to be coming out right ...

 

[NoPoke]

The elephant in the room for a reward only system is that players with the access to the most events win the invites. You can try and reduce this by capping the gain from each series but it does not remove the advantage of access. Only got a 1st and a 2nd form two city championships? never mind just play another five to max out the firsts.

There is more. TEAM PLAY. Once I've got my two wins or whatever the cap is for an event series I can now choose which of my friends to lose too as there is no penalty to myself. Teams can literally take it in turns to win events. Or is OP to try and reintroduced the hated win one and done policy? You don't have the choice of sitting out you are told you can't play.

ELO isn't working well at the moment, it has worked better for pokemon in the past in the less speed obsessed days. ELO as currently implemented needs to be changed. However if elo can not be altered to work better then there is no obvious replacement that can do a better job of exposing underlying skill. Just because it is not working right now does not mean that it is broken in principle rather than its implementation is not a good match for the recent environment.

Where I do agree is that it has to change. The current implementation is not good enough. It is even possible that the format is too luck based for any rating/ranking system to work. I don't believe that is the case but it does make the task harder.

In the end any system has its flaws, has its unfairnesses, encourages some undesired behaviours. Just because the grass looks greener on the other side does not mean it is.

---------- Post added 07/05/2011 at 09:04 AM ----------

@PokeDad: There are rating systems that only reward success. But they all use a fixed number of teams and a fixed number of events. Soccer, and Formula 1 racing for example.

 

[thisguy]

Maybe every event state level+ should give invites. Something like Top 2 at States, Top 4 at Regionals and Top 8 at Nationals or something along those lines. If that doesn't seem generous enough you could possibly increase the invites from a Nationals event to 16. Countries without a Regionals or equivalent could have the Top 4 from states get invites.

 

[nnaann]

My problem is that I don't feel there's an easy solution. If you only reward winning, then it just gives an advantage to those who can attend more tournaments. I like the idea of having an ELO system for Swiss rounds but introducing something else for Top Cut. It doesn't make sense that people in Top Cut are being punished if they lose, over those who bubble and keep their points.

 

[NoPoke]

The idea that top cuts are special needs close examination. At every point in a tournament half the players in the room just lost their match. This applies to the last round of swiss just as much to the first round of the top cut. You can end your day on two losses: no different to the swiss rounds where a quarter of the players will likely end with a loss in the last two rounds.

Since top cuts are typically match play a win in the top cut ought to be stronger evidence of a difference in skill than the single game swiss results. Unfortunately we throw away the game data in the top cuts and make the problem much harder than it could be.

Play at risk is an important feature of the rating system. Play at risk keeps a lot of team play tactics from occurring. Play at risk does mean that we should try and have a representative value for the risk - something that the current elo does not attempt

Zero sum is another important feature to control the rampant growth in ratings that those with access to the most events would quickly demonstrate. However some relaxation of zero sum in the first round of the top cut does not wreck that important characteristic of a fair rating system. Any system that focuses primarily on wins or even on wins-losses strongly favours those with access to the most tournaments.

It isn't like there aren't options to address many of the issues that strain the current system. But unless some action is taken then the only alternative is to scrap the current system and hope that its replacement is not fatally flawed.

And just in case anyone is unsure where I stand I do not want the current system to continue much longer.

 

[FunnyBear]

I know that chess uses the Elo system -practically invented for it. Thankfully chess doesn't rotate every year. Chess leagues have dealt with the same type of problem when it comes to people playing or dropping to protect their rating. I suppose you could say that the system is gamed no matter what game is being played. Maybe adjusting metrics would help, but I can't pretend to understand the problem since I focus more on non-ranked events.

In organized chess, you keep your rating year after year, so it stabilizes after 20 or so games and then moves based on performance. In Pokemon, you start fresh every year, so have to get through the initial part of the curve via early tournaments. This distorts results up through Regionals where top players can have ratings in the 1600s by not playing in more than States. The whole problem of protecting ratings would go away if they just wouldn't reset it each year. The game changes, but the top players are still the top players after a format change.

Oh, and organized chess has only K=32, and it helps stabilize the ratings faster.

 

[EHLERZ5]

Would RAISING the K Value of NATIONALS in all countries be a solution? Make it where people back in the pack can Win games and snag a "sitters" spot at Worlds?

 

[vanderbilt_grad]

It’s been said before but might as well go down this road again. I do favor change & I also feel pretty strongly that the OP is on the money about the source of the problem being the system not the players.

The current system has the following issues, it:

• Discourages highly ranked players from actually participating meaningfully in big events.
• Encourages “gaming the system” by deliberately missing the first round of an event.
• Punishes top cut losses too severely.
• Rewards luck disproportionately.
• Fails to accurately measure the value of a win/loss for or against a good player who doesn’t play before States, Nationals, or Regionals since ELO is reset every year.

There seem to me to be several ways fixes could be attempted.

• Tweak the system itself. Suggestions like not measuring ELO in top cut matches would fall here as would altering K values to try to accommodate for luck more.
• Make major changes to the system. Doing something about the yearly reset would be in this category as would moving to a reward only system like pro-points.
• Alter the reward. Separating Worlds Invites from ELO would remove the disincentives for playing at Nats for instance.

My suggestion would be to try several different things.

1. Tweak the system so that Top Cut matches either do not count for ELO or so that they count less. Similarly avoid setting K values for States, Regionals, and Nats too high … perhaps even lower than Cities given the next item.

2. Introduce a Pro Points system similar to MtG that functions alongside ELO but is less comprehensive. Pro Point events would be States, Regionals, and Nationals - or equivalent international events. Cities and BRs would not grant Pro Points. This would have to be tweaked for international play to ensure that their Pro Point opportunities were as similar as possible to the US.

3. Alter the rewards system to give Worlds invites to winners or near winners of Regionals and Nats. There would have to be some decisions on how this would work internationally, but for the US I’m thinking all Regionals winners and T4 at Nats. This means fewer invites for ELO or Pro Points.

4. Have ELO based World Invites by region similar to what is had now, but with fewer available overall. This rewards people who have had strong & consistently good seasons but haven’t actually won some of the biggest events.

5. Have a small number of Pro Point based World Invites that are NOT based on region but rather strictly on the number Pro Points gained in a given season (remember Pro Points only for States, Regionals, & Nats). This helps ensure players who do well in ALL the big pre-worlds events, where the competition is the toughest, have a shot.

What this does is give strong competitors several potential paths to Worlds. Pro Points & multiple potential invites encourages play at Regionals & Nats to counterbalance ELO working against it. ELO rewards the folks who grind out a long good season over the folks who just play at the big events. Luck is still a factor but there is more of an opportunity to have a bad day at one event without the dream being dead … hence more of an incentive to actually play & play the right way.

 

[mysterioustrainer]

I am beginning to agree the current Championship Series is not quite working the way it is suppose to be. I realized this when the terrible attendances for Battle Roads Spring came to pass. When I first played in the 03-04 and 04-05 seasons, the system was greatly different with Gym Challenges the easiest way to earn the critical invite to Worlds. Obviously ever since Worlds started going to Hawaii every other year, the budget for the organized play programs had to be rearranged, leaving the National Championships the only paid trip invite method, and a wide opening of invites with rankings. I also found it logical to restrict ranking tournaments to tournaments in the Championship series to prevent TO abuse.

If I had the opportunty to rearrange the Championship Series, here is how I would do it:

I first would introduce the Fall Battle Road just like how it is, 4K and all. There needs to be no changes there. City Championships would come up then, however since City Championships are also introductory, I would keep it at 4k. Then I would re-introduce, Gym Challenges, however instead of inviting to Worlds, they are an invite to Nationals, with a small travel stripend, however it also has a 4K value. Then would come State Championships. City Winners get one bye, 8K value, winners earn invite to Nationals, with a small travel stripend. Then Spring Battle Roads would come to pass, again 4K. I would remove Regional Championships and re-introduce Stadium Challenges in their place and Top Winner earns invite and paid trip to Worlds and Nationals, and top four earn invite and travel stripend to Nationals. Stadiums would be 16K. Nationals would be invitation only, and this is to help cut down the long 3-day long tournament. Of course there would be a day before last chance qualifer. Nationals is 32K. Top 8 earn invites, top 4 earn paid trips to Worlds. Then comes the ranking cut, which would be top 0-4 per country (based on country's level of participation). No exceptions, keeping the ranking system from being overly abused. Ranking players earn invites but no travel awards. The World Championships come in, 32K to improve rankings for show, in general there should be up to 20 players representing each country (not counting returning champions).

I do not know if this system would be any better than the current one, but I think it would help stop banking rating points and encourage more play in the events themselves, and by making Nationals invite only, it drives more focus to working the road to get to Nationals.

 

[NoPoke]

There is nothing inherently wrong with players sitting out tournaments.
There is nothing inherently wrong with a system that discourages a few players from playing.
You do want to minimise that quantity but not at the expense of breaking the system for a larger number of players

Given that tournament access varies by region, often very dramatically. Any system which rewards those with access to the most tournaments to such a degree that those with significantly less access can not achieve the rewards available is fatally flawed. It is really easy to accidentally design a pure rewards system with this characteristic. P!P want more tournaments for their players not fewer. Yet the more tournaments you add to a pure reward system the worse the problems of favoured access become.

 

[espeon200]

I remember a discussion on Pro Points happening on The Gym before. The way it was described then seemed to make a lot of sense.

The biggest problem with using a pro points solution is, as NoPoke has already pointed out, that players are rewarded not for being good players, but for being able to attend lots of events. I do think that only using Pro Points at Regionals or higher would solve this problem, especially if the amount of Pro Points that could be earned doubled at each event level.

I also on't think that ELO would have to be abandoned entirely. I think that ELO should be used for lower events and before the top cut in every tournament below Nationals level. We already see a lot of players who drop from top cut because they want to preserve their ratings. This would take ratings out of the equation for those players and give extra incentive to keep playing proving who is the best of the best.

Now, I would suggest that Nationals would be played entirely for Pro Points although only the top 16 or better would earn them. Again, this rewards the best of the best without penalizing those who are having a bad day. Just because you get three autolosses in a row does not mean that you lose a bunch of ratings points.

So what happens to ELO? Well, as I said, it continues to be used for lower events. It also is used when determining invites. Basically the way I'd like to see it is that at the time invites to Worlds are decided a conversion amount is decided (depending on a curve of the top and bottom rated players in the country) that converts ELO rankings into reward points. This means that playing at lower events still has an impact on your ability to get to Worlds, but it's not the end-all-be-all deciding factor.

 

[Yoshi-]

Just an idea, what if we keep everything as it is but at the end of the tournament if youre below +-0 you dont loose anything?

 

[espeon200]

The problem is that ELO is a closed system, which makes it easy to check if it's right. Basically the sum of all points should be equal to 1600 times the number of players in the system. Makes it simple to check but impossible to change by ignoring results.
Posted with Mobile style...

 

[Ice'Cold]

I'm definitely not a fan of a system that discriminates against players geographically. (being able to attend something like the Georgia marathon is flat out unfair for people who don't live in an area like that).

Cap "Battle Points" per series, such that player X (Kettler I'm looking at you) who lives in a state where they can attend 7+ battle roads easily cannot earn more than a player who can attend fewer battle roads. I'm thinking their best 3 is a good number. Yes they have more chances to make a strong performance but at least the player in the other area has an opportunity to earn just as many points. Cities could be best 2, and states/regs/nats would obv be just the 1.

I'm also a fan of not resetting ELO each year (making rankings meaningful again) but having a minimum participation requirement over the year in order to qualify for worlds invites.

 

[Yoshi-]

The problem is that ELO is a closed system, which makes it easy to check if it's right. Basically the sum of all points should be equal to 1600 times the number of players in the system. Makes it simple to check but impossible to change by ignoring results.
Posted with Mobile style...

So we need to have a savety thing because otherwise we cant be sure the system is running flawlessly? Thats a pretty weak argument IMO, the system should work flawlessly and we shouldnt need this.

 

[MattPL]

So we need to have a savety thing because otherwise we cant be sure the system is running flawlessly? Thats a pretty weak argument IMO, the system should work flawlessly and we shouldnt need this.

It's difficult to create a flawless system for a game that has a notable amount of luck involved.

 

[Yoshi-]

It's difficult to create a flawless system for a game that has a notable amount of luck involved.

Im talking about the nessesarity that all points need to add up to 1600

 

[espeon200]

So we need to have a savety thing because otherwise we cant be sure the system is running flawlessly? Thats a pretty weak argument IMO, the system should work flawlessly and we shouldnt need this.

Apparently we do. Not saying it's a good reason, and I'm definitely not the only reason that TCPi uses ELO. It is however, the reason that was given last time someone made a similar suggestion. :smile:
Posted with Mobile style...