Ratings and Rankings: Top Cut

[getjosh]

I didn't know were else to ask this question so I thought I would put it up to the Professors.

My son and I have been in dispute over this issue since our last Cities, two weeks ago. We both play in the master division and we both went 4-2. With the way the tournament rankings washed out, he came in eighth place and I came in ninth. There were enough people in the division for Top Cut to be the top eight players. So he made the cut and I just missed it.

He was playing a Machamp deck and lost to a Flygon/GG deck in the first round, two games to one.

So...after the tournament he said that I was better off, in regards to the Premier Rating, because now he had two extra losses. I said that making Top Cut should never punish or put a Premier Rating in jeopardy, and that if anything he should be rewarded for making the cut and playing the extra games.

Here are my questions;

1. How does making Top Cut affect your point total for the tournament?

2. Do all three (or two) games plyed in a 'match' count separately for points, or just the one 'match' that is played?

3. Is it possible that I (coming in ninth and missing the cut) garnished more points from this tournament than he did (coming in eighth and losing the first match)?

Thanks in advance for your help.

 

[Lawman]

1. Yes.
2. Each match counts as either a W or a L. Individual games within the Bo3 dont give/take away points.
3. You did gain more points by missing topcut if he lost in 1st rd of SEF. 4-2 vs 4-3.

Keith

 

[bullados]

Every match that you play in a Premier level tournament affects your Premier Ranking. Yes, it is possible (and likely) that a player making the top cut winds up with fewer Premier Points gained than a player that misses the top cut. This is balanced by the fact that the player in the top cut gets to play more matches, and has more of a chance to raise his or her rating.

Notice that I also avoided the word "game" in this explanation. In most Swiss rounds, Game == Match. however, in the Top Cut, Matches are comprised of 3 Games.

This is yet another example of how More Rounds does not necessarily mean More Points.

 

[Ruiner]

Yes, it is a slight flaw in the system, as a matter of fact. By slight, I mean a huge glaring error that really REALLY needs fixed. I dropped at Regionals last year at 5-2 to protect my rating before top cut because if I lost in top 16, top 8, OR top 4, I'd have wound up WORSE OFF than had I 5-2 wiffed cut. I think the best way to handle the situation is to award bonus points for top cutting, than apply standard k value modifications based on performance.

 

[bullados]

Then you remove the biggest asset to the Elo system from an administrative standpoint: Zero Sum. If you average every player in any closed Elo rating system, you will get 1600, guaranteed. From an administrative standpoint, that makes everything much easier to manage, especially now that they've gone into fractional points.

How high can your rating be that you need 3 wins to offset a single loss at an event with a K-value as high as Regionals? If you're in the top cut at Regionals, you can pretty well be assured that everybody (assuming an average 16 gain per win, 16 gain per loss) has a minimum rating of 1648, and if you're 5-2, you're likely facing somebody with a rating of 1680. Your rating would have to be somewhere in the mid-1800s to make that kind of spot remotely possible. Plus, if a player goes 6-1 at Regionals, it's pretty well assured that they've done fairly well up until then, and probably have a ranking well into the 1700s. You can't lose more than 32 points in any single round at Regionals, and it's usually closer to 20-25, depending on the quality of your opponent.

I also don't understand why simply potentially getting the invite (no money, no trip, no nothing) into Worlds could be more important than potentially getting the trip and hotel to Nationals. You'd pretty much have to play Nationals anyway to get your ratings invite unless you're all but undefeated through the 3 big events and at least 3 mid-sized events.

 

[SteveP]

Yes, it is a slight flaw in the system, as a matter of fact. By slight, I mean a huge glaring error that really REALLY needs fixed....

By the same accord, the person who wins the top cut gets a nice bonus in points, which "should" happen.

I do agree that top cut ratings need to be done differently from swiss. Assuming two players with near-rankings, it's a bit disheartening to think that the one who makes the playoffs, then loses can actually earn fewer points than the one who didn't make the top cut.

Nevertheless, if rating points were the only thing "up for grabs" in the top cut at championship-level events, then I'd say, "Fix the problem!" But, that's not the case. Boosters, trophies, trips, scholarships, etc., provide the incentive to risk points in hopes of winning the "big prizes."

Back to back posts merged. The following information has been added:

Then you remove the biggest asset to the Elo system from an administrative standpoint: Zero Sum....

I agree, but I also think it would be nice to somehow supplement rankings like they do in Magic and used to do in Decipher TCGs -- use a separate tally for "qualifier" or "premier" points. For example, in Lord of the Rings (LOTR) premier events, the champion got 2 premier points and everyone else who made the top cut got 1 point. In LOTR, you had two ranking systems, ELO and premier points. Each ranking system gave you a different perspective as to who the top players were.

 

[Diaz]

How high can your rating be that you need 3 wins to offset a single loss at an event with a K-value as high as Regionals? If you're in the top cut at Regionals, you can pretty well be assured that everybody (assuming an average 16 gain per win, 16 gain per loss) has a minimum rating of 1648, and if you're 5-2, you're likely facing somebody with a rating of 1680. Your rating would have to be somewhere in the mid-1800s to make that kind of spot remotely possible. Plus, if a player goes 6-1 at Regionals, it's pretty well assured that they've done fairly well up until then, and probably have a ranking well into the 1700s. You can't lose more than 32 points in any single round at Regionals, and it's usually closer to 20-25, depending on the quality of your opponent.

Yeah. You would probably have to be in 1800+ range. But if your serious about getting a rating invite, by that point in regionals, thats the kind of rating you probably have.

It's very common to need 3 wins to off set a loss. By around the end of the first states every year, thats where I'm at. I'm sure there are people out there in even better positions, that need near 4. At that point you go into events with the idea that you can go x-1 and break even, but you still play because theres a chance at being undefeated.

If you want an invite, you need to be at 1830+ at the end, so everyone who's gotten a rating invite has been in a position where they need a 3-1 win ratio to break even, on average.

For many players, getting a worlds invite is the whole point of the entire season. Nothing else really matters, as the whole season builds up to worlds. If you have a rating that will get you in, you have to consider dropping to protect it.

 

[Biggie]

Yes, it is a slight flaw in the system, as a matter of fact. By slight, I mean a huge glaring error that really REALLY needs fixed.

I believe the Professor Core Values has a thing or two to say about how we prefer Professors to pass along this kind of feedback.

 

[Ruiner]

Biggie: So what is the point of the Professor Forum if it isn't to discuss our opinions about issues such as this?

I'd like to see a good argument how one will argue that it is appropriate that a player who misses cut deserves to earn more points than one who makes cut. I understand that it is an inherent flaw in the system, an argument I guess I'd have to accept, but perhaps a strict ELO system is not the best means by which to handle the invite structure? As was pointed out, perhaps a similiar system to Magics Pro Points would be better used. Although that would render ratings altogether useless, unless those were used as alternatives for an invite as well. Perhaps use "Pro Points" to earn trips, and rating for an invite? Not sure...

Bullados: How could my rating be so high that I'd need 3 wins to offset a loss? At the end of Cities, I was sitting at 1785. After taking 3rd at two states ( X-1 at the one ) I was sitting at 1885 heading into Regionals, and I believe 2nd in N.A. Estimations put it that 1860 was the SAFE cut off point to make an invite ( In reality it was far lower, but 1860 was basically a guarantee ) At 5-2 in swiss, I knew I'd lose about 10 points ( I finished at 1875 ) and if I lost in top 16, I'd risk losing my invite, so I dropped. A 3 to 1 ratio is very common place above the 1800 threshold.

 

[SteveP]

We can debate the goods and bads of ELO, but in the end, it does what it's supposed to do -- provide a rough scale of who the most consistant players are for the season.

In LOTR, all you needed was at least 1 Premier Point to "qualify" for Worlds. 1 point got you a seat in the LOTR Worlds Qualifier, similiar to our LCQ but with a few more invitations. High rankings, more Premier Points, Continental Champs, and returning Worlds TopX players got you a seat in the LOTR Worlds Championship.

And Ruiner, just a word of advice about how you say things when you have the Professor title: Your opinion is important, but the "powers that be" will be more inclined to listen if your words are more positive (based on personal experience).

 

[Biggie]

Biggie: So what is the point of the Professor Forum if it isn't to discuss our opinions about issues such as this?

This Professor forum is provided by the fine folks who run the Gym so you might ask them.
The key difference between the Gym and the OP forum that we provide is that this forum is open to be read by the public while the OP forum is not. If you still are not sure why this would cause me to bring up the Professor Core Values please feel free to contact me via email or PM.

Your opinion is important, but the "powers that be" will be more inclined to listen if your words are more positive (based on personal experience).

I don't always expect a Professors response to be positive because I don’t imagine that we can please all of our Professors all the time. What I do expect is that any dissatisfaction is either stated with tact or delivered via the proper channels. After all, honest feedback is healthy but what I call "soapboxing" serves no positive purpose. To be clear I don't think that Ruiner was soapboxing in this thread; my comment earlier today was just meant to be a reminder not a rebuke.

 

[Ruiner]

I'm not sure what soapbox means but I think I have an idea based on context.

I feel that the rating system ( ELO being a common, and overall effective way to do this ) is a great idea. I was huge proponent of it when we switched from the Gym Challenge structure over to a rating invite structure, and to this day I'd fight tooth and nail in its defense as being the most fair way to handle worlds invites.

Yet I also feel I have more experience playing competitively than any other Professor by a significant margin, and I've gotten a chance to see a lot of the flaws in the system. I don't mean to say the system as a whole is "bad" as I feel it is resoundingly great. If I see something I think I'd like to see improved on, I will fight for that. Usually fairly aggressively, and I apologize if that comes off as crude at times. The current structure was a huge jump in the right direction when it was implemented, and those inherent flaws in it were ( and to a degree still are ) largely negligible. The system has been in place for years now, and now maybe its time to look at a way to work out some of the kinks in the system.

Let me rephrase this. If I'm ranking our current invite structure ( use 08-09 as reference ) I'm giving it a 9 out of 10. Yet I think it would be fairly easy to envision ways to make it a 10 out of 10. Would it be easy to get those ideas approved and past some of the red tape involved in an international effort? Probably not as easy as it is brainstorming the ideas.

If I try and address something I feel could be improved, even if stated with a more aggressive tone, I don't feel it is sending any sort of strong negative message about the game. Anyone reading these forums is not going to change their stance on the game because its been pointed out that some people don't feel the rating system is perfect. It might, on the other hand, get some of those people to get more involved and try and get changes enacted. Thats all I'm trying to do. If attention can be drawn to potential improvements, I really do not feel like that is doing any innate harm or disservice.

Perhaps I am not the "ideal" Professor in every sense of the word, but trust me when I say I have absolutely nothing but the best desires for this game, and maybe I'll find a way to not anger a bunch of people along the way fighting for that

 

[Pete De Shaw]

We can debate the goods and bads of ELO, but in the end, it does what it's supposed to do -- provide a rough scale of who the most consistant players are for the season.

Steve has hit on the essential point here. ELO is far from perfect and our implementation of ELO has some holes, but it does function. Even with all its flaws it provides usefull results. I will go Steve one further and state that our ELO based system is a remarkably good system- given the data set that we have to work with.

There have been dozens of proposals tossed around on the Gym for improving the way we calculate ratings or for non-ratings based invite systems. POP doesn't pay serious attention to them because they universally assume data we don't have, or at least don't have in a format that can be used for large scale calculations.

We don't use pro points because we cannot gather the necessary data from our website and we will not rely on hand entered data. We don't count top cut rounds differently from Swiss rounds because they are not distinguishable in our data set. We use ELO because it has a proven track record of results using data sets very similar to what is available to us. We don't count just the first X Cities or just the best X Cities because that breaks ELO.

 

[Diaz]

We don't use pro points because we cannot gather the necessary data from our website and we will not rely on hand entered data. We don't count top cut rounds differently from Swiss rounds because they are not distinguishable in our data set. We use ELO because it has a proven track record of results using data sets very similar to what is available to us. We don't count just the first X Cities or just the best X Cities because that breaks ELO.

In this case, it would seem that maybe the information gathering can be improved upon. If moving toward a better system is just an issue of better data collection, then better data should be collected.

A change in TOM could have it recognize the winner of an event. I believe this would not be too complex a thing to keep track of. The winner of the last round of top cut, or the highest ranked swiss player (if there is no cut), could be recorded be tom. Then this could be used to establish a pro-point system.

Furthermore, it would seem that a change to TOM could have it recognize the difference between a swiss and top cut match. Then, this additional data could be used to significantly better the system. This would require work under the hood on both TOM and the OP website, but I feel that over time, it could be something that could be implemented.

 

[Pete De Shaw]

*sigh* I guess I should have expected that.

No it is not quite that easy. All of your suggestions have occured to us.

 

[Biggie]

*sigh* I guess I should have expected that.

No it is not quite that easy. All of your suggestions have occured to us.

To add to Pete's response:
We are constantly looking at ways to improve the program but the process of tracking events globally is often significantly more complex than it appears on the surface. If we have not implemented a seemingly easy fix it is a pretty safe bet that we have a good reason for not doing so. That is not to say that you should stop offering suggestions. We just ask that you give us the benefit of the doubt and understand when we can't get into too much detail on our internal processes.

@Ruiner:
There is a substantial difference between a comment like:

Yes, it is a slight flaw in the system, as a matter of fact. By slight, I mean a huge glaring error that really REALLY needs fixed.

and

The system has been in place for years now, and now maybe its time to look at a way to work out some of the kinks in the system.

One is just fine on a public forum, the other I prefer be brought to us directly. Note that neither of those options is “keep it to yourself”. This is not an issue with what you have to say, it is an issue with where you have chosen to say it.

It might, on the other hand, get some of those people to get more involved and try and get changes enacted. Thats all I'm trying to do. If attention can be drawn to potential improvements, I really do not feel like that is doing any innate harm or disservice.

This is soapboxing.
Change is not enacted because some portion of the player base is stirred up enough about it, it is enacted when:

1) It is the best thing for the program
2) We have the resources and tools to make it happen

Generally speaking, getting players stirred up over things that possibly should not be changed or possibly cannot be changed is simply not helpful. Using the direct contact you have with us to provide feedback and thoughts that will help us develop our programs so that the appropriate changes are put in place is helpful.
The key here is that the POP team is not sitting around trying to resist change. As I said above, we never stop looking for ways to improve our program. We welcome ideas, suggestions, brainstorming, and general thoughts and concerns. That said actions which create or foster divisions, dissent, or dissatisfaction with the program is not welcome whether they are intended to have those effects or not.

 

[bullados]

I know that NoPoke has proposed a Beyesian (sp) Luck modifier to Elo several times before. Would a modifier like that work with the current system in place?

 

[SteveP]

ELO works fine. No need to modifiy it; otherwise, it's not ELO anymore.

If the data collection doesn't provide the data needed to give Pro Points, and TPCi won't accept manually-entered data, then modifications to the tournament software is the only solution.

Years ago, before TOM/TMS, I developed a Pokemon tournament program that mimicked WOTC Reporter. In my program, I was able to determine the TOP X finishers, with or without playoffs. Pro Points can be automated. It's just a matter of cost, time, and effort (and priorities).

Suggestions are "free," but improvements aren't. Pro Points would be nice, but would it really give us a different picture than ELO of who the top players are? In LOTR, there were slight differences, but differences nonetheless.

 

[Ruiner]

A few points, I guess.

I see a significant difference between drawing attention to something and what is being called "soapboxing". There is a significant difference between spreading information and alerting people to something, and trying to start a riot. One of the unfortunate side effects of this lovely "internet age" is how quickly people dismiss constructive, pulled back suggestions. A lot of times, the more passively something is mentioned, the more likely a majority of readers are to simply overlook or disregard it. Ask anyone who knows me, I do not try to provide information in person the same way I do online. What is an effective approach changes depending on the medium its presented over, and unfortunately, most information posted on message boards gets disregarded.

I understand the negative impact of someone saying " I dislike how Battle Roads are structured, don't go to them out of protest! ", but saying " There are significant problems with how Battle Roads are structured, something should be done to make them more worth attending " is still powerfully worded, but with a completely different intent in mind. It draws attention " riling people up" even. Words obviously are left up to interpretation, of course.

I'll use the current situation as an example. Lets give benefit of the doubt that the flaws in the system are known, and if a simple solution existed it would have been implemented. If enough people feel strongly about a certain issue, alternatives that exist but aren't " a simple solution" would get looked at due to demand. A larger change, such as the shift from Gym Challenges to Rating Invites, could occur. I am certainly not calling for something like that, but if a very small minority nitpick about something than it is less likely that a more difficult solution may be looked at. It that vocal minority gains enough support, different options may be looked at. I understand at the core of things, Pokemon is a business, and businesses respond better to larger portions of their demographic. Its simple public relations.

Opinion on these boards have influenced decisions in the past, and with precedent set, I don't think it is unrealistic to use these forums as a means to try and gauge the player bases' opinion.

I am fully aware that POP isn't resistant of change. In fact, I've seen a lot of positive changes over the pas 3-4 years alone. I am also very glad to see that what is being said is being so closely watched, even if it means I'm getting singled out for my words.

I am a bit unsure how a Pro Point system would be so difficult to implement. If a rag tag group of forum members are able to keep track of a majority of tournament finishes, WITHOUT any of the official data, I cannot imagine it would be excessively difficult to do so WITH that data. Especially since, at CCs, we sign forms for 1st-3rd place. Simply having PTOs turn in forms such as that and having someone manually log those isn't that difficult. Over the course of 2 months there are, max, 300 CCs that occur around the World? Probably less, maybe more ( my international knowledge is lacking here ) and with 3 age groups, thats 900 bits of data to be entered over the course of 2 months. It seems like it would require work, but a reasonable amount of it only at worst.

 

[Pete De Shaw]

I know that NoPoke has proposed a Beyesian (sp) Luck modifier to Elo several times before. Would a modifier like that work with the current system in place?

In theory, yes but not without extensive modifcation. In practice, to much effort, expense and risk for too little gain.