You are hereThe KenPom Wisconsin Problem


The KenPom Wisconsin Problem


By hagrin - Posted on 09 January 2012

Update 1/12/2011: Here's yet another explanation as to why KenPom is dealing with the Wisconsin issue and possibly how to fix it by Dan Honner.

It's early in the 2011-12 college basketball season and we already have a huge controversy on our hands! No, it's not the Xavier-Cincinnati brawl or even anything that is directly happening on the basketball court. Instead, the controversy is over Ken Pomeroy's ranking of Wisconsin #1 and #2 throughout the year even though they have already lost 5 of their first 17 games. It's been such a hot topic that KenPom has released a FAQ explaining why the system is rating Wisconsin so favorably.

I love KenPom's work. I am a paid subscriber to his site and it's the best $20 that a college hoops stat head can spend. I loved KenPom's work so much that I have built an in-house rankings system using scraped data from all over the Internet so I know exactly how hard his task is, how unforgiving the data is and how hard it is to "correct" outliers and, even more simply, "should" they be corrected. Being a computer scientist, building a ranking/predictive system really is one of the hardest things to do because you will be wrong and wrong often in a field like predicting sports outcomes and trying to objectively rank teams based solely on statistics.

All that said, the non-computer scientist in me (also known as the @GoodmanCBS side), can't seem to rationalize the current Wisconsin ranking even after the KenPom FAQ. Let's try and talk out why I have problem with my "idol's" rankings.

1. Problems at the top are significantly more magnified than problems near the bottom

I love college basketball more than 99.99999% of the humans on this planet. I'm the guy who watches every single minute of ESPN's 24/25 hours of basketball ... live. That said, even I am more interested with the Wisconsins of the world as opposed to the Towsons of the world. Let's take Towson. Towson hasn't even won a game in their last 35 games. Ouch. That said, Towson still isn't ranked at the bottom of the KenPom rankings almost definitely due to the fact that they play in the relatively strong CAA. If Towson doesn't win their next game at home vs William & Mary, they will have almost a 50/50 shot at going winless in conference play this year (currently a predicted 35.1%). Since December 30th, Towson has lost 3 games ... and moved up 3 spots in the KenPom rankings! There are actually 4 teams ranked worse than a team that has lost 35 in a row and who has a 35.1% chance of going winless in their conference.

Does anyone care? Not really. In fact, Towson's ranking could in fact be correct although part of me doesn't think it is (it's either too low or too high, but it shouldn't be 340). However, this emphasizes the point - Outliers at the top of the rankings are magnified by factors of ten.

College basketball is a sport where really the majority of people only follow the top 40 teams and maybe the hardcore fan follows the top 100-150. With only 68 teams making the tournament, fans subconsciously draw a line at those top teams and pretty much ignore the other teams. Therefore, when there is such a glaring outlier at the very top of your rankings, there is a natural reaction to invalidate everything else underneath which is why this seems to have generated so much interest.

2. The ability to win is being undervalued

KenPom is 100% right when he states "The point is, for predictive purposes, simply looking at who a team has beaten or lost to is short-sighted". However, I don't think it's as simple as winning or losing. Some teams, despite their talent, have an innate ability to win and lose games that I do think is measurable by the amount of wins and losses you have.

They say that "winning on the road in college basketball is one of the hardest things to do". Therefore, the reverse must also be true - that winning at home should be one of the easiest things to do. Wisconsin, in the past, has always played beyond their talent level at home. Bo Ryan, the Wisconsin coach, had a record of 152-11 (93.3%) at home compared to an overall record of 242-91 (72.7%). That means, doing some quick math, Wisconsin is 90-80 on the road which is basically a few games within a margin of error of being a .500 team.

With all that said, if a team already has THREE home losses and one to a vastly inferior opponent in Iowa (Wisconsin was a KenPom 98% predicted to win for that game), the ratings must adjust accordingly. Where KenPom and I diverge is that I do think that a loss at home to Iowa and three home losses already, when this is a program that has only lost 11 in 11 seasons, is in fact a predictor of future outcomes since this team has proven more often than not that it doesn't have the same ability to win.

3. KenPom is (one of) the Google of the Hoops Prediction World

Google does everything through cold, unbiased algorithm data crunching. Therefore, when you search for something on Google, you don't always get the best result first. Does this make Google worthless as a search engine? Obviously, no. The same goes for KenPom - don't invalidate all the work just because of one team. In fact, all KenPom really gives its users is just another "tool" and college hoops fans need to make sure they use the full toolset when evaluating teams. Algorithms are an ever evolving thing and I am sure KenPom, in all his brilliance, will try and tinker with his algorithm, see how those changes affect past data and we'll see an ever more complex formula.

One other note - just like the Internet and search results, college basketball is a "moving target". The sport of men's college basketball constantly evolves. The players change, the strategies change, the arenas change and sometimes even the rules change. There is no Golden Forumla to rule us all, but it can provide us a tool to help us sort through the massive amounts of data.

Bonus Note - Gamblers Rejoice

Look, if we all know KenPom is overvaluing Wisconsin, yet Vegas continues to set lines very close to KenPom's simulated predictions, there's value there so stop complaining. Use the "mistake"/outlier to your advantage and fade Wisconsin when the lines don't make sense. KenPom already stated that Wisconsin is going to stay in the top few spots all year long so handicappers will have a great opportunity to fade a disappointing Wisconsin team.

Conclusion - DON'T PANIC

With all the above being said, who is to say that Wisconsin doesn't rip off 15 wins in a row now? It's January 8th, relax. One outlier in 345 data points is pretty good and I think most of us would say that KenPom's rankings are a very useful tool. Use the information given to you to draw your own conclusions, not piggy back the conclusions of other based on very good, but incomplete information.