Philosophy of hockey sabermetrics: Can hockey accurately be measured?

[Iain Fyffe]

What are your thoughts on players being "hot" or "cold?" Is it simply a pattern that is based on random chance?

Mostly, most likely, yes. Which is to say, it may appear to be a pattern, but it's not. As I said human brains are excellent pattern-detection tools, and are in fact so good at it that sometimes they invent patterns to see where none really are.

You do obviously have to consider things like the player's situational usage, minor injuries, fatigue and quality of teammates and opponents. But events in hockey are more context-dependant than in basketball, for example. Once you get a shot off in basketball, for the most part you just have to wait and see how good your aim and execution was. Take a shot in hockey, and you have to see if it gets by any defenders first, and then of course there's the goalie, who has a significant influence on the results of most shots. The more things beyond your control involved in the results, the more of a role variation plays.

If there is something to being hot and cold, then there needs to be something. The usual explanations are things like "things are just clicking" or "nothing is going his way", which have no real explanatory power. Those are based on results, not causes. If players get hot and cold, why do they do so, and if there is something making them hot, why can't they continue that instead of eventually becoming cold?

 

[alanschu]

You do obviously have to consider things like the player's situational usage, minor injuries, fatigue and quality of teammates and opponents. But events in hockey are more context-dependant than in basketball, for example. Once you get a shot off in basketball, for the most part you just have to wait and see how good your aim and execution was. Take a shot in hockey, and you have to see if it gets by any defenders first, and then of course there's the goalie, who has a significant influence on the results of most shots. The more things beyond your control involved in the results, the more of a role variation plays.

I agree. I just used the basketball example as an illustration. Having said that, though, players are still responsible for their own actions. While any "hot/cold" effect in hockey is going to be less pronounced than in basketball, if there is any effect then it will still be there (whether or not it's significant is another story).

If there is something to being hot and cold, then there needs to be something. The usual explanations are things like "things are just clicking" or "nothing is going his way", which have no real explanatory power. Those are based on results, not causes. If players get hot and cold, why do they do so, and if there is something making them hot, why can't they continue that instead of eventually becoming cold?

I think psychology also has something to do with it, and how our mental state can affect our own physiology (for example, biofeedback shows that people can adjust their heart rate to slow down more effectively if they are actively hooked up to a heart monitor rather than not).

I think a tricky part in measuring this is that we don't hook up equipment to athletes that are "in the zone" during live play. If there's a psychological impact, experimentation becomes problematic because of an observer bias where the athlete behaves in different ways (even subconsciously) due to being in an experimental setting.

To me, it does seem clear that psychology can play a key role in athletic performance. Chuck Knoblauch was regarded as a great fielder until he inexplicably starting having troubles making throws to 1st base. He attempted many forms of treatment to try to deal with it but the situation continued to get worse. He was in a "cold slump" that he was never able to recover from.

It's harder to say if this affected the rest of his game (which began to deteriorate as well as the throwing woes continued). He was getting older (just entering his 30s), but the rate at which his throwing accuracy deteriorated places him as a definite outlier in terms of typical progression.


I think part of it lies with what things affect our focus while playing. Ironically, telling someone to focus on something often has opposite effects. For example, spend the next 5 minutes ensuring that you never think of pink elephants ever. Most people fail this test, despite the fact that they routinely go 5+ minutes without thinking about pink elephants. Focus is the same way, where people will often find themselves spending more time on making sure they don't lose focus (and hence, focusing on things that may cause them to lose focus), rather than keeping focus on the task at hand.

In my experience in competitive basketball, three times I have had the "on fire" where I made a disproportionately large number of scoring plays, and while this is just an anecdote, my recall of them was more that my mental acuity during the game seemed different, in that it felt like I was more aware of the where all the players were on the court and how both teams were reacting to the play developing.

In fact, I had that happen just yesterday in Volleyball where my teammates complimented me as I was making fantastic reads of the opposing team and attempting shots I don't typically try because there were holes in the defense. Ironically, this was against one of the better teams, and my team always seems to be one that plays to the level of our competition (we play we worse against poorer teams than we do against really good teams, even if we lose more to the better teams).


I don't think psychology of sport can be wholly discounted, and due to the complexity of measuring such things (not to mention the inconvenience of measuring it), it's not something that can be easily tested. In order to do so we'd need some sort of ability to read biometrics either without the player noticing, or without the player caring.


Until then, though, the idea is pretty unfalsifiable so I can understand your reticence to buy into it. Heck, it affects mine too!

 

[Czech Your Math]

Wilt Chamberlain scored 100 points one game, and in doing so he ended up shooting an uncharacteristic 28/32 from the free throw line. He was roughly a 50% free throw shooter (using 50% for simplicity), and it we're just using a random variable to determine the likelihood of him going 28/32 from the free throw line, then the odds of him doing that are about 0.000837% (underestimate since his FT% over his career was 51.1%)

In this case, I don't think it's as unlikely as it first appears.

Wilt shot 61.3% from the FT line that year, which increases his chances of doing that dramatically. I calculate ~3%, but may be off in that calculation. He also shot 58.2% two years earlier and 59.3% the next season, so he was better at FTs early in his career.

Also, there may be other factors at work due to the unusual circumstances. Adrenalin could be a factor, although one might not expect that to help with something like shooting FTs. Because there was constant fouling by both teams in the latter stages of the game, perhaps Wilt had more rest and could relax more at the line. Also, because he was shooting so many FGs and FTs, perhaps he got more in the rhythm.

It's not something that appears to be particularly measurable, and very likely not something that can be easily predicted, but at this point I'm not keen on outright dismissing it. Sam Gagner's 8 point night is likely something he'll never do again and the odds that he ever accomplish it in his career were already quite low, but are there personal factors from within him that may have tipped the scales rather than pure random variation?

This is much more difficult to explain. The one minor factor I notice upon cursory review is something outside Gagner's control: his opponent. Chicago was one of the weaker teams in GA, finishing 22nd, but still gave up less than 3 GA/game. However, they were going through a bad streak during which they gave up 3+ goals in 9 straight games, including 32 in 7 games, mostly during a western road trip. This was also after the all-star game, so they had only played 1 game in the previous 8 days, and could have been a bit rusty. After almost a week off, they played an OT game at Vancouver, had a day between the game in Edmonton and had a game the following night in Calgary. Still, it's hard to really explain how he scored 8 points in one game.

Sometimes athletes and artists just get in the zone. There are many anecdotes of an athlete having the flu and having a great performance, or an artist with a fever or illness writing a masterpiece. I think there's a lot of luck involved, but there are often other factors involved, which could increase the chances of such things happening. There may have been something going on with Gagner that we don't know, even something seemingly irrelevant, that at least would help to explain such a performance a little more.

 

[Czech Your Math]

To me, it does seem clear that psychology can play a key role in athletic performance. Chuck Knoblauch was regarded as a great fielder until he inexplicably starting having troubles making throws to 1st base. He attempted many forms of treatment to try to deal with it but the situation continued to get worse. He was in a "cold slump" that he was never able to recover from.

Steve Sax had the same trouble with such a seemingly simple task. It's definitely psychological in those cases.

It may have been psychological in Chamberlain's case too, but he was able to overcome that for a short period. The question may be why his FT% declined so much in later years, instead of improve with practice.

 

[Iain Fyffe]

In my experience in competitive basketball, three times I have had the "on fire" where I made a disproportionately large number of scoring plays, and while this is just an anecdote, my recall of them was more that my mental acuity during the game seemed different, in that it felt like I was more aware of the where all the players were on the court and how both teams were reacting to the play developing.

It's certainly something worth considering. Of course, if I were researching it, my question here would be: is this how you remember it, or is this how it was? To some extent at least, memory serves its user, not the truth.

 

[Iain Fyffe]

This was also after the all-star game, so they had only played 1 game in the previous 8 days, and could have been a bit rusty.

This is an issue with this kind of analysis. If the team does badly, this amount of rest means they're rusty. If they do well, it means they're well-rested and refreshed. So many of these explanations rely on the result. As a Habs fans, I hear the same thing about Carey Price all the time. When he's playing well, his calm demeanor is called an advantage - he keeps his cool and doesn't lose his head. When he's not, him calm demeanor is called a disadvantage - he needs to play with more fire to break out of his rut. The result cart is put before the explanation horse.

 

[Czech Your Math]

This is an issue with this kind of analysis. If the team does badly, this amount of rest means they're rusty. If they do well, it means they're well-rested and refreshed. So many of these explanations rely on the result. As a Habs fans, I hear the same thing about Carey Price all the time. When he's playing well, his calm demeanor is called an advantage - he keeps his cool and doesn't lose his head. When he's not, him calm demeanor is called a disadvantage - he needs to play with more fire to break out of his rut. The result cart is put before the explanation horse.

You're right, it's easy to try to explain after the fact. Still, I don't think Gagner's performance was close to being predictable or easily explainable. I was just trying to find anything out of the ordinary, and as I said it may have been a "minor factor." That game was in the middle of a 9 game losing streak by Chicago and in the middle of a time when they were giving up more goals than normal. That doesn't explain all of it, maybe just 10% or something, IDK.

In contrast, while Wilt's FT% in that game was significantly above his normal %, it wasn't nearly as improbable and more easily explainable. That was unusual game due to how close he was getting to 100 points. Also, 100 points for Wilt Chamberlain (and I know this wasn't directly used as an example of an outlier performance) could have been and actually was predicted to happen at some point (by Elgin Baylor at least).

 

[alanschu]

There may have been something going on with Gagner that we don't know, even something seemingly irrelevant, that at least would help to explain such a performance a little more.

The thing I find interesting about Gagner's game is that for many of the goals, the play seemed to appear to be more of the "high skill" variety rather than "fortunate bounce" type. Granted, there's still elements of luck even in just finding yourself open to make a nice play.

Confidence is an interesting thing, and I would suspect that part of Gagner's good fortunes came from a desperate Blackhawks team that was not playing disciplined as well. Mistakes were made and Gagner was able to capitalize.

Could just be a perfect storm, but the random chance of something like that would be truly remarkable that I have a hard time believing it was just an exceptionally rare event.

And yes you're right that Wilt was better at FT% that season (which actually makes a huge difference in the probability).

 

[Czech Your Math]

The thing I find interesting about Gagner's game is that for many of the goals, the play seemed to appear to be more of the "high skill" variety rather than "fortunate bounce" type. Granted, there's still elements of luck even in just finding yourself open to make a nice play.

Confidence is an interesting thing, and I would suspect that part of Gagner's good fortunes came from a desperate Blackhawks team that was not playing disciplined as well. Mistakes were made and Gagner was able to capitalize.

Could just be a perfect storm, but the random chance of something like that would be truly remarkable that I have a hard time believing it was just an exceptionally rare event.

And yes you're right that Wilt was better at FT% that season (which actually makes a huge difference in the probability).

To me, it's basically the random chance of any 40-50 point scorer doing something like that.

What sticks out as most unusual to me (aside from it being a relatively mediocre player) is that he scored points on all 8 of his team's goals and it was a close game until late. If it was an 11-3 blowout or something, it wouldn't be as surprising to me.

Still, it's more like a mediocre pitcher throwing a no-hitter. Great single game feat, takes some talent for sure, but not an indication of true greatness on an all-time level or anything. I'd say the same about Sittler's 10 point game, even though he was a much better player.

 

[alanschu]

Fair. It's certainly not unusual for some non-superstar player to have one of the ages.

So while the odds of Gagner specifically doing it may be rare, the odds of one of the many 40-50 point players doing it does go up significantly.

 

[Iain Fyffe]

Fair. It's certainly not unusual for some non-superstar player to have one of the ages.

So while the odds of Gagner specifically doing it may be rare, the odds of one of the many 40-50 point players doing it does go up significantly.

Precisely. Given enough observations even the most unlikely things will happen, just by chance.

 

[Hammer Time]

Precisely. Given enough observations even the most unlikely things will happen, just by chance.

And therein lies the beauty of sports. If it was completely predictable, there would be no point watching it.

 

[Dalton]

To me, it's basically the random chance of any 40-50 point scorer doing something like that.

What sticks out as most unusual to me (aside from it being a relatively mediocre player) is that he scored points on all 8 of his team's goals and it was a close game until late. If it was an 11-3 blowout or something, it wouldn't be as surprising to me.

Still, it's more like a mediocre pitcher throwing a no-hitter. Great single game feat, takes some talent for sure, but not an indication of true greatness on an all-time level or anything. I'd say the same about Sittler's 10 point game, even though he was a much better player.

Sittler also scored 5 goals in a playoff game and set a couple team records on an O6 team. Guy was pretty lucky eh?

To the OP's question. After reading and skimming this thread I think the KISS rule needs to be applied. Desperately. I don't think we can quantify human behaviour. I don't think we can safely use analytics on databases comprised of judgement calls from observers. But we can look at raw data. But I think the question should be- can we accurately analyze the data- before we ponder the value of collecting more.

I think there is too much over analysis that just paralyzes the debate. Start simple. What question are you trying to answer?

Recently I've considered the question of how to compare players across eras. I came across a study in Human Resources that showed that the top 20% or so of performers are responsible for about 80% of the output. Within that elite group the same rule applies. This elite group influence the mean disproportionately. IOW if the top 20% of year A outperform the top 20% of year B then the mean of year A will be higher than the mean of year B. The same is true of the bottom 20%.

Comparing the mean of two different seasons does not answer any questions. One must look at and compare the top and bottom 20% to see what their effect is. A year in which the worst defensive teams were worse than other years would result in more goals scored by everyone, not just the top 20%. So the top 20% would not have scored a higher proportion of goals than the rest of the players compared to other seasons but the league as a whole would have seen more goals scored. The bottom 20% of goalies would have allowed a higher percentage of goals than the remaining goalies compared to other seasons. The other goalies would be comparable to other seasons.

The same argument could be true of scorers as well. This needs to be tested and I need data to do so.

My point here is that this idea came from Human resources not math. Math alone is not the answer here. There are many fields that study human preformance that can be mined for ideas to analyze hockey. The math comes into play when it presents as the tool for the job. In this case the necessary math is just plain old fractions.

I dl'd last season and found that this rule appeared to hold for points. The top twenty percent had indeed sored a disproportionate amount of the points. It was pointed out to me that this was because of the distribution of scorers on a the teams but that doesn't impact it's possibilities for comparing seasons. It could in fact support that poster's opinion if perhaps scoring is more spread out in a given season or era compared to another. In that case the top 20% wouldn't have scored a disproportionate amount of points compared to the rest of the league as they did last season. Or at least the proportions would differ in a manner supporting the contention.

Just for kicks I added a typical Gretzky season of more assists than the next guys points and 55 goals IIRC. There was a pretty clear lead for Gretzky among the top 20%. It would be interesting to use this to compare other top performers and perhaps see who scored the highest proportion of points or goals among their peers in given seasons, eras or careers. I think it's a useful tool that coud contribute to many debates and even displace some of the dubious formulae that litter the debate.

In a small league the number of players in the top 20% would be smaller as well which could see some aparently bizarre results such as a single player or a very few responsible for all or most of the goals scored in that league. In that case (or maybe all cases) one should use an arbitray number such as what percentage of league scoring does a ppg represent. A baseline to compare seasons and performances. Every ruler needs a 0.

Without data it's just MHO. But OTOH- KISS. HR study. Raw data. Simple fractions. Higher order math need not apply.

The measurement question is simply a matter of careful definition of events and counting them. I think we need to analyze current data better first. That will lead to questions which should lead to more or better data collection for analysis. Just applying math to madly collected data is not the best analysis.

Ask questions. Research. The answer to the OP's query will follow in time.

 

[JoelWarlord]

I find the statistical analysis of hockey fascinating. I think we're at a rudimentary level of understanding at this point. Corsi is IMO the best overall stat we have right now, it works very logically. Shots directed at the net are a good measure of puck possession, if you're directing the puck on net you have possession of the puck. Stats like PDO are a good way of looking into seasons that seem a little out of the ordinary, it also corresponds roughly with perceived quality of players, stars generally have a PDO of slightly above 1000, and less talented guys will regress closer to 1000 or slightly under. I also really like the Quality of Competition metric, it's useful when comparing players on the same team. As a Canadiens fan, it makes Subban's progression even more impressive, he faced very tough competition last year (although some would pin it on Gorges, I think both players contribute to that pairing's effectiveness pretty equally).

They're not be all and end all measurements either, but they provide a good rough outline of players. For example, Gomez's Corsi stats are still quite high, and it would be silly to assume that he should displace other forwards just on the basis of his Corsi numbers. It's a lot like the way we measure economics, it's impossible to completely quantify, but that doesn't mean it doesn't provide a good outlook and a way to measure a team or player's effectiveness. I think there has to be a middle ground, some guys almost exclusively look at advanced stats, and some guys completely throw them out. Most of the advanced metrics have a very logical background, but they can't quantify anything.

Hockey's a very fluid game, and it's not possible to entirely quantify it, but I really feel like we're just scratching the surface of this kind of analysis. The turning point will be when NHL teams start devoting resources to "moneypuck" staff. I think eventually there will be a stathead employed in every team's front office, and the possibilities are incredible. It's going to be cool to watch the field develop in the next few years.

 

[Dalton]

That's demonstrably not true. One only has to look at a binomial distribution to refute your point.

Can you explain this clearly or at least point to a link. This kind of comment can be a thread killer for fans wanting to contribute.

I'll throw this in. http://en.wikipedia.org/wiki/Binomial_distribution as an example.

IIRC Gauss was the last renaisance man and Poincare was the last math renaissance man.

 

[Gobias Industries]

Hockey's a very fluid game, and it's not possible to entirely quantify it, but I really feel like we're just scratching the surface of this kind of analysis. The turning point will be when NHL teams start devoting resources to "moneypuck" staff. I think eventually there will be a stathead employed in every team's front office, and the possibilities are incredible. It's going to be cool to watch the field develop in the next few years.

I still think at the end of the day, the fluidity you mentioned will mean data is next to impossible to acquire on a broad scale. That "stathead" is more likely to be a slave watching a lot of hockey to get the data he needs as opposed to spending time breaking down statistics.

If there was ever a sport where I would value opinion over statistics, it would be hockey.

 

[Feed Me A Stray Cat]

I still think at the end of the day, the fluidity you mentioned will mean data is next to impossible to acquire on a broad scale. That "stathead" is more likely to be a slave watching a lot of hockey to get the data he needs as opposed to spending time breaking down statistics.

If there was ever a sport where I would value opinion over statistics, it would be hockey.

Once the NHL starts tracking puck possession, puck touches, carries, passes complete, pass distance, etc., we will see big progress.

 

[Gobias Industries]

Once the NHL starts tracking puck possession, puck touches, carries, passes complete, pass distance, etc., we will see big progress.

Yeah, I look forward to those statistics as well.

Still a ton of work going into those statistics, I don't envy the people mining that data.

 

[TieClark]

I'm not a big believer in statistics for any sport but that is especially true for hockey. There are so many intangible variables in hockey that cannot be measured with numbers

 

[Micklebot]

Once the NHL starts tracking puck possession, puck touches, carries, passes complete, pass distance, etc., we will see big progress.

If only we still had the Fox Puck...

 

[kmad]

Seems that Brian Burke's answer is "no".

 

[LuGBuG]

Advanced Stats Debates

As much as I don't think they tell the entire story of the game and a lot of the time they are bull crap, that's just my opinion.

I will say though that i find these debates interesting with all the different stats thrown around and thought this could be a thread where we debate all these "advanced" stats.

Or maybe no one cares and this thread will just disappear or be locked.

Basically if you can't take the debates about your team suggested as "no good" because of these stats, and are just going to attack people then don't bother staying.

If you're going to add to a productive debate then it should be interesting.


I would like to see one of those charts from the Leafs Wild debate posted with just this years teams thus far? Anyone have that?

 

[Christian Litscher]

Mean nothing. For people who can't WATCH the games and know what's happening.

 

[Legionnaire]

Title should have been. "Yo! Statgeeks!"

 

[kmad]

Advanced stats are useful to illuminate trends, but they don't tell the whole picture.

I think the most useful statistical developments are going to be in puck possession. Stats can help identify players that contribute to continued puck possession, and those players can be targeted by GMs. I guarantee this is why the Canucks went after Derek Roy, which has been paying off greatly.

Still, every stat needs the eye test. The game of hockey is too fluid - nothing can be measured accurately by just a number.