Philosophy of hockey sabermetrics: Can hockey accurately be measured?

[Doctor No]

For variance to be a factor it would have to be shown that from season to season a goalies global SV% is relatively constant globally, but the component sub SV%s vary greatly. His performance on breakaways may vary by 25-30% from season to season, by 40% on shots from the LW(strong one/weak the next) vs RW(weak one/strong the next),etc.

That's demonstrably not true. One only has to look at a binomial distribution to refute your point.

 

[Iain Fyffe]

For variance to be a factor it would have to be shown that from season to season a goalies global SV% is relatively constant globally, but the component sub SV%s vary greatly. His performance on breakaways may vary by 25-30% from season to season, by 40% on shots from the LW(strong one/weak the next) vs RW(weak one/strong the next),etc.

No, that doesn't make any sense. Why would "sub" save percentages be subject to variance but overall save percentage not be?

You're also assuming that a season's worth of data is a meaningful unit for evaluating goaltender performance, in the "global" sense, which is not necessarily the case.

 

[metalfoot]

I guess one of the fundamental questions here is what sample size in hockey is sufficient to develop statistics with adequate confidence levels? Is a season enough? a career? how long a career? Player A only played 180 games, but Player B played 290 so his numbers work for projection and extrapolation?

 

[Hardyvan123]

I'll toss up a weak first opinion, to echo what I said in the Introduce Yourself thread:

I don't think it can. It's not segmented like baseball or football. It's too fluid, too complicated, and there are far too many variables to consider. For each statistic shown, it has to be given at least two layers of context to have any meaning. Usually far more.

Thoughts?

I agree it's not as easy to isolate stats, and defer meaning, in hockey as it is in baseball or to a lesser extent football. there is more segmentation and isolation of players and outcomes in baseball and football than hockey.

My answer to those who think that it can't be: you're trying to measure too much all at once (or equivalently, you're trying to take too large of a bite).

Start by measuring things that you can measure.

Or work on measuring something better - even in baseball, you can't get a perfectly isolated statistic.

Perfection is the enemy of good. If you know that you can't achieve "perfection", and that stops you from aiming for "good", then we lose in the process.

I agree with what you are saying here but stats are more relevant in baseball, then football, then hockey IMO due to what was said above.

It's an area that should be pursued but in context.

Don't you think some players start to care and try harder in the playoffs? Not really 'clutchness', but it can't be attributed to randomness either.

some can and other players games are more suited to the playoffs when they is more emphasis on intensity and physical play and intimidation and more often than not they "let more go" in the playoffs than regular season which benefits certain players more than others.

Also the regular season is more of a grind with 82 games, the playoffs have an end in sight and more at stake with a maximum of only 16-24 games.

 

[Hardyvan123]

I guess one of the fundamental questions here is what sample size in hockey is sufficient to develop statistics with adequate confidence levels? Is a season enough? a career? how long a career? Player A only played 180 games, but Player B played 290 so his numbers work for projection and extrapolation?

I'm not a stats expert but the larger the size the less likelihood for variance makes sense.

 

[Doctor No]

I agree it's not as easy to isolate stats, and defer meaning, in hockey as it is in baseball or to a lesser extent football. there is more segmentation and isolation of players and outcomes in baseball and football than hockey.

True. But also keep in mind that where we currently are with hockey metrics, baseball was behind that in the early 1980s.

I have a friend who, when we're climbing mountains, likes to ask "How do you eat an elephant?". Answer: "One bite at a time". If we try to eat the entire elephant all at once, we're never going to get anywhere.

 

[metalfoot]

I'm not a stats expert but the larger the size the less likelihood for variance makes sense.

But the player (health, etc) is himself a constantly changing variable? So player A at 22 is not the same as player A at 32, you know?

 

[cptjeff]

I guess one of the fundamental questions here is what sample size in hockey is sufficient to develop statistics with adequate confidence levels? Is a season enough? a career? how long a career? Player A only played 180 games, but Player B played 290 so his numbers work for projection and extrapolation?

It depends. For example, in baseball, the various stats on range aren't considered to be particularly accurate below 3 seasons worth of data, but a batting average stabilizes to something usable after after a week or two's worth of game. It depends on how many data points you have, and how wide the variation is within your population. When you're performing operations with any number with uncertainty attached, the error multiplies.

The way statistical analysis works though, once you're above a threshold, how many points don't matter. Going from 95% certainty to 99% isn't a big jump, but going from 66% to 95%. If I'm remembering my bell curves right, 1 sigma gets you the 66%, 2 sigma gets you the 95%, and three the 99%. The returns you get in increased accuracy from adding new data diminish pretty rapidly, so that if 1 sigma turns out to be 150 games or so, player A's number won't be reliable while B's will. If it turns out to be 50, both will be equally valid.

 

[metalfoot]

That's kind of what I remember, too, but my course in experimental design keeps flashing warning signs in my head with as many variables as hockey seemingly has. Maybe I'm just too much of a hockey mystic to see its discrete testable variables clearly.

 

[BoHorvatFan]

A lot of these hockey stats come down to coaching. One coach may give player A more offensive zone starts than player B even though player B is better.

Also, players numbers in these made up stats will be different on different teams with different line mates in different systems.

 

[cptjeff]

That's kind of what I remember, too, but my course in experimental design keeps flashing warning signs in my head with as many variables as hockey seemingly has. Maybe I'm just too much of a hockey mystic to see its discrete testable variables clearly.

That's my biggest concern. So much of how hockey is played simply can't be isolated without massive investment in time and metrics by a very large, dedicated staff. And for what? Increasingly dubious metrics? I really think watching the game with an educated eye is gonna give you a much better result.

 

[unknown33]

Maybe, but until someone can demonstrate that that is true, rather than simply assert that it true, it shouldn't be part of the analysis. All we know for a fact is that the same player plays in the regular season and in the playoffs. If you want to assert that there is something inherently and predictably different about this player's ability to perform based on what time of year that is, you need to provide evidence. Otherwise you fail to show anything that cannot be explained by normal variance in game-to-game performance.

Normal variance in game-to-game performance is often not caused by randomness.

Claude Lemieux scored .34 goals per game in the playoffs, and .31 in the regular season. He had some big playoff years, but also years where he scored 1 in 11, 3 in 19 and 4 in 23. When looking at players with clutch reputations, proponents tend to focus on the data that supports the assertion while downplaying that which does not. It's classic confirmation bias: I believe that clutch play exists, therefore I will find and present data to support it. It's starting with the conclusion and then looking for supporting evidence. You need to look at the evidence first, and derive your conclusion from that.

I'm not talking about clutchness.

And then, moving beyond all that, is the underlying value judgement. If certain players try harder in the playoffs, that means they are not giving 100% in the regular season. Yet they are held up as heroes, while those who give the same effort year-round are not.

That was my point.

 

[kmad]

A lot of these hockey stats come down to coaching. One coach may give player A more offensive zone starts than player B even though player B is better.

That's something I noticed with the Sedins.

Recently saw something on Kukla's Korner saying Daniel Sedin (followed by Henrik) has the best CORSI rating in the league. CORSI is one of the more respected stats, but people were quick to point out that CORSI can be flawed when certain players get a tremendous advantage from the majority of their starts being in the offensive zone (which is true for the Sedins).

So off the bat, CORSI has to be adjusted for quality of competition and zone starts to have any meaning. And when the sample size for one kind of zone start is so small (Sedins in the defensive zone), the stat can't have any meaning anyways.

 

[BoHorvatFan]

That's something I noticed with the Sedins.

Recently saw something on Kukla's Korner saying Daniel Sedin (followed by Henrik) has the best CORSI rating in the league. CORSI is one of the more respected stats, but people were quick to point out that CORSI can be flawed when certain players get a tremendous advantage from the majority of their starts being in the offensive zone (which is true for the Sedins).

So off the bat, CORSI has to be adjusted for quality of competition and zone starts to have any meaning. And when the sample size for one kind of zone start is so small (Sedins in the defensive zone), the stat can't have any meaning anyways.

Exactly, Sedins never take defensive zone face-offs so obviously they will spend more time in the offensive zone, less shots against, and so on.

They are actually poor defensive players IMO, some other Canucks fans may not agree but I find them weak in their own zone. Don't need a stat to tell me. I mean Im all for people trying to find useful stats, it is interesting and useful in some sports, but I think the best way to evaluate hockey players is to watch them not look for a % or number to describe their play.

 

[Hardyvan123]

But the player (health, etc) is himself a constantly changing variable? So player A at 22 is not the same as player A at 32, you know?

Your example gave two players with 180 and 290 games, without other evidence they were treated as over the same seasons or time periods.

That's why more information and larger sample sizes give better results.

 

[Hardyvan123]

True. But also keep in mind that where we currently are with hockey metrics, baseball was behind that in the early 1980s.

I have a friend who, when we're climbing mountains, likes to ask "How do you eat an elephant?". Answer: "One bite at a time". If we try to eat the entire elephant all at once, we're never going to get anywhere.

Yes point taken but would you not agree that we can isolate events in baseball better than in hockey and there is less flow and offense and defense take place at the same time in hockey when in baseball these things are segregated.

One can look at two mountains and see that one is easier to climb than the other in most cases as well even if we know little about mountain climbing.

If nothing else the pursuit of advanced stats causes us to ask even more questions and try to gain a better understanding and this is good in of itself IMO.

 

[Krnuckfan]

Exactly, Sedins never take defensive zone face-offs so obviously they will spend more time in the offensive zone, less shots against, and so on.

They are actually poor defensive players IMO, some other Canucks fans may not agree but I find them weak in their own zone. Don't need a stat to tell me. I mean Im all for people trying to find useful stats, it is interesting and useful in some sports, but I think the best way to evaluate hockey players is to watch them not look for a % or number to describe their play.

Strongly disagree with the bolded. I believe your average hockey viewer does a poor job of evaluating defensive play in hockey. Fans will tend to notice the big mistakes, like a dman blowing his coverage and ending up in no mans land while an opposing player is left alone in the slot, or a bad pinch leading to a odd man rush. However, many won't be able to notice things like solid positioning, covering passing lanes, using his stick effectively and other subtleties of defence.

The Sedins are not bad defensively. They may seem like they are bad because they are average skaters and do not play aggressively, but they are almost always in position and definitely pull their weight defensively.

 

[BoHorvatFan]

Strongly disagree with the bolded. I believe your average hockey viewer does a poor job of evaluating defensive play in hockey. Fans will tend to notice the big mistakes, like a dman blowing his coverage and ending up in no mans land while an opposing player is left alone in the slot, or a bad pinch leading to a odd man rush. However, many won't be able to notice things like solid positioning, covering passing lanes, using his stick effectively and other subtleties of defence.

The Sedins are not bad defensively. They may seem like they are bad because they are average skaters and do not play aggressively, but they are almost always in position and definitely pull their weight defensively.

Sedins struggle to get the puck back and do a poor job of winning puck battles. Positioning is fine but if they're just standing there watching it really isn't good enough. Good defensive players are in the right position and separate the opponent from the puck. A line of 5 Sedins would be hemmed in it's zone until the goalie froze it or the other team scored.

They play with burrows who's great defensively and always play with a top defensive pairing. They watch them do the work in the d-zone.

 

[ssh]

Exactly, Sedins never take defensive zone face-offs so obviously they will spend more time in the offensive zone, less shots against, and so on.

That's obviously an exaggeration though not by much. The Canucks have been micromanaging zone starts much more than any other team.

There seems to be a gradual shift going on from line matching towards optimizing zone starts and this is most certainly due to advances in statistical analysis.

 

[Canadiens1958]

Not Recent

That's obviously an exaggeration though not by much. The Canucks have been micromanaging zone starts much more than any other team.

There seems to be a gradual shift going on from line matching towards optimizing zone starts and this is most certainly due to advances in statistical analysis.

Started in the fifties. Close game and the team's best defensive center/line would take the defensive zone faceoffs.

 

[ssh]

Started in the fifties. Close game and the team's best defensive center/line would take the defensive zone faceoffs.

What the Canucks in particular have been doing recently is optimizing zone starts the whole game, every game.


Edit: Was this something that was not done before the 50's and was the tactic pioneered by a specific team/coach or was it something that gradually became a part of normal procedure.

In the first decade after the 67 expansion some teams seemingly low on talent had defensive specialists playing almost all the PK time and a group of offensive players (5 forwards in some cases) playing the whole PPs. Were there teams that tried to optimize zone starts based on players' perceived abilities on ES as well?

 

[Doctor No]

Don't need a stat to tell me. I mean Im all for people trying to find useful stats, it is interesting and useful in some sports, but I think the best way to evaluate hockey players is to watch them not look for a % or number to describe their play.

How do you account for the selection biases inherent with (1) you not being able to see all of a player's data yourself, (2) you not being able to see all of a player's colleagues' data to get the benchmark right, and (3) your brain being biased towards what it remembers?

These are major problems with your proposed method.

(For what it's worth, no one's suggesting the elimination of watching players play.)

 

[overpass]

Strongly disagree with the bolded. I believe your average hockey viewer does a poor job of evaluating defensive play in hockey. Fans will tend to notice the big mistakes, like a dman blowing his coverage and ending up in no mans land while an opposing player is left alone in the slot, or a bad pinch leading to a odd man rush. However, many won't be able to notice things like solid positioning, covering passing lanes, using his stick effectively and other subtleties of defence.

The Sedins are not bad defensively. They may seem like they are bad because they are average skaters and do not play aggressively, but they are almost always in position and definitely pull their weight defensively.

You've only addressed the average hockey viewer, probably watching the game on TV. What about experienced hockey viewers watching the game at the rink? They'll probably be able to give a pretty good evaluation of a player.

 

[Canadiens1958]

Zone Starts

What the Canucks in particular have been doing recently is optimizing zone starts the whole game, every game.


Edit: Was this something that was not done before the 50's and was the tactic pioneered by a specific team/coach or was it something that gradually became a part of normal procedure.

In the first decade after the 67 expansion some teams seemingly low on talent had defensive specialists playing almost all the PK time and a group of offensive players (5 forwards in some cases) playing the whole PPs. Were there teams that tried to optimize zone starts based on players' perceived abilities on ES as well?

Zone starts are a function of standardizing where face-offs are held - designated closest face-off spot and increased game skater rosters combined with rules dictating when players may or may not be changed.

The importance of face-offs evolved as did the importance of the PP/PK after the introduction of the Red Line at the start of the 1943-44 season combine with the increase in game skater roster size between the 50 and 70 game schedule. As is the case today or the post 1967 expansion era all teams nuance this within the rules and their available talent.

Toe Blake was probably the most influential coach in this area since he used the SH situation to rest his star forwards while getting line match-up advantages post SH situation. Also his positioning of Doug Harvey on defensive zone face-offs combined with centers used for the draw was quite advanced. Likewise the positioning of Doug Harvey on the PP - left point as opposed to Harvey's usual RD position advanced how dmen were used on the PP.

 

[seventieslord]

Intangibles, clutchness, will to win, these are not things that can be statistically defined. Often the players who become heroes in the playoffs do so precisely because there's no reason to think based on their stats and their career trajectories that they will come through in a huge moment.

Would statistical analysis tell you that Dustin Penner would be one of the most key guys in the playoffs this season? The stats would probably tell you Penner sucks. But he came through when it mattered. What about a Max Talbot, or a Sean Bergenheim, or a Claude Lemieux, or a Bill Barilko, guys like that are legends (not Bergenheim but he totally came out of nowhere that one year, you know what I mean)... precisely because the immense value of their contributions was surprising and unexpected, because their lackluster overall stats didn't matter and they stepped it up when it really mattered. That's a part of hockey, it's intangible and it transcends "Sabremetric" statistical analysis, it exists on another level, another plane.

or it's random.