Philosophy of hockey sabermetrics: Can hockey accurately be measured?

[Fixed to Ruin]

What i find interesting is how do you measure players in terms of a certain quality when some players are a product of their own environment.

I like to use Renberg as an example. If you take the early part of his career he was a consistant 50 pts + player with the Flyers. Most of his success was because he played on one of the best lines in the league. Take him out of that environment and he's barely an NHL caliber player. How does one measure this? Anson Carter would be another good example.

Same with goaltenders. In gereral, a goaltender playing in a defensive system will get better numbers statistically. They will also in most cases get more shutouts than a team playing a more offensive style.

Is Mike Smith better than Bryzgalov? Did Brian Elliott become an all star caliber goaltender overnight or are they the primary benefactor in playing a system that makes life easier for them.

I find sabremetrics incredibly fascinating because I find it difficult to measure something when it seems there are endless variables.

 

[seventieslord]

Lonny Bohonos probably would have been a 50-point player playing all year with Sundin and Thomas; so yes, the argument holds that anyone within reason can play on that top line and get points and look good while getting "carried" by better players. The question you must ask yourself is, "WHY did Renberg get this opportunity while Bohonos didn't?"

 

[Fixed to Ruin]

Lonny Bohonos probably would have been a 50-point player playing all year with Sundin and Thomas; so yes, the argument holds that anyone within reason can play on that top line and get points and look good while getting "carried" by better players. The question you must ask yourself is, "WHY did Renberg get this opportunity while Bohonos didn't?"

True.

I was thinking about the goalies while eating my lunch today and i think many of the goalie stats are more or less meaningless.

I think for goalies it's a bit easier to judge them based on save percentage and saves rather than Wins, Losses, GAA. I feel that most of the goalie stats are too dependent on team performance rather than individual merits.

 

[metalfoot]

And even save percentage doesn't really address the difficulty of the save; a soft wrister into the chest = a diving post-to-post sprawl.

 

[TheDevilMadeMe]

True.

I was thinking about the goalies while eating my lunch today and i think many of the goalie stats are more or less meaningless.

I think for goalies it's a bit easier to judge them based on save percentage and saves rather than Wins, Losses, GAA. I feel that most of the goalie stats are too dependent on team performance rather than individual merits.

Goalie stats are ALL team dependent to varying degrees. Save percentage is less team dependent than others, which in theory should make it a better stat, but in reality has led to it being abused. Much of hockey analytics sees the goalie as completely seperate from the rest of the hockey team and considers save % a perfect measure of goalie and shot differential as a perfect measure of the skaters.

At least goals against average isn't nearly so prone to abuse, as quality of team is explicitly a part of the stat

 

[Fixed to Ruin]

And even save percentage doesn't really address the difficulty of the save; a soft wrister into the chest = a diving post-to-post sprawl.

I agree however I think its even more misleading to evaluate a guy like Manny Legace getting 37 wins playing on the Red Wings where he faces 30% less shots and probably 40-50% less quality scoring chances than another goalie getting only 25 wins but 30% more shots and significantly more scoring chances.

 

[Fixed to Ruin]

Goalie stats are ALL team dependent to varying degrees. Save percentage is less team dependent than others, which in theory should make it a better stat, but in reality has led to it being abused. Much of hockey analytics sees the goalie as completely seperate from the rest of the hockey team and considers save % a perfect measure of goalie and shot differential as a perfect measure of the skaters.

At least goals against average isn't nearly so prone to abuse, as quality of team is explicitly a part of the stat

After reading your post i went and looked at who was the top ten for save percentage in the NHL last year.

All of them play in what I would consider a defense first system. Thus making my first argument less valid. The only exception would be Kiprusoff. Mike Smith led starters in save percentage. Elliott is technically 1st and Halak is 6th but neither played 60 games.

I was expecting guys like Ward, Pavelec and Price to have higher save percentages than the likes of Howard and Backstrom.

Well back to the drawing board.

 

[Canadiens1958]

Post #18

No, that doesn't make any sense. Why would "sub" save percentages be subject to variance but overall save percentage not be?

You're also assuming that a season's worth of data is a meaningful unit for evaluating goaltender performance, in the "global" sense, which is not necessarily the case.

See your post #18 in this thread. the global sv% may remain fixed at 55B:45A or about but the "sub" sv%s that are the component strings of As and Bs may vary widely. Example globally sv% .920 < sv% < .925 but the component SV%s by zone, net, offensive style etc may vary greatly.

A season is just a starting reference point that is a base for a game, games, opponents, career. View it as the first bite of many.

 

[team_alex]

Baseball Sabremetrics focuses on the value of the individual, which might possibly translate to hockey, but it could be difficult to effectively model.

With hockey's complexity it may be also effective to focus on the team as a whole. As in what style, characteristics, or stats can predict success. Some proper specified OLS regressions could possibly yield some pretty interesting results.

 

[Doctor No]

See your post #18 in this thread. the global sv% may remain fixed at 55B:45A or about but the "sub" sv%s that are the component strings of As and Bs may vary widely. Example globally sv% .920 < sv% < .925 but the component SV%s by zone, net, offensive style etc may vary greatly.

In your contorted example, *why* would the global ratio remain fixed at 55B:45A?

 

[GKJ]

And even save percentage doesn't really address the difficulty of the save; a soft wrister into the chest = a diving post-to-post sprawl.

If the goaltender is making too many difficult saves, it should show up in team (SA/60) and skater stats (Corsi/Fenwick, etc.)

 

[seventieslord]

After reading your post i went and looked at who was the top ten for save percentage in the NHL last year.

All of them play in what I would consider a defense first system. Thus making my first argument less valid. The only exception would be Kiprusoff. Mike Smith led starters in save percentage. Elliott is technically 1st and Halak is 6th but neither played 60 games.

I was expecting guys like Ward, Pavelec and Price to have higher save percentages than the likes of Howard and Backstrom.

Well back to the drawing board.

One problem with that is that one season has proven to be a poor sample size with which to get a true picture of a goalie's talent level. If you look at the post-lockout sv% leaders with a GP threshold put in place, you'll like the results a lot more.

 

[metalfoot]

Right, but it doesn't show up in SPC itself. Which was my point. I love the idea of advanced statistics but there are inherent weaknesses in pretty much every statistic which currently exist.

 

[Canadiens1958]

May

In your contorted example, *why* would the global ratio remain fixed at 55B:45A?

It may remained fixed around that ratio.

 

[Doctor No]

It may remained fixed around that ratio.

"May"?

And if you flip a coin 100 times, it "may" come up heads 100 times. If I were you, I wouldn't sit around all day waiting for it to happen.

I want you to look up variance formulas for a linear combination of random variables - it's in any good stats book.

If you then come back and suggest that anyone would *expect* what you describe to happen, I'm going to assume that you're either not paying attention or have an axe to grind (in either case, I'll direct you to forum specific rule #1).

 

[seventieslord]

Right, but it doesn't show up in SPC itself. .

but it does. just maybe not season by season.

 

[GKJ]

Right, but it doesn't show up in SPC itself. Which was my point. I love the idea of advanced statistics but there are inherent weaknesses in pretty much every statistic which currently exist.

I don't think anyone argues that any stat is absolute. Just better than +/-.

 

[TheDevilMadeMe]

If the goaltender is making too many difficult saves, it should show up in team (SA/60) and skater stats (Corsi/Fenwick, etc.)

I don't understand how, since both are measure of shot quantity, not quality

 

[Iain Fyffe]

Sedins struggle to get the puck back and do a poor job of winning puck battles.

What's the relative value of puck-battle-winning to good-defensive-positioning? This is the forest for the trees problem. If you look only at a player's weaknesses, you undervalue them. It happens all the time. Martin St. Louis went undrafted not because of what he was, but he wasn't. People saw his size and wrote him off, regardless of what he could actually do on the ice.

The converse is also true. Looking only at a player's strengths overvalues them. It's a common bias of perception, and is one of the reasons that relying only on your perceptions is flawed. Similarly, if you over- or under-value certain aspects of the game, you under- or over-value players who are good/bad at those things.

People into statistical analysis are often admonished to keep it in context, that the numbers can't tell you everything. Put the same is true for those who tout watching the players. You can't see everything, and you can't always tell what something means even if you see it.

See your post #18 in this thread. the global sv% may remain fixed at 55B:45A or about but the "sub" sv%s that are the component strings of As and Bs may vary widely. Example globally sv% .920 < sv% < .925 but the component SV%s by zone, net, offensive style etc may vary greatly.

If you're trying to say that a set of data is less variable than various subsets of that same data, then yes, that's obviously true. That's sample sizes and regression to the mean at work.

Which is irrelevant, of course, unless variance is an important consideration in your analysis.

A season is just a starting reference point that is a base for a game, games, opponents, career. View it as the first bite of many.

Indeed, it's again sample size at work. But sample size is irrelevant unless variance is a significant consideration in your analysis.

The smaller the number of games, the less reliable the results are for drawing conclusions from that data. So if you try to explain why some team won a seven-game series (a tiny number of games from an analytical perspective), you need to consider that. You can try to say it's because all the centres on one team were right-handed, but demonstrating that that factor is significantly more important than simple variance due to the very small number of games is an uphill battle.

 

[seventieslord]

because all the centres on one team were right-handed,

ah, but the onus is on you to prove that wasn't the reason

 

[Master_Of_Districts]

That's something I noticed with the Sedins.

Recently saw something on Kukla's Korner saying Daniel Sedin (followed by Henrik) has the best CORSI rating in the league. CORSI is one of the more respected stats, but people were quick to point out that CORSI can be flawed when certain players get a tremendous advantage from the majority of their starts being in the offensive zone (which is true for the Sedins).

So off the bat, CORSI has to be adjusted for quality of competition and zone starts to have any meaning. And when the sample size for one kind of zone start is so small (Sedins in the defensive zone), the stat can't have any meaning anyways.

Don't kid yourself - the fact that a stat can be improved upon via adjustment doesn't render the unadjusted stat useless. The signal is merely weaker.

That's all.

 

[MadLuke]

When (If they do) every player will were wireless monitor and the puck too, maybe we will access a lot more of stats easily.

 

[The Mighty Duck Man]

I don't understand how, since both are measure of shot quantity, not quality

Exactly. A great example that I am familiar with is the San Jose-Anaheim series from 2009. Everyone raved how great Hiller was and the Sharks were badly outshooting the Ducks, but if you watched the games Hiller had it fairly easy and the Ducks were easily the better team. They just played great defense and let the Sharks take low percentage shots from the circles all game.

Scoring chances over shots is a much better indicator, but even then it's flawed. Partially because of the same problem, a semi-decent scoring chance is rated the same as a great one, but also because tallying scoring chances is hard in itself to do, as it's based on judgement(not unlike errors in baseball).

In the end, like defense in baseball, the only way to get a full grasp on it is scouting. The average Joe might be a bit off base with some of it, but someone who knows what he's looking for(when scouting actually matters) will be fine.

 

[Master_Of_Districts]

Exactly. A great example that I am familiar with is the San Jose-Anaheim series from 2009. Everyone raved how great Hiller was and the Sharks were badly outshooting the Ducks, but if you watched the games Hiller had it fairly easy and the Ducks were easily the better team. They just played great defense and let the Sharks take low percentage shots from the circles all game.

Scoring chances over shots is a much better indicator, but even then it's flawed. Partially because of the same problem, a semi-decent scoring chance is rated the same as a great one, but also because tallying scoring chances is hard in itself to do, as it's based on judgement(not unlike errors in baseball).

In the end, like defense in baseball, the only way to get a full grasp on it is scouting. The average Joe might be a bit off base with some of it, but someone who knows what he's looking for(when scouting actually matters) will be fine.

No, the Ducks weren't "easily the better team."

At best, they were even.

The Ducks had the lead for a lot of the series, and that hurts them with respect to the shot count, which invariably favors the trailing team.

But the Sharks still had more shots when the score was close. And by a fair margin too (154-122 in terms of attempted shots at EV).

If you think that the Sharks were shooting from further away, then prove it.

The data is available.

 

[The Mighty Duck Man]

No, the Ducks weren't "easily the better team."

At best, they were even.

The Ducks had the lead for a lot of the series, and that hurts them with respect to the shot count, which invariably favors the trailing team.

But the Sharks still had more shots when the score was close. And by a fair margin too (154-122 in terms of attempted shots at EV).

If you think that the Sharks were shooting from further away, then prove it.

The data is available.

Just go back and watch the games. If they weren't shooting further away, they were shooting from terrible angles. Obviously not all the time, but way too much of their offense was low-percentage stuff. Everyone praised Hiller, but if anything he let in a couple weak ones. The Sharks were absolutely pathetic that series, it was embarassing. For all the flack they get about being playoff chokers, for the most part it isn't that accurate. Except for 2009, where they fit the bill perfectly.