Do 'Expected' goals statistics suck?

  • Xenforo Cloud has upgraded us to version 2.3.6. Please report any issues you experience.
  • We are currently aware of "log in/security error" issues that are affecting some users. We apologize and ask for your patience as we try to get these issues fixed.
I can only speak to what these two guys had to say.

I will say the they didn't feel that analytics were very useful, at all, for football. They never mentioned basketball at all.

It was 5 years ago so they may have changed their opinion. NFL coaches pretty much call up the analytical team every time there is a 4th down to see what their model suggests.

I was thinking about what I said and I came to a conclusion that there just isn't enough money in the NHL. There are competitive advantages to gain from statistics but they usually come in small increments. Is it really worth the costs of keeping a statistics team for a couple wins, when those wins don't bring in nearly as much financially as the other leagues?
 
It's not about juicing numbers. It's about assigning a blanket value to a shooting event almost entirely based on from where the shooting event took place, and then deciding you can conclude how many goals you are expected to score based on that data. It's true that a 4th liner doesn't often get that many rebound chances but it does happen, and not all that rarely either. You can't dismiss these occurrences as not statistically significant and at the same time accept xG conclusions at their face value.

Expected goals, at least the publicly available models, don't sufficiently account for where the other players are on the ice. How one can possibly then call it a comprehensive appraisal of what has happened then?



View attachment 990987

So even more than 10x what an Ovechkin one timer from his office at the top of the circle is worth in the model then. Totally flawless definitely.


Could really do without the condescension of "well you must not understand it if you think it's bad". I understand the model just fine and it's because I understand it I don't think it's worth as much as many of the advanced stat zealots do. The entire premise is flawed in its current form and paints a woefully incomplete picture of what occurred during a game.

You're telling me you are applying shooter-dependent and goalie-dependant modifiers for every expected goals-value you come across? Is that what you're doing? Come on. Frankly ridiculous is right. That's no longer a stat that is useful if you have to do a bunch of stuff to it after it's been recorded to make it meaningful. That's not rocket science, merely common sense.



Well if we're accepting a baseline blanket 'expected goals' stat applicable to all players and all situations it really shouldn't matter who is taking the shot should it? But it does. I never argued the the elite players aren't generating the most volume either.


Since you all took issue with my initial scenario lets explore a couple more for how xG is calculated. These are common occurrences in NHL games.
*I know different sites use differing models and apply different filters and gradients etc but they are all at their basic form built upon something similar to the posted diagram above.*

1a) Kevin Rooney starts his shift in the DZ. Loses the faceoff and spends the next 73 seconds along with his linemates chasing the puck around under pressure from his opponents. His goalie bails him out several times. Finally the opponent's fwd line tire and go for a line change leaving Rooney's defenseman in the corner with the puck for an easy breakout with lots of open ice. He passes to Rooney while the rest of his line changes. Rooney, being the responsible and boiler plate nothing NHL player he is knows he must get the puck deep to allow for the next line to establish themselves on the ice. He takes the open ice and crosses the blueline with both opponent defenders in front of him. Rather than just dumping it in the corner Kev decides he should probably get at least one SoG this week so he wrists a 72mph shot at the goalie from just inside the blueline. Unless he shot some heroin between periods the goalie gloves it easily and sets the table for the weakside D to start a breakout.

This is deemed worthy of 0.03 xGs.

1b) Cale Makar starts his shift in the OZ. MacKinnon wins the faceoff and the winger gets it back to Makar. Makar controls the puck and fakes sending it back down low to open up the middle of the ice. He walks the blueline as layers of traffic gather in front of his opponent's net. The goalie can't see the puck through the traffic and stands taller in his perimeter stance to try to find it. Makar sees this and sees an opening through the moving bodies. With no windup he wrists a 72mph shot from just inside the blueline. The shot is well placed and he scores under the goalie's glove. Goalie never saw it and curses his winger for missing his assignment and not blocking the point shot.

This is deemed worthy of 0.03 xGs.

2a) Ryan Lomberg takes a pass with speed in the NZ after some nice work by his other winger to win a high board battle. He has some room so he takes the defender wide and executes some respectable net drive by keeping his feet moving. He's by himself as his linemates are busy picking fleas off each other by the bench. He takes a contested shot along the ice from in tight to the net, the goalie has the angle covered easily and the shot bounces harmlessly to the corner.

This is deemed worthy of 0.3 xGs.

2b) Patrick Kane vacating his defensive responsibilities as he senses the puck is about to turn over creeps past the mouth breathing weakside defenseman still standing on the blueline trying to keep the pressure on. Sure enough Kane's center wins the puck along the boards and lifts a nice outlet area pass to center ice. Kane grabs the puck and is in alone on the goalie. He fakes going high glove and freezes our poor tender before making a move to the backhand and then crossbar down roofs it.

This is deemed worthy of 0.3 xGs.


Go ahead and handwave away these differences if you want but to me they are not reconcilable. It's not that xG models tell us nothing or aren't worth considering, it's that they are incomplete and aren't actually providing what they are claiming to. Over season long samples they can definitely be useful and predictive but stuff like the deserve-to-win'o'meter or NST's charting for individual games just isn't that useful without the context of seeing the game happen. It's not one or the other eye test vs advanced stats, it should be considering both as parts of an overall picture.
The fundamental problem with any shot attempt based stat is that while they may be decent approximations for an entire population (all NHL players) over large samples, they are not great metrics for analyzing individuals in that population
 
Is this the same Moneypuck that put out their power rankings yesterday with Rangers ranked 7th and Blue Jackets ranked 28th in the league??? 😂😂😂😂




That's actually hillarious as a Ranger fan but to be fair it's not based on xG.

If you read what those rankins are based on it's basically based on Regulation Wins, Goaltending (Rangers usually get good goaltending) and scoring chances. Rangers are up there in Regulation Wins (largely because of goaltending).
 
  • Like
Reactions: Bear of Bad News
As far as why it's valued, good xGF% teams win the Cup a lot more than teams that aren't. And now we're seeing teams like the Panthers, who were positing elite xG metrics years before they were good. The predictive value is proven.

The models are not designed on a whim nor are they designed with lack of theory on how goals are scored. Despite the internet meme of a bunch of nerds who have never seen hockey, a vast amount of hockey knowledge goes into creating models and tracking events.

There's this character in Moneyball, Pete. Pete is the meme. He's this out of shape nerd from some fancy college and it's heavily implied he knows nothing about the game. That's not reality. The real guy that Pete is based on was in his second front office at that point (previously worked for the Cleveland Indians) and played baseball at a high level.

He did go to Harvard for economics, that part is true. The part they leave out is that he played for Harvard's baseball team and football team. That doesn't fit the nerd meme.

The analytics people care about hockey a lot and know a ton about it.
I'm not sure that the Panthers having good xGF before their team was good is making the argument you think.
 
I believe expected goals as a predictor for goals are like a 0.36 for correlation.

They're based on historical data, so it means that shots with the same parameters as that one have historically ended up becoming a goal around 16% of the time.

It's a statistical model, they don't actually look at the videos, they just go by the event data.

Also, here the goalie didn't play it correctly at all. If he had moved even a little bit, it wouldn't have looked nearly as free as it was.

I'm sorry, wat?! The goalie is the last guy to put any blame on that sequence. He had to square up on and respect Werenski. Danforth was never really open until the last second when he popped out from net front to back post.

Goalie had no chance. Entirely on the dreadful Ranger defense.
 
I think my main problem with xGoals and other advanced stats is people's rampant misinterpretation of the data.

Then again, it's kind of understandable how people would misinterpret that due to the name.

Like Tesla's "autopilot" (which isn't actually capable of being an autopilot), xGoals isn't actually comprehensive as to what should be expected. Shooting talent is repetitive enough to become reasonably expected, except xGoals is agnostic to shooting talent despite shooting talent being a demonstrable thing.
 
Last edited:
  • Like
Reactions: Summer Rose
I feel like everytime I go under the hood and really look at them, they just do not represent reality.

For example, Moneypuck had this Justin Danforth goal as .16 XG, which to my understanding is them saying this goal should only go in 16/100 times on an average NHL goalie (please correct me if I am wrong). IMO, it's closer to 96/100 than it is 16.




Idk who watched the Rangers/Jackets tonight but it's absolutely laughable that these stats 'say' the Rangers should have won that game most of the time. They were absolutely dreadful defensively and in pretty much all phases of the game.

I feel like I laugh my ass off at the 'Win O Meter' more than half the time they get posted in GDTs/PGT's (again as being wrong and unreflective of reality)
LIrWIuE.png



Help me. Am I misinterpreting the information? Should these stats and source (Moneypuck) be thrown out the window? Is there a better source (where you can actually pinpoint specific shots/goals and see their 'expectedness')

Depends on if you understand the question it answers.

The stat essentially indicates the effectiveness of the systems. That’s it. Like all stats it doesn’t stand on its own, it requires further knowledge, context and nuance in the analysis. Teams have used versions of this stat for decades, it’s not new. What has changed is that it’s now available to fans who most use the information incorrectly.
 
It's pointless when you pin your hopes and dreams on analysis like this. These guys are expected to put the biscuit in the basket. The fact they almost did or should have doesn't mean anything.
 
  • Like
Reactions: VivaLasVegas
People are also quite good at identifying boundary cases with these sorts of metrics (and yes, Chris Dingman taking a shot from the slot won't have the same chance of scoring as Alexander Ovechkin).

Yet they seem to ignore (or say "it doesn't matter") with boundary cases - of which there are far more - on mainstream statistics.

(For evidence, see the "we can assume all shots are of the same quality" discussion earlier in this thread.)
Yup.

Good thing I can look at ovis career and see he consistently shoots above expectation.

In fact, moneypuck, as meh as it is in terms of "advanced numbers" (which really aren't that advanced tbh), has a stat for this. It's called shooting talent adjusted expected goals and uses individual player data to adjust for this exact complaint.

You can do the same with pretty much any player above 30.


The real things that it's tough for is young guys and not having the sample size to figure out if this guy shooting at a ridiculously high or low rate is lucky/unlucky, or it simply is who is as a player. Finding that out just requires more data and time tho
 
It's pointless when you pin your hopes and dreams on analysis like this. These guys are expected to put the biscuit in the basket. The fact they almost did or should have doesn't mean anything.
It means a tonne if you are running an NHL team.

If your front office can't recognize that Hoglander wasn't going to come close to repeating his previous season and you end up with a poor contract, perhaps your management needs to be looking at the numbers a little deeper.
 
  • Like
Reactions: VivaLasVegas
It's not about juicing numbers. It's about assigning a blanket value to a shooting event almost entirely based on from where the shooting event took place, and then deciding you can conclude how many goals you are expected to score based on that data. It's true that a 4th liner doesn't often get that many rebound chances but it does happen, and not all that rarely either. You can't dismiss these occurrences as not statistically significant and at the same time accept xG conclusions at their face value.

Expected goals, at least the publicly available models, don't sufficiently account for where the other players are on the ice. How one can possibly then call it a comprehensive appraisal of what has happened then?



View attachment 990987

So even more than 10x what an Ovechkin one timer from his office at the top of the circle is worth in the model then. Totally flawless definitely.


Could really do without the condescension of "well you must not understand it if you think it's bad". I understand the model just fine and it's because I understand it I don't think it's worth as much as many of the advanced stat zealots do. The entire premise is flawed in its current form and paints a woefully incomplete picture of what occurred during a game.

You're telling me you are applying shooter-dependent and goalie-dependant modifiers for every expected goals-value you come across? Is that what you're doing? Come on. Frankly ridiculous is right. That's no longer a stat that is useful if you have to do a bunch of stuff to it after it's been recorded to make it meaningful. That's not rocket science, merely common sense.



Well if we're accepting a baseline blanket 'expected goals' stat applicable to all players and all situations it really shouldn't matter who is taking the shot should it? But it does. I never argued the the elite players aren't generating the most volume either.


Since you all took issue with my initial scenario lets explore a couple more for how xG is calculated. These are common occurrences in NHL games.
*I know different sites use differing models and apply different filters and gradients etc but they are all at their basic form built upon something similar to the posted diagram above.*

1a) Kevin Rooney starts his shift in the DZ. Loses the faceoff and spends the next 73 seconds along with his linemates chasing the puck around under pressure from his opponents. His goalie bails him out several times. Finally the opponent's fwd line tire and go for a line change leaving Rooney's defenseman in the corner with the puck for an easy breakout with lots of open ice. He passes to Rooney while the rest of his line changes. Rooney, being the responsible and boiler plate nothing NHL player he is knows he must get the puck deep to allow for the next line to establish themselves on the ice. He takes the open ice and crosses the blueline with both opponent defenders in front of him. Rather than just dumping it in the corner Kev decides he should probably get at least one SoG this week so he wrists a 72mph shot at the goalie from just inside the blueline. Unless he shot some heroin between periods the goalie gloves it easily and sets the table for the weakside D to start a breakout.

This is deemed worthy of 0.03 xGs.

1b) Cale Makar starts his shift in the OZ. MacKinnon wins the faceoff and the winger gets it back to Makar. Makar controls the puck and fakes sending it back down low to open up the middle of the ice. He walks the blueline as layers of traffic gather in front of his opponent's net. The goalie can't see the puck through the traffic and stands taller in his perimeter stance to try to find it. Makar sees this and sees an opening through the moving bodies. With no windup he wrists a 72mph shot from just inside the blueline. The shot is well placed and he scores under the goalie's glove. Goalie never saw it and curses his winger for missing his assignment and not blocking the point shot.

This is deemed worthy of 0.03 xGs.

2a) Ryan Lomberg takes a pass with speed in the NZ after some nice work by his other winger to win a high board battle. He has some room so he takes the defender wide and executes some respectable net drive by keeping his feet moving. He's by himself as his linemates are busy picking fleas off each other by the bench. He takes a contested shot along the ice from in tight to the net, the goalie has the angle covered easily and the shot bounces harmlessly to the corner.

This is deemed worthy of 0.3 xGs.

2b) Patrick Kane vacating his defensive responsibilities as he senses the puck is about to turn over creeps past the mouth breathing weakside defenseman still standing on the blueline trying to keep the pressure on. Sure enough Kane's center wins the puck along the boards and lifts a nice outlet area pass to center ice. Kane grabs the puck and is in alone on the goalie. He fakes going high glove and freezes our poor tender before making a move to the backhand and then crossbar down roofs it.

This is deemed worthy of 0.3 xGs.


Go ahead and handwave away these differences if you want but to me they are not reconcilable. It's not that xG models tell us nothing or aren't worth considering, it's that they are incomplete and aren't actually providing what they are claiming to. Over season long samples they can definitely be useful and predictive but stuff like the deserve-to-win'o'meter or NST's charting for individual games just isn't that useful without the context of seeing the game happen. It's not one or the other eye test vs advanced stats, it should be considering both as parts of an overall picture.
One day you'll scroll all the way to the right and find this
1741707066308.png
 
I have an irrational personal disdain for all "advanced" stats. Expected goals are not exempt from this.

What I will concede, considering I find myself defending plus/minus a lot, is that most statistics actually can be useful, as long as they're interpreted correctly, rather than taken as the be all and end all of everything. "Expected goals" clearly falls into this category. However, another thing that I will say, is that very rarely do I find that advanced stats tell me something I don't already know by actually watching the game or player in question.

I have a very mixed relationship with statistics in general because of all this, in short. :laugh:
 
No perfect stat, and has multipe flaws, but... still better than OP's IMO, it's closer to 96/100 than it is 16/100.

LOL, I don't even know if 96% situation in hockey exists- and surely there is no such thing as 96% situation with goalie in the crease.
 
I have an irrational personal disdain for all "advanced" stats. Expected goals are not exempt from this.

What I will concede, considering I find myself defending plus/minus a lot, is that most statistics actually can be useful, as long as they're interpreted correctly, rather than taken as the be all and end all of everything. "Expected goals" clearly falls into this category. However, another thing that I will say, is that very rarely do I find that advanced stats tell me something I don't already know by actually watching the game or player in question.

I have a very mixed relationship with statistics in general because of all this, in short. :laugh:

Everyone who understands them even slightly should have a mixed relationship, at best, with statistics, because that interpretation part is the key.

My biggest issue with advanced stats is that there's no way to test if the formulas give the right answer. If it can't be tested, it's a made up number that shouldn't be trusted.
 
  • Like
Reactions: Summer Rose
I have an irrational personal disdain for all "advanced" stats. Expected goals are not exempt from this.

What I will concede, considering I find myself defending plus/minus a lot, is that most statistics actually can be useful, as long as they're interpreted correctly, rather than taken as the be all and end all of everything. "Expected goals" clearly falls into this category. However, another thing that I will say, is that very rarely do I find that advanced stats tell me something I don't already know by actually watching the game or player in question.

I have a very mixed relationship with statistics in general because of all this, in short. :laugh:
See, it's interesting you mention this, because I really think the issue is just semantics sometimes.

Whether you subscribe to stats or not, we all have an idea of who's carrying the play in a game.

Some people will just beat themselves over the head with the old "the team that won played better" club all day (and then those same people will talk about "having a goalie that can steal a series" like WTF), but most of us will concede that teams getting outplayed win all the time, and vice versa.

In a series, anything can happen in one game, two games; over the course of that entire series, the team getting the lion's share of the chances probably wins that series. I don't think anybody really disagrees with that.

Once you call it "expected goals," they're mad about it.
 
  • Like
Reactions: Summer Rose
See, it's interesting you mention this, because I really think the issue is just semantics sometimes.

Whether you subscribe to stats or not, we all have an idea of who's carrying the play in a game.

Some people will just beat themselves over the head with the old "the team that won played better" club all day (and then those same people will talk about "having a goalie that can steal a series" like WTF), but most of us will concede that teams getting outplayed win all the time, and vice versa.

In a series, anything can happen in one game, two games; over the course of that entire series, the team getting the lion's share of the chances probably wins that series. I don't think anybody really disagrees with that.

Once you call it "expected goals," they're mad about it.

I wish I could "like" this post more than once, because it deserves it.
 
  • Love
Reactions: Machinehead
Everyone who understands them even slightly should have a mixed relationship, at best, with statistics, because that interpretation part is the key.

My biggest issue with advanced stats is that there's no way to test if the formulas give the right answer. If it can't be tested, it's a made up number that shouldn't be trusted.
What do you mean, "give the right answer"? It depends on what you want to gain from the data. You can easily test their worth as a predictor for machine learning. This only requires a couple dozen lines of Python code, and you can probably tell ChatGPT to write the whole thing for you even if you can't code.
 
What do you mean, "give the right answer"? It depends on what you want to gain from the data. You can easily test their worth as a predictor for machine learning. This only requires a couple dozen lines of Python code, and you can probably tell ChatGPT to write the whole thing for you even if you can't code.

I mean, can you test their expected goal numbers against anything you can measure to make sure they didn't screw up the calculation somehow? Like missing a variable that should have been accounted for, or using the wrong formula. I certainly wouldn't trust GPT to be able to get everything right.
 
Exactly. What is this, Reddit?

It would be an extremely odd story to lie about, so while I do think it's a little outlandish as a story, I'll take his word for it. Also, don't underestimate what golfers shoot the breeze about during a round. It's a borderline faux pas to mention how somebody else is playing if you're paired up with someone for a tee time (the only way to avoid this is to bring a foursome of friends in the first place) so you end up talking about almost anything other than golf, even though you're playing golf. I've learned some very random things I never expected to know about while playing golf. Some of the random shit my fellow golfers have said may turn out to be not true, but it's definitely an interesting experience to play a round with someone with a completely different life experience you've had. I've definitely played plenty of rounds with people from completely different backgrounds.
 
Expected goals largely correlates with on ice results. Players who get real goals at a way higher or
lower rate than expected tend to level back.
I'm not sure that's true. I remember looking up Crosby's career goals versus xG and I believe it was something like 15 out of 17 seasons (when I last checked), he outperformed his xG and sometimes by quite a bit.

The problem with expected goals is, at least the ones the general public has access to, it doesn't seem to take into account a player's actual skill. An unscreened shot from the slot has the same expected chance of going in regardless of whether the person shooting is Auston Matthews or Ryan Reaves.

I also believe (though not sure on this second one) that it doesn't take into account who is defending the shot. For instance, if Jaccob Slavin is standing between the shooter and the net or Erik Gudbranson is defending. Slavin likely either gets a stick on the shot or angles his body that the shooter has to shoot wide, while Gudbranson probably plays it horribly and the shooter gets a clear shot on net.

The stat just seems to come up with a percentage based on shot location without taking into account all the other variables that can affect the danger of a shot from going in.
 

Ad

Ad