Against PDO

[Hunn]

First off, let's all remind what PDO is:

From Wikipedia

PDO, called SPSV% by the NHL, is the sum of a team's shooting percentage and its save percentage. PDO is usually measured at even strength, and based on the theory [ha-ha – my note] that most teams will ultimately regress toward a sum of 100, is often viewed as a proxy for how lucky a team is. According to Wilson [who the is that? – my note], a player or team with a PDO over 102 is "probably not as good as they seem", while a player or team below 98 likely is better than they appear.

In other words, PDO is the sum of shooting percentage (the number of goals scored divided by the number of shots for) and save percentage (the number of shots saved divided by the number of shots against). Usually, PDO is evaluated at 5-on-5, but for the discussion below it's irrelevant. PDO is a team statistics, but apparently ("a player" in the quote above) it can be evaluated for players too, I suppose similarly to +/−.

Myth No 1
PDO converges to 100

[A side note: actually, I've seen references to the law of large numbers and even to the central limit theorem. From that it is claimed that this myth follows from mathematics. A half-educated nerd is worse than one hundred fools.]

Imagine a league made of Team Canada and Team Kuwait (no disrespect, I just took the first team and the last team from the IIHF rankings) playing 82 games. Each game finishes 10:1, with shots 50:10. Team Canada PDO is 10/50 + 9/10 = 110. Team Kuwait PDO is 1/10 + 40/50 = 90.

This is of course very generous, most probably it will be like 20:0 and 80:5, but you get the point: it is league-average PDO that "converges" to 100, not individual team PDO (league-average PDO does not even "converge" in layperson's terms – it's always equal to 100 by definition).

In a real, competitive league like NHL, individual PDO converges to something between 98 and 102: in the last 6 seasons, only 14 teams out of 182 finished outside this range, roughly 8%.

This very narrow range in PDO (root-mean-square deviation of 1.13 points over the last 6 seasons) gives the impression that any difference in PDO is simply due to statistical noise – i. e. "luck".

Myth No 2
PDO is a measure of "luck"

From the discussion above one can see that PDO in NHL varies within about 4 points. Keep in mind that this difference is due to shooting and save percentages.

4-point difference in SV is the difference between Vezina-level goaltending and a bad backup. Imagine an average team with Tuukka Rask in goal. Replace Rask with whoever is current Toronto backup. It's clear that PDO will go down because Rask is better than the Toronto backup, otherwise there is no point to pay them vastly different salaries.

It is quality, not luck (apart from being "unlucky" to have a Toronto backup instead of Rask in first place). Good teams (like good goalies) have higher PDO (SV) and bad teams (like bad goalies) have lower PDO (SV).

The same applies to shooting percentages.

Myth No 3
Teams with high PDO crush next season

This myth highlights a very interesting fallacy, the one that statisticians call "confirmation bias": we look at high PDO teams that crush, but we disregard high PDO teams that DON'T crush.

Let's look at the data for the last 6 seasons (excluding 2018-19 for obvious reasons):

Top-5 PDO teams: 9 didn't make the playoffs next season, 16 made.

For comparison:

Top-5 CF% teams: 8 didn't make the playoffs next season, 17 made.

On top of that, 7 out of 30 Top-5 CF% teams didn't even make the playoffs in the season they posted Top-5 CF% numbers (compare this with just 3 teams PDO-wise). Clearly, PDO is not much worse than the cherished Corsi in predicting postseason success.

[A side note: based on intuitively clear definition of "postseason success", I calculated correlation coefficients for how far a team went in the postseason versus its regular season PDO and CF%. PDO appears to be a BETTER predictor than CF% (correlation coefficients of 0.414 and 0.339 respectively over the last 6 years) – but this result is for another discussion.]

Is there any luck in PDO? Yes, in short term, there is some luck there too (as almost in any other number), but usually one is not required to parse through PDO to see that. For example, right now the Islanders are in a 15-1-2 point streak, and the Stars are in a 14-1-1 of their own, their PDO over that span must be huge (102.7 and 103.7 respectively, via nhl.com), but I don't need PDO to understand that close to 150-point pace is unsustainable for ANY team.

So, let's give a rest to PDO as a measure of luck, leave it to uneducated folks like Lambert and Luszczyszyn.

Click to read more...

 

[Just Linda]

First off, let's all remind what PDO is:

From Wikipedia


In other words, PDO is the sum of shooting percentage (the number of goals scored divided by the number of shots for) and save percentage (the number of shots saved divided by the number of shots against). Usually, PDO is evaluated at 5-on-5, but for the discussion below it's irrelevant. PDO is a team statistics, but apparently ("a player" in the quote above) it can be evaluated for players too, I suppose similarly to +/−.

Myth No 1
PDO converges to 100

[A side note: actually, I've seen references to the law of large numbers and even to the central limit theorem. From that it is claimed that this myth follows from mathematics. A half-educated nerd is worse than one hundred fools.]

Imagine a league made of Team Canada and Team Kuwait (no disrespect, I just took the first team and the last team from the IIHF rankings) playing 82 games. Each game finishes 10:1, with shots 50:10. Team Canada PDO is 10/50 + 9/10 = 110. Team Kuwait PDO is 1/10 + 40/50 = 90.

This is of course very generous, most probably it will be like 20:0 and 80:5, but you get the point: it is league-average PDO that "converges" to 100, not individual team PDO (league-average PDO does not even "converge" in layperson's terms – it's always equal to 100 by definition).

In a real, competitive league like NHL, individual PDO converges to something between 98 and 102: in the last 6 seasons, only 14 teams out of 182 finished outside this range, roughly 8%.

This very narrow range in PDO (root-mean-square deviation of 1.13 points over the last 6 seasons) gives the impression that any difference in PDO is simply due to statistical noise – i. e. "luck".

Myth No 2
PDO is a measure of "luck"

From the discussion above one can see that PDO in NHL varies within about 4 points. Keep in mind that this difference is due to shooting and save percentages.

4-point difference in SV is the difference between Vezina-level goaltending and a bad backup. Imagine an average team with Tuukka Rask in goal. Replace Rask with whoever is current Toronto backup. It's clear that PDO will go down because Rask is better than the Toronto backup, otherwise there is no point to pay them vastly different salaries.

It is quality, not luck (apart from being "unlucky" to have a Toronto backup instead of Rask in first place). Good teams (like good goalies) have higher PDO (SV) and bad teams (like bad goalies) have lower PDO (SV).

The same applies to shooting percentages.

Myth No 3
Teams with high PDO crush next season

This myth highlights a very interesting fallacy, the one that statisticians call "confirmation bias": we look at high PDO teams that crush, but we disregard high PDO teams that DON'T crush.

Let's look at the data for the last 6 seasons (excluding 2018-19 for obvious reasons):

Top-5 PDO teams: 9 didn't make the playoffs next season, 16 made.

For comparison:

Top-5 CF% teams: 8 didn't make the playoffs next season, 17 made.

On top of that, 7 out of 30 Top-5 CF% teams didn't even make the playoffs in the season they posted Top-5 CF% numbers (compare this with just 3 teams PDO-wise). Clearly, PDO is not much worse than the cherished Corsi in predicting postseason success.

[A side note: based on intuitively clear definition of "postseason success", I calculated correlation coefficients for how far a team went in the postseason versus its regular season PDO and CF%. PDO appears to be a BETTER predictor than CF% (correlation coefficients of 0.414 and 0.339 respectively over the last 6 years) – but this result is for another discussion.]

Is there any luck in PDO? Yes, in short term, there is some luck there too (as almost in any other number), but usually one is not required to parse through PDO to see that. For example, right now the Islanders are in a 15-1-2 point streak, and the Stars are in a 14-1-1 of their own, their PDO over that span must be huge (102.7 and 103.7 respectively, via nhl.com), but I don't need PDO to understand that close to 150-point pace is unsustainable for ANY team.

So, let's give a rest to PDO as a measure of luck, leave it to uneducated folks like Lambert and Luszczyszyn.

So yeah, you took a micro based argument and attempted to disprove it by explaining the macro.

Like all things, it's a tool and one of many. It can be used to see which players are having good luck and bad luck to an extent short term but nobody claims it's the Bible of stats. Yes it can show someone is unlucky short term if used in context

 

[Hunn]

So yeah, you took a micro based argument and attempted to disprove it by explaining the macro.

Could you please elaborate what you mean by "micro" and "macro" (in this context), I'm lost?

 

[iCanada]

I like this post, a fair bit.

The one I always point out is this; who'd you rather have shooting, Ovechkin or the 13th forward on the last place club in the league?

This is my large problem with NHL analytics; they are largely built on two assumptions that generally have held true for the entire dead puck era.

1. All NHL players are roughly equal in skill, to the point that there are not vast skill disparities between different clubs or players. Thus:
2. All shots and shot attempts are equal. Thus your shooting percentage is a function of luck.
3. All saves and save attempts are created equally. This your save percentage is a function of luck.

Generally speaking, I think the assumptions made are accurate enough that they are a good tool for looking at the game, because the league has generally been par enough that low level talent is being sucked out and replaced, and high level talent has been stifled so much due to clutch and grab. But in edge cases they really fall apart; Colby Cave for example is an analytics darling.

And notably, as discussed by you above... Analytics has no way to account for the fact that McDrai are McDrai. Forgetting that they have better shots then the mean by a ridiculous margin in terms of release / accuracy / power, they also are just getting better opportunities then other players. Because they play with a player also so much better than other players, they have more open ice, they have an easier time being picky with chances and creating stronger opportunities. Any shot Drai takes is blessed with so many advantages compared to any shot Khaira takes, and it's not remotely close.

Case in point... You look at a guy like Getzlaf and he's maintained a "lucky" PDO of 101.5 over his entire career. Is that because he's creating / preventing opportunities more than most and is better at scoring? Or does that mean his entire career he's been a lucky mother? What about Crosby's 102.4?

I think to a certain degree, a lot of analytics value should be in finding edge cases to the above; analytics as we know it generally focuses on possession and generating chances, but good players cash on a greater percentage of chances, too. Sure, generally speaking better players generate more chances, also, but I think if you just focus on generation you're going the wrong way.... Ultimately I don't care how many chances to score someone has, I care how many times they score. Scoring chances don't win games, scoring does. If I had a guy who rarely generated chances but cashed on everything, I'd take him over a guy that generates a lot but never cashes.

 

[supsens]

Because all stats are taken after the games not before, the next game does not have a PDO or corsi yet. They are just like keeping score and you can’t win them all

 

[aufheben]

It’s not just a team statistic.

So yeah, you took a micro based argument and attempted to disprove it by explaining the macro.

Like all things, it's a tool and one of many. It can be used to see which players are having good luck and bad luck to an extent short term but nobody claims it's the Bible of stats. Yes it can show someone is unlucky short term if used in context

For instance Ryan Strome has a 108.4 PDO. I could assume he’s not going to score 82 P this year, but his current production alone doesn’t show me how lucky he’s been or why, or even if there’s luck involved at all. What if—despite his 82 P-pace—his PDO was 95.0 or his SH% was 5%? Maybe he’s not getting lucky, maybe he’s just taking twice as many shots this year.

Teams, skaters, and goalies all go through hot/cold streaks. Yes you can often assume they’re going to end, but some of us just prefer having a clearer picture of how and why.

 

[tarheelhockey]

I've been ranting against PDO for years.

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Either of these should be considered fatal. To have both combined into one stat is just silly.

A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.

 

[Hunn]

It’s not just a team statistic.

It is not indeed, I indicated that explicitly that there is also PDO for players. My point is to criticise the 3 "myths" about individual team PDO: (1) it converges to 100, (2) it's luck, (3) high PDO = overachievement. The first one is false, whereas (2) and (3) are somewhat true, but over a short-term period only, like a point streak, or a lull.

 

[deckercky]

PDO, like any other statistic, is useful in proper context. Using fake theoretical counterexamples is silly - do the groundwork and find practical counterexamples.

Worth noting that your assumptions are wrong. There are goaltenders who will naturally have above average save percentages and shooters who will naturally have above average shooting percentages, but even those players have variances (shorthanded save percentages, for example, is highly variable).

Not all of PDO is luck - some of it comes down to usage. For example, Oates changes how Ovechkin is used and his shooting percentage plummets - there's a non-luck factor that lowered Ovechkin's PDO. Playing injured is another non-luck factor that will drop PDO.

 

[Nick Hansen]

I've been ranting against PDO for years.

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Either of these should be considered fatal. To have both combined into one stat is just silly.

A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.

*like*

Always rubbed me the wrong way.

 

[Hunn]

I've been ranting against PDO for years.

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Either of these should be considered fatal. To have both combined into one stat is just silly.

A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.

Exactly! One does not need PDO to see that 14-1-1 streak is ... a streak, it can't be maintained over the whole season. Why then rely on PDO?

 

[SnuggaRUDE]

[A side note: actually, I've seen references to the law of large numbers and even to the central limit theorem. From that it is claimed that this myth follows from mathematics. A half-educated nerd is worse than one hundred fools.]

A timely observation.

 

[Finnish your Czech]

I agree with the content of this post, but I don't think you're revealing anything crazy. Yes, you can't use PDO to make an overarching opinion on something/someone. But that's the same for every stat. PDO has it's uses.

 

[Hunn]

PDO, like any other statistic, is useful in proper context. Using fake theoretical counterexamples is silly - do the groundwork and find practical counterexamples.

The only fake theoretical counterexample is Team Canada vs Team Kuwait. Rask vs Sparks is sadly quite real one.

Worth noting that your assumptions are wrong. There are goaltenders who will naturally have above average save percentages and shooters who will naturally have above average shooting percentages, but even those players have variances (shorthanded save percentages, for example, is highly variable).

This is what I am talking about.

Not all of PDO is luck - some of it comes down to usage. For example, Oates changes how Ovechkin is used and his shooting percentage plummets - there's a non-luck factor that lowered Ovechkin's PDO. Playing injured is another non-luck factor that will drop PDO.

Exactly so. It also depends on style. PDO in recent years was dominated by teams like Washington much the same way CF% was dominated by LA 5-10 years ago.

 

[aufheben]

Exactly! One does not need PDO to see that 14-1-1 streak is ... a streak, it can't be maintained over the whole season. Why then rely on PDO?

Well for one thing this isn’t always apparent to everyone. Lots of people were trashing the Dallas Stars in October, saying they called it during the off-season and so on. This happens all the time.

Beyond the yes/no, people like to know how and why, to what degree can a streak not be maintained.

 

[Aceboogie]

Meh, your railing against something in which noone is on the other side to argue. Noone with an appetite for "advanced stats" is going to the on the other side of the arguement.

There are a ton of great teams who have maintain high PDOs, because they have good goalies (sv%) or shooters (sh%)

PDO is still relevant for predicting future success. Rarely have bottom 10 CF%, top 10 PDO teams maintained consistent success.

I find PD) is just scary because people dont understand it, nor what to. But if you break it down to its components (shooting %, save %), its very intuitive. A career,3rd line, 6% sh % player whos scores 35 goals shooting 18% one season is likely to regress in goals next season. Vs a career 15% career player wont, because hes a better shooter

 

[Aceboogie]

Well for one thing this isn’t always apparent to everyone. Lots of people were trashing the Dallas Stars in October, saying they called it during the off-season and so on. This happens all the time.

Beyond the yes/no, people like to know how and why, to what degree can a streak not be maintained.

Bingo.

I have yet to see any PDO ultra defenders saying its the be all and end all. Its a useful tool though

 

[Bounces R Way]

I've always found the suggestion that every shooting and save percentage will inevitably regress or progress towards some arbitrary mean pretty ridiculous. Some players are better than other players, some teams are better than other teams. That doesn't make PDO worthless or anything.

The world is not simply an unending sequence of 1's and 0's and neither is hockey.

 

[iCanada]

I've always found the suggestion that every shooting and save percentage will inevitably regress or progress towards some arbitrary mean pretty ridiculous. Some players are better than other players, some teams are better than other teams. That doesn't make PDO worthless or anything.

The world is not simply an unending sequence of 1's and 0's and neither is hockey.

The good old gamblers fallacy. Clearly if I flip a coin after getting three tails, there is less than 50% chance that next coin is Tails, so bet Heads, baby.

Or as I like to call it... "The HF regression to the mean" fallacy.

 

[ijuka]

Yeah, it's interesting - PDO's a team's goalie performance in relation to shooting performance, it's not really a luck stat. Also, even if PDO was only about luck, it still wouldn't regress to 100. Say you're starting at 115 due to a fortunate early season performance. For the rest of the season, you're predicted to be performing at 100 PDO. How you did in the past doesn't affect that. As such, it wouldn't be regressing towards the mean anyway.

The good old gamblers fallacy. Clearly if I flip a coin after getting three tails, there is less than 50% chance that next coin is Tails, so bet Heads, baby.

I actually tested these things when I was a kid haha, flip the coin until I get heads or tails 4 times in a row, then flip it 200 times in a row. Then compare to what happens if I just flip it 200 times without doing anything specific before. Interestingly enough, zero correlation found.

 

[kladorf2005]

Top-5 PDO teams: 9 didn't make the playoffs next season, 16 made.

Maybe I'm not following your argument, but if 16/25 (high PDO) teams made the playoffs, isn't that better than 16/31 (random) teams that make the playoffs?

i.e. higher PDO = higher chance of making the playoffs? 64% vs 52%?

 

[TheDoldrums]

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Most of the time people criticize advanced stats it's because they're using it out of context or expecting it to explain everything.

PDO says nothing about a high/low quality play. It assumes nothing. Maybe people use it to form their own assumptions, but that's a human problem.

 

[TheOtherOne]

PDO is weird but OP's argument is also weird.

This is more or less what I was thinking of saying:

I've been ranting against PDO for years.

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Either of these should be considered fatal. To have both combined into one stat is just silly.

A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.

To reiterate: Why in the world would you add together two unlike things? This is like saying 3 kilometers + 4 grams = 7 squirrels.

If you have a goalie who's letting everything in all season (as a Red Wings fan I would know a lot about this), PDO goes down. What does that have to do with anything? Why would you make that a team stat? Why would you relate it to shooting %?

If you have a player who's very good at picking his shots, PDO goes up for him and his linemates. Okay? What does that have to do with the team's save %?

If one player does a very good job of clearing the crease but has zero offense, and another player cherrypicks all game and snipes well, they both get the same PDO? That just seems really awkward and useless.

 

[Kamiccolo]

I've been ranting against PDO for years.

There are two severely flawed assumptions behind using PDO as a predictive stat:

1) Shooting % and save % are connected
2) There is no such thing as high/low quality play, just lucky/unlucky variation

Either of these should be considered fatal. To have both combined into one stat is just silly.

A final point: much like +/-, the last resort in defending PDO is usually along the lines of: "Yeah but if you see an extreme outlier, it probably means something".
And my response to this is the same as to +/-: "You can find that outlier more easily using the original stats that make up PDO, than by using PDO itself".
If a team is shooting 25%, that tells you all you need to know about unsustainable shooting. There's no point looking at their PDO, then backtracking to figure out which element of that formula is wildly out of whack.

Disagree.

1) PDO is meant to show how sustainable a teams play is. No different than when a player goes on a hot streak and shoots 40%, you know eventually it will even out. Look no further than Buffalo, they were a high PDO team through the first 10 games and then regressed back to 100 over the past 12.

2). Sure there is. A shot from the side of the net from along the boards that has almost 0 chance to go in shouldn't be counted the same as someone right in the slot taking a shot.

I would argue PDO is better than it ever has been with parity what it is. In general, teams are going to score generally the same and have goaltending generally the same. Over an 82 game season these things usually even out, but where PDO shines is even bigger sample sizes than that.

In fact, using PDO is almost always a regression litmus test for teams who unexpectedly shot up the standings.

 

[Machinehead]

Nobody is saying there's no such thing as quality. When Holtby was at his peak, a high-scoring team like the Caps could easily hit 103.

It becomes a luck issue when your PDO is egregious. What constitutes egregious depends on your roster.

Everyone was surprised at what the 8th seeded Kings did in 2012 but they had a PDO below 99 with a multi-Cup roster.