Study - relationship between PP ice time & point production

[Hockey Outsider]

Objective

Recently there's been some discussion on the main board about the relationship between ice time and production. The purpose of this study is to examine the relationship between those two variables in a way that's objective, transparent, and systematic.

My overall conclusion is, although a positive relationship exists between the two variables, there's some complexity that isn't captured in the popular "per 60 minutes" productivity stats. There's clearly some value in that metric, but people who use it need to understand its limitations.

Click to read more...

 

[Hockey Outsider]

Defining the population

I'm looking at 5-on-4 powerplay production, for forwards only. (Why? Because this is the game state where nearly all of the focus is on generating offense and, to the extent there's a relationship between ice time and production, it would be most apparent here. I also don't want the data to be skewed by other types of powerplay situations, ie 4-on-3, which are even higher-scoring). I have data for the thirteen most recent seasons (2007-08 through 2019-20). The source for the data is naturalstattrick.com.

Data validation

I've used naturalstattrick.com ("NST") in the past and have never identified any significant errors. However I cross-referred their data to Evolving-Hockey.com ("EH"). I couldn't compare it to NHL.com because, as far as I'm aware, they only publish total PP stats - nothing specifically for 5-on-4.

For example, the NST data shows Patrick Kane as having 274 points in 3,175 minutes over the past thirteen years. The EH data shows him having 274 points in 3,172 minutes. NST shows Steve Stamkos having 237 points in 2,768 minutes. EH shows him as having 239 points in 2,763 minutes. I'm not going to present all the data validation that I've done, but the differences I've identified are minimal. For that reason, I'm confident I have an appropriate starting point for this analysis.

Paired data

The majority of this analysis is based on paired data. In other words, I look at a player in a given season, then I looked at how their ice time and production changed in the following season. For most of the analysis, I'm using a threshold of 160 minutes of 5-on-4 ice time Year N and Year N+1. That threshold equals approximately two minutes of 5-on-4 ice time per game. The goal was to have it high enough that the data presented is meaningful (as opposed to be skewed by low-minute seasons) and low enough that I can still have hundreds of data points to analyze.

As an example of what paired data looks like - Gabriel Landeskog played 186 minutes at 5-on-4 in 2015. In 2016, he played 193 minutes - therefore there's a data pair for "Gabriel Landeskog 2015" and "Gabriel Landeskog 2016". As we'll see below, this gets included in the analysis. In 2017, his 5-on-4 ice time dropped to 157 minutes, below the threshold. Therefore there's no data point between his 2016 and 2017 (or 2017 and 2018) season.

In total, I have 996 paired data points (that is - 1,992 player seasons). I didn't set the threshold to give me around a thousand pairs, but it works out nicely.

 

[Hockey Outsider]

Before we get into the in-depth analysis, we need to examine some basic principles. The first question is - do players who get more PP ice time tend to be more productive on a per-minute basis than those who receive fewer minutes? Based on all of the hockey I've watched and played, it seems obvious to me that the answer is yes. Coaches tend to play their best offensive talents during the game state when there's a clear opportunity to attack. But let's see if we can demonstrate this with data.

There can be significant variations in PP opportunities from team to team, so doing a league-wide comparison would introduce distortion into the results. Instead, let's look at each team-season from the past 13 years. In total, there are 393 team-seasons (13 years * 30 teams = 390 + 3 for Las Vegas). Can we identify a relationship between who receives more PP TOI, and who produces more? This table shows how each player ranks on their team in powerplay time, and their per-60 production:

Rank	Average
1st	4.59
2nd	4.30
3rd	4.07
4th	3.76
5th	3.37
6th	3.20
7th	2.93

[TBODY] [/TBODY]

What this table shows is the forward with the most PP TOI on a team (in a given season) average 4.59 points per 60 minutes; the forwards with the second-most PP TOI average 4.30 points per 60 minutes; and so on. This provides a clear and unequivocal answer to the question. Yes - in general, the players who receive the most PP TOI are the most productive. The data follows a fairly clean pattern, with productivity dropping off between 5% and 10% as you head down each rank.

I'll emphasize that the table above is showing per-minute production, not total production. The data isn't telling us that the players with more PP TOI score more points in total (even though that's true). It's telling us that the players with more PP TOI are more productive on a per-minute basis. This suggests that, on a given team, PP time is earned through productivity. (Instead, if the data showed us that the forwards with the 1st and 6th most PP TOI scored at the same rates, that would imply that the number of points scored was strictly due to ice time rather than ability - but that's simply not the case).

Observation 1 - in general, coaches give their most productive players (per minute) the most ice time on the powerplay.

 

[Hockey Outsider]

Here's an excerpt of the paired data that I mentioned before:

Some of the data supports the theory that there's a clean, linear relationship between ice time and production. For example, look at Paul Stastny. In 2010 he scored 20 points in 279 minutes. In 2011, he scored 15 points (a 25% decrease) in 210 minutes (a 25% decrease). If all the data followed such a clean pattern, we'd only need to plug a player's per-60 minute production into the desired level of ice time, and we could predict his point totals with perfect accuracy!

Of course, reality is more complex than that. Look at Phil Kessel as a counter-example. In 2009, he scored 15 points in 163 minutes. In 2010, despite getting a huge increase in ice time (251 minutes - a 54% increase) he actually saw his production fall to 12 points (a 20% decrease). Ouch.

It's easy to cherry-pick examples. Let's look at the data holistically. The correlation coefficient, comparing the percentage change in ice time to the percentage change in points, is 0.50. As far as correlations go, it's moderate. That means there's clearly some relationship between the two variables, but it doesn't have a lot of explanatory power. Overall this relationship explains 25% of the variance - meaning that 75% is attributable to other factors.

Observation 2 - there's clearly some linkage between ice time and production, but it only explains a small portion of the variance.

 

[Hockey Outsider]

Here's the data from the previous post, presented in chart form:

This is a scatterplot showing the percentage change in ice time along the x-axis (horizontal), and the percentage change in production along the y-axis (vertical). This helps explain why there's relatively little explanatory power looking at the relationship between ice time and production. There's a fairly wide scatter on the data. If there really were a strong, linear relationship between these two variables, the 996 data points would sit closely on the red line. The fact that they don't, demonstrates that the explanatory power is small.

(Someone might argue that paired data is flawed because circumstances change each year. Maybe a player gets more ice time, but scores fewer points, because he's getting older; or he was struggling with injuries; or he had bad linemates; or he simply had bad luck. All of those points are valid - but it's equally likely that the opposite scenarios are true. That's why we're looking at nearly a thousand paired data points. There are surely random factors that exist which skew individual pairings, but the likelihood if it impacting all of the data in a systematic way is remote).

 

[Hockey Outsider]

One of the more interesting questions is - can we use a player's per-minute productivity to forecast his production when his ice time changes? Here's a simple model where we take a player's per-minute productivity in Year 1, plug it into his actual ice time in Year 2, and compare his actual to predicted ice time. An excerpt of the data is as follows:

This model gives us a correlation of +0.60. This means the model explains 36% of the variation from year to year. That leaves a lot of the variance unexplained, but it's not bad for a basic model.

Of course, the unspoken implication is each player has a per-minute productivity rate that's inherent to him - and all you need to do is change his TOI and he'll still maintain that rate. Let's challenge that assumption. In this second model, we'll forecast each player's Year 2 production based on his actual ice time that year, and the population's average productivity rate. In other words - we assume that no player has an intrinsic ability to produce on the powerplay, and that any player in the population I've identified would score at the same rate, regardless of how they produced in Year 1:

This model gives us a correlation of +0.64 (with 41% explanatory power). In other words, ignoring a player's Year 1 production rate gives us a more accurate forecast than using a player's actual productivity. This concept seems obviously wrong - but it produces a more accurate forecast. How can that be?

 

[Hockey Outsider]

The phenomenon (that I referred to at the end of the previous post) occurs due to a concept called regression towards the mean. Essentially this means that large variations aren't likely to recur in the future - so a player with a particularly high (or low) per-60 productivity rate in one year is likely to see that result decrease (or increase) towards the average in the next year. That's a bit of a simplification, but it illustrates the point. The implication is a player who has an exceptional result in Year 1 is more likely to produce at an average pace in Year 2.

As an example - the highest per-60 production in this population was Mitch Marner in 2018, when he scored a staggering 8.98 points per 60 minutes. Nobody can sustain that level of production - I'm not even sure if Gretzky did that in the 1980's. Anyway, predictably, Marner was rewarded with more ice time in 2019 (a 19% increase). His per-minute production plummeted (to 5.81 points per 60 minutes), and his total production also dropped (a 23% decrease). That's a perfect example of regression towards the mean - his production in Year 1 was unsustainable, and then it dropped to a more normal level in Year 2. Someone who predicted that he'd continue to score almost nine points per hour over 206 minutes was setting themselves up for disappointment. This shows the folly in simply plugging in a player's per-60 production into a different number of minutes - it assumes that there's a fixed, knowable level of production that holds constant.

If we look at the 25 highest production rates in Year 1, all but one of them drop in Year 2. 44 of the top 50 highest production rates in Year 1 drop in Year 2. And the same pattern holds looking at the players with the lowest rates (46 of of the 50 lowest production rates rose in Year 2).

Are there exceptions to this principle? Some might suggest Alex Ovechkin - he has an uncanny ability to produce at excellent rates on the powerplay year after year, and his numbers are forecasted more accurately under the first model. But he's just one player. Even if I exclude him entirely from both models, the results barely change.

Returning to the model - you can build an even more accurate forecast (correlation of +0.67, with explanatory power of 45%) basing the forecast on a 50/50 split between Year 1 production and the group's average production. (You can play around with the variables and use something other than a 50/50 split, but any improvements from there are minimal). To be clear, we're not applying that averaging factor only to extreme observations - that's applied to every player. When you need a mean-reversion factor applied to every piece of data to maximize the accuracy of the forecast, that suggests there are obvious limitations to how much weight to can put on a player's productivity.

Observation 3 - it's questionable if players really have a fixed, repeatable level of per-minute production (at least one that's strong enough to be a better predictor than the population's average production).

Observation 4 - the better (or worse) the player's production, the more likely it is to regress towards the mean in the next year.

 

[Hockey Outsider]

The last question is - do players demonstrate the effects of fatigue when their TOI increases? This was a tough one to tackle because I've demonstrated that more productive players generally receive more ice time. (In other words, if a player gets more ice time in Year 2 compared to Year 1, it's probably because, for whatever reason, he was more productive on a per-minute basis in Year 2, and therefore earned the increase in TOI).

What we want to do is isolate players who led their team in PP TOI in Year 1 and Year 2, and see the relationship between the change in ice time and change in production. There were 152 players who led their team in ice time both years (thus 42% of the time, the same player leads a team in TOI two years in a row). Here's a graph looking at the changes:

There's a trend in the data. Among the players who were the #1 forward on their team's PP in both years, we see a negative relationship between the change in TOI and the change in production in Year 2. It's a weak correlation, but it's there. In other words - as these top forwards, who were already logging heavy minutes, received even more ice time, their per-minute production rate declined.

If we look at the players whose TOI increased at least 10% in Year 2, their per-60 production dropped by 0.23 points/60 minutes on average. Players with relatively stable ice time (rising or falling by no more than 9%) basically maintained their production - there was a small 0.04 points/60 minute drop. Finally, the players who saw a decrease of at least 10% in their ice time saw a small increase in their productivity (a small 0.06 points/60 minute rise).

This is statistical evidence showing the impact of fatigue. Looking at huge swings in TOI from year to year (ie examining a player who jumped from the #6 forward to the #2 forward - or the opposite) isn't meaningful, since it's been established that, in general, players' ice time is commensurate with their ability. However, by looking at a player in the same role (#1 forward) in both seasons, we see that players who are given significant increases in ice time experience a drop in their per-minute productivity.

Some might object - aren't I seeing patterns in the data that are really the result of regression towards the mean? It's easy to demonstrate that the answer is no. If this result was due to regression towards the mean, we'd expect the production rates for the players in those three categories to be different. (In other words, if the players who got the biggest boost in ice time had scored at an unsustainably high rate in Year 1, that implies that the decrease in Year 2 was inevitable, regardless of whether their ice time increased or not). But the data shows that explanation is false. The first group (with large increases in ice time and an overall drop in production in Year 2) averaged 4.74 points/60 in Year 1; all the other players averaged 4.73 points/60. In other words, since both groups had essentially the same production, regression towards the mean wouldn't explain why we see a decrease in one group but not the other.

Observation 5 - when looking at the top forwards, an increase in ice time reduces a player's per-minute production (presumably - but not conclusively - due to fatigue) - and we know that regression towards the mean isn't causing this phenomenon.

 

[Hockey Outsider]

Overall conclusions

The data that I presented supports, quite straightforwardly, each of these conclusions:

• There's a positive relationship between PP ice time and production. As a player gets more minutes, they produce more points. However, the correlation is only moderate, and this means that the relationship between the two variables has much less predictive power than one might expect.
• At the team level, the players who are more productive per minute get the most ice time. In other words, ice time (relative to the team) is "earned" based on production.
• A player's production in Year N+1 can be estimated somewhat well based on their production in Year N and their ice time in Year N+1. However the forecast is even more accurate if the player's production rate is ignored entirely (and the "group" production rate is used) - suggesting there's a limit to how sustainable a player's production rate actually is.
• Regression towards the mean exists - meaning that a player with a particularly strong (poor) level of per-minute production in Year N is very likely to see their production decrease (increase) the next year.
• There's some evidence that, for players who are already heavily utilized on the powerplay, their per-minute production drops off if they're given a large increase in ice time (which is evidence of the impact of fatigue).

 

[NHL WAR]

Great work. Should be required reading for posters on this site

 

[Filthy Dangles]

Nice work per usual.

Overall conclusions

The data that I presented supports, quite straightforwardly, each of these conclusions:

• There's a positive relationship between PP ice time and production. As a player gets more minutes, they produce more points. However, the correlation is only moderate, and this means that the relationship between the two variables has much less predictive power than one might expect.
• At the team level, the players who are more productive per minute get the most ice time. In other words, ice time (relative to the team) is "earned" based on production.
• A player's production in Year N+1 can be estimated somewhat well based on their production in Year N and their ice time in Year N+1. However the forecast is even more accurate if the player's production rate is ignored entirely (and the "group" production rate is used) - suggesting there's a limit to how sustainable a player's production rate actually is.
• Regression towards the mean exists - meaning that a player with a particularly strong (poor) level of per-minute production in Year N is very likely to see their production decrease (increase) the next year.
• There's some evidence that, for players who are already heavily utilized on the powerplay, their per-minute production drops off if they're given a large increase in ice time (which is evidence of the impact of fatigue).

Could somewhat of the bolded be attributed to what i call accumulating 'garbage ice time' so to speak? I notice someone like Ovechkin will frequently stay out on the ice when the others on the 1st PP unit depart. So he'll be out there with guys like Lars Eller and sometimes even two defensemen where they aren't really generating chances or perhaps even "trying" to score as much as just keep the puck down in that end for like 30 seconds while they prime themselves to get back to Even Strength. That can add up if it happens 2-3 times a game, even less.

I've seen it hypothesized on the mainboards/poll board when comparing PPP/60 between guys who get a ton of TOI like Ovechkin and someone who might not get as much as the other superstar players like Auston Matthews.

I guess the fatigue thing also comes into play there.

 

[SA16]

Nice work per usual.



Could somewhat of the bolded be attributed to what i call accumulating 'garbage ice time' so to speak? I notice someone like Ovechkin will frequently stay out on the ice when the others on the 1st PP unit depart. So he'll be out there with guys like Lars Eller and sometimes even two defensemen where they aren't really generating chances or perhaps even "trying" to score as much as just keep the puck down in that end for like 30 seconds while they prime themselves to get back to Even Strength. That can add up if it happens 2-3 times a game, even less.

I've seen it hypothesized on the mainboards/poll board when comparing PPP/60 between guys who get a ton of TOI like Ovechkin and someone who might not get as much as the other superstar players like Auston Matthews.

I guess the fatigue thing also comes into play there.

I just read this entire thread and was going to comment the exact same thing. The loss of production due to an increase in TOI for heavily used players could be due to fatigue but it also could be due to an increase in ice time with the PP2. PP2s are far less efficient than the PP1. If a player's extra ice time is coming from that his production will be driven down because he's staying on the ice after the puck is cleared out of the zone, waiting for the line to change, and then remaining on the ice with a worse unit. I lean towards it being the latter because for the most part the PP is not a high energy state that would tire you out. For example, Ovechkin sits in the left circle most of the time waiting around for the right shot. He's not constantly bursting from side to side. I don't think he's too tired at the end but at the end he's playing with guys like Eller, Wilson, Orlov, Vrana instead of Backstrom, Kuznetsov, Carlson, and Oshie.

 

[decma]

The last question is - do players demonstrate the effects of fatigue when their TOI increases?
...
What we want to do is isolate players who led their team in PP TOI in Year 1 and Year 2, and see the relationship between the change in ice time and change in production. There were 152 players who led their team in ice time both years (thus 42% of the time, the same player leads a team in TOI two years in a row). Here's a graph looking at the changes:
...
There's a trend in the data. Among the players who were the #1 forward on their team's PP in both years, we see a negative relationship between the change in TOI and the change in production in Year 2.

Are you using "TOI" as shorthand for PP TOI? Or does each reference to "TOI" (rather than "PP TOI") refer to total TOI?
If the former, did total TOI also increase for the players in the sample, or only PP TOI?

Thanks.

 

[Hockey Outsider]

@Filthy Dangles and @SA16 - thanks for the feedback. I think your comments are spot-on.

@decma - any reference to ice time in this thread is to PP ice time (I realize I got imprecise towards the end).

 

[decma]

@decma - any reference to ice time in this thread is to PP ice time (I realize I got imprecise towards the end).

Thanks for clarifying.

In addressing question 5 (do players demonstrate the effects of fatigue when their TOI increases?), did you also look at overall TOI? It is probably fairly highly correlated with PP TOI, but I can see situations where a player's PP TOI increased from season N to N+1, but his overall TOI declined (in which case there may not be any increase in fatigue).

 

[Fatass]

@Hockey Outsider
Very interesting analysis. It created questions for me.
Can we say there are certain players who help their teams on the PP, but hurt their teams at ES? And (highly likely) those same players don’t PK, so they are actually only useful on the PP? Hence the tag - PP specialist? Could those players value on the PP be so high they are overall helping their teams win, or are those types of players tending to be on the crappy teams? Is this tag craperolla, or does it have merit?

 

[Hockey Outsider]

Thanks for clarifying.

In addressing question 5 (do players demonstrate the effects of fatigue when their TOI increases?), did you also look at overall TOI? It is probably fairly highly correlated with PP TOI, but I can see situations where a player's PP TOI increased from season N to N+1, but his overall TOI declined (in which case there may not be any increase in fatigue).

That's a great concept. No, I haven't looked at that yet, but I should have time later in the month.

 

[Hockey Outsider]

@Hockey Outsider
Very interesting analysis. It created questions for me.
Can we say there are certain players who help their teams on the PP, but hurt their teams at ES? And (highly likely) those same players don’t PK, so they are actually only useful on the PP? Hence the tag - PP specialist? Could those players value on the PP be so high they are overall helping their teams win, or are those types of players tending to be on the crappy teams? Is this tag craperolla, or does it have merit?

There's no question that PP specialists exist. Whether they're a net benefit to their team depends on the circumstances.

The earliest PP specialist that I'm aware of is Camille Henry, who won the Calder in 1954. Across 1958 and 1959, he tied for the league lead in PP scoring and was only 34th (in a six-team league) in ES scoring. Of course, there was no ice time or goals for/goals against data back then.

An example of a PP specialist who probably didn't help his team win very much is post-prime Dave Andreychuk (let's focus on those last three full seasons in Tampa - 2002 to 2004). He was T-259th in ES scoring among forwards (in the same range as Kevyn Adams and Serge Aubin) and 40th in PP scoring (in the same vicinity, in roughly the same number of games, as Alexei Kovalev, Bill Guein, and Owen Nolan). And we all know the limitations to plus/minus, but he had the lowest rating on the team, with the only players anywhere close to him being either an enforcer with minimal offense (Chris Dingman) or a dedicated penalty killer who took far tougher matchups (Tim Taylor).

Andreychuk was decently productive with the man advantage, but it makes you wonder if, say, Vincent LeCavalier would have developed a bit quick had he been given some of Andreychuk's ice time. (If not for that PP ice time, which allowed him to push his career goals total over 600, you'd have to think he'd never have gotten a spot in the Hall).