Byron Bader draft rankings

[93LEAFS]

I'm not sure you responded to the wrong person here, but I say that pretty clear. I understand its just a model. It's just numbers. But the data he uses to come to an output is flawed. That's my biggest problem with it. It's a bad analytical model. You can go through a bunch of examples of him not weighing certain factors properly, such as:

- Calling Devon Levi a low probability NHLer, even after his near-record setting NCAA year, due mostly to the fact that he was drafted out of the CCHL which he calls a tier three junior league. Clearly there's an over-reliance on the CCHL part, and not weighing enough a .952 SV% in division 1 hockey.

- Calling JJ Peterka a low probability NHLer because "Nobody (in 30 years) who's had a similar development path as Peterka has ever turned into a star. And over half of them don't make the NHL." (that's his exact tweet). Clearly, an overreliance on his DEL production and not nearly enough at his near PPG pace in the AHL in his D+2 season (and over PPG in the playoffs).

It could be argued that I'm a salty Sabres fan that doesn't like how he's ranked our prospects - but remove that part from the equation I would still firmly believe that the weights he assigns to CCHL drafted prospects + players that are drafted out of the DEL (Seider + Stutzle included) should not factor as much as what they've done against top-tier competition in their D+1/D+2 years.

Most recent years should be measured essentially by a reversion method. Anything older than 3 years is essentially worthless. So, most recent times by 5, season before 2.5 and three years ago 1 then divided by 8.5 for example. Now, this draft that is complex especially with the OHL guys (and was even worse last year). But, historically, that is vitally important. Yes, track record at a young age matters, but more recent outstanding stuff matters more with the importance there is an age multiplier involved.

 

[TopC0rner]

I think the biggest problem is those analytics are only good if they are good predictors. Otherwise, it's essentially a stats mumbo-jumbo which isn't necessarily better than a PPG-weighted random generator.

Imo, his model is just a variation of NHLe that tries to match prospects with historical players. It does both suffer from NHLe problems and from the potential lack of accuracy of using historical data as a predictor (such as weight/height/production at the same age).

I would take the results as amusement more than as an accurate read on prospects' future.

In essence, it poses this question... If you have two prospects of equal height and weight from the same league producing the same PPG that were drafted in the past, and one became an elite NHLer while the other completely busted, does that mean that a prospect this year of equal height and weight from the same league with the same PPG at the same age as those others has 50% chances of becoming an NHL star and 50% of becoming a bust?

 

[93LEAFS]

He totally is, but are you going to sit here and tell me the concensus is always right? Just because he’s off of the general opinion doesn’t mean it will be wrong in the end.

He’s only been doing the model for a few years so it’s tough to judge it on past success, but these rankings are put out for free with the intention to add some statistical input to peoples knowledge of the draft. He’s not claiming this should be the be all end all list of who you should draft

But his model is demonstrably flawed. He's messed up a significant informational input on a system that produces around 10-15 draftable prospects every year. Many who are the best Americans in the draft.

And seriously, anyone with any level of competency with Excel who has read the chapter on this in Statshot published in 2016 could create pretty much this. This might have been revolutionary in 2014 (removing the flaws), but this info has been publicly available for 6 years. I read about this before Matthews played his first NHL game. If he's measuring all the USNTDP games under one league multiplying factor instead of accounting for the many different level they play in a given season it has serious issues.

The other serious issue with it is this. The goal of NHLe as it has been created is to predict year to year. It is not designed to predict longterm growth. Your goal would be to predict what scoring in these leagues between the ages of 16-21 mean 5 years down the road. NHLe is not designed as a predictive tool outside of what a player will do next year. Pretty much NHLe only accounts for elite players at the CHL level, as they are the only ones who jump right into the NHL. Whereas comparably, you have massive sample sizes for guys going from the AHL to NHL at 23 to 25.

 

[BondraTime]

Imagine charging people for a service and not even knowing how to read the stats you're inputting.

Plugging in 101 goals for Caulfield and not even double checking anything has me cracking up

Not to mention 92 games (with no playoffs) plus all international games. How the f*** do you not realize that isn't correct when your inputting data into something you're supposed to understand

 

[Goldenhands]

There were mainstream scouts who had him ranked in that neighborhood as well. As some have pointed out, the last couple drafts are probably going to be screwy looking back because of the limited playing time many prospects had during the pandemic. I don't think it would be wise to draw broad conclusions during that time.


And scouts touted Alexander daigle as the next Gretzky.

It's funny what is and isn't acceptable when ranking prospects.

Scouting has evolved over the years and kids are more easily projectable since most junior level teams play with much more structure than 10-20 yrs ago, so easier to assess players' weaknessess. In a draft like 2012, first it was a pretty weak one, but when you look at Sarnia, there was 0 structure there, a kid like Yakupov could cheat on the wrong side of the puck all he wanted, he was getting alot of pts but costing alot as well. Im pretty sure that these days, his weaknessess would be huge red flags and that a player like that would fall in the rankings.

 

[Brodeur]

And scouts touted Alexander daigle as the next Gretzky.

It's funny what is and isn't acceptable when ranking prospects.

Somebody uploaded the '93 Draft to YouTube and it's an interesting rewatch. Paraphrasing but I'm pretty sure Bob McKenzie said something to the effect that Daigle was not a Gretzky/Lemieux/Lindros level talent at the top of the draft.

 

[Static]

Somebody uploaded the '93 Draft to YouTube and it's an interesting rewatch. Paraphrasing but I'm pretty sure Bob McKenzie said something to the effect that Daigle was not a Gretzky/Lemieux/Lindros level talent at the top of the draft.

There must have been some polarizing opinions at that time then.

 

[Poppy Whoa Sonnet]

Somebody uploaded the '93 Draft to YouTube and it's an interesting rewatch. Paraphrasing but I'm pretty sure Bob McKenzie said something to the effect that Daigle was not a Gretzky/Lemieux/Lindros level talent at the top of the draft.

Let's go the tape!

 

[Brodeur]

Let's go the tape!

Best part is the weird round table talk that they made Daigle, Pronger, Gratton, and Niedermayer do. Not sure if there is a consensus, but I would say Gratton had the best mullet of the draft class.

~31 minute mark Bob says the 'not Gretzky/Lemieux' part but then offers up lofty comparisons to Jeremy Roenick, Pat LaFontaine, Steve Yzerman, and Joe Sakic.

 

[BondraTime]

When you don't like doing something, but you're good at it, you're less likely to succeed at the highest levels compared to those who enjoy and want to continue getting better.

If Daigle liked hockey, I'm sure his career is much, much different. There are not many guys in the NHL who don't enjoy playing the game and would rather be something else.

 

[bossram]

Dude what are you talking about? Look at his 2020 and 2021 rankings:

The Hockey Prospecting Top 32 (2020)

We introduce the final Hockey Prospecting rankings of the top 32 for the 2020 draft! The rankings factor in the probabilities from the model, but also age, runway left to improve, league-specific p…

hockeyprospecting.com

The Hockey Prospecting Top 32 (2021)

Introducing the final Hockey Prospecting rankings of the top 32 for the 2021 draft! The rankings factor in the probabilities from the model, but also age, runway left to improve, league-specific pa…

hockeyprospecting.com

The rankings generated by his model are laughably bad.

I don't really get what conclusion you're taking from looking at his previous lists. Some guys he was higher on than consensus have progressed much better than expected. And some of the guys he was higher on haven't. You'll find the same with any scout.

The model output isn't a mock-draft. It's just a list of players ranked by NHLe. You can take whatever away from that what you will, but a lot of the vitriol generated toward his models are criticisms that don't land.

 

[David Suzuki]

Most of the people who do analytics are insufferable and some of the worst kinds of people but their models have some merit. Not sure if this guy is a decent guy (seems like it from twitter fwiw) but his model is iffy but then again so are most scouts lol.

 

[AlternateSideParking]

I mean, his first round has 26 guys who are in my first round, and I've been watching 4-16 hours of hockey a day for the last seven years. And if you think I'm lying or exaggerating, ask my wife and kids.

Only issues I would have are Hutson that high, Slafkovsky that low, and I'm not that bullish on Howard. If he wants to go the stats-only route and charge you for it, there always will be people who are willing to pay.

 

[Dack]

For those who are wondering the model takes into account age at the draft, production and generates an NHLer and star likelihood based on former projects who hit similar thresholds.

Here's how it ranked past drafts:

He's actually done that before reranking the drafts from 2011-2016 just based on what the model valued them at in their draft year (excluding the 1st overall which he never changed). It has some big misses and it also identified a lot of the "steals" in the NHL draft just based on pre draft data.
Click to view each year

2011
View attachment 452599
2012
View attachment 452600
2013
View attachment 452601
2014
View attachment 452602
2015
View attachment 452603
2016
View attachment 452604

FWIW Star is defined by Points per game 0.6 or greater for forwards and 0.4 for D-men which is why a guy like Tony Deangelo is a "star". EDIT:Forgot to mention that overages also weren't included so that's likely why you see no Mangiapane in 2015. Additionally the team Tom Novak played for routinely had players produce at a high level before falling off and seems to be an outlier.

 

[David Suzuki]

For those who are wondering the model takes into account age at the draft, production and generates an NHLer and star likelihood based on former projects who hit similar thresholds.

Here's how it ranked past drafts:

Pretty fair. I don't think anything will ever be better or more important than watching a player (lol), but models like this can help teams avoid over thinking certain players for otherwise arbitrary reasons.

 

[R2010]

I don't really get what conclusion you're taking from looking at his previous lists. Some guys he was higher on than consensus have progressed much better than expected. And some of the guys he was higher on haven't. You'll find the same with any scout.

The model output isn't a mock-draft. It's just a list of players ranked by NHLe. You can take whatever away from that what you will, but a lot of the vitriol generated toward his models are criticisms that don't land.

Its a fine effort at doing something different from an analytics point of view but there are lots of issues and too much faux statistical rigour.

I mean he doesn't provide any uncertainties and claims to be a stats person. He doesn't include international events when NHLe itself would say that things like the WJC and U18s have predictive value. Like if we believe in empirical data then throwing out sources and choosing to value one over another is subjective. Include everything or nothing. Don't pick and choose which to include.

He makes mistakes counting the stats even... His model is already outperformed by Patrick Bacon's because Bacon's model actually recovers substantially more of the signal than Byron's because it targets WAR instead of PTS.

I'm not a huge fan of draft analytics for more than helping you make initial lists and not miss gems who fell too far. Byron's models does show value but like it should be completely complementary not instead of. And even then you should use a better model.

 

[bossram]

Its a fine effort at doing something different from an analytics point of view but there are lots of issues and too much faux statistical rigour.

I mean he doesn't provide any uncertainties and claims to be a stats person. He doesn't include international events when NHLe itself would say that things like the WJC and U18s have predictive value. Like if we believe in empirical data then throwing out sources and choosing to value one over another is subjective. Include everything or nothing. Don't pick and choose which to include.
View attachment 561620


He makes mistakes counting the stats even... His model is already outperformed by Patrick Bacon's because Bacon's model actually recovers substantially more of the signal than Byron's because it targets WAR instead of PTS.

View attachment 561619


I'm not a huge fan of draft analytics for more than helping you make initial lists and not miss gems who fell too far. Byron's models does show value but like it should be completely complementary not instead of. And even then you should use a better model.

Your criticisms are definitely valid, and I already preferred TDH's draft model for the reasons stated above.

My point is that most of the vitriol directed at him is just because of "analytics", and vanishingly few are able to actually identify the flaws as you've done. A lot of people have criticized him for not correctly guessing the draft order, which makes absolutely no sense as the model is not a mock-draft.

 

[R2010]

Your criticisms are definitely valid, and I already preferred TDH's draft model for the reasons stated above.

My point is that most of the vitriol directed at him is just because of "analytics", and vanishingly few are able to actually identify the flaws as you've done. A lot of people have criticized him for not correctly guessing the draft order, which makes absolutely no sense as the model is not a mock-draft.

Fair point. Most people also assume the teams have similar lists to the consensus rankings as well. A lot of the challenges faced by portions of the analytics community is the confidence with which statements are made about things.

 

[jc17]

Its a fine effort at doing something different from an analytics point of view but there are lots of issues and too much faux statistical rigour.

I mean he doesn't provide any uncertainties and claims to be a stats person. He doesn't include international events when NHLe itself would say that things like the WJC and U18s have predictive value. Like if we believe in empirical data then throwing out sources and choosing to value one over another is subjective. Include everything or nothing. Don't pick and choose which to include.
View attachment 561620


He makes mistakes counting the stats even... His model is already outperformed by Patrick Bacon's because Bacon's model actually recovers substantially more of the signal than Byron's because it targets WAR instead of PTS.

View attachment 561619


I'm not a huge fan of draft analytics for more than helping you make initial lists and not miss gems who fell too far. Byron's models does show value but like it should be completely complementary not instead of. And even then you should use a better model.

This is a good post, but I'm not sure his model targets WAR. The charts you posted just show WAR is more correlated with standings success than scoring, but his NHLE model is only based on points, unless I'm missing something.

 

[UnSandvich]

He had some downright silly takes on Othmann last year iirc

 

[R2010]

This is a good post, but I'm not sure his model targets WAR. The charts you posted just show WAR is more correlated with standings success than scoring, but his NHLE model is only based on points, unless I'm missing something.

If I understand correctly he uses his WAR model after calculating NHLe

 

[jc17]

If I understand correctly he uses his WAR model after calculating NHLe
View attachment 561953

Interesting.

I imagine it can't be much different than Bader's star probability though, because they use the same type of inputs. So you might have instances like a Monahan vs O'Reilly, where Monahan is considered the bigger star in Bader's and Oreilly in Bacon's. This might result in something like the model favoring size or something a bit more, but I'm guessing it would only be a slight difference in terms of the end output

 

[MoneyManny]

There was an actual TVA article on their website this morning talking about this guys mock draft and calling it "very credible", i lol'd.

Juraj Slafkovsky «classé beaucoup trop haut»? - TVA Sports

Un outil analytique utilisé par des équipes de la LNH place le gros ailier slovaque aussi loin qu'au... 16e rang.

www.tvasports.ca

 

[Static]

 

[R2010]

Interesting.

I imagine it can't be much different than Bader's star probability though, because they use the same type of inputs. So you might have instances like a Monahan vs O'Reilly, where Monahan is considered the bigger star in Bader's and Oreilly in Bacon's. This might result in something like the model favoring size or something a bit more, but I'm guessing it would only be a slight difference in terms of the end output

I mean on its face it considers defensive contributions more than the points only model.