I have not seen an analysis that shows models are inherently more accurate than team scouts. I view them as a guiding tool that requires knowledge and insight to interpret them properly. Further, every NHL team has an analytics department in 2024, though I will acknowledge that I am sure some teams rely on them more than others.
For instance:
Draft pick A puts up huge numbers (but he was playing with the presumptive #1 overall pick and another guy who was drafted last year in the first round)
Draft pick B puts up mediocre numbers (but he was playing mostly with 3rd/4th liners and majorly stepped up his production when given a chance)
The model, at least as I understand it (focused on age and production), would look at Draft pick A as a significantly better player. A scout might realize that player B actually has a higher long-term upside when put in a position to succeed. I am very pro-analytics in general, but I find there are some who rely on data without understanding the context (though I have been extremely impressed by how some of the models are rapidly improving on incorporating context into their models)
I'd love to take a deeper look at this a few years from now. The earliest draft I can find reliable data on Bacon's model is 2021, which is far too recent to properly evaluate. (ideally 2018 or before) Looking specifically at 2018, I'd love to see how the data looked at guys who were drafted high and haven't done super well and vice versa (Hayton, Kotkaniemi, Zadina, Kravtsov, Wahlstrom on the negative side and Miller, Romanov, Durzi, Fehervary, Sharangovich, and others on the positive side).
www.nhl.com
