100% with you, which is why I have been against the speed at which the Sabres have elevated some prospects into the NHL.
I too am 5'7" and played high level sports thru high school, the military, college, and then city leagues until I couldn't anymore. I would love Savoie to make it, because I was that guy. I was 5'7" and 180-185 lbs from the time I was 16 till I was about 28. I always surprised people and outplayed my size, because while I was shorter than most on the field, I was built like 6'2" 210 lb guy, just with shortened bones.
Now, I can tell you without any question, that my 25-28 year old body would have completely dominated my 18-20 year old body. I was stronger, tougher, more balanced, more grizzled, and the interaction between my brain and body made me quicker.
18-20 year olds are still in teenagers brains and bodies. They need to stay in an environment where their bodies are not being overmatched, so they can have more success, build confidence, and grow up mentally as well. People can always point out exceptions and outliers, but that's all they are.