As knowledgeable as the scouts are about the players they scout, players prove them wrong on every turn, either by busting, or becoming a "diamonds in the rough". Looking back on HF's previous years' top 5 prospects for each team, it's been shown that about half of them turned out to be busts, and from the rest, anyone with half a brain could have picked out players were going to be stars in the league.
When you take that less than 50% prediction accuracy from scouts observing a player, and you collect the observations from numerous scouts (with no real good way to standardize those observations), you end up with a pretty unreliable result. If they had to re-compile teams' prospect rankings from 5 years ago with 20-20 hindsight, the results would be vastly different.
They're doing the best they can, better than any of us could, most likely, but I wouldn't bet my life savings on any of their rankings.