I think it can make a difference.
This topic is one that people will argue on both sides of it. But personally I think there is something to 'being in the building', to every morning being the guy who you can walk up to their door, knock on it and sit down for a meeting....rather than having to wait to schedule a zoom meeting with.
And here is the thing for me, If he was the best hockey mind in the world, someone with a ton of experience and led teams to cup wins in management, and you wanted to allow him to do that, fine. But, I don't see him as a huge asset. If you are even going to be 'just as good as the next guy' or even 'just slightly better' in your role....then I'll take the next guy. The guy who will be in the building. Be available for face-to-face meetings by knocking on the door. A guy who shows up at practice, who meets the guys in management and on the team and gets to know them. Are all of those things NEEDED in the 'remote world' we live in? No they aren't NEEDED, but if I have a choice I would rather have them than not.