Two things
Detroit probably had three of the four best players in the NHL in the early 1950s in Howe, Sawchuk, and Kelly. Inevitably it will hurt Hart voting to have three teammates who are top four players in the NHL.
Early 1950s Hart voting is not the place to look for an "outstanding player" award.
In 1950 Rayner got the Hart, Kennedy finished second in Hart voting, neither even made the first all star team. Richard was very likely the best player in the NHL.
In 1951 Schmidt gets the Hart in what some people consider a career achievement award. I think Schmidt's award is justified in the value sense of the Hart, and at least he was first team all star, but Howe dominated NHL scoring (~30% higher than number 2) and was clearly the best player. Howe finished fourth in Hart voting. Kelly likely gets the Norris had it existed.
In 1952 Howe again dominates league scoring and this time is given the Hart. Sawchuk is fourth in Hart voting but makes the first all star team over third place Hart finisher Henry. The Norris doesn't exist yet but Kelly, first all star, probably gets it that year. Hard spot for Sawchuk to garner Hart support in.
In 1953 Howe destroys the league in scoring to a historic degree again and gets the Hart. Second in Hart voting, narrowly, is Rollins, who doesn't make the first or second all star team. Sawchuk is first team all star. Kelly finishes a close third in Hart voting. McNeil, second team all star behind Sawchuk, finishes fourth in Hart voting with some significant support.
In 1954 Howe again comfortably leads the NHL in scoring but Rollins gets the Hart. Rollins was not a first or second team all star and there is belief that he in part got it due to how he performed for a terrible team the year before. Kelly is a close second in Hart voting and gets the inaugural Norris. Howe finishes fourth in Hart voting. Sawchuk was second all star that year, Howe or Kelly were likely the best player in the NHL.
In 1955 Kennedy gets the Hart despite not making the first or second all star team, many believe the award was a career achievement award. His teammate Lumley, first team all star again, finishes second and three Montreal players round out the top five. Sawchuk is second team all star on the down swing of his prime and Kelly has been supplanted by Harvey as the NHL's best defenceman.
Basically it was a weird period for the award voting, at least compared to how we usually view it. It's clear that voters were not just ranking the best players in Hart voting. This is always true to some degree, but it seems especially pronounced in the early 1950s.