Population size is pretty much irrelevant though. What counts is the number of players and the number of and access to rinks, and Germany doesn't really have lots of those.
While the local hockey teams do have a bit of a pull in larger cities, the biggest draw for hockey is a climate that is fitting for it, meaning cold winters with enough frozen lakes. Germany doesn't really have that anymore, outside of some smaller and not particularly populated regions near the mountains. And those regions tend to be the basis for all winter sports, many of which are quite big in Germany.