EDIT: that Islanders team wasn't in Albany. It was in a tiny building in Troy. Then they built a new building in a better location and the Devils wanted in. It was going great until Lou stopped caring.
A historical correction here: The Capital District Islanders were indeed in the Houston Fieldhouse at RPI in Troy - a building that seats about 4,800 people (i.e. not tiny). They started there in 1990-91. The Houston Fieldhouse was not up to pro standards, but it was a good-sized building, heated (most college rinks at the time were not), and had several concessions areas and plenty of parking. They did not draw well, in my opinion, because they were marketed (and named as) the Islanders farm team, which alienated many Rangers and Devils fans in the area, and also because they competed for attention with the RPI Engineers Division 1 hockey team (Union had only recently become Division 1, and was winning less than 15% of its D1 games)
The building that was built as the Knickerbocker Arena, later known as the Pepsi Arena, now the Times-Union Center, was also opened in 1990-91, and housed the Albany Choppers of the IHL. The Choppers were a dismal failure, folding mid-year, with announced attendances far higher than actual fans in the building (I remember one reporter counting just 75 people in the building one night).
The CD Islanders weren't much better, because any existing pro fans in that area supported the Adirondack Red Wings, 45 minutes to the north in Glens Falls, and other hockey fans supported RPI, which would typically get 4,500 per game, with some games selling out. CDI averaged just around 2,000 per game in its final two seasons.
The CD Islanders played three seasons in Troy before moving to into the Knickerbocker arena in Albany, grabbing the Devils affiliation from Utica in the process. They debuted with a highly touted logo which transformed minor league hockey from a marketing standpoint. They were considered moderately successful at the time - you have to remember, 1991-92 was a very typical AHL attendance year, with the median team (Maine) averaging 3,828. This was before the Providence Bruins and the Philadelphia Phantoms, and before Hershey got their new arena. 3,000 fans was considered a viable operation in those days, so when the River Rats came in at 3,751, that was a big deal.
One other thing to remember when comparing attendance - it is heavily dependent on ticket sale promotions, so it is difficult to compare apples to oranges and use the reported numbers as a proxy for viability. A $8 ticket is counted the same as a $22 ticket. I don't know if free tickets are counted toward attendance, but I think they must be since Providence offers Buy One, Get One Free nights and they have high attendances reported.
Also, there is one other city that outdid Albany for on-ice futility: Springfield missed the playoffs for 11 out of 12 seasons, with that one season only being due to a qualification round being introduced. Several seasons had "official" winning percentages under .400 - with "true" winning percentages even worse - the 2004-05 Springfield Falcons spent the period from January to the end of the season under .300, mostly in the .275 range. It was such a bad run that when the Falcons finally made the playoffs in 2012-13, the front office staff had no idea what to even do to get tickets sold - it had been nine years since the team had made the playoffs.
As a hockey fan and season ticket holder, I can tell you that even I don't go out of my way to attend games when the team is that bad during a season, and I know that as those seasons mount, people stop buying season tickets because they realize that they aren't getting their money's worth when they don't go to over half the games.