You can also compare Allen to the other goalies in the league over the full course of the year and point out that if he had performed at the NHL average the Blues would have safely been in the playoffs. The Blues had one of the top defenses in the league last year in terms of both shot count and quality, but Jake Allen was below average in nearly every stat possible. There's no real argument that Allen's performance over the year wasn't a primary factor in the team's finish last season. There are obviously other places to improve because Hockey isn't so simple that you can blame a single thing, but for some reason people seem to use that as a shield to not hold the goalie accountable for his performance. Average goaltending is not an unreasonable expectation.
I don't think our defense played nearly as well as you're suggesting and I don't think this statement is at all true.
Allen's expected 5 on 5 SV% last year was .9251 and his actual 5 on 5 SV% was .9231, which means the difference between his actual 5 on 5 SV% and expected was -.2 for the year. That is obviously below average, as the entire point of "expected save percentage" is to quantify what the save percentage of an average goalie would be against the actual shots faced. However, among the 32 goalies with 1600 minutes played or more, his expected 5 on 5 save percentage was 22nd. Had he performed at his expected save percentage, he would have allowed 2.5 fewer goals at 5 on 5.
His shorthanded stats were brutal, but PK stats for goalies are wildly volatile and are often tossed out when evaluating goalies (similar to the way most people within the stats community view shooting percentages as inherently volatile and not necessarily a measure of a player's performance). I think Allen contributed to the Blues mediocre PK last year, but not to the extent that his PK goals against tanked his overall numbers.
Overall, I think that Allen playing at NHL average would have saved the team somewhere between 5 and 8 goals throughout his starts. It also probably would have prevented the team from starting Hutton a few extra games, which may have cost the team a couple goals since Hutton was statistically the league's best goalie last season and performed well, well above average.
Allen deserves some blame for last year. He wasn't good enough and I don't think anyone disagrees with that. But him playing NHL average would not have made us safely a playoff team. Realistically it probably would have gotten the team 2 or 3 more points in the standings. Hockey Reference's "point shares" stat suggests that Allen's play contributed 7.9 points to the team standings based on his play. The average starter contributes 10, so that suggests that we lost out on about 2 standings points based on Allen's sub-average performance. That's still putting us in the last wild card spot and uncertainty around a playoff spot in the last week of the season.
There are a number of things you can point from last season that cost this team 2-3 points in the standings. An average PP nets us 11 more PP goals over the course of the year. Tarasenko having a normal season by his standards give us another 6 or 7 goals. A competent 3rd line nets us another 6-10 goals. As an overall team, a league average offense provides 15-20 more goals than we scored last year.
Allen was one of many flaws last year and we absolutely would not have cruised into a playoff spot if he was simply NHL average.