There are definitely people with a bee up their butt about how Brady plays, which in parallel with losing games, is a lightning rod for negative attention. When the team is mired in an extended slump, getting pissed off at late game antics is natural from a fan perspective.
Truthfully, I don't really know if fixing this group will take one move or two moves, or one big move etc etc. I do not envy Steve Staios right now.
But I do know that despite changing a lot of the peripheral players and the coaching staff, this team still effectively plays the same way and has the same problems. I know that simply relying on the problem to fix itself isn't going to work.
Naturally the attention falls on the core guys. Unfortunately given the contract situation and his appeal around the league, I don't think it's unfair to say trading Brady would provide the most value (and would be the most prudent option) in acquiring the assets necessary to change the direction of the team and hopefully get things back on track.