His record as an OC without Brady is average at best.
Look at what he's worked with as an oc without Brady.
2008 - Matt Cassel as qb with really good weapons. 7th best scoring offense in the league
Rams 2011 - Sam Bradford as qb with Brandon Lloyd as WR1 and Brandon Gibson WR2. Their tight ends that year were Stephen Spach and Billy Bajema. Who?
2020 - Cam Newton who couldn't throw the ball after week 2, Jakobi Meyers as WR1 and Damiere Byrd as WR2, with the worst tight end room in the league.
2021 - Mac Jones as qb, Jakobi Meyers WR1, Kendrick Bourne WR2 and Hunter Henry as the main tight end. 6th highest scoring offense in the league.
You give him trash like Brandon Gibson, Billy Bajema, Damiere Byrd and Ryan Izzo to work with, you'll get a trash offense. You give him nfl talent, doesn't even have to be great but nfl talent that deserves to be on the field and he's shown as an oc he can cook up an offense. Bill O'Brien is a competent oc and he couldn't get the offense close to what McDaniels had it running at.
McDaniels flaming out as a head coach for a second time has made people judge him negatively as an oc. There's something personality wise that doesn't fit to being an nfl head coach but he wouldn't be the head coach here. If Mayo wants him he's one of the best candidates and maybe the best considering he won't be gone in a year if he does a good job like other coordinators.