I'm definitely in this camp.
I'm a believer that expected goal stats are interesting, as a single tool to be evaluated against others, including actual (real life goal) events and eye test.
To me, there are just so, so many variables that make ALL the difference, that I don't trust them for evaluating individuals, especially goalies. You gotta keep in mind these xG stats were predominantly developed to evaluate TEAMS, not players, so their validity holds up over large n-size with large number of events where the noise can get washed out.
In terms of application to real life...
Like any hot-headed goalie, my blood pressure can be orders of magnitude different on two "identical in the eyes of xG" plays.... a player receives a pass from down low in the slot. My defender is between me and the shot. xG stats would see these two situations the same, but one scenario has me fuming and my defender buying me a beer and the other is entirely savable:
A) Defender charges out toward shooter (let's say from the right), forces a rushed shot and gets his legs together as the shot comes. I'm somewhat screened, at least xG stats say I am, but in reality the shot can only go to the left (or wide right) because my defender has covered the right side of the net. I can cheat left, look around the defender and make an easy save into the crest.
B) Defender doesn't charge out, but tries to awkwardly block the shot closer to me than to the shooter. Legs are open, kinda going down with one knee. Shooter has time. I can't see and because there is space to score on either side of defender AND between his legs AND plenty of angles to deflect the puck in any number of directions... I'm F'd. It's definitely going in.