Yep. The whole Yoko broke up the Beatles narrative is too simplistic to explain a large event, but I do think she ruined John.
I was recently reading about the period when John and Yoko separated and John was with May Pang, and during that year and a half John had a great relationship with Julian for really the only time in his life, he was making music prolifically (some of his best solo work too), collaborating with the likes of Elton John and Nilsson, and even reconciled with Paul. Then in 1975 Yoko shows up again and he becomes a recluse again, cuts off his relationship with his first son, stops making music, etc. Like you say they tried to claim he was being a stay-at-home-dad for Sean, but he was definitely on some substances despite how much he denied it up to like 1979. At the time of his death he was rail thin and withering away, I'm certain he was still on heroin to some extent and that Yoko was feeding him it (she introduced him to it in the first place in the 60s, as is documented).
Whatever it is it's a really weird dynamic. Sometimes people are just uncontrollably attracted to each other despite how toxic they really are together, I guess.