For me the core issue is healthcare. We are very lucky with what we currently have in Waterloo and as you get older that matters. I am not nearly as confident that we would be in the same place if we came back. They are also building an absolute state of the art hospital about 5 minutes from where we live.
We have though about getting place down south. I formally retire in June and my wife just retired in September so its still an option. We thought about San Diego where we have a very good friend we visit. It's pretty expensive though. Financially, I would make a lot more sense to just rent a nice place for a month or two in the winter or to look for a cheaper community.