Ugh, I was typing up a larger response but accidentally closed out the tab...
Anyway, I mostly agree with everything you have written. However, George Lucas (a creative mind you reference and wish the sequels had) seemingly always had Luke becoming disillusioned with the Jedi Order. Though I do think his portrayal in TLJ was too harsh.
Regarding the EU, there's no way they could have kept all of it. I don't even think they could have kept a significant amount of it. The EU was always "Legends", unless it came from the mouth of George Lucas. Lucas allowed people to play within his universe (and made money off of it). Whenever Lucas and Lucasfilm decided something in canon needed change in order to meet George's vision, they had no issues with changing it. For example, Boba Fett in his short stories and books, to Shadows of the Empire, and then to the prequels.
For example, imagine if they kept all of the EU and the majority of Star Wars fans go to the new movies expecting to see Chewbacca. Nope, got crushed by a moon. And who is this Luuke?
When I say "majority of Star Wars fans", think of people who aren't us on an internet message board talking about Star Wars.
But I do get what you're saying about pick and choose parts of the EU to keep. That statement from KK about how Marvel had so much material to pick from was such bullshit. Though I do think they're starting to take the good parts from the EU.
Hopefully that gets my point across. Still mad I closed this tab earlier......