Tolkien > Rowlings in writing but both arent great "authors" or writers imo. They imagined and then put to paper these huge imaginative awesome worlds .. that was their magic. Not their actual writing ability. LOTR is an overly detailed death-by-minut description writing. Rowlings was all over the place.
I see where you're coming from, but these are subjective takes. Creating a dazzling world is certainly a hallmark of great writing. Rowling was also pretty strong with character and pretty exceptional at quirky little details that added a playfulness to her world. Tolkien's race/faction/class structures and balance were vivid and important to the narrative, as was the importance of the history of his world and people.
GRRM, Herbert, and Asimov had similar strengths. And to one level or another, most of the writers who truly excel in this area tend to be a little less fixated on characterization and dialogue. Then there's the other way around -- character over setting/world -- where you have writers like Austen, Ishiguro, King, Rendell, Tolstoy, etc.
And even among those extremes you have extreme variation. If Rowling wrote GoT Arya's wolf would have been a ferret named Slinkleboom. And if GRRM wrote Harry Potter you'd get the occasional 2-page scene of Ron taking a dump. His world is gritty and lived-in where hers is so perfectly ornate that it sometimes feels like it'd evaporate if you touched it wrong.
There's also just being a painstakingly good writer of prose. In terms of pure technique, Nabokov's work might be the best I've ever read. But while there's a lot of great stuff there, none would make my top 10 favorite books. The text itself is magical though. Twain, Achebe, Bronte, Shelley, Cervantes... Similar talents for sure. Their stuff is just a pleasure to read because the prose is so good. (And pick the Bronte sister of you choice.)
Then there's the writers who influenced generations of writing. Hemingway can be polarizing, but there's no question he's near the top of that list. Flaubert might be the only challenger. And when style is analyzed as its own thing, you get Proust, Defoe, and Melville.
Writers excel in a bunch of wildly different categories, and world building is equal to any other. Sometimes a thinly-drawn world is crucial and sometimes it's the foundation for every ounce of a story's impact. And sometimes you can do both -- build a great world and tell a great story -- without writing 500,000 words. Matheson did both in I Am Legend, also created the modern vampire and maybe accidentally created zombie fiction, in about 25,000 words. Feel how you will about the book but that's a shitload of quality per word.
I went the Tolstoy route with this post. My apologies for that. TL;DR: World building is as important as most other things, and those writers aren't limited to that one particular strength. There's a reason their work mesmerized generations...