Probably similar to how ribs and brisket were trash cuts up until recently, as they were too tough unless you actually figure out what to do with them.
I would imagine that lobster wasn't really appreciated until both preparation techniques as well as transportation/handling came up to par. Lobsters have a bunch of enzymes that break down the protein that go to work when it dies that can give it a woolly or cottony texture unless it heated up right after death to something north of like 140F, but doing so kinda overcooks the meat. That is why they are usually stored live and cooked right before eating, so the enzymes don't **** it up.
I don't eat much seafood, but from what I have had of crab and the like the meat is mostly about the texture... So it makes sense back when they used to not handle it right and it just tasted like sea flavor wool that it wasn't appreciated much.