I'm not sure I'd personally get behind Elon Musk as a leader, but people respond to vision and are willing to put in work and ensure hardship, to be part of a compelling vision. Not everybody is just going to work to collect their pay cheques.
Have you at any point seen Elon provide anything close to a compelling vision for Twitter? I haven't.
Other than trollish comments about free speech that he's since walked back, he's given no sign of having any coherent vision whatsoever.
The problem here isn't just that he's a terrible person; it's that it becomes clearer by the day that he's really just winging it. Smart, capable people don't want to work around that.