I definitely understand and largely agree with your sentiment. But this is the internet, hate is not limited by just name calling. What starts with just a verbal argument can snowball into cyber stalking and bullying. An lgbt artist could see every artwork downvoted to oblivion. Bots can send subscribers of targeted communities daily death treats. Public data can often be de-anonymized (no matter how carefull you are) to extend the bullying to the real world.
Its not just about growing a thick skin to a few slurs now and then but the exhaustion of the daily reminder people want you dead. Personally i am rarely the target of such hate but ive seen in action how quickly a fandom can destroy itself just because someone well known. changes their own name to match their person.
For lemmy: If facist make their own instance and largly keep their racist opinion for their own safe space then the big public servers dont need to defederate or block them so they are welcome to join in on the public space just like the Minority insrances can. Even if they defederated eachother. Most people would see and be able to interact with anyone, only the intolerant and their potential targets would not with eachother.
But take for example what OPs post is about the large public server allows a facist community on its own instance. Now the only way for a minority to protect themselves is to defederate with the larger instance excluding themselves from the bigger public while the facists roam free as a subgroup within it. which is exactly how they like it.
There might be many other solutions, but as for now this one seems to be the only i can think off with the tools provided.