Another thing to consider is that there will invariably be alternative communities if one goes away. There's a "tech" or "technology" community on just about every lemmy instance right now. You might find that the one on Lemmy.ml is the biggest, but if that one gets out of hand, people can migrate to another one fairly easily. And in fact, people will probably be subscribed to more than one "tech" community, so they won't even need to migrate. They just become more active in one of the other dozen tech communities.
For communities that are more niche, this will be more difficult but I think the same concept rings true. When something starts getting out of hand in a community or on an instance, there's much less friction to creating another community and getting the word out for people to migrate there. It happened plenty with subreddits fracturing into multiple subs or a new sub after a mod threw a tantrum. Now it isn't just mods, but also admins that we can escape.