The ActivityPub #fediverse we've built couldn't have happened with an invite-only federation protocol. The only way to build up enough fedizens (beyond true believers like me) to make the network useful and attractive, was to make joining it as frictionless as possible. The network only exchanges messages online, so the potential for harm is limited, and mute/block tools are adequate to mitigate abuse.
