When people go to Mastodon, Kbin, Lemmy, Firefish, Misskey, etc., they do so knowing they're going to the fediverse. When people go to Threads, most do so because they have an Instagram account.
This is my main concern.
Personally, I don't care if the fediverse grows. I just care what it grows into. The fediverse has a nice community at the moment because everybody on it made a conscious decision to be here and not somewhere else. Threads users will not have made that decision. Furthermore, they'll outnumber the rest of us enough as to have no incentive to try and fit into the preexisting community here (which isn't helped by the fact that they've already been their own isolated community for awhile).