In fairness, I think we might already be the rest who don't matter. Threads has just passed 100 million users in like three days. The entire fediverse, in about ten years (it's tough to pin down an exact start date because "When did it become the fediverse?"), has accrued around 12 million users, of which less than 4 million are active. There's any number of things Meta might want, but I don't think greater access to 4 million geeks is at the top of their list.
I do think the embrace, extend, extinguish concerns have some merit. Meta isn't threatened by the fediverse now, but maybe they do want to kill it before it becomes a problem. In the short term, though, we're not overtaking Threads. Personally, I think another plausible theory is that Threads is using ActivityPub to demonstrate that they're not running a monopoly or gatekeeping control of social media (which the EU's new Digital Markets Act now regulates) by pointing to the fediverse--which will soon also include direct competitors Tumblr--and saying "See, we're all on equal footing! We don't control social media! Look over there at those 4 million geeks and whatever number of Tumblr users."