From what I understand, those shadow profiles are based on information that Meta gets about you from other users, or through things like cookies and tracking pixels.
Suppose that Meta builds a shadow profile for each fedi user, then how would they link it back to you? They cannot get your IP address from just being federated with your home server, and they cannot inject a tracking pixel to your server’s website. Is there another way that they could use your fedi activity to serve you targeted ads?