I did some more research after your comment and it does indeed sound like it’s not for the feint of heart.
Spam seems to be one of the biggest challenges, both incoming and outgoing. For incoming, it’s a constant arms race with spammers to circumvent spam filtering techniques. But at least that’s something you have control over, you can just turn off your spam filtering and ensure you receive all important email. The real problem is ending up in other people’s spam filters, which you have very little control over once you’ve decided on your mail server domain/certificate.
The crux of the issue seems to be that SMTP is ancient insecure tech designed for an innocent era when email was for universities only. We desperately need a more secure open source email protocol designed for the modern era, but capitalism isn’t having it - instead we’ve got corporations wrestling for control of the next big thing with proprietary protocols… Discord, Slack etc. And big tech companies that continue using SMTP (Gmail, Outlook etc.) simply treat any servers outside their sphere with a high level of suspicion.