you can use the latest and greatest upstream versions (even versions under active development)
you can let the compiler optimize for your system (especially useful with PGO, but honestly only worth it for a few applications, e.g. video encoders)
you have full flexibility over optional features
And it’s a good learning experience, sooner or later things won’t just work and you have to learn about compilers, linkers, various build systems and script languages etc.
But yeah, for most people it’s not worth the effort