I think a lot of it comes from schools, and in particular physical education and competitive sports. There is nothing wrong with competitive sports but the attitudes around it in schools can be so toxic, and in particular it can be used to create hierarchies. The idea of being good at sports and that being masculine was something I certainly experienced a lot at school. Also people who weren't as academic but thrived in sports were lauded.
My school had various sports teams and clubs, and fuck all academic activities. Sports aren't toxic but the attitudes around them can be, and particularly adults who feed in toxic attitudes and values around it.
SuiXi3D