“Feminism” is just a sloppy term. It’s “egalitarianism”: people deserve rights, your demographic shouldn’t decrease your rights. Those who you’re referring to when you use the term “feminists” will insist upon this interpretation, for good reason.
“Feminism”, as a term, conjures images of the uplifting of women, which was a potent image when women weren’t allowed to vote or work most jobs. Now, with many of those low-hanging battles won, equality is largely the case, and the image of uplifting women feels a lot more like favoritism and bias than leveling the field.
Yes there are gender specific issues, but those exist in both directions much more equally than when the “feminism” label was solidified. The goal should not be to uplift women, the goal should be to trivialize the influence of gender and sex on the involuntary conditions of life. When that results in the uplifting of women, great. But men face struggles intrinsic to being men too, and naming your egalitarian movement after femininity only deepens the divide with marginalized men.