Feminism is for feminists - who specifically want to advance the cause of women in regard of gender equality. Most of the people who'd define themselves as feminists are likely to be women.
Whereas, if someone of either sex is against gender discrimination, and if they regard that discrimination as affecting both sexes, then they can't they just say they're 'against gender discrimination'? Of 'for gender equality'? Or, if a snappier -ism is required, then 'anti-sexist' might do at a pinch.
(Although I would personally wonder whether proudly claiming any of the above labels would make me sound a bit of a self-important unctuous berk, even though I'm sympathetic to feminism.)