No of course it’s reasonable that they wouldn’t understand the ins and outs. The op and commenter you’re replying to are talking about the connotation of the word, not the fact that a hobbyist understands their hobby.
Take the term Trekkie for example – people who are into star trek can become Trekkies which symbolizes that they’ve joined a community. That term can be used to mean that two people both belong to a community (i.e., “we’re Trekkies”) or it can be used to refer negatively to people in that community by those who aren’t in it (i.e., “Trekkies smell bad”).
There are (at least) two things happening here that people are picking up on. One is that context matters, and the way that the term normie is often used is not a positive one. I’ve personally never seen anyone refer to themself proudly as a normie, have you? And the other is that we’re referring to normies, a group we ostensibly don’t belong to, as a homogeneous blob which is obviously not accurate.
I doubt anyone’s feelings are especially hurt if they’re called a normie, but that doesn’t mean it isn’t a dismissive and usually negatively valenced term used to refer to a massive and diverse group of people.