Generally, votes are overrated. Especially if you’re not mainstream, by definition most people won’t support you, won’t agree with you, won’t understand you.
Some things may be downvoted because they’re too stupid. But occasionally, you might be downvoted simply because you’re a bit too early. Like, if you’d said “being gay is not crime” or something 50 years ago, you might have got downvoted… Just a thought.