Yeah, "sloppy word choice of people letting off steam" is a common argument used when caught in bigotry.
"I'm not a racist. I was just angry. That doesn't represent who I really am." Etc.
That's just it. It does reflect one's biases, when one is stressed enough—as in, when one would let of steam—to speak without filter. Those words came from somewhere.
"Acting against their own interest" is also problematic, as it presumes authority on what another person's interest are.