That’s literally when to use it, it’s specifically to use when you could come off like an asshole and do not want to.
You don’t use it in my best guess judging by your responses so far is because you get to act like the other person is crazy for taking you seriously.
Being the guy that jokingly hates Jews can be hilarious especially if they are Jewish but everyone has to know otherwise they might seem a bit… Mmm Hitlery right?