By treating tolerance as a binary (it's either completely present or completely absent) you've removed your argument very far from reality. The goal in reality is to be as tolerant as possible, and the most tolerant stable state simply has some (limited) amount of (very specific) intolerance in it.