Under the asymmetry hypothesis, political tolerance and intolerance differ in their underlying psychology, making it easier to persuade the tolerant to become less tolerant than to convince the intolerant to become more tolerant. Using a representative sample of the Dutch population (N = 546), we examined this hypothesis for people’s tolerance or intolerance of socially disruptive protest actions of their least-liked group. Focusing on the relevant contrasting values of freedom of speech and public order, we found empirical evidence for the asymmetry of political tolerance: it was easier to persuade the tolerant to become less tolerant than to convince the intolerant to become more tolerant. In fact, we found a backlash effect among the intolerant participants with them showing higher intolerance as a result. These findings support the notion that tolerance is more fragile than intolerance because of the required self-restraint that involves psychological discomfort and uneasiness. However, tolerance is indispensable for our increasingly polarized liberal democratic societies making further research on the social psychology of tolerance and intolerance topical and urgent.