There's an indefinable line, or gray area, between being polite toward others and not being forced to kowtow to their every whim.
Some of political correctness is just general decency, but those who try to pretend it's no more than that are gaslighting you.
Unfortunately, there's no guarantee that even reasonable people can agree on where to draw the line, so as much as I'd like to offer a solution, I don't have one, except to ask both sides to not be inconsiderate.