The way I see it, people have the right to criticize anything they want for any reason. If someone finds something racist, sexist, insensitive, or whatever, they are allowed to say so, Likewise, if someone thinks it's stupid to accuse something of being racist, sexist, or insensitive, they have the right to say so too. If they wanna get upset over a simple disagreement and call each other idiots over the internet I don't care. If it leads to actual fighting, that could become a problem.

Personally I think some things are offensive, but I usually just choose not to look at it or be around it. If it really bothers me I'll say so then leave not really expecting it to change. If all else fails I could just go home. But I do think some things like little jokes can be blown out of proportion, and if I get accused of being racist, sexist or insensitive over something I legitimately believe isn't I just say, "too bad, so sad, go whine about on tumblr"

The problem comes from deeming stuff offensive. What is offensive to you, might not be at all for your neighbour.