The important thing is to confront it when you find it. To speak out when others say such things around you.
I've known many people with different, contrasting viewpoints of mine on racial matters...so you think when you hear someone utter a dissenting viewpoint they should immediately start arguing with that person? Really? If I started remonstrating with people who had different views on things from mine, I'd have spent the majority of my life doing nothing but that.
And for the record I have argued with racists, haters about the topic...it didn't do anything to change their minds.
But if you like to do that (argue with people), go ahead. See how many minds you change.
But one thing...stop blaming people who don't like to argue for the nasty opinions of others. They, the haters, are themselves responsible. We have enough liberal SJWs in the country as it is...we don't need conservatives ones.