While I agree with your assertion that BLM and Antifa are counterproductive, they don't have the body count of millions (Nazis) and thousands (KKK) of people murdered under their respective banners.
The historical record matters. Those individuals in Charlottesville chose to represent those heinous organizations because they embrace the blood-stained history of said groups.
The individuals were there to protest the destruction of monuments to Confederate generals and troops (essentially, war memorials).
Either Rights are Rights for everyone or they are not Rights at all, they are privileges exercised at the whim of those in power.
Most of the perpetrators of crimes you describe are long dead. Should we wait until the Antifa (Communists, really, with an impressive body count as a group if you aren't picky about the name they use this month) or BLM rack up the tombstones or build ovens and send people to camp to do anything about them?
Otherwise, the KKK and the Neonazis just need a different name and a new flag, right?
Or do we learn from history and nip this sh*t in the bud all around and demand that such protests, lawfully conducted, go unmolested?
That way, other groups retain the right to protest, too. Otherwise, the ladies garden club won't even be able to protest flower bed regulations, and we all lose a fundamental Right.
We have a Constitution which respects the Right to peaceably assemble and the Right to free speech. Encumbered as it was with permit requirements, one group got the required permit to assemble in peaceful protest and was attacked, forced from that venue and attacked some more. That is the essence of the issue, and regardless of the name or ideology of the groups involved, one was in the right and one was in the wrong--whether or not we agree with what they have to say, it is their Right to say it. We can find their viewpoint despicable, but to deprive them of their Right to hold that belief, to express it, to assemble peacefully to do so places all of our Liberty in far greater peril than to endure and ignore and even refute in open forum (without infringing on their Right) their message or their core beliefs. Regardless of the core beliefs of the Klan or White Supremacist groups, I do agree that the war memorials should stand and do not see where the beliefs of those groups should be used to oppose the preservation of out nation's history.
If others want other memorials, they can put them up, too.
The actions of one individual with an automobile were definitely wrong, without argument.