I think many minority groups would disagree that there haven’t been problems in the very recent past.

I mean there are problems for minorities, but not for America as a whole I mean.

because otherwise we’ll always have a group who is willing to burn democracy to the ground because they hate people who have a different skin color.

I think it’s debatable they only do it because they hate people with a different skin color. They also hate women and poor people, among other things, so while the rhetoric would be different I don’t think being a 100% white country would do much to curb the American far right.