Our politics in this country are binary......Conservative/Liberal....Republican/Democrat.....so you are talking about trying to take the whole political spectrum and place them into two boxes. This is the result, the kooks go into one of the two boxes, and they don't have their own box, that's isolated from the others.
The "Powers That Be" have decided that this is preferable to having many smaller parties, and instability as parties try constantly to forge coalitions, as is the case in other countries. So this is the price we pay for having the Two-Party System, the coalitions have to come within the party, and sometimes that includes elements we consider to be unsavory.