And just where does it say in our Constitution that someone should not have the right to refuse service to anyone on any basis he chooses? If I don't want your business why should I not have the right to tell you so?
The things not said in the Constitution far outnumber those that are said. Congress is granted the power to "... make all Laws which shall be necessary and proper for carrying into Execution the foregoing Powers, and all other Powers vested by this Constitution in the Government of the United States, or in any Department or Officer thereof", so Congress has passed many, many laws since that First Congress passed the Bill of Rights and all subsequent legislation passed during the institution's Freshman sessions.
If all we needed was the Constitution, you'd think those were the guys to best understand that, and in having that understanding, they would have passed no laws. But they did.
In theory, I do believe that anyone should have the right to refuse service to anyone for any reason whatsoever, but as a person who could feasibly be the object of that refusal, and whose children could doubly so be the same, I don't feel that it is in the best interest of the "general welfare" of the nation to promote any manner of discrimination, irrespective of the reason.
I don't think it a good idea that an ambulance driver be permitted to refuse transportation to someone based on race, gender, religion, national origin, or age, so I don't think allowing others to do so is a good idea.
Some time ago, as a nation we decided to eliminate discrimination, so if I want to own a hotel, run a taxi service, a nail salon, a restaurant, etc, I need to be able to either set my prejudices aside, or not run a business.
Now, some people will make the argument that their religious beliefs are protected, and that they have the right to stand on those beliefs, and live their lives accordingly. They will be right to say that of course.
But just like you pointed out that nowhere in the Constitution does it say that someone should not have the right to refuse service, that very same Constitution does not say that anyone has the right to bake a cake or own a bakery.