Unfortunately he (the baker) is on uncertain ground, particularly under the Supreme Court's jurisprudence. While that jurisprudence is not directly on point - i.e., it isn't controlling - since it generally deals with federal anti-discrimination laws, it is very strong persuasive precedent and it indicates that in all likelihood the baker would not prevail if he claimed that the Colorado actions violated his federal Constitutional rights. One of the key Sup. Ct. cases in this area is Heart of Atlanta Motel Inc. v. United States, 379 U.S. 241 (1964). In that case the Supreme Court held that the federal law that prohibited racial discrimination in public accomodations - such as hotels and motels - did not, as is relevant to this case, violate either the Fifth Amendment (freedom of association and taking of property without due process) or the Thirteenth Amendment (involuntary servitude). The baker is also unlikely to succeed on a claim that his First Amendment rights (freedom of expression and free exercise of religion) have been violated because the rule applies to his commercial activities and the Supreme Court has generally held that the exercise of constitutional rights in the realm of commercial enterprises are accorded less protection than in the realm of private noncommerical activity; e.g., states can place a lot more restrictions on commercial speech under the First Amendment then they can on private, noncommercial speech.
The baker needs to find a more nuanced means of achieving his ends, e.g., by requiring that wedding cakes be ordered through the church or minister performing the ceremony, or by using some other sort of reason for why he's unable to provide a cake to someone (e.g., his schedule is too tight and he already has too many cakes, etc, that have to be delivered on the day in question - of course, this sets him up for a potential sting operation, so he'd better be extra careful). That, or he has to either close up shop or move to a more receptive state. Then again, he can choose to be ruined financially by the fines to be imposed or choose to rot in jail.