He did not refuse them service. He refused to support their political belief. It was pure entrapment. The same baker would have turned down a same-sex wedding cake for a straight couple.
It would be like a Muslim baker turning down a cake for an anti-Muslim function. Muslim bakers turn down same sex wedding cakes all the time. Gays just know it would be politically inconvenient to sue them for it. They would lose gravitas. So they only choose white Christian bakers to make their political statement.