Let's not be so enthusiastic about our criticisms of climate change (née global warming) alarmism that we start making false statements.
The assertion, "there is no such physical place with 'average global temperature,'" is provably false (though that place will wander around the globe over time): temperature varies continuously from place to place and the earth's surface is compact, any continuous function on a compact space attains a maximum and a minimum value (the Extreme Value Theorem) and attains any value between any two different values it attains (the Intermediate Value Theorem). And bounded continous functions on compact sets have averages, once a measure is chosen (the physically meaningful one for global temperature being area). As the average value of the function is between the maximum and minimum, it must be attained somewhere. Thus at any given time, there is a physical place with the average global temperature.
The problem with "average global temperature" as a summary measure of something about the earth's climate is we do not actually have enough data to determine it, and thus estimates of it are subject to bias in the methods selected to make them (and guess which way certain people bias their estimates), not that it does not exist.