Allen was pretty good, too, though I haven't seen quite as much of him. From what I understand, he was a guy who loved the surreal. You take a look at a lot of the early stuff of David Letterman (when he was still sharp and before he got washed up) and it shows a lot of influence from Allen. (He also made a point of being a clean comic, too, which most comics don't bother to do these days.)
Carson was OK. He's definitely dry, and he was great with characters, but he had some lousy monologues. I'm not going to mock him too much, since he did last about 30 years and was able to choke out pretty much any competition he had, but he's not the kind of guy I'd consider a great comedian, although he was a great emcee. Carson was an expert straight man.
He's only overrated because people consider him this sort of king of the genre, just out of longevity and reputation.