September 19, 2012, 5:24 pm100 Comments
Obama’s Lead Looks Stronger in Polls That Include Cellphones
By NATE SILVER
As I observed on Tuesday, and as The New Republic’s Nate Cohn also found, Barack Obama seems to have received a much clearer bounce in some types of polls than others.
Although there are exceptions on either side, like the Gallup national tracking poll, for the most part Mr. Obama seems to be getting stronger results in polls that use live interviewers and that include cellphones in their samples — enough to suggest that he has a clear advantage in the race.
In the polls that use an automated dialing method (“robopolls”) or which exclude cellphones, Mr. Obama’s bounce has been much harder to discern, and the race looks considerably closer.
The difference seems especially pronounced at the state level. Mr. Obama got very strong results in a series of NBC News/Marist College polls last week in Ohio, Florida and Virginia, which included cellphones and used live interviewers. Likewise, Tuesday morning’s series of New York Times / CBS News / Quinnipiac polls had reasonably good news for Mr. Obama in Virginia and Wisconsin.
By contrast, the automated polling firm Rasmussen Reports has recently released polls showing Mr. Obama two points behind Mitt Romney in New Hampshire and three points behind in Colorado — the worst results that it has shown for him in those states all year. Another automated polling firm, Gravis Marketing, recently put Mr. Obama at a five-point deficit in Virginia, in contrast to three traditional polls that put him ahead by margins ranging from four to eight points there. A third automated polling firm, Public Policy Polling, has shown stronger results for Mr. Obama, but they also had him with good results before the conventions, and so haven’t shown him gaining much ground.
These results are consistent with some past research. Roughly one third of American households rely solely on mobile phones and do not have landlines, meaning they will simply be excluded by polls that call landlines only. Potential voters who rely on cellphones belong to more Democratic-leaning demographic groups than those which don’t, and there is reasonably strong empirical evidence that the failure to include them in polls can bias the results against Democrats, even after demographic weightings are applied.
We keep track of which polling firms include cellphones in their samples and which do not. So on Monday night, I decided to run two alternate versions of the FiveThirtyEight forecast. (Note that all results are based on polls that were in our database as of Monday night, and so will not include Tuesday morning’s New York Times polls or others published on Tuesday.)
In one of the forecasts, I ran the numbers based solely on polls that do include cellphones in their samples. The vast majority of these polls also use live interviewers, since federal law prohibits automated calls to cellphones under most circumstances. (Note, however, that one or two mostly automated polling firms, like SurveyUSA, use a separate sample based on live interviewers to reach cellphone-only voters; these were included in the model run.)
In this universe, Mr. Obama seems poised for victory. The model forecasts him for a 4.1 percentage points win in the national popular vote. That compares with 2.9 percentage points in the regular FiveThirtyEight forecast, which includes polls both with and without cellphones.
Mr. Obama’s advantage is also clearer in the swing states. The cellphone-inclusive polls give him an 80 percent chance to win Virginia, a 79 percent chance in Ohio, and a 68 percent chance to win Florida, all considerably higher than in the official FiveThirtyEight forecast.
Overall, this version of the model gives Mr. Obama an 83 percent chance of winning the Electoral College, a full 10 percentage points higher than the 73 percent chance that the official FiveThirtyEight forecast gave him as of Monday night. So the methodological differences are showing up in a big way this year.
The other version of the model consisted solely of polls that do not include cellphones. Although there are a handful of live-interview polls that still omit cellphones, most of the no-cellphone polls used automated dialers, while some were conducted online.
Here, Mr. Obama’s position is much more tenuous. The no-cellphone polls project him to a popular vote margin of just 1.5 percentage points, 2.6 points lower than in the cellphone-inclusive polls.
Mr. Obama is also losing Florida and Virginia in this version of the model, and has a much slimmer lead, of about one percentage point, in Ohio. Thus, his chances of winning the Electoral College are just 61 percent, as compared with 83 percent in the polls that do include cellphones.
Could the exclusion of cellphones really make that much difference? This gets tricky to analyze because, as I’ve mentioned, the inclusion of cellphones also tends to correlate with other methodological characteristics, like using live interviewers rather than an automated dialer, and other industry-standard methodological practices. Thus, the usage of cellphones may be an indicator of overall poll quality, in addition to making a difference unto itself.
Either way, we also detected a tendency for polls that include cellphones to show better results for Democrats in both 2008 and 2010.
In 2008, there was very little overall bias in the polls. In 2010, however, polling averages partial adjustment based on whether polling firms include cellphones or not. (We assume that a polling firm does not call cellphones unless there is evidence that they do.) But as you can see from these results, the adjustment is a fairly conservative one relative to the practice of omitting polls that skip cellphone votes entirely.
My view is that this adjustment strikes a reasonably good balance for now – this is the first year that we’ve made any sort of direct adjustment for the inclusion or exclusion of cellphones at all.
But if Mr. Obama over-performs the polling averages on Nov. 6, we’ll have to reconsider things for 2014 and 2016, by which time number of Americans who rely solely on cellphones may have grown to almost half of the population.