We ran three polls of Montana: May 12, 19, and 25 (12 PM MT to 7 PM MT). On the May 12 poll we had Republican Gianforte +12 and we had Gianforte holding that lead a week later on May 19. We were not planning on a running an additional poll until the Gainforte assaulted a reporter on Wednesday night, May 24. We ran a poll from 12 PM MT to 7 PM MT on Election Day and posted the results at 7:45 PM MT (15 minutes before the polls closed). The key findings:
High level of accuracy. We sent out Gianforte +5.5 and he won by +6.
1) Effect of Assault: 70 percent or so of voters had already voted by mail, but still Gianforte fell from +12 to +5.5 (he actually won +6). Omitting non voters, we polled: Gianforte 48% and Quist 43% . The ground truth was Gianforte 50%, Quist 44%. So, we are very confident that the assault had a big impact, but it was somewhat dulled because Gianforte voters did not switch in high numbers to Democrat Quist, they moved to the Libertarian or did not vote. If there was a new vote today, with no early voting, we believe the election would be a perfect toss-up (leaning Quist by +1).
Only 17 percent of Republicans said that Gianforte was not justified in assaulting the reporter. Let that sink in for a second; not great for the democracy. 85 percent of Democrats believed that he was not justified.
2) Polling Methods: During the 2016 election we were reporting on our blog on our experimental polling with online and mobile-based polls (for Montana we added in IVR) that were then modeled and post-stratified using “big data”. A primer of our methodology:
First, we model the raw responses to each question, given each respondent’s age, gender,, education level, race, and party identification. This information divides the population into thousands of demographic categories. For each sub-group and poll question, we predict the percent of people that would provide each answer if the entire country showed up to the poll using a variant of shrinkage Machine Learning classifiers (in this case, L1 regression with shrinkage varying by parameter).
Second, we project our estimates for each sub-group onto our best estimate of the likely adult population, a process known as post-stratification. Specifically, we weight our predictions by the fraction of that sub-group in the overall target population. We derived the target population starting from Census data, the American Community Survey of American citizens, and then imputed turnout propensity of the 2016 presidential election and partisan identification from the full voter file. Individual-level party identification is derived from a predictive machine learning algorithm leveraging polling data with N>10,000. In this case, we calibrated by the known outcome of the 2016 presidential election in Montana. While this gives us a projection space that does not inform us about the total expected voter population, it gives us the relative composition of the electorate. Two things made polling in MT very hard in addition: Basing the likely voter population off of 2016, we knew we were skewing Democratic, and the nature of the election-day poll, after the Gianforte incident, would lead to some social desirability bias, in that some respondents would claim they voted Quist but a) already voted (almost 70% of vote was cast early), or b) did not really change the intention. Hence, we also included vote intention of the 2016 gubernatorial race between Bullock and Gianforte, for which the outcome was known. Getting this question relatively right based on our methodology gave us calibration and confidence for our top-line results for the MT special election.
In 2016’s presidential race, we did not have the strength of conviction in our experimental polling to push it hard, but we had Trump sweeping through the upper-Midwest, and even leading in Florida and North Carolina. We will not make the same mistake again; we want this polling disseminated widely. This polling is fast (as we showed last night), cheap (1/10 or so the cost of traditional poll), flexible in what we can ask, and, as we have shown again and again: accurate! It will open the door to more questions being asked about more detailed constituencies. It can move us all past the topline horse-race into more depth on issues, underlying indexes of populism, and knowledge of facts and information.
3) Populism: In the same polls we asked questions about healthcare and taxes (May 12 and 19: results below are May 19). What we see is that Montana voters have overwhelming support for the public option and higher taxes on household income over $250,000. It is very encouraging that 45 percent of Montana voters think/know that GOP/Trumpcare will raise rate of uninsured, but only 15 percent think that at least half the Trump tax cuts will go to households that make $250,000 or more. Estimates of earlier plans had about half going to households making $700,000 or more. While this gap is a suitable target for massaging campaigns, the source of this divergence can be either lack of information or partisan motivated reasoning. Of course, only the former can lead to positive net change.
4) Trump doing OK: Trump won Montana by 22 pp, but now has a positive approval of 51 percent to 41 percent (steady across all three polls). 10 percent of Trump voters disapprove of him and 8 percent neither approve nor disapprove. Only 2 percent of Clinton voters approve of him at all and 91 percent “Disapprove Strongly”. Most interesting is that Trump’s strong approval and disapproval is the same, as many Trump supporters “Approve Weakly”. Trump has lost support in Montana, but is still above water.