Turn on the news or read through much of the analysis put out by some of our friends, and you’re likely to hear a lot of talk about “momentum”: the term is used about 60 times per day by major media outlets in conjunction with articles about polling.
When people say a particular candidate has momentum, what they are implying is that present trends are likely to perpetuate themselves into the future. Say, for instance, that a candidate trailed by 10 points in a poll three weeks ago — and now a new poll comes out showing the candidate down by just 5 points. It will frequently be said that this candidate “has the momentum”, “is gaining ground,” “is closing his deficit,” or something similar.
Each of these phrases are in the present tense. They create the impression that — if the candidate has gone from being 10 points down to 5 points down, then by next week, he’ll have closed his deficit further: perhaps he’ll even be ahead!
There’s just one problem with this. It has no particular tendency toward being true.
Say that a candidate has improved her position in the polls from August to September. Is her position more likely than not to improve further from September to October?
This constitutes a testable hypothesis. I took our database of Senate polls, which includes all nonpartisan polling in Senate races going back to 1998. I then divided the polls roughly into fifths, based on the number of days before the election that they were in the field:
Period A: Polls conducted 1-10 days before election.
Period B: Polls conducted 11-25 days before election.
Period C: Polls conducted 26-60 days before election.
Period D: Polls conducted 61-150 days before election.
Period E: Polls conducted 151+ days before election.
Let’s start with the period that is probably of the most interest to us right now. Say that a candidate has gained ground in the polls from Period C (which consists of polls conducted a month or two before the election) to Period B (polls conducted a couple weeks before the election). Is she likely to continue gaining ground as we move from Period B to Period A (polls conducted immediately before the election)?
The best way to look at this is via a scatterplot. On the horizontal axis, we’ll plot the change in the polling average from Period C to Period B. So, a figure of “D+5″, for instance, indicates that the Democrat had gained 5 points in the polls over this period (say, for instance, she’d gone from being ahead by 4 points to being ahead by 9). A figure of “R+5″, likewise, indicates that the Republican had gained ground over this time frame. On the vertical axis, we’ll plot the same figure for the change in the polling average between Period B and Period A.
(By “polling average”, by the way, I mean a simple average of all polls conducted during that period — nothing fancy at all — although I do require that at least 2 polls were conducted in each period for the race to be included.)
Now, if there is “momentum”, what we should expect to see are the dots lining up (at least to some extent) along a diagonal line from bottom left-hand corner of the chart to the top right-hand corner. That would indicate a positive correlation (specifically, a positive serial correlation) between the direction in which polls are moving from one period to the next.
But this is not what we see at all:
Instead, the dots are all over the place. Sometimes, a candidate who has gained ground in the polls continues to do so; otherwise, the trend reverses itself, or the race simply flatlines.
If anything, there is a slightly negative serial correlation — that is, a candidate who gains ground in the polls from Period C to Period B tends to lose ground from Period B to Period A — although it is not quite statistically significant.
There is also no sign of momentum we look at the change in polling between other periods. For instance, there is no correlation between the change in polling between Period D and Period C and the one between Period C and Period B.
Nor is there a relationship between the direction in which the polls move between Periods E and D, and the direction in which they move between Periods D and C.
Although this analysis might seem modestly technical, the conclusion is really pretty simple. In general elections, the direction in which polls have moved is not predictive of the direction in which they will move.
Thus, it is usually wrong to say that a candidate is gaining ground in the polls — present tense — or that her position is improving. Instead, you should say that the candidate has gained ground or that her position has improved.
Now, I don’t mean to suggest that “momentum” never exists. Surely there are cases where it does. But it does not exist as a general rule on the basis of the polling — at least not in general elections.
(Primaries may be a different story entirely; I haven’t studied primary polling in nearly as much detail as general election polling, but I would expect that you might find fairly strong serial correlation there.)
If you’re suggesting that a candidate has the momentum, rather, you are advancing a hypothesis rather than stating a fact — and that hypothesis should probably be rooted in factors above and beyond the direction in which the polls have moved in the past.
For instance, you could say something like:
The polls have moved in Mr. Strickland’s direction. This is because he is a stronger retail politician than Mr. Kasich. Since this will continue to be an advantage for him, the polls should continue to move in his direction as the campaign progresses.
Or:
Generic ballot polls have moved in Republicans’ direction. This is because the unemployment picture has not improved, but voters are assigning more and more blame to President Obama for unemployment the longer that he has been in office. Since most economists expect employment to remain flat, but voters will continue to assign more responsibility to Mr. Obama each month, I’d expect this trend to continue.
These are perfectly reasonable hypotheses — although only time can usually tell whether or not they have the virtue of being true.
There is one last test that we ought to perform. So far, we have only been talking about how polls move from one point in time to the next. What we’re ultimately concerned with, though, is what the polls tell us about the ultimate outcome of the election.
Say that a candidate improves his position in the polls in the final 10 days of the campaign — in our terminology, this means he gains ground from Period B to Period A. Do such candidates tend to overperform their polls on Election Day?
Again, the answer is no.
There were 34 Senate candidates in our sample whose position in the polls improved by at least 3 points between Period B and Period A: that is, they had a late “surge” in the polls. Of these candidates, 22 of the 34 — roughly two thirds — actually underperformed the late polls on Election Day (although the tendency is not quite statistically significant).
Now, this does get a bit more complicated because the number of polls conducted in each race is also something one must consider. In particular, the slight tendency toward negative serial correlation (that is, the tendency of “momentum” to reverse itself or to overshoot the mark) tends to dissipate in races with abundant polling. In races with lots of polling, instead, the most robust assumption is usually that polling is essentially a random walk, i.e., that the polls are about equally likely to move toward one or another candidate, regardless of which way they have moved in the past.
In races where there isn’t a lot of polling, however, you should not place that much of a premium on the recentness of a poll. Instead, you’ll make notably better forecasts over the long-run by continuing to include “old” polls in your average when a sufficient number of newer ones aren’t available.
Our forecasting models account for this tendency. We sometimes take a lot of heat for the fact that our forecasts can seem “sluggish” –that is, that our models can need a fair amount of evidence before they are convinced that the likely outcome of a race is significantly changed. We’re hearing a lot of this criticism for our forecast in Pennsylvania, for instance, where Joe Sestak has almost certainly closed his deficit with Pat Toomey, but where we still consider Mr. Toomey the favorite.
But our models are rooted in the empirical evidence — whereas this is another place that our intuitions about polling can sometimes lead us astray. We can look at a pretty chart, and see the lines converging toward one another: our brains tend to assume that they are eventually bound to cross.
But at least as often, they do not. And a candidate’s “momentum” proves to be ephemeral.