Sept. 9: Call It as You See It

I’ve been picking up some sentiment from analysts and journalists in my Twitter feed recently, who correctly note that polling around the party conventions can be volatile. They suggest that we ought to wait for more data before concluding very much about the bounces that the conventions have produced.

I’d love to have more data. I’d love it if we had a dozen national tracking polls rather than four. I’d love it if we had a pollster who was spending tens of thousands of dollars to poll every single swing state every single day.

It’s also the case that we’ll know more about the state of the race in two weeks than we know now — and we’ll know more about in four weeks than we do in two.

But we publish our forecasts every day. The goal is to make what we hope is the most accurate possible forecast given the information available at that time.

Saying “wait for more data” sort of misses the point. What about the data that we have on hand already? Is it compelling enough to suggest that there has been some change in condition of the race? Or isn’t it?

That’s really the central question that we seek to answer with our forecast models. On some days, the trend is a little more obvious than on others. But we make forecasts when it is easy to do so, and we make forecasts when it’s hard.

One of my pet peeves, when people go back and look at how different forecasting methods have performed, is that they tend to focus only on the last forecast that was produced just in advance of the election. Our forecasts did pretty well by that standard in 2008, and in 2010. But the forecasts we make on Election Day morning are the easy ones! There are tons of polls in every state, very few undecided voters left in the race, and no time remaining for global or economic events to intervene. There is still some skill required to go through the electoral math in elections that are close enough for it to matter. But otherwise, our forecasts are likely to be very, very close to those of our competitors by Election Day.

Instead, where a forecasting model has the potential to add more value is precisely when the information is murkier — either by (correctly) discerning a trend a few days before other methods do, or (just as importantly) by avoiding attaching too much significance to one based on flimsy evidence.

The most impressive thing that our model did in 2008, in my view, was not in “calling” all but one of the states right on Election Day. There was very little doubt about who was favored in perhaps 46 or 47 of these states. The other three or four were tossups — and whether you guessed the winner right had as much to do with luck as skill.

Rather, it was what the model did in September of that year, when it detected very, very quickly after the collapse of Lehman Brothers that John McCain’s goose was cooked, with Barack Obama’s projected probability of winning the Electoral College increasing by about 25 percent in a period of just 48 hours.

We’re not seeing anything quite that dramatic in the polls right now. Nevertheless, the polling movement that we have seen over the past three days represents the most substantial shift that we’ve seen in the race all year, with the polls moving toward Mr. Obama since his convention.

How far will Mr. Obama’s numbers rise, and how long will his bounce last? We don’t know that, of course. But the range of possible outcomes reads pretty favorably for him.

As I wrote on Saturday night, Mr. Obama’s polls could easily cool off quickly. If we return to the equilibrium where Mr. Obama is about two points ahead in the polls — about where they were for months on end heading into the conventions — then Mitt Romney’s position won’t be too badly damaged. Still, Mr. Romney will be the underdog, and he’ll have had two or three weeks of time run off the clock.

Or, if Mr. Obama’s bounce is more prolonged and more pronounced, Mr. Romney could be in pretty bad shape.

Our goal is not to err on the side of caution any more (or any less) than we think is warranted by the data. There is absolutely, positively the chance that we will jump too far ahead of the trend. I can all but guarantee you, in fact, that if we run these forecasts for enough years, there will be a few instances in which the model makes an aggressive move — and is totally and completely wrong about it.

It’s just that we see being too slow to detect a trend as being an equally bad mistake. We don’t want to be like the credit-rating bureaus, which often slowly (and predictably) downgrade a country in a series of a half-dozen or more steps, gradually reacting to highly salient information long after canny investors had already figured it out.

Finally, keep in mind that our forecasts normally are pretty stable. More often than not, our forecast model’s calculations suggest that a trend that gets the pundits excited probably isn’t all that meaningful once you examine it more carefully.

An apparent trend in early August, for instance, in which Mr. Obama had a streak in which he posted strong-looking numbers in a series of national polls, was one that the forecast model was largely dismissive of. There hadn’t been much of a shift toward Mr. Obama in the national tracking polls, nor much movement one direction or the other in the state polls — and those are richer data sets in many respects because they allow you to make comparisons against a constant baseline. Once we weighed all of the evidence, it looked as if these polls contained more noise than signal.

What’s different in this case is that there has been fairly unambiguous movement toward Mr. Obama in each of the four national tracking polls. On Sunday, Mr. Obama further expanded his lead in the tracking polls run by Gallup, Rasmussen and the RAND Corporation. His lead held at four points among likely voters in the Ipsos tracking poll, however. (Although he gained ground among Ipsos’s broader sample of registered voters).

Some Silver Linings for Romney

And yet, I don’t think Sunday’s data was as unambiguously good for Mr. Obama as the polls published on Friday or Saturday. His approval rating declined to 50 percent from 52 percent in the Gallup poll, for instance. Since Gallup’s approval rating tracking is based on a three-day average, whereas its horse-race polling is based on a seven-day average, the approval numbers can sometimes be a leading indicator for where their horse-race numbers will go.

A 50 percent approval rating is still reasonably strong for Mr. Obama, especially since Gallup has normally had middling approval ratings for him. But perhaps there is the hint in this data that Mr. Obama’s best polling days came late last week, and his numbers over the weekend were decent but not great. The counterpoint to this would be the Rasmussen poll, which is conducted over a shorter, three-day window, and in which both Mr. Obama’s approval ratings and his head-to-head numbers improved in the edition of the poll published on Sunday.

In addition, a pair of state polls published by Public Policy Polling on Sunday night were decent for Mr. Obama, but not really more than that.

Public Policy Polling put Mr. Obama ahead by five points in Ohio, and by one point in North Carolina. But once you consider that Public Policy Polling’s surveys have been a point or two Democratic-leaning this cycle, his numbers don’t look quite as strong, and Mr. Obama made only modest gains from the numbers Public Policy Polling had been publishing in these states earlier in the summer.

Further, our forecast model does assume that Mr. Obama’s numbers are likely to be slightly inflated by the polls right now — perhaps by a point or two, is our best guess — because of his convention.

What makes it difficult to get an intuitive read on the polls right now is that some of them (like the Gallup tracking poll) still contain data from before Mr. Obama’s convention, during what should have been the midst of a Republican convention bounce.

The forecast model, however, does consider this level of detail in adjusting for the effects of the party conventions. It considers exactly when the polls were in the field, and what fraction of their interviews were conducted at different stages of the convention period.

The upshot is that the forecast model is anticipating a gain of another point or two for Mr. Obama in the slowly-updating polls like Gallup, as they fully roll over to post-convention data. If Mr. Obama’s numbers just hold steady in the Gallup poll instead, it will count that as a slight negative for him.

Still, I don’t want to lose the larger point that the data flow over the past several days has been very positive for Mr. Obama, especially in comparison to the small bounce that Mr. Romney received after his convention.

As for waiting for more data? Well, the odds are that we won’t have to wait for very long. The volume of polling typically picks up substantially after the conventions, and by the middle of this week, we should have a fairly keen sense, at least, for where the race stands right now — even if we can never be totally sure where it’s headed.

Comments are no longer being accepted.