Gallup Whines About Nate Silver

It couldn't have been only me when night after night approaching the election the Gallup poll repeatedly showed Romney leads on the order of 51%-45%. This was when most other polls either showed the race very close or Barack Obama with a 2-3 pt. lead. The most accurate consistent polling analyst has been Nate Silver who ended up predicting an Obama victory with 332 electoral votes, with a 90.9% probability.

Meanwhile, Rachel Maddow, in a show after the election, covered the results of some 24 polling organizations and pollsters across the campaign, and lo and behold, good ol' Gallup came in around 21st. They were consistently one of the least accurate polling groups, maybe next to the Repuke interior pollsters. What gives?

Well, rather than commend Nate Silver Gallup honcho Frank Newport, published  a comment on Friday, to give a decided impression of sour grapes. He wrote in part:

"Some of this will result from a variant of the venerable “law of the commons.” Individual farmers can each made a perfectly rational decision to graze their cows on the town commons. But all of these rational decisions together mean that the commons became overgrazed and, in the end, there is no grass left for any cow to graze. Many individual rational decisions can end up in a collective mess.


We have a reverse law of the commons with polls. It’s not easy nor cheap to conduct traditional random sample polls. It’s much easier, cheaper, and mostly less risky to focus on aggregating and analyzing others’ polls. Organizations that traditionally go to the expense and effort to conduct individual polls could, in theory, decide to put their efforts into aggregation and statistical analyses of other people’s polls in the next election cycle and cut out their own polling. If many organizations make this seemingly rational decision, we could quickly be in a situation in which there are fewer and fewer polls left to aggregate and put into statistical models. Many individual rational decisions could result in a loss for the collective interest of those interested in public opinion.

This will develop into a significant issue for the industry going forward.”


Well, first of all, let me say that the first paragraph and opening sentences of the 2nd amount to a cheap shot. They basically denigrate Silver for not carrying out his own "original" polling, and instead using readily available numbers, data and subject them to his own analyses. In doing so, Newport gives no credit at all for the considerable quantification skills needed to use existing data to arrive at very firm, consistent and quality predictions. It would be analogous to another solar researcher bitching that my superior solar flare prediction model (say forecasting a given flare with 95% accuracy) was "inferior" because the data I used to formulate my statistical model wasn't done by me personally, but instead assembled from others' data.

Question: Then if my approach and data assembly techniques were so inferior, how come no one else thought of doing it to enhance the quality of flare predictions overall? Isn't it preferable to have a superior prediction model, however it is crafted, than to settle for inferior models .....because they arrive from one's "original" tabulations of data?

Newport's barely veiled threat of putting Gallup's poll results "in aggregation" and "cutting out their own polling" is laughable to be sure. Given Gallup's low performance many people will simply say "RIGHT on! Do it!" Save us the trouble of looking at your stupid, outlier polls for which we're then left scratching our heads, and responding 'HUH?' Same goes with some of Gallup's crazy polls showing a significant Romney lead among women when anyone looking at the entire array of polls knew that was bollocks.

Newport's last point that "if many organizations make this seemingly rational decision, we could quickly be in a situation in which there are fewer and fewer polls left to aggregate and put into statistical model" appears reasonable as far as it goes, but takes no account of the fact that too many polls already exist, and most are redundant. Rachel on her 11/7 show displayed 24 polls and differing poll performance levels,. and just from a statistician's eye view I'd say that perhaps one fourth or more of them are redundant. Hence, fewer quality polls would arguably be better, and also provide Nate Silver with an enhanced probability of improved future forecasts for his five thirty eight blog.

So, Nate didn't pick up his own phone and call voters himself. Big deal! So when I have done solar flare data forecast analyses I didn't actually take all the H-alpha or sunspot images myself but often used those from other sources.  This doesn't change the fact that a standard method of analysis must still be applied to the data, irrespective of source! Again, Newport mixes apples and oranges, losing sight of the basis for a quality forecast by being overly obsessed with the modus operandi.

As far as Nate goes, to his credit he consistently pointed out that Gallup’s results were oddly inconsistent with what other pollsters were finding, and he was right — Gallup got it wrong.  Way wrong in its final polls, perhaps seeding massive delusions in certain repuke brains that an Obama rout was imminent.

Nate Silver is the one who deserves congrats, and Frank Newport and Gallup a bucket of way sour grapes!