Who gets your vote in 2012?

Barack Obama (BO)	Mike Huckabee (MH)
Email:

<< Previous | Next >>

Sep. 8, 2007 - News

How to Estimate the Confidence Level of a National Poll

Is it possible to come up with a measurement of how confident we are in the results of a national poll based on state polls? It is, if we make some assumptions. Let us assume that we can come up with out own estimate of the national percentage of registered Democrats or Republicans who will vote for a given candidate based on the state poll results for a given month. This can be done, for instance, by computing a weighted average of the percentages of each candidate in each state, weighted by the population of each state. Suppose we have d candidates (other not included). This corresponds to d degrees of freedom. Let E_i denote the expected numbered of surveyed registered voters who will vote for candidate i, where i ranges from 1 to d. Now consider a national poll of N registered voters which reports a total of O_i polled registered voters supporting candidate i. From this information, we may then compute the chi-squared statistic from the following formula:

chi-squared = sum((O_i - E_i)^2/E_i), (1)

where the sum is taken over the N candidates, i.e. the index i ranges from 1 to N. Finally, we may use a chi-squared table to compute the confidence level of the national survey.

Consider the following hypothetical example. Suppose Polling Organization X reports the following national poll results for the Democratic candidates:

Now suppose we compute the following expected percentages based on our weighted average of the state polls for the past month:

From our data, we see that d=3. We also compute the following:

Using formula (1), we compute chi-squared= 5.804. Using an online chi-squared calculator, we find the confidence level to be p=0.1215. This means that there is a 12.2% probability that assuming our estimate is correct and that Organization X performed a truly random sample of registered Democratic voters nationwide, the chi-squared statistic would be at least as large as what we calculated. Although this probability is rather small, it is not considered small enough to conclude that Organization X's poll is not reliable. For this to be the case, the confidence level p would need to be less than 5%.