At CNN, Jennifer Agiesta reports on CNN's standards for reporting on polls.
There is not one correct way to answer the 16 questions that make up our standards questionnaire, but there are some things that have not met CNN's standards for reporting in the past and will not now. And there are some new, clear lines that define the way polls should be done today. We will not cover:
- Polls that are conducted by campaigns or by those who have a financial or advocacy interest in the outcome
- Polls that are conducted by telephone using robocalls rather than live interviewers
- Polls that are conducted without any type of sampling, where anyone who chooses to can participate
- Polls that are conducted solely using unrepresentative sample sources
- Polls that are conducted without accounting for people who take surveys on their cellphones, either over the phone or by web
- Polls on some topics which do not ensure that respondents of all education levels are adequately reflected
CNN’s Transparency Questionnaire
1. What survey firm conducted the poll?
2. How were respondents interviewed – by live interviewers on the phone, IVR, online, self-administered questionnaire or another method?
3. Who paid for the survey and why was it done?
4. How many people were interviewed for this survey?
5. In what language(s) were respondents interviewed?
6. Please provide a copy of the full text and interviewer instructions/programming for all questions included in this survey release.
7. When was your survey conducted?
8. What is the source of your sample for this survey, and by what method were respondents selected? Please be as specific as possible, and if via web panel(s), please include a description of how the panelists were recruited. If your study was conducted online and included respondents chosen via routers, approximately what percentage of respondents were directed to the survey via routers?
9. If any quotas were applied to sampling or interviewing, at what stage were they applied, what variables and targets were used, and what is the source of your estimate of the target quota?
10. What is the universe of people you are trying to survey, and what makes you confident that the sample source represents that universe?
11. If surveys were conducted by telephone, what percentage of interviews were conducted via calls to cellphones? If surveys were conducted online, were respondents allowed to complete the survey via mobile browsers, and approximately what share of your respondents did so?
12. If surveys were conducted by telephone, how many callback attempts did a sampled number receive before being retired?
13. If surveys were not conducted by a live interviewer, what do you do to ensure your respondents are real people and are paying attention to the survey?
14. What is your estimate of this survey’s error, how is it calculated, and why is this an appropriate error estimation for your survey? If you are reporting a margin of sampling error, has it been adjusted for design effects?
15. If your survey has been weighted, please list the weighting variables and the source of the weighting parameters. If your survey has not been adjusted for education, please explain why and provide an unweighted frequency for education distribution among your respondents.
16. Is there a minimum unweighted sample size you require before releasing any subset estimates, and if so, what is it?
