Many posts have discussed the problems of surveying public opinion in the 21st century.
Online opt-in polls have become increasingly popular. And for some purposes, such as election polling, they can perform similarly to more traditional survey approaches.
There is evidence, however, that the online environment in which they operate is somewhat unstable.
In particular, several recent studies have documented large errors in online opt-in surveys due to the presence of so-called “bogus respondents.” These respondents do not answer questions sincerely; instead, they attempt to complete surveys with as little effort as possible to earn money or other rewards.
Studies have shown that bogus respondents can cause opt-in surveys to overestimate rare attitudes and behaviors, such as ingesting bleach to protect against COVID-19, belief in conspiracies like Pizzagate or support for political violence.
At Pew Research Center, we’ve found that this type of overreporting tends to be especially concentrated in estimates for adults under 30, as well as Hispanic adults. Bogus respondents may be identifying this way in order to bypass screening questions that might otherwise prevent them from receiving a reward, though the precise reasons are difficult to pin down. Whatever the underlying cause, the result can be unreliable estimates for those groups.
For example, in a February 2022 survey experiment, we asked opt-in respondents if they were licensed to operate a class SSGN (nuclear) submarine. In the opt-in survey, 12% of adults under 30 claimed this qualification, significantly higher than the share among older respondents. In reality, the share of Americans with this type of submarine license rounds to 0%.
