The online survey market is soaring. In 2007 alone, more than 1 billion surveys were completed, leading to concerns about respondent and survey design quality, especially in online panels. With online research influencing billions of dollars in marketing and product decisions, organizations need to be certain their decisions are based on accurate, high-quality data, or they risk impacting their bottom line. Researchers have sought to identify the elements leading to bad data and to quantify the impact these elements have on business decisions. Michael Conklin will discuss methods for quantifying the risk-benefit trade-off of taking actions to eliminate “bad” respondents. He will also discuss compiled research findings including:

• Even a small proportion of bad respondents caused research risk to increase exponentially. For example, if a survey is based on responses from sample that has 30 percent invalidated respondents, the risk of making a wrong decision is 100 percent higher.
• As sample size increases, risk increases. Surprisingly, increasing the sample size, a tactic many companies employ in an attempt to counteract the effect of bad respondents, actually increases the risk of making a wrong decision instead of diminishing it. 
• Poor survey design increases risk. Creating a survey that causes respondents to speed, straight-line or even abandon a survey can skew results and lead to an increased risk of making a wrong decision.