If you found yourself nodding in agreement to last week's post -- about the differences between good online respondents and bad ones -- you're probably looking for some tips about how to get more of the good. We've identified three potential problem areas, and how to take measures to improve the effectiveness in each one.
Where are your respondents being recruited from? Broadly speaking, the options are to recruit from
If budget and criteria allow, we prefer to recruit respondents from lists, ideally with a phone conversation being part of the recruiting process.
Sometimes panels are necessary, for example when rapid survey turnaround is needed or sometimes for cost reasons. But the hard truth is that panel members are often motivated primarily by incentive rather than being interested in the brand or the topic at hand. This leads to poor engagement and requires careful recruiting and management.
To mitigate the issues of panels, here are some rules of thumb:
The problems at the participant-level were outlined last week. To summarize,
A comprehensive online screener needs to include the following:
If you only listen to the companies that are paid to recruit respondents, you might start to believe that "you get what you pay for." They'll tell you the way to get better information is to pay the respondent more money.
There is some truth in that. If a respondent feels like they are spending more time on our online study than they thought they would have to spend, and the compensation is low, you're likely to get cursory responses and little followup.
At the same time, though, there's an interesting paradox. We have found that if incentives are too high it also creates a problem. A high incentive can attract the least desirable study respondent, the type who will say anything to get into the study just so that they can do as little as possible to get the compensation.
The sweet spot, then, is a dollar amount where participants feel adequately respected for the time they are spending to participate in the study, but is not so high as to tempt in the riffraff. What that dollar amount actually is varies widely. A study of lower-income homeowners will not need to pay anything near what is needed for a study of corporate tax lawyers. But in either case, that balance must be found.