Optimizing the Invitation to Increase the Response Rate in Online Surveys

Low response rates in online surveys threaten the validity of the data collected. This challenge can be effectively overcome by optimizing the invitation of survey participants.

Risks of a Low Response Rate

The numerous availability of online panels consisting of people with a high willingness to participate in online surveys makes it easier to achieve a high response rate. However, for certain target groups, the use of these resources is out of the question. This is the case, for
example, when the opinion of certain buyers is to be collected for the purpose of measuring customer satisfaction.

In studies of this kind, it is not uncommon for the challenge to arise that only a small proportion of those invited take part in the survey. A low response rate results in a smaller sample size than originally planned, which carries different risks:

• Non-response error ² : It is conceivable that in a small sample certain subgroups of the population are underrepresented, e. g. dissatisfied rather than satisfied customers participate in a customer satisfaction survey. This leads to a distortion of the results at the total level.
• Sampling error ² : When sample size decreases, standard error increases. This puts the validity of the insights gained at risk.
• Cost per interview (CPI): A high rate of people not responding to invitations leads to a high CPI.

Reasons for Participating in (Online) Surveys

To analyze the willingness to participate in (online) surveys, the social exchange theory is often used on the one hand⁶. According to the theory, people are most likely to participate in a survey if they trust that the rewards of participation exceed the associated costs⁶. The
rewards or costs are assessed subjectively⁶ and can be of a tangible or intangible nature¹ (for example see Figure 1).

On the other hand, it is argued that survey participation is not always a rational decision but the result of mental shortcuts⁶ . For example, people follow the norm of reciprocity and voluntarily fill out a questionnaire in return after receiving an incentive⁶ .

• Against this background, an effective invitation should aim to: reduce the perceived costs, increase the perception of the rewards associated with participation in the survey as well as enhance the trust in a positive cost-reward balance² , and/or
• trigger mental shortcuts in favor of participation⁶ .

Three-stage Mixed-Mode Invitation to an Online Survey

Ideally, the invitation is made via several contacts to the defined target group, as the number of contacts is positively related to the response rate². Different media, e. g. email, postal letter or postcard, etc., should be used to benefit from their respective advantages and attract the attention of the invitees².

The Prenotification

The first contact with potential participants in the form of a prenotification is already made a few days to a week before the invitation email is sent². A prenotification provides basic details about the study, e.g. about its topic and usefulness, and informs respondents about the upcoming email, making this second contact predictable and serious^2,9. It should be delivered by post, as a postal letter is more credible and has a positive impact on the perception of the importance and benefits of the survey.

A postal prenotification has a positive effect on the response rate⁹. The proportion of participants increases further if an incentive in the form of cash is sent along with this letter⁹. Sending a prenotification leads to higher total costs for the invitation, but may mean lower costs for the invitation per participant due to the strongly increasing response rate⁹.

Table 1: Costs (in €) for different variants of the invitation in the field experiment of van Veen et al. (2015)⁹

The positive effect of prepaid cash can be explained in different ways. An amount of money paid in advance represents a tangible reward for those who are invited⁹. Furthermore, this incentive could initiate a trustful relationship between the researchers and the participants and motivate them, following the norm of reciprocity, to complete the questionnaire in return^6,9.

Before using a prepaid incentive, the research manager should consider the number of potential respondents and the expected increase in response rate to ensure that this additional investment is not disproportionate.

The Invitation

The invitation sent by email contains the link or a QR code to the survey which ensures easy access to the questionnaire. The following recommendations result from current research findings:

• The subject line should contain a plea for help, e. g. “Please help us with your advice and opinions”⁷.
• A personalized salutation is desirable 4.
• Information on what added value can be generated from the data motivates invitees to participate in the survey 3.
• The link to the survey should be positioned at the end of the invitation⁵. This increases the chance that respondents will read the other contents of the email⁵.

The Reminder

One week after the invitation, people who have not yet completed the questionnaire are reminded of the survey². A postcard is suitable for this purpose, which contains only the most important information about the study and can be read quickly².

A reminder increases the response rate⁸. However, there is evidence that post-invitation contacts do not help to diversify the sample, i.e. respondents before and after the reminder tend to belong to the same subgroups⁸. For this reason, reminders are especially recommended if the sample size is to be increased⁸ .

Conclusion

The response rate in online surveys has direct implications for data quality as well as project costs. For this reason, an effective invitation consisting of multiple contacts through different media should be used².

 

The article is based on Hieu Nguyen’s own research and opinion.

 

[1] Albaum, G., & Smith, S. M. (2012). Why people agree to participate in surveys. In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 179–193). Springer Science+Business Media.

[2] Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.). John Wiley & Sons.

[3] Eggleston, C., Childs, J. H., & Nichols. E. Conjoint analysis of survey invitation messages: A new method to understand what moves people to respond [PowerPoint slides]. United States Census Bureau. https://www.census.gov/fedcasic/fc2016/ppt/2_5_Conjoint.pdf

[4] Heerwegh, D. (2003). Effects of personal salutations in e-mail invitations to participate in a web survey. Public Opinion Quarterly, 69(4), 588–598. https://doi.org/10.1093/poq/nfi053

[5] Kaplowitz, M. D., Lupi, F., Couper, M. P., & Thorp, L. (2011). The effect of invitation design on web survey response rates. Social Science Computer Review, 30(3), 339–349. https://doi.org/10.1177/0894439311419084

[6] Manzo, A. N., & Burke, J. M. (2012). Increasing response rate in web-based/internet surveys. In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 327–343). Springer Science+Business Media.

[7] Trouteaud, A. R. (2004). How you ask counts: A test of internet-related components of response rates to a web-based survey. Social Science Computer Review, 22(3), 385-392. https://doi.org/10.1177/0894439304265650

[8] van Mol, C. (2017). Improving web survey efficiency: the impact of an extra reminder and reminder content on web survey response. International Journal of Social Research Methodology, 20(4), 317–327. https://doi.org/10.1080/13645579.2016.1185255

[9] van Veen, F., Göritz, A. S., & Sattler. S. (2015). Response effects of prenotification, prepaid cash, prepaid vouchers, and postpaid vouchers: An experimental comparison. Social Science Computer Review, 34(3), 333–346. https://doi.org/10.1177/0894439315585074