Descriptive, but not Injunctive, Normative Appeals Increase Response Rates In Web-based Surveys

Abstract

Background: In an earlier experiment Misra, Stokols, & Marino (2012) found that participants who received a descriptive normative prompt in the message requesting them to complete an online survey were more likely to comply with the request compared to participants who did not receive any normative prompts.

Purpose: Building on that earlier study, the present field experiment compared the separate and additive effects of descriptive and injunctive norm- based persuasive messages on response rates of online surveys. We also investigate the influence of email reminders on response rates.

Intervention: Participants in an interdisciplinary conference were assigned to one of four groups. The three experimental groups received one of the following messages asking them to complete an online survey that highlighted: (1) a descriptive social norm indicating typical response rates among attendees of prior similar conferences; (2) an injunctive norm appealing them to join fellow participants in completing the survey; or (3) both social norms. The control group received a generic request to complete the online survey without any norm-based appeals.

Research Design: This study used an experimental design which afforded a within subjects replication of participants’ assignment to each of two treatment groups and the control prompts for two successive surveys.

Data Collection and Analysis: One hundred and twenty nine conference participants were requested to complete an online questionnaire about their experiences of an interdisciplinary conference in Fall 2011, immediately following the conference (Time 1) and then three months later (Time 2). At both time points, participants were given three weeks to complete the online surveys. For both time points, participants who had not completed the survey one week after they were sent the initial request to complete the survey received an e-mail reminder. One week after the first email reminder, participants who had still not answered the survey were sent a second email reminder. Once the online surveys were closed at both time points, response rates were calculated. To evaluate the differences in response rates among the various treatment and control groups at both time points, we conducted one-tailed z-tests for proportions to ascertain the z-scores.

Findings: Participants receiving a message highlighting the descriptive social norm when asked to complete an online survey were more likely to comply with the request compared to all the other groups. Additionally, one and two email reminders were found to be effective in improving response rates of online surveys.

Keywords: response rate; online surveys; norm-based persuasive messages; descriptive norm; injunctive norm; research on evaluation