Can Survey Instructions Relieve Respondent Burden

Survey designers aiming to reduce respondent burden often choose to reduce the number of questions asked of the respondent. However, such a choice may result in a trade-off between respondent burden and data utility. This research investigates whether survey instructions can affect ratings of burden by manipulating the context in which the survey experience is judged. Participants were recruited from Amazon Mechanical Turk to complete an online survey (n=544). Two factors were manipulated in a between-groups design: actual number of questions asked (24, 42) and burden frame (screened in to an extra survey section, screened out of an extra survey section, no screener instructions). The results show a main effect of survey length whereby the objectively longer survey was associated with higher respondent burden. However, there also was a main effect of burden frame whereby being told one was screened out of a longer survey lowers ratings of burden compared to being told one was screened in. Survey instructions are able to influence respondents’ perceptions of the survey experience, regardless of the objective features of the survey, and ultimately affect respondent burden.