This report summarizes the preliminary results of research into the effect of the removal of the mandatory message “YOUR RESPONSE IS REQUIRED BY LAW” from the envelopes for the initial mailing and paper questionnaire packages on return rates.
Feedback from stakeholders and respondents has raised concerns about the prominent references to the mandatory participation in the American Community Survey (ACS). In order to be responsive to these concerns we conducted this test. The results of this test show that eliminating the phrase “YOUR RESPONSE IS REQUIRED BY LAW” from the initial mail package envelope and the paper questionnaire package envelope does significantly lower the self-response return rate by 5.4 percentage points. This rate is before the start of computer-assisted telephone interviewing and it pushes additional cases into the more expensive follow-up modes.
We estimate the preliminary cost impact of eliminating mandatory messages from the envelopes in the manner we tested to be an increase in the annual costs of the survey by roughly $9.5 million. This estimate will be refined for the final report. The final report will also provide additional analysis on the impact to hard-to-count populations and the impact on ACS estimates.
It is important to recognize that this test is just the first step in our research to examine the impact of modifying the mandatory messages contained in the ACS mail materials. Additional testing that is being conducted in connection with the September 2015 ACS sample will provide important insights on ways to revise the mandatory messages in a more comprehensive manner throughout the various mail pieces, and other messaging and design enhancements to continue to be responsive to the feedback from stakeholders and respondents while preserving the quality of the survey.
Others in Series
Research for Replacing or Supplementing the ROYA Question on the ACS
Learn more about the potential for using IRS data to determine whether a respondent moved in the past year.
User Preferences for Measures of Sampling Error
Examined two related issues associated with the presentation of information on sampling error for ACS estimates.
Effect on Item Nonresponse of Changes to Failed Edit Follow-up
Conducted evaluation to understand the effect the cut back in follow-up had on data quality.