U.S. flag

An official website of the United States government

Skip Header


Cognitive Testing of 2020 American Community Survey (ACS) Strategic Framework Mail Materials

Written by:
Working Paper Number rsm2023-14

Abstract

This report presents findings from the cognitive pretesting of redesigned mail materials that are mailed to respondents of the American Community Survey (ACS). The mail materials were redesigned as part of the ACS Strategic Framework Evaluation. The Strategic Framework was developed by the Census Bureau as a framework for evaluating ACS mail materials and developing new materials based on findings from relevant research literature (Oliver et al., 2017). The analytic framework was used to evaluate the various messages presented in the English-language mail materials in use by the ACS at that time (Schreiner et al., 2020). A cross-directorate team was formed to develop new mail materials based on the literature and informed by the analysis of production ACS materials. Part of this strategy involved selecting unique thematic messages to communicate in each of the five ACS mailings. The Census Bureau’s ACS Strategic Framework Mail Materials Design Team developed three new sets of materials, or treatments, to include in the 2021 Strategic Framework Mail Materials Test (2021 SFMMT). A fourth treatment, referred to as Outside Researcher (OR), was developed by researchers outside of the Census Bureau. All four treatments focused on using plain language principles, including the standard language required by the Census Bureau, and incorporating the intended thematic message of each mail piece recommended by the Strategic Framework. In addition to the changes to content, the Design Team incorporated design features such as the use of color, icons, pictures, and novel structure in their three treatments. Each of the three treatments focused on using these novel features in a particular way consistently across mailings to highlight key information and thematic messages in each mail piece. Overall, the iterative rounds of testing confirmed that all four treatments worked well to convey the messages outlined in the Strategic Framework. While the use of color, images, and novel structure in three of the treatments diverged from the typical approach to mail materials taken by the Census Bureau for the ACS, our findings showed that participants still viewed the materials as legitimate and motivating. Post-round revisions ensured that all materials were in good shape for an eventual experimental field test.

Page Last Revised - November 6, 2023
Is this page helpful?
Thumbs Up Image Yes Thumbs Down Image No
NO THANKS
255 characters maximum 255 characters maximum reached
Thank you for your feedback.
Comments or suggestions?

Top

Back to Header