Cognitive Testing of Translations of ACS CAPI Materials in Multiple Languages
Yuling Pan, Marjorie Hinsdale, Alisú Schoua-Glusberg, and Hyunjoo Park
KEY WORDS: Translation of survey materials, cognitive testing, multiple languages
In an effort to ensure high-quality data from linguistically isolated populations included in the American Community Survey (ACS) sample, the ACS language team at the Census Bureau took the initiative to develop translations of ACS supporting documents into five languages: Spanish, Chinese, Korean, Russian, and Vietnamese. This report documents a study conducted by the U.S. Census Bureau, in collaboration with RTI International and Research Support Services, that cognitively pretested translations of survey letters and brochures in multiple languages (Spanish, Chinese, Korean, and Russian).
The study was designed to include the following steps: 1) organize a panel of experts in each language consisting of survey methodologists and language experts; 2) adopt the committee approach in translating interview protocols from English into target languages; train language experts for cognitive interviews; 3) conduct cognitive interviews in multiple languages; and 4) recommend changes for translations through the panel of experts. Using this approach and design, we successfully identified a set of issues that are not normally informed by the traditional translation-review process. In addition to translation issues, we identified problems caused by differences in preferred norms of communication or presentation styles, and variations in culturally-driven expectations regarding certain topics. These issues affect respondents’ reaction to and interpretation of the messages conveyed in the translated materials.
Findings from this project demonstrate the importance of pretesting translation in the target language, effectiveness of the committee approach in survey translation, and the value of pairing survey methodologists with language experts for cognitive testing in non-English languages. This report documents all aspects of this research, from recruitment through protocol development to results of cognitive interviews.