Methodology for Cognitive Testing of Translations in Multiple Languages
Yuling Pan, Ashley Landreth, Marjorie Hinsdale, Hyunjoo Park, and Alisú Schoua-Glusberg
KEY WORDS: methodology for cognitive testing, translation, multilingual, cultural issues
ABSTRACT
With the growth of multinational and multilingual surveys, pre-contact letters and informational brochures are
translated from the source language into target languages to convey legally required information and to
encourage survey participation. Due to variations in linguistic conventions and communication styles across
cultures, translated materials may have different effects on target populations. In order to ensure that translated
documents convey the same messages and have the equivalent communicative effect as original texts, survey
researchers need to develop sound methods to pretest translations in target languages.
This paper reports a study conducted by the U.S. Census Bureau that cognitively pretested translations of
survey letters and brochures in multiple languages (Spanish, Chinese, Korean, and Russian). The study was
designed to include the following steps: organize a panel of experts in each language consisting of survey
methodologists and language experts; adopt the committee approach in translating interview protocols from
English into target languages; train language experts for cognitive interviews; conduct cognitive interviews in
multiple languages; and recommend changes for translations through the panel of experts. Using this approach
and design, we successfully identified a set of issues that are not normally informed by the traditional
translation-review process. In addition to translation issues, we identified problems caused by differences in
preferred norms of communication or presentation styles, and variations in culturally-driven expectations
regarding certain topics. These issues affect respondents’ reaction to and interpretation of the messages
conveyed in the translated materials.
Findings from this project demonstrate the importance of pretesting translation in the target language, the
effectiveness of the committee approach in survey translations, and the value of pairing survey methodologists
with language experts for cognitive testing in non-English languages. This study thus aims to explore
methodologies and best practices for pretesting the appropriateness and adequacy of survey translations in
multiple languages.
CITATION:
