U.S. Department of Commerce

Research Reports

You are here: Census.govSubjects A to ZResearch Reports Sorted by Year › Abstract of RSM2007/18
Skip top of page navigation

The Impact of Instructions on Survey Translation: An Experimental Study

Brian Kleiner, Jerelyn Bouic, Yuling Pan

KEY WORDS: Survey translation, translation quality, translation instructions


In calling for faithfulness to intended meaning as a guiding principle of survey translation, Harkness and Schoua-Glusberg (1998) argue further for the need to provide translators with documentation so that they can better understand the intended readings and research aims of survey items. The authors also point out that translators should be given guidelines regarding an acceptable degree of adaptation. In actuality, however, translators are rarely provided with such documentation and guidelines, and while doing so seems reasonable in theory, there is currently little empirical work that lends support to the utility of this practice. The experimental study described in this paper examined the impact of providing special instructions and supporting material to translators. Specifically, it addressed whether translators provided with explanatory text and guidelines were able to produce translations that were more faithful to the intended meaning of source survey items and more culturally appropriate and natural sounding compared to translators who received no such guidance.

To assess the impact of different types of instructions, 27 professional translators translated an English source instrument into three target languages—Mandarin Chinese, Spanish, and Canadian French (9 translators per language)—following one of three sets of instructions (translators were randomly assigned to three instructional subgroups). The first subgroup received a core set of instructions. The second and third subgroups received the core instructions as well as question by question explanations of the intended meaning of source items, and were told to translate in a way that was faithful to the intended meaning of the source items. The third subgroup received an additional instruction to translate in a natural sounding way with respect to the cultural and linguistic norms of the target population.

Once the translations were completed, 15 professional survey researchers (5 per language) who were native speakers of the target languages conducted blind evaluations, with each examining three translated versions of survey items (that followed the three sets of instructions). The evaluation involved rating each of the translated survey items on Likert scales along three dimensions—overall quality, faithfulness to intended meaning, and cultural appropriateness. Telephone interviews were subsequently conducted with the evaluators to better understand their beliefs and practices regarding survey translation and how they approached the evaluation task.

Study findings indicate that while the provision of special instructions and documentation to translators had a significant impact on their translations, the direction of the impact (positive or negative) differed across the target languages, according to ratings of the survey researchers. We attribute these differences to the beliefs of the survey researchers and their level of commitment to two conflicting general types of equivalence in survey translation: equivalence of stimulus and equivalence of effect. Our research suggests that the issue of providing translators with documentation and guidelines is more complex than assumed, and that researchers should consider carefully in advance whether this is necessary, given the nature of the survey as well as their own beliefs about adaptation and the primacy of equivalence of stimulus or equivalence of effect.


Source: U.S. Census Bureau, Statistical Research Division

Created: June 15, 2007
Last revised: June 15, 2007

[PDF] or PDF denotes a file in Adobe’s Portable Document Format. To view the file, you will need the Adobe® Reader® Off Site available free from Adobe.

This symbol Off Site indicates a link to a non-government web site. Our linking to these sites does not constitute an endorsement of any products, services or the information found on them. Once you link to another site you are subject to the policies of the new site.

Source: U.S. Census Bureau | Statistical Research Division | (301) 763-3215 (or chad.eric.russell@census.gov) |   Last Revised: October 08, 2010