Race, Hispanic origin, Cognitive Testing, Validity measurement
Staff from the Statistical Research Division (SRD) assisted staff from Population Division (POP), Decennial Management Division (DMD), and Decennial Statistical Studies Division (DSSD) to develop and pretest the 2010 Census Race and Hispanic Origin Alternative Questionnaire Experiment (AQE) Reinterview questionnaire. This report summarizes the findings and recommendations from the three rounds of this pretest. The results and recommendations of this report will inform the 2010 Census Program for Evaluations and Experiments (CPEX). Census Bureau staff, including six members of SRD, two members of POP and one member of DSSD conducted 37 cognitive interviews in the Greater Washington D.C. and Baltimore Metropolitan areas from May through July of 2009. Six of these interviews were conducted in Spanish using the Spanish translation of the instrument. This report highlights the findings and recommendations from each round. During the Pilot Test, it was quickly determined that the questionnaire as initially drafted was overly burdensome to respondents and that respondents were likely to mention other races or origins from their heritage, even if they did not self-identify with them. After a revision to the questionnaire, subsequent cognitive interviews allowed us to conclude that this problem was resolved. Three measures of race and origin were developed and tested that, we believe, provides a comprehensive view of respondents’ reported races and origins. The first measure is an open-ended question that allows the respondent to self-identify with any races or origins. The second is a series of yes/no questions aimed at measuring identification with the Office of Management and Budget (OMB) race and origin categories. The final measure is a summary measure which attempts to gather the respondent’s “usual” response to questions like these. The series of three measures was found to often capture respondents’ self-identification under different circumstances and will be used to assess bias in the 2010 AQE. Overall, what was demonstrated is that there is no single “truth” of race. This is reflected in the OMB definition that considers race a socio-politically constructed and “not anthropologically or scientifically based” (1997). In this light, we note that the open-ended measure of race seems to most closely match respondent’s self-identification. Interestingly, for non-Hispanics, the reported race of their parents agrees strongly with self-identified race; even stronger than self-identification across measures, but this is not true for the Hispanics in this study. When asked “yes/no” to each OMB race category, respondents sometimes provided a more ancestral or genealogical response. Next, and importantly, the race and origins respondents choose may differ depending on which of the measures is used. About one out of every four individuals in our convenience sample changed their answer for any two of the measures asked. The agreement across measures was higher for non-Hispanics than for Hispanics, suggesting that the questions are being interpreted differently by non-Hispanics and Hispanics. This speaks directly to the difficulty of using the current measures of race and Hispanic origin.
Jennifer Hunter Childs, Rodney Terry, Nathan Jurgenson, Matthew Clifton, and George Higbie. (2010). Iterative Cognitive Testing of the 2010 Race and Hispanic Origin Alternative Questionnaire Experiment (AQE) Reinterview. Statistical Research Division Study Series (Survey Methodology #2010-13). U.S. Census Bureau. Available online at <http://www.census.gov/srd/papers/pdf/ssm2010-13.pdf>.
This symbol indicates a link to a non-government web site. Our linking to these sites does not constitute an endorsement of any products, services or the information found on them. Once you link to another site you are subject to the policies of the new site.