In theory, experts in a domain of interest could perform an in-depth interview using scripted and
unscripted questions to obtain information needed to answer research questions. These experts
would feel confident that the data they collected was of high quality since they could supplement
the scripted probes with unscripted probes to determine the truth (see Conrad and Schober, 2000,
and Schober and Conrad, 1997 for studies on conversational interviewing). If they felt the
respondent was confused by questions, or if the respondent indicated there was more pertinent
information available, then experts could inquire further. This is largely the method employed
by experts doing cognitive testing of a questionnaire. However, this method is impractical for a
large data collection.
This paper explores a method that uses expert respondent debriefings as a means of evaluating
the quality of the production data. This is especially important for surveys for which an
assessment of "truth" is not otherwise available. In July of 2006, a small group of experts in
Census residence rules observed 169 Census Coverage Measurement Person Interviews in two
sites for the 2006 Census Test. Immediately following 50 interviews where a complex living
situation was described during the interview, these experts conducted qualitative, ethnographictype
respondent debriefings asking open-ended questions about where the person of interest had
been staying over the last year. The goal of the respondent debriefing was to determine the "true"
residence for each person. We compare this "true" residence against the residence status obtained
by the questionnaire alone. By comparing the two sets of data, we determine if the questionnaire
was collecting accurate information, and for what situations the questionnaire needs to be
modified. This paper discusses our methodology for these expert respondent debriefings.
Source: U.S. Census Bureau, Statistical Research Division
Created: October 3, 2007
Last revised: October 3, 2007
This symbol indicates a link to a non-government web site. Our linking to these sites does not constitute an endorsement of any products, services or the information found on them. Once you link to another site you are subject to the policies of the new site.