CFU CPEX Experimental Question Cognitive Testing: Undercount, Overcount and Duplicate Experimental Question Sequences
Jennifer Hunter Childs, Leticia Fernández, Matt Clifton, and Mikelyn Meyers
KEY WORDS: Coverage, Cognitive Testing, Duplication, Census
This test included 3 different situations that were tested somewhat separately. Cognitive interviews were conducted using cases from the 2008 Census Dress Rehearsal that fit at least one of three criteria: 1) the household marked “yes” to the undercount question in the initial census form, but did not list any potential additional people during the CFU interview; 2) the household marked an affirmative overcount category for at least one household member on the initial census form, but did not flag that person as having another place to stay during the CFU interview; and 3) a potential person duplicate was identified through computer matching of the census data and the CFU interview did not reveal a potential additional address for that person. Cognitive interviews were conducted with nine undercount cases. This question series was fairly successful. It elicited the name of a person in eight out of nine cases. In six of those cases, the respondent mentioned a person who was not already on the roster. In two cases, the respondent reported that the “undercounted” person was already on the roster. In the case where a person was not identified during the survey, during the debriefing, the respondent reported that she had probably been thinking of someone who was already on the roster. We had minor recommendations to the question text that we think will clear up the confusion a few respondents experienced as well as the situations where the “undercounted” person was actually already counted. Cognitive interviews were conducted with eight overcount cases, all of which involved someone who lived or stayed somewhere else “while in the military.” There were two cases (which were also undercount cases) where the respondent indicated that a person lived or stayed somewhere else “for another reason.” The overcount question series also worked reasonably well, and we think can to perform well in 2010 with fairly minor revisions. The question series yielded an address or a place for five of the ten tested cases. In three of the other cases, it yielded information that the person sometimes deploys for the military, but not during the time frame posed in the CFU questions. This testing allowed us to propose some options for response categories that we think will account for a large portion of the followed-up people. Finally, cognitive interviews were conducted with seven duplicate cases, the majority of which were children that were duplicated in two different housing units (4 of 7). The first question in this series identified the duplicate person by name and said “NAME may have been counted at another residence (in state) as well as on your census form.” Researchers found this phrasing was very sensitive, at least for one respondent – a father with sole custody of his children – who was upset because he inferred from this question that the mother of his children had reported the children on her census form. Though we saw this sensitive situation with children, we believe that this situation could also happen with adults, particularly when relationships are involved. In one case of an adult son who stayed sometimes with his girlfriend, the respondent (his mother) made some very critical and hostile remarks and reported inconsistently about how much time he spent with the girlfriend. We deem this to be a sensitive reaction as well. Because we did not have the opportunity to ask about out-of-state duplicates, we did this during the debriefing by asking “Would it bother you if we said that our records indicate that you may have been counted in Texas as well as on your census form?” Two respondents immediately said they would worry about identity theft. Because of these two areas of sensitivity, we do not recommend directly saying that our records indicate that a specific person may have been duplicated. While this text does not violate our confidentiality mandates, we feel that it might be perceived as violating confidentiality if the respondent figures out who duplicated them. Additionally, we do not recommend using the state in the question wording for out-of state duplicates. We think the risk is too high for those rare cases for which it is not a real duplicate.
CITATION: Childs, Jennifer Hunter , Leticia Fernández, Matt Clifton, and Mikelyn Meyers. (2009). CFU CPEX Experimental Question Cognitive Testing: Undercount, Overcount and Duplicate Experimental Question Sequences . Statistical Research Division Study Series (Survey Methodology #2009-17). U.S. Census Bureau. Available online at <http://www.census.gov/srd/papers/pdf/ssm2009-17.pdf>.