Research Reports

Final Report of an Experiment: Effects of a Revised Instruction, Deadline, and Final Question Series in the Decennial Mail Short Form

Elizabeth Martin

KEY WORDS: coverage errors, householder, questionnaire design, mail response rate

ABSTRACT

This study evaluates potential improvements in the census mail form, including a revised instruction about whom to list as Person 1, a Final Question series to reduce and identify coverage errors, and a deadline for return of the form.

Westat conducted a national mailout test for the Census Bureau in March and April of 2006. Three experimental factors were evaluated in four panels, with 7,095 sample addresses allocated to each panel. Census questionnaires were mailed to a national sample of addresses, excluding Austin TX, and 13,703 were returned by the cutoff date. A sample of about 600 cases was sent for a Coverage Followup (CFU) interview during July of 2006.

The instruction about whom to list as Person 1 confuses respondents, leading some to inadvertently leave themselves off the form. However, the frequency of this error did not differ between the control (Census 2000 version) and experimental wordings, so this test provides no basis for preferring either version of the instruction.

This study developed and tested an experimental “Final Questions for Everyone” section that reminded respondents of the date of Census Day and of people who might inadvertently be left off the form and asked two “coverage questions.” A random portion of the sample received the experimental version, and a random portion received the control version that did not include the introduction and reminders or coverage questions. To evaluate how effectively the final questions identify coverage errors, all cases with responses indicating a possible coverage problem were selected for a Coverage Followup (CFU) interview, along with a random subsample of cases that were not so flagged. CFU added someone in 4.5% of flagged households, compared to 0.7% of non-flagged households, and deleted someone in 7.0% of flagged households, compared to 2.8% of non-flagged households; both differences are statistically significant. Thus, the Final Questions help discriminate between households in which a coverage follow-up interview is productive from those in which it is much less so.

Deadline messages in the mailing pieces, coupled with mailings a week later than the conventional schedule, obtained a significantly higher mail response rate, by 2.0 percentage points, along with improvements in several measures of data completeness and coverage.

CITATION: Elizabeth Martin. (2007) Final Report of an Experiment: Effects of a Revised Instruction, Deadline, and Final Question Series in the Decennial Mail Short Form. 2010 Census Memoranda Series #55. U. S. Census Bureau.

Source: U.S. Census Bureau, Statistical Research Division

Created: July 25, 2007
Last revised: July 25, 2007