The Census 2000 Dress Rehearsal was designed to conduct operational testing of methods planned for Census 2000 in three locations across the country: Columbia, South Carolina, and 11 surrounding counties; enominee County, Wisconsin, including the Menominee American Indian Reservation; and Sacramento, California. The overview section offers a description of the methodologies. This report provides information comparing the data from the dress rehearsal with a set of standards for each site. The standards, set prior to the dress rehearsal, focus on six areas considered critical for measuring the success of the Census 2000 Dress Rehearsal:
On-time completion of the Integrated Coverage Measurement and Post Enumeration Surveys, and consistency of census results with independent demographic benchmarks.
Although many of the standards were based on the 1990 Census, there are differences between a census and a dress rehearsal. For example, Census 2000 Dress Rehearsal sites did not have a 100 percent block canvass of the address list as was conducted in the 1990 Census and as will be done prior to Census 2000. Such factors may have had an impact on our ability to meet the 1990 standard. This and other limitations are noted in the appropriate sections of this report.
The data contained in this Report Card were drawn from the Census 2000 Dress Rehearsal Mid-Term Status Report, which documented the progress of dress rehearsal activities, and evaluation studies, which provided information on the quality of census results. The Report Card presents data separately for South Carolina, Menominee, and Sacramento. The data are not comparable across sites because of the differing methods and procedures used at each site, and differing demographic and geographic characteristics of the sites.
Columbia, South Carolina, and 11 surrounding counties have approximately 252,000 housing units and a re-dress rehearsal estimated population of 667,000 people [Reference 6]. Offering a mix of areas with varying degrees of enumeration difficulty, the site provided an opportunity to test the integration of procedures planned for Census 2000, in addition to some procedures that would be part of a census that does not include sampling and statistical estimation.
Table 1 provides a summary report card for Columbia recording the performance of the Census 2000 Dress Rehearsal in comparison with the standards set prior to the dress rehearsal. The report card results are explained in the subsequent sections.
| Standard | Met | Partially Met | Not Met |
|---|---|---|---|
| Master Address File | X | ||
| Questionnaire Mailing Strategy | X | ||
| Paid Advertising | X | ||
| Followup for Nonresponse | X | ||
| Multiple Response Resolution | X | ||
| Coverage Measurement | X |
An integral part of a successful census is the development of a complete address list. For the Census 2000 Dress Rehearsal, a series of steps was developed to create the Master Address File. This file was the source of addresses for the census enumeration. In order to present some information about the completeness of the Master Address File at this time, a preliminary dress rehearsal evaluation examined, to the extent possible, how well the Master Address File covered existing housing units at the time of the census enumeration.
This preliminary assessment of the completeness and accuracy of the Master Address File at the South Carolina site was determined in the Census 2000 Dress Rehearsal by information from the initial housing unit match between the Post Enumeration Survey (PES) list and the Master Address File. In the PES, enumerators created independent lists of housing units that were then matched against the Decennial Master Address File, as it existed in March of 1998. After a series of computer matches, clerical matches, and field followup, match codes were assigned. From these initial operations, measurements of the coverage of housing units on the March 1998 version of the Decennial Master Address File were produced.
Census operations that followed this initial housing unit match altered the coverage of housing units by adding and deleting housing units. Estimates of the number of these census additions and deletions are additional components of this preliminary assessment. We cannot compute an actual final estimate of the coverage of housing units in the dress rehearsal because of the uncertainty of the accuracy of these additions and deletions.
In mailout/mailback areas, the Census 2000 Dress Rehearsal standard for completeness of the Master Address File was a net undercount of 1.0 percent [10]. It was based on the 1990 Housing Unit Coverage Study, which had a net undercount in urban areas of 0.91 percent with a standard error of 0.42 percent [7].
A net undercount of 1.0 percent is an optimal standard for the mailout/mailback areas. A 100 percent block canvass, which was conducted in the 1990 Census, was not implemented in the Census 2000 Dress Rehearsal. Lack of a 100-percent block canvass may have had an impact on the ability to meet the 1990 standard in mailout/mailback areas.
The Census 2000 Dress Rehearsal standard for completeness of the Master Address File in update/leave areas was a net undercount of 3.2 percent [10]. It was based on the 1990 Housing Unit Coverage Study, which had a net undercount in rural areas of 3.16 percent with a standard error of 0.53 percent [7].
Initial estimates of net undercount were not available separately for the mailout/mailback areas and the update/leave areas in the site. Also, estimates of additions and deletions were available only at the site level. We therefore computed a weighted standard for the entire site to be used for comparison purposes. The weighted standard is based on the number of census housing units in the mailout/mailback areas versus update/leave areas of the South Carolina site as identified in the dress rehearsal. The weighted standard for the entire site is 1.5 percent.
The standard was not met. In the entire site, the net undercount of housing units after the initial housing unit match was 10.5% (Table 2). Following the initial match, there was a weighted estimate of 13,760 added housing units and 24,213 deleted housing units [18]. The weighted standard is less than the initial net undercount estimates and the number of additions and deletions after the initial housing unit match do not give us an indication that the net undercount was improved greatly by the operations that followed the initial housing unit match. However, without knowing how much the mailout/mailback areas and the update/leave areas individually contributed to the additions and deletions, we cannot say definitively whether or not each type of enumeration area individually met the standard [18].
| Process | Census 2000 Dress Rehearsal South Carolina Site |
1990 Census National |
||
|---|---|---|---|---|
| Standard | Net Undercount from Initial Match | Result | Standard Error | |
| Mailout/Mailback | 1.0% | - | 0.91% | 0.42% |
| Update/Leave | 3.2% | - | 3.16% | 0.53% |
| Entire Site | 1.5% | 10.5% | - | - |
The initial questionnaire was mailed to arrive at all households in the mailout universe about two and one half weeks prior to Census Day, April 18, 1998. The replacement questionnaire was mailed to all those households just prior to Census Day, regardless of whether a household had returned an initial questionnaire. The dress rehearsal was the first time that replacement questionnaires were mailed to all households, regardless of whether they responded to the initial questionnaire mailing.
For the South Carolina dress rehearsal, the standard for the increase in the mail response rate between the time frame of two days after the mailout of the replacement questionnaire (April 17) until the late cut-off for nonresponse followup (May 7) was at least a six percentage point increase [10]. That increase was consistent with the assumptions used in the Census 2000 cost model.
The standard was met. The actual increase in response rate is not ascertainable, but it is at least 8.2 percentage points, based solely on the number of replacement questionnaires received during the time window for which no corresponding initial questionnaire was ever received (Table 3). For the short form, the response rate increased at least 8.6 percentage points. For the long form, the response rate increased at least 6.2 percentage points [8]. Note that when considering the entire mailout/mailback dress rehearsal response period, of about 63,000 replacement mailings received at both Columbia and Sacramento, the proportion that were duplicates of initial mail forms was approximately 40% [6].
| Form Type | Number of Housing Units in Mailout | Replacement Questionnaires Received between April 17 and May 7 with No Corresponding Initial Questionnaire |
|
|---|---|---|---|
| Number | Rate | ||
| Short | 188,128 | 16,155 | 8.6% |
| Long | 35,881 | 2,232 | 6.2% |
| Total | 224,009 | 18,387 | 8.2% |
A paid advertising campaign was conducted to increase awareness of the Census 2000 Dress Rehearsal among both the general public and hard-to-reach minority subgroups. The marketing strategy drew from the following components: print media, radio, television, out-of-home media (billboards, bus shelters, posters, mobile billboards, advertisements on shopping carts and in beauty salons, convenience stores, and check-cashing establishments), and a special school-based public information campaign.
The standard for the paid advertising campaign was at least a 30 percentage point increase in awareness of the dress rehearsal following the advertising campaign [10]. An increase of that size would be consistent with the results of the 1990 Outreach Evaluation Survey [1].
The standard was met. The proportion of South Carolina residents who had recently seen or heard anything about the census increased from 29 percent before the paid advertising campaign to 89 percent after the campaign (Table 4), an increase of 60 percentage points [17].
| Measures | Number of Respondents | Respondents Aware of Census | |
|---|---|---|---|
| Number | Rate | ||
| Pre-Campaign Awareness | 817 | 237 | 29% |
| Post-Campaign Awareness | 1,506 | 1,340 | 89% |
Although the evaluation concentrated specifically on the efforts of paid advertising for the dress rehearsal, there were outreach and promotion activities independent of the advertising campaign (e.g., local Partnership Program activities) and receipt of census materials (pre-notice letter, census forms, a reminder postcard) that undoubtedly influenced awareness and were reflected to some degree in evaluation results. In addition, both before and during the campaign, there was national media coverage of the debate over sampling for Census 2000, and this coverage may have increased awareness of the census as well. There is no reason to believe, however, that it had a bigger impact on post-campaign respondents than on pre-campaign respondents.
Advertising began the first week of March and continued as late as the last week in June for some media. Before and after the campaign, a telephone survey of South Carolina residents was conducted to determine their awareness of the dress rehearsal. Within each sampled household, an interview was conducted with a household member who opens the mail for that household. There were 817 respondents to the pre-Census Day survey (a 28 percent response rate), and 1,506 respondents to the post-Census Day survey (a 64 percent response rate).
Completing the pre-campaign survey before the advertising campaign began allowed a field period much shorter than that required to obtain a high response rate to that survey. Interviewing began on February 10, 1998, allowing only 19 days to complete the interviews before the March 1 initiation of campaign activities [17]. Survey methods that are usually used to maximize the response rate, such as re-contacting respondents who declined to participate the first time they were called, were dramatically limited by this short field period. This, combined with the need to complete the largest number of baseline interviews possible before March 1, resulted in the low response rate for that survey.
The Census 2000 Dress Rehearsal standards for nonresponse followup were completion of the nonresponse followup operation on time and meeting quality criteria for final attempt cases, proxy respondents, and unclassified cases.
Nonresponse followup was conducted with all housing units in the mailout/mailback and update/leave universes for which we did not receive or check in a completed questionnaire by May 7. All housing units designated as undeliverable-as-addressed by the U.S. Postal Service also became part of the nonresponse followup universe.
The standard for completion of the nonresponse followup operation on time was met. Nonresponse followup was scheduled to be completed by July 10, 1998; it was actually completed on July 2, six work days early [9]. There were 1,726 field enumerators hired to omplete nonresponse followup. The staff turnover rate was 13.1 percent.
Final Attempt Cases. An intense effort was made to get a completed questionnaire for each remaining unit in nonresponse followup. Final attempt procedures for nonresponse followup began when the completion rate for a crew leader district reached 95 percent. The standard for the percentage of final attempt cases was less than 5.0 percent of nonresponse followup cases [10]. This standard is consistent with similar standards from the 1990 Census.
The standard was met. The proportion of final attempt cases in the dress rehearsal was equal to 3.2 percent of all nonresponse followup cases (Table 5). Long-form housing units appeared to be more difficult to enumerate than short-form housing units: the proportion of final attempt cases in the dress rehearsal was equal to 2.7 percent of the cases with the short form and 5.6 percent of the cases with the long form [9]. Although the long-form universe failed to meet the operational goal, the proportion of the entire universe enumerated via final attempt procedures met the standard.
Proxy Respondents. Data obtained by proxy response are not always as reliable as data obtained firsthand. A proxy response is one provided by someone who is not a member of the household. Interviews by proxy are expected to increase in frequency toward the end of the nonresponse followup procedures. The standard for the percentage of cases enumerated with proxy respondents was less than 6.0 percent [10, 2]. The methods used to identify proxy responses were quite different from 1990 to the Census 2000 Dress Rehearsal. In 1990, proxy response was determined by review of the data on a questionnaire. This review was performed only for cases in an evaluation, not the entire census. For the dress rehearsal, the Census Bureau added a proxy response check-box on the form field representatives completed.
The standard for proxy respondents was not met. The proportion of cases enumerated with proxy respondents in the dress rehearsal was equal to16.4 percent of occupied units in the nonresponse followup universe (Table 5) [9].
Unclassified Cases. The census unedited file included all units that were subject to mailback procedures. The following classifications were used:
At the end of nonresponse followup, all units should have had a classification of occupied, vacant, or deleted. The percentage remaining unclassified reflected the success of nonresponse followup. The standard for the percentage of cases which were unclassified -- that is, there was no housing unit status after nonresponse followup -- was less than 0.05 percent of the units on the census unedited file [10]. The standard for unclassified cases was consistent with similar data from the 1990 Census.
The standard was not met. The proportion of unclassified cases in the dress rehearsal was equal to 1.1 percent of the address universe (Table 5). There were 3,353 housing units left unclassified in South Carolina (2,546 unclassified mail returns and 807 unclassified enumerator returns) out of the universe of 294,446 housing units [9]. Evidence suggests that unclassified mail returns were largely caused by errors in the data capture and data processing.
| Cases | Standard | Overall | Short Form | Long Form | |||
|---|---|---|---|---|---|---|---|
| Number | Percent | Number | Percent | Number | Percent | ||
| Final attempt | <5.0% | 4,200 (of 132,566) | 3.2% | 2,970 (of 110,638) | 2.7% | 1,230 (of 21,928) | 5.6% |
| Proxy respondents (occupied units) | <6.0% | 14,941 (of 91,296) | 16.4% | 12,808 (of 75,326) | 17.0% | 2,133 (of 15,970) | 13.4% |
| No housing unit status (unclassified) |
<0.05% | 3,353 (of 294,446) | 1.1% | - | - | - | - |
For the initial phase of the dress rehearsal, the standard for the percentage of erroneously enumerated persons resulting from specific rules used to unduplicate multiple responses was set to be small in comparison to the percentage of erroneous enumerations in the 1990 Census, as measured by the PES [4]. Based on the 1990 PES, the standard for the percentage of erroneously enumerated persons selected in error was small in comparison to 4.6 percent [10].
The standard was met. The percentage of persons included in error by the multiple response resolution algorithm for the dress rehearsal was0.3 percent (Table 6) [12]. This indicates that the multiple response resolution operation did not create a problem relative to total erroneous enumerations in the dress rehearsal.
For the percentage of persons excluded in error as a result of specific rules used to unduplicate multiple responses, the standard was set to be small in comparison to the percentage of omissions in the 1990 Census, as measured by the PES [4]. Based on the 1990 PES, the standard for the omission rate for persons in housing units that were enumerated in the census by mail was small in comparison to 1.3 percent. [10].
The standard was met. The percentage of persons excluded in error for the dress rehearsal as a result of specific rules used to unduplicate responses was 0.3 percent (Table 6) [12]. This indicates that the use of these specific rules did not create a problem for omissions in the dress rehearsal.
The multiple response resolution errors shown here could be considered additions to the 1990 PES enumeration errors. The findings indicate that additional error caused by the multiple response options should be minimal both in overall effect and as a percent of all errors.
In every test including multiple response options that the Census Bureau has conducted to date, the use of Be Counted Forms and other multiple response options has been minimal. If use grows substantially in Census 2000, these errors may be magnified.
| Cases | Actual | Standard |
|---|---|---|
| Included in error | 0.3% | small relative to 4.6% |
| Excluded in error | 0.3% | small relative to 1.3% |
The Census 2000 Dress Rehearsal standards for coverage measurement were consistency with key demographic characteristics for the site and completion of the Post Enumeration Survey on time.
An independent study was implemented to evaluate the consistency of the Census 2000 Dress Rehearsal estimates and the effectiveness of the Post Enumeration Survey (PES) in achieving a reduction in the differential undercount. A battery of demographic tools was used in this evaluation: population, housing, and group quarters estimates; vacancy rates; school enrollment data; Medicare data; age and sex ratios; household distributions; and the detailed characteristics for the dress rehearsal site.
The key demographic characteristics examined are consistent with expectations for this site. The demographic distributions (e.g., age, race, sex, Hispanic Origin, group quarters) and rates (e.g., vacancy rates, persons per household) are in line with previous census results and expected trends since 1990 [15].
The housing and population totals fall below expected levels for this site. The census population result of 662,140 is about 4.5 percent below the independent demographic benchmark not adjusted for undercoverage (Table 7). Population coverage in 1998 declined relative to 1990 -- attributable in large part to a shortfall of housing units in the site. The undercoverage in the dress rehearsal results implied by the demographic benchmarks is broadly consistent with the undercoverage measured by the PES results for this site [15].
The PES is a series of data collection and processing operations that provides a measurement of the net undercount or overcount. The central component of the PES is an independent enumeration of about 18,000 households in the South Carolina site.
The standard for completion of the PES on time was met. The PES interview consisted of the telephone interview phase, the personal visit interview phase, and the nonresponse conversion phase. The telephone interview was scheduled to begin on May 11, 1998, and end on June 9, 1998. The personal visit interview (including nonresponse conversion) was scheduled to be completed by September 21, 1998. All PES interviewing was actually completed on August 27, 1998, 16 workdays ahead of schedule [13]. The response rate was at least 95 percent, but is not directly comparable to 1990 due to differing methodologies. A complete evaluation of the PES results will be provided later this spring.
| 1990 Census | 1998 Dress Rehearsal | 1998 Demographic Estimate | |
|---|---|---|---|
| Population | 655,066 | 662,140 | 693,626 |
| Housing | 253,285 | 273,497 | 285,952 |
Menominee County has approximately 1,700 housing units and a pre-dress rehearsal estimated population of 4,600 people [Reference 6]. The population includes a high proportion of American Indians. The locale was selected for the dress rehearsal based on a recommendation from the Census Bureau's Advisory Committee on the American Indian and Alaska Native Population. This site was used to test the integration of operations planned to conduct
Census 2000 in the most efficient and high quality manner possible, with an emphasis on methods for improving the count for American Indians living on reservations.
Table 8 provides a summary report card for Menominee County, Wisconsin, recording the performance of the Census 2000 Dress Rehearsal in comparison with the standards set prior to the dress rehearsal. The report card results are explained in subsequent sections.
| Standard | Met | Partially Met | Not Met |
|---|---|---|---|
| Master Address File | X | ||
| Followup for Nonresponse | X | ||
| Coverage Measurement | X |
An integral part of a successful census is the development of a complete address list. For the Census 2000 Dress Rehearsal, a series of steps was developed to create the Master Address File. This file was the source of addresses for the census enumeration. In order to present some information about the completeness of the Master Address File at this time, a preliminary dress rehearsal evaluation examined, to the extent possible, how well the Master Address File covered existing housing units at the time of the census enumeration.
This preliminary assessment of the completeness and accuracy of the Master Address File at the Menominee County site was determined in the Census 2000 Dress Rehearsal by information from the initial housing unit match between the Integrated Coverage Measurement (ICM) and the Master Address File. The ICM survey enumerators created independent lists of housing units that were then matched against the Decennial Master Address File as it existed in March of 1998. After a series of computer matches, clerical matches, and field followup, match codes were assigned. From these initial operations, measurements of the coverage of housing units on the Decennial Master Address File (as of March 1998) were produced.
Census operations that followed this initial housing unit match altered the coverage of housing units by adding and deleting housing units. Estimates of the number of these census additions and deletions are additional components of this preliminary assessment. We cannot compute an actual final estimate of the coverage of housing units in the dress rehearsal because of the uncertainty of the accuracy of these additions and deletions.
The Census 2000 Dress Rehearsal standard for completeness of the Master Address File in update/leave areas was a net undercount of 3.2 percent [10]. It was based on the 1990 Housing Unit Coverage Study, which had a net undercount in rural areas of 3.16 percent with a standard error of 0.53 percent [7].
The standard was met. The net undercount of housing units after the initial housing unit match was 0.0%. (Table 9). Following the initial match there was a weighted estimate of 90 added housing units and 93 deleted housing units [18]. The standard is above the initial net undercount estimates, and the numbers of additions and deletions after the initial housing unit match do not give us an indication that the net undercount was changed greatly by the operations that followed the initial housing unit match.
| Process | Census 2000 Dress Rehearsal Menominee Site | 1990 Census Rural National | ||
|---|---|---|---|---|
| Standard | Net Undercount From Initial Match | Result | Standard Error | |
| Update/Leave | 3.2% | 0.0% | 3.16% | 0.53% |
The Census 2000 Dress Rehearsal standards for nonresponse followup were completion of the nonresponse followup operation on time and meeting quality criteria for final attempt cases, proxy respondents, and unclassified cases.
The standard for completion of the nonresponse followup operation on time was met. Nonresponse followup was conducted with all housing units in the update/leave universe for which we did not receive or check in a completed questionnaire. Nonresponse followup began on May 14, 1998, and was scheduled to be completed by June 26, 1998. It was completed on time [6]. There were 27 field enumerators hired to complete nonresponse followup. The staff turnover rate was 41.7 percent [6].
Final Attempt Cases. An intense effort was made to get a completed questionnaire for each remaining unit in nonresponse followup. Final attempt procedures for nonresponse followup began when the completion rate for a crew leader district reached 95 percent. The standard for the percentage of final attempt cases was less than 5.0 percent of nonresponse followup cases [10]. The standard was consistent with similar ones from the 1990 Census.
The standard was met. The proportion of final attempt cases in the dress rehearsal was equal to 0.1 percent of nonresponse followup cases (Table 10). Since only one questionnaire indicated having been completed during final attempt procedures, it appears that either the final attempt procedure was not used or unnecessary in Menominee, or that the enumerators did not properly complete the item on the questionnaire [9].
Proxy Respondents. Data obtained by proxy response are not always as reliable as data obtained firsthand. A proxy response is one provided by someone who is not a member of the household. Interviews by proxy tend to increase in frequency toward the end of the nonresponse followup procedures [9]. The standard for the percentage of cases enumerated with proxy respondents was less than 6.0 percent [10, 2]. The methods used to identify proxy responses were quite different from 1990 to the Census 2000 Dress Rehearsal. In 1990, proxy response was determined by review of the data on a questionnaire. This review was performed only for the cases in an evaluation, not the entire census. For the dress rehearsal, the Census Bureau added a proxy response check-box on the form field representatives completed.
The standard was not met. The proportion of cases enumerated with proxy respondents in the dress rehearsal was equal to 11.5 percentof the occupied portion of the nonresponse followup universe (Table 10). Interviews with household members were more difficult to obtain than had been hoped, and as a result the quality of the data is a concern [9].
Unclassified Cases. The census unedited file included all units that were subject to mailback procedures [9]. The following classifications were used:
At the end of nonresponse followup, all units should have had a classification of occupied, vacant, or deleted. The percentage remaining unclassified reflected the success of nonresponse followup. The standard for the percentage of cases which were unclassified -- that is, there was no housing unit status after nonresponse followup -- was less than 0.05 percent of the units on the census unedited file [10]. The standard for unclassified cases was consistent with similar data from the 1990 Census.
The standard was not met. The proportion of unclassified cases in the dress rehearsal was equal to 0.8 percent of the address universe (Table 10). In Menominee, 17 housing units (13 unclassified mail returns and 4 unclassified enumerator returns) out of the nonresponse followup universe of 2,061 housing units were left unclassified [9]. Evidence suggests that unclassified mail returns were largely caused by errors in the data capture and data processing.
| Cases | Standard | Overall | Short Form | Long Form | |||
|---|---|---|---|---|---|---|---|
| Number | Percent | Number | Percent | Number | Percent | ||
| Final attempt | <5.0% | 1 (out of 1,162) | 0.1% | 1 (out of 1,027) | 0.1% | 0 (out of 135) | 0.0% |
| Proxy respondents (occupied units) | <6.0% | 62 (out of 539) | 11.5% | 58 (out of 469) | 12.4% | 4 (out of 70) | 5.7% |
| No housing unit status (unclassified) |
<0.05% | 17 (out of 2,061) | 0.8% | - | - | - | - |
The Census 2000 Dress Rehearsal standards for coverage measurement were consistency with key demographic characteristics for the site and completion of the Integrated Coverage Measurement Survey on time.
An independent study was implemented to evaluate the consistency of the Census 2000 Dress Rehearsal estimates and the effectiveness of the ICM in achieving a reduction in the differential undercount. A battery of demographic tools was used in this evaluation: population, housing, and group quarters estimates; vacancy rates; school enrollment data; Medicare data; age and sex ratios; household distributions; and the detailed characteristics for the dress rehearsal site.
The key demographic characteristics examined are consistent with the expectations for this site. The demographic distributions (e.g., age, sex, race, Hispanic Origin, group quarters) and rates (e.g., vacancy rates, persons per household) are in line with previous census results and expected trends since 1990 [15].
The housing and population totals are broadly consistent with independent estimates for 1998. The one-number census result released on January 14, 1999 (population of 4,738) is consistent with the demographic benchmarks not adjusted for undercoverage (Table 11). The estimate falls between the estimate provided by the State of Wisconsin and the Census Bureau demographic estimates. These are excellent results for the Menominee site. We do not believe this should be taken as indication that the enumerating problems on reservations are "solved."
The ICM is a series of data collection and processing operations that provides a measurement of the net undercount and be used to produce an accurate one-number census. The central component of the ICM is an independent enumeration of about 800 households in Menominee.
The standard for completion of the ICM on time was met. The ICM interview consisted of the telephone interview phase, the personal visit interview phase, and the nonresponse conversion phase. The telephone interview was scheduled to begin on May 11, 1998 and end on May 26, 1998. The personal visit interview (including nonresponse conversion) was scheduled to be completed by September 4, 1998 [13]. All ICM interviewing was actually completed on September 3, 1998, one workday ahead of schedule [13]. The response rate was at least 98.5 percent, but is not directly comparable to 1990 because of differing methodologies. A complete evaluation of the ICM results will be provided later this spring.
| 1990 Census | 1998 Dress Rehearsal | 1998 Demographic Estimate | 1998 Wisconsin Estimate | |
|---|---|---|---|---|
| Population | 3,890 | 4,738 | 4,872 | 4,299 |
| Housing | 1,742 | 2,046 | 1,944 | n/a |
Sacramento, California, is a large, urban area with approximately 153,000 housing units and a pre-dress rehearsal estimated population of 374,000 people [Reference 6]. The population is diverse with significant African American, Hispanic, and Asian and Pacific Islander populations. This site was used to test the integration of operations planned for Census 2000 in the most efficient and high quality manner possible with emphasis on methods designed to improve the count for the population groups that historically have been undercounted.
Table 12 provides a summary report card for Sacramento, California, recording the performance of the Census 2000 Dress Rehearsal in comparison with standards set prior to the dress rehearsal. The report card results are explained in subsequent sections.
| Standard | Met | Partially Met | Not Met |
|---|---|---|---|
| Master Address File* | |||
| Questionnaire Mailing Strategy | X | ||
| Paid Advertising | X | ||
| Followup for Nonresponse | X | ||
| Multiple Response Resolution | X | ||
| Coverage Measurement | X |
*insufficient information to determine
An integral part of a successful census is the development of a complete address list. For the Census 2000 Dress Rehearsal, a series of steps was developed to create the Master Address File. This file was the source of addresses for the census enumeration. In order to present some information about the completeness of the Master Address File at this time, a preliminary dress rehearsal evaluation examined, to the extent possible, how well the Master Address File covered existing housing units at the time of the census enumeration.
This preliminary assessment of the completeness and accuracy of the Master Address File at the Sacramento site was determined in the Census 2000 Dress Rehearsal by information from the initial housing unit match between the Integrated Coverage Measurement (ICM) listing and the Master Address File. In the ICM, survey enumerators created independent lists of housing units that were then matched against the Decennial Master Address File, as it existed in March of 1998. After a series of computer matches, clerical matches, and field followup, match codes were assigned. From these initial operations, measurements of the coverage of housing units on the March 1998 version of the Decennial Master Address File were produced.
Census operations that followed this initial housing unit match altered the coverage of housing units by adding and deleting housing units. Estimates of the number of these census additions and deletions are additional components of this preliminary assessment. We cannot compute an actual final estimate of the coverage of housing units in the dress rehearsal because of the uncertainty of the accuracy of these additions and deletions.
In Sacramento, the Census 2000 Dress Rehearsal standard for completeness of the Master Address File was a net undercount of 1.5 percent [10]. It was based on the 1990 Housing Unit Coverage Study, which had a net undercount for the West Region of 1.48 percent with a standard error of 0.58 percent [7].
A net undercount of 1.5 percent is an optimal standard for the mailout/mailback areas: a 100-percent block canvass, which was conducted in the 1990 Census, was not implemented in the Census 2000 Dress Rehearsal. Lack of a 100-percent block canvass may have had an impact on the bureau's ability to meet the 1990 standard in mailout/mailback areas.
We do not have enough information to determine if the standard was met. The net undercount of housing units after the initial housing unit match was -0.5% (Table 13). Following the initial match, there was a weighted estimate of 3,560 added housing units and 12,794 deleted housing units [18]. The standard is above the initial net undercount estimates, but the numbers of additions and deletions after the initial housing unit match give us an indication that the net undercount could have been changed enough by the operations that followed the initial housing unit match to keep us from meeting the standard.
| Process | Census 2000 Dress Rehearsal Sacramento Site | 1990 Census West Region | ||
|---|---|---|---|---|
| Standard | Net Undercount From Initial Match | Result | Standard Error | |
| Mailout/ Mailback | 1.5% | -0.5% | 1.48% | 0.58% |
The initial questionnaire was mailed to arrive at all households in the mailout universe about two and one-half weeks prior to Census Day, April 18, 1998. The replacement questionnaire was mailed to all households just prior to Census Day, regardless of whether they had returned an initial questionnaire. This test was the first time that replacement questionnaires were mailed to all households, regardless of whether they responded to the initial questionnaire mailing.
For the Sacramento dress rehearsal, the standard for the increase in the initial mail response rate between the time frame of two days after the mailout of the replacement questionnaire (April 17) until the late cut-off for nonresponse followup (May 7) was at least a six percentage point increase [10]. The increase was consistent with the assumptions used in the Census 2000 cost model.
The standard was met. The actual increase in response rate is not ascertainable, but it is at least 7.5 percentage points, based solely on the number of replacement questionnaires received during the time window for which no corresponding initial questionnaire was received (Table 14). For the short form, the response rate increased at least 7.9 percentage points. For the long form, the response rate increased at least 5.2 percentage points [8]. Note that when considering the entire mailout/mailback dress rehearsal response period, of about 63,000 replacement mailings received at both Columbia and Sacramento, the proportion that were duplicates of initial mail forms was approximately 40% [6].
| Form Type | Number of Housing Units | Replacement Questionnaires Received between April 17 and May 7 with No Corresponding Initial Questionnaire |
|
|---|---|---|---|
| Number | Rate | ||
| Short | 145,329 | 11,513 | 7.9% |
| Long | 28,154 | 1,458 | 5.2% |
| Total | 173,483 | 12,971 | 7.5% |
A paid advertising campaign was conducted to increase awareness of the Census 2000 Dress Rehearsal among both the general public and hard-to-reach minority subgroups. The marketing strategy drew from the following components: print media, radio, television, out-of-home media (billboards, bus shelters, posters, and mobile billboards), and a special school-based public information campaign.
The standard for the paid advertising campaign was at least a 30 percentage point increase in awareness of the dress rehearsal following the advertising campaign [10]. An increase of that size is consistent with the results of the 1990 Outreach Evaluation Survey [1].
The standard was met. The proportion of Sacramento residents who had recently seen or heard anything about the census increased from 28 percent before the paid advertising campaign to 80 percent after the campaign (Table 15), an increase of 52 percentage points [16].
| Measures | Number of Respondents | Respondents Aware of Census | |
|---|---|---|---|
| Number | Rate | ||
| Pre-Campaign Awareness | 565 | 158 | 28% |
| Post-Campaign Awareness | 1,504 | 1,203 | 80% |
Although the evaluation concentrated specifically on the efforts of paid advertising for the dress rehearsal, there were outreach and promotion activities independent of the advertising campaign (e.g., local Partnership Program activities) and receipt of census materials (pre-notice letter, census forms, a reminder postcard) that undoubtedly influenced awareness and were reflected to some degree in evaluation results. In addition, both before and during the campaign, there was national media coverage of the debate over sampling for Census 2000, and this coverage may have increased awareness of the census as well. There is no reason to believe, however, that it had a bigger impact on post-campaign respondents than on pre-campaign respondents.
Advertising began the first week of March and continued as late as the last week in June for some media. Before and after the campaign, a telephone survey of Sacramento residents was conducted to determine their awareness of the dress rehearsal. Within each sampled household, an interview was conducted with a household member who opens the mail for that household. There were 565 respondents to the pre-survey (a 25 percent response rate), and 1,504 respondents to the post-survey (a 54 percent response rate).
Completing the pre-campaign survey before the advertising campaign began allowed a field period much shorter than that required to obtain a high response rate to that survey. Interviewing began on February 10, 1998, allowing only 19 days to complete the interviews before the March 1 initiation of campaign activities [16]. Survey methods that are usually used to maximize the response rate, such as re-contacting respondents who declined to participate the first time they were called, were dramatically limited by this short field period. This, combined with the need to complete the largest number of baseline interviews possible before March 1, resulted in the low response rate for that survey.
The Census 2000 Dress Rehearsal standards for nonresponse followup were completion of the nonresponse followup operation on time and meeting quality criteria for final attempt cases, proxy respondents, and unclassified cases.
A direct sample followup, using differential sampling fractions by census tract to achieve a 90 percent response rate, was performed in Sacramento. This six-week operation was anticipated to be a challenge for field recruitment and retention.
The standard for completion of the nonresponse followup operation on time was met. Nonresponse followup began on May 14, 1998, and was scheduled to be completed by June 26, 1998. It was completed on time [6]. There were 864 field enumerators hired to complete nonresponse followup. The staff turnover rate was 19.3 percent.
Final Attempt Cases. An intense effort was made to get a completed questionnaire for each remaining unit in nonresponse followup. Final attempt procedures for nonresponse followup began when the completion rate for a crew leader district reached 95 percent. The standard for the percentage of final attempt cases was less than 5.0 percent of nonresponse followup cases [10]. This standard is consistent with similar standards from the 1990 Census.
The standard was not met. The proportion of final attempt cases in the dress rehearsal was equal to 8.9 percent of nonresponse followup cases (Table 16). Presuming that enumerators properly marked the final attempt procedure item, it appears that the final attempt procedure in Sacramento was not followed properly (final attempt procedures should not have begun until 95 percent of the housing units in each crew leader district had information that had been checked in). Long-form housing units were more likely than short-form housing units to be enumerated later: 20.4 percent of long-form housing units were enumerated via final attempt procedures, compared to 6.9 percent of short-form housing units.
Proxy Respondents. Data obtained by proxy response are not always as reliable as data obtained firsthand. A proxy response is one obtained from someone who is not a member of the household. Interviews by proxy are expected to increase in frequency toward the end of the nonresponse followup procedures. The standard for the percentage of cases enumerated with proxy respondents was less than 6.0 percent [10, 2]. The methods used to identify proxy responses were quite different from 1990 to the Census 2000 Dress Rehearsal. In 1990, proxy response was determined by review of the data on a questionnaire. This review was performed only for cases in an evaluation, not the entire census. For the dress rehearsal, the Census Bureau added a proxy response check-box on the form field representatives completed.
The standard was not met. The proportion of cases enumerated with proxy respondents in the dress rehearsal was equal to 20.1 percent of occupied units in the nonresponse followup universe (Table 16). This implies that data acquisition in Sacramento was more difficult than expected and raises a concern about the reliability of the data. In particular, the quality of the long-form detailed questions is an issue when responses were obtained from a non-household member.
Unclassified Cases. The census unedited file included all units that were subject to mailback procedures. The following classifications were used:
At the end of nonresponse followup, all units should have had a classification of occupied, vacant, or deleted. The percentage remaining unclassified reflected the success of nonresponse followup. The standard for the percentage of cases which were unclassified -- that is, there was no housing unit status after nonresponse followup -- was less than 0.05 percent of the units on the census unedited file [11].
The standard was not met. The proportion of unclassified cases in the dress rehearsal was equal to 1.0 percent of the address universe (Table 16). In Sacramento, 1,569 housing units (1,147 unclassified mail returns and 422 unclassified enumerator returns) out of the universe of 157,432 housing units were left unclassified [9]. Evidence suggests that unclassified mail returns were largely caused by errors in the data capture and data processing.
| Cases | Standard | Overall | Short Form | Long Form | |||
|---|---|---|---|---|---|---|---|
| Number | Percent | Number | Percent | Number | Percent | ||
| Final attempt | <5.0% | 5,075 (out of 56,828) | 8.9% | 3,299 (out of 48,109) | 6.9% | 1,776 (out of 8,719) | 20.4% |
| Proxy respondents (occupied units) | <6.0% | 8,914 (out of 44,420) | 20.1% | 7,402 (out of 37,132) | 19.9% | 1,512 (out of 7,288) | 20.7% |
| No housing unit status (unclassified) |
<0.05% | 1,569 (out of 157,432) | 1.0% | - | - | - | - |
For the initial phase of the dress rehearsal, the standard for the percentage of erroneously enumerated persons resulting from specific rules used to unduplicate multiple responses was set to be small in comparison to the percentage of erroneous enumerations in the 1990 Census, as measured by the PES [4]. Based on the 1990 PES, the standard for the percentage of erroneously enumerated persons was small in comparison to 4.6 percent [10].
The standard was met. The percentage of persons included in error in the multiple response resolution algorithm for the dress rehearsal was0.3 percent (Table 17) [12]. This indicates that the multiple response resolution operation did not create a problem for total erroneous enumerations in the dress rehearsal.
For the percentage of persons excluded as a result of specific rules used to unduplicate multiple responses, the standard was set to be small in comparison to the percent of omissions in the 1990 Census, as measured by the PES [4]. The standard for the omission rate for persons in housing units that were enumerated in the census by mail was small in comparison to 1.3 percent [10].
The standard was met. The percentage of persons excluded in error for the dress rehearsal as a result of specific rules used to unduplicate responses was 0.4 percent (Table 17) [12]. This indicates that the use of these specific rules did not create a problem for total omissions in the dress rehearsal.
The multiple response resolution errors shown here could be considered additions to the 1990 PES enumeration errors. The findings indicate that additional error caused by the multiple response options should be minimal both in overall effect and as a percentage of all errors.
In every test including multiple response options that the Census Bureau has conducted to date, the use of Be Counted Forms and other multiple response options has been minimal. If use grows substantially in Census 2000, these errors may be magnified.
| Cases | Actual | Standard |
|---|---|---|
| Included in error | 0.3% | small relative to 4.6% |
| Excluded in error | 0.4% | small relative to 1.3% |
The Census 2000 Dress Rehearsal standards for coverage measurement were consistency with key demographic characteristics for the site and completion of the Integrated Coverage Measurement Survey on time.
An independent study was implemented to evaluate the consistency of the Census 2000 Dress Rehearsal estimates and the effectiveness of the ICM in achieving a reduction in the differential undercount. A battery of demographic tools was used in this evaluation: population, housing, and group quarters estimates; vacancy rates; school enrollment data; Medicare data; age and sex ratios; household distributions; and the detailed characteristics for the dress rehearsal site.
The key demographic characteristics examined are consistent with expectations for this site. The demographic distributions (e.g., age, sex, race, Hispanic Origin, group quarters) and rates (e.g., vacancy rates, persons per household) are in line with previous census results and expected trends since 1990 [15].
The housing and population totals are broadly consistent with independent estimates for 1998 for Sacramento. The one-number census result released on January 14, 1999 (population of 403,313), is consistent with the State of California estimate while it is 6.9 percent greater than the Census Bureau demographic estimates, not adjusted for undercoverage (Table 18). The results are consistent with past census data and expected trends [15].
The ICM program is a series of data collection and processing operations that provides a measurement of the net undercount and be used to produce an accurate one-number census. The central component of the ICM is an independent enumeration of about 16,400 households in Sacramento.
The standard for completion of the ICM on time was met. The ICM interview consisted of the telephone interview phase, the personal visit interview phase, and the nonresponse conversion phase. The telephone interview was scheduled to begin on May 11, 1998, and end on May 27, 1998. The ICM personal visit interview (including nonresponse conversion) was scheduled to be completed by September 4, 1998 [13]. All ICM interviewing was actually completed on September 3, 1998, one workday ahead of schedule [6]. The response rate was at least 95 percent, but is not directly comparable due to differing methodologies. A complete evaluation of the ICM results will be provided later this spring.
| 1990 Population | 1998 Dress Rehearsal | 1998 Demographic Estimate | 1998 California Estimate | |
|---|---|---|---|---|
| Population | 369,365 | 403,313 | 377,002 | 393,264 |
| Housing | 153,362 | 158,281 | 160,253 | 161,278 |
The Census 2000 Dress Rehearsal was the culmination of the Census 2000 testing program which began shortly after the 1990 Census was completed. It was conducted in Columbia, South Carolina, and eleven surrounding counties; Menominee County, Wisconsin; and Sacramento, California. Each dress rehearsal site was selected because of its demographic and geographic characteristics to provide experience with some of the expected Census 2000 environments. Each site is using a different mix of census and statistical procedures. The dress rehearsal provided information to assess procedures used in the individual sites, but not for comparisons between sites.
The Columbia, South Carolina, site consisted of the city of Columbia in its entirety, including a small portion in Lexington County; the entire town of Irmo, which is in Richland and Lexington Counties; and the following 11 contiguous counties in North-central South Carolina:
This site was selected because it contains living situations and socioeconomic characteristics that are not found in a predominately urban environment. The site provided the only opportunity to test procedures for developing the census address list in an area containing a mixture of house number/street name and rural route and box number addresses. Since this site has a relatively high proportion of African Americans, it provided an opportunity to test Census 2000 procedures for reducing the differential undercount for this racial group. The site selected represented the size of typical local census offices planned for Census 2000, which was necessary to provide an understanding of the effectiveness of census operations.
The Menominee County, Wisconsin, site included the Menominee American Indian Reservation. This site was selected because of the Menominee Reservation. A very high proportion of the population living on the Reservation are American Indians. Also, the Census Advisory Committee on the American Indian and Alaska Native Populations recommended the Menominee Reservation. Conducting the dress rehearsal on an American Indian reservation allowed for the testing of Census 2000 methodologies for reducing the differential undercount for reservations, which exceeded 12 percent in the 1990 Census.
The Sacramento, California, site was selected because it contains great diversity among racial and ethnic groups. Selecting a site with a diverse population provided the opportunity to test Census 2000 methods designed to reduce the differential undercount and produce an accurate census for all components of the population. Sacramento is a primary media market, which allowed a full test of the paid promotion program. Also, this site represented the size of typical urban local census offices planned for Census 2000, which allowed for an understanding of the effectiveness of census operations in this type of office.
The relative sizes of each of the sites were estimated to be:
| Sacramento | South Carolina | Menominee | |
|---|---|---|---|
| Housing Units | 153,000 | 252,000 | 1,700 |
| Population | 374,000 | 667,000 | 4,600 |
The dress rehearsal involved operational testing of the Headquarters, Regional Census Center, Local Census Office, and Data Capture Center procedures and systems in a census-like environment. Several procedures new and enhanced since the 1990 Census, such as user-friendly forms, digital capture of forms, statistical sampling and estimation, and paid advertising, were tested individually prior to the dress rehearsal.
| Sacramento | South Carolina | Menominee | |
|---|---|---|---|
| Mailout/mailback | X | X | |
| Update/leave/mailback | X | X | |
| 100% Nonresponse followup | X | X | |
| Sample Nonresponse followup | X | ||
| Integrated Coverage Measurement | X | X | |
| Post Enumeration Survey | X |
Two questionnaire delivery methodologies were used in the Dress Rehearsal. The mailout/mailback methodology involved delivery by the United States Postal Service. There were four components of the mailout/mailback delivery: an advance letter, an initial questionnaire, a reminder card, and a "blanket" replacement questionnaire (mailed to all addresses). The update/leave/mailback methodology involved Census Bureau enumerators delivering the questionnaires at the same time they updated maps and the list of addresses. In addition to the delivery of questionnaires by census enumerators, the U.S. Postal Service delivered an advance letter and a reminder card to all "Postal Patrons" within the update/leave area. Under both delivery methodologies, respondents were asked to mail back their questionnaires in provided envelopes.
Short-form and long-form questionnaires were included in both delivery methodologies. Every household received either a short or a long form. The observed rates for the dress rehearsal sites varied, since the sampling rate varied by size of place. Using the same sampling plan for the entire nation for Census 2000, we would expect about 17 percent of all households to receive a long form.
In Columbia, South Carolina, and Menominee County, Wisconsin, a 100 percent followup for households not returning their census questionnaires was used. This procedure sent enumerators to all addresses and/or locations that received a mailed questionnaire, or had a questionnaire delivered in person, and did not return it by a specified date. This followup operation continued until a response from the household was received, or proxy information was obtained.
In Sacramento, California, a sample nonresponse followup was used. Instead of following up all addresses that did not return a census form, enumerators followed up a sample of nonrespondents. The sample was designed so that each census tract reached a final completion rate of at least 90 percent.
For example, if the initial completion rate in a census tract, defined as: # of addresses responding x 100 Total number of addresses was 60 percent, then a 3-in-4 systematic sample of nonrespondents was selected to reach the 90 percent completion target. If a census tract had at least an 85 percent initial completion rate, then the sampling rate was 1-in-3.
Followup continued for these sample-selected households until a response was received, or proxy information was obtained. Through statistical estimation techniques, responses for all of the other nonresponse households were derived from the sample responses. Additionally, all nonrespondent addresses that were added to the address list too late to be mailed a census form were also included in the nonresponse followup sample.
This operation was independent of the dress rehearsal operation and was used in Sacramento, California, and Menominee County, Wisconsin. There were three phases to this operation: Housing Unit phase, ICM Interview phase, and Person Matching and Reconciliation phase. In the Housing Unit phase, a list of housing units (within selected sample blocks) confirmed to be in existence on census day was compiled. This list was created independently of the dress rehearsal, using a workforce separate from the one used in the dress rehearsal. Enumerators used this list to conduct the second phase. During the ICM Interview phase, enumerators collected information about current residents and those who moved out of the sample block between census day and the time of the interview. This interview was done by personal visit or telephone. The Person Matching and Reconciliation phase involved matching persons enumerated in the ICM process with persons enumerated during the dress rehearsal operations. Some cases warranted additional interviews to reconcile differences in recorded data.
After completing all matching, estimates were developed of the number of people missed or duplicated during the dress rehearsal operations by means of statistical methods. These ICM results were incorporated into the final dress rehearsal enumeration to produce "one number" estimates of the population.
This operation was independent of dress rehearsal operations and was used in Columbia, South Carolina. Like the ICM, it was used to develop an estimate of the undercount from dress rehearsal operations.
The field procedures used for PES were very similar to those used during the ICM. The difference between the two is that the PES estimates were not used to produce the final dress rehearsal results.
Advance Letter - A letter sent to all addresses on the Decennial Master Address File a few days before the initial questionnaire as part of the questionnaire mailing strategy.
Blanket Replacement Questionnaire - A second questionnaire sent to all addresses on the Decennial Master Address File in mailout/mailback areas, including both respondent and nonrespondent households, to increase mail response rates.
Crew Leader District - The geographic area assigned to a first-line supervisor for field data collection operations.
Final Attempt Cases - Final attempt procedures for nonresponse followup cases begin when the completion rate for a crew leader district reaches 95 percent. An intense effort is used to get a completed questionnaire for each remaining unit in nonresponse followup.
ICM - Integrated Coverage Measurement
Initial Questionnaire - The census questionnaire delivered to all addresses on the decennial Master Address File as part of the questionnaire mailing strategy.
Integrated Coverage Measurement - A coverage measurement methodology, also known as the Quality Check Survey, that will be used to determine the number of people and housing units missed or counted more than once in Census 2000. This information is combined with enumeration results before producing a single set of official census results (the one-number census estimates).
Long Form - The census questionnaire containing 100 percent and sample questions.
MAF - Master Address File
Mailout/Mailback - A type of enumeration method in which the United States Postal Service delivers addressed census questionnaires to residents who are asked to complete and mail back the questionnaires.
Mail Response Rate - The total number of questionnaires returned by mail divided by the number of questionnaires mailed by the United States Postal Service or delivered by census enumerators.
Master Address File - A computer file of residential addresses based on a combination of the addresses in the 1990 Address Control File and current versions of the Delivery Sequence File, supplemented by address information provided by state, local, and tribal governments. The Master Address File is updated throughout the decade to provide a basis for producing address labels needed to deliver Census 2000 questionnaires and keep track of which forms have been returned and which need followup.
Nonresponse Followup - A census followup operation in which temporary field staff, known as enumerators, visit addresses from which no questionnaire was returned.
Person Interview - A face-to-face meeting or telephone conversation in which information (e.g., names, characteristics, and residence status on census day) about members of a given household is collected. Enumerators attempt to contact a member of the household - but if that is not possible, then they collect information from a proxy.
PES - Post Enumeration Survey
Post Enumeration Survey - A coverage measurement survey used to determine the number of people missed or counted more than once in the census in order to evaluate the final census results.
Production Rate - A performance measure calculated as the number of cases completed within a specified time period; for example, cases completed per hour or cases completed per day.
Proxy Return - A questionnaire that was completed based on information from someone other than a household member.
Reminder Card - A postcard sent to all addresses on the Decennial Master Address File to remind respondents to return their completed census questionnaires as part of the questionnaire mailing strategy.
Replacement Questionnaire - A second questionnaire sent to all addresses on the Decennial Master Address File in mailout/mailback areas to increase mail response rates as part of the questionnaire mailing strategy.
Short Form - The census questionnaire containing only 100 percent questions.
Standard Deviation - A measure of the dispersion of values from the average in a frequency distribution.
Turnover Rate - The turnover rate is the total number of enumerators who quit during a field operation divided by the total number of enumerators who were hired for that operation.
Update/Leave/Mailback - A type of enumeration method in which enumerators deliver census questionnaires to households to be completed and returned by mail and at the same time update the address list.
[ 1] 1990 Decennial Census Preliminary Research and Evaluation Memorandum Number 97, "Reaching Everyone: Encouraging Participation in the 1990 Census," October 31, 1991.
[ 2] 1990 Decennial Census Preliminary Research and Evaluation Memorandum Number 179, "Characteristics of Census Errors," September 15, 1992.
[ 3] 1990 Decennial Census Preliminary Research and Evaluation Memorandum Number 240, "Characteristics of Census Error - Additional Results," July 23, 1993.
[ 4] 1990 Decennial Census Preliminary Research and Evaluation Memorandum Number 251, "Analysis of Census Omissions: Preliminary Results," September 29, 1993.
[ 5] Bates, Nancy and Sara K. Buckley. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. E1b, "Effectiveness of the Paid Advertising Campaign: Reported Exposure to Advertising and Likelihood of Returning a Census Form."
[ 6] Census 2000 Dress Rehearsal Mid-Term Status Report. September 1998.
[ 7] Childers, Danny R. October 4, 1993. HUCS Results Memorandum Number 4, "Coverage of Housing in the 1990 Census," October 4, 1993.
[ 8] Dimitri, Charles R. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. A1a, "Evaluation of the Mail Implementation Strategy."
[ 9] Dimitri, Charles R. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. A1b, "Evaluation of the Nonresponse Followup Operation."
[10] Holmes, James F. May 28, 1998. Census 2000 Dress Rehearsal Evaluation Program Memorandum. May 15, 1998, Attachment: Standards for Measuring Success at Each Dress Rehearsal Site.
[11] Jones, Charles D. to Michael R. Darby. January 8, 1991. Imputation of Counts in the 1990 Census.
[12] Kearney, Anne, Michael Ikeda and Maria Garcia. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. F1ab, "Evaluation of the Primary Selection Algorithm."
[13] National Opinion Research Center, January 31, 1999; Census 2000 Dress Rehearsal Results Memorandum No. C1,"Risk Assessment of the Integrated Coverage Measurement Field Data Collection and Processing Schedule of Operations."
[14] Petroni, Rita. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. C1, "Risk Assessment of the Integrated Coverage Measurement Field Data Collection and Processing Schedule of Operations."
[15] Robinson, J. Gregory. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. C7, "Assessment of Consistency of Census Estimates with Demographic Benchmarks."
[16] Roper Starch Worldwide. December 31, 1998. Census 2000 Dress Rehearsal Evaluation Results Memorandum No. E1a, "Promotion Evaluation: Evaluating the Effectiveness of Paid Advertising in Sacramento."
[17] Roper Starch Worldwide. December 31, 1998. Census 2000 Dress Rehearsal Evaluation Results Memorandum No. E1a, "Promotion Evaluation: Evaluating the Effectiveness of Paid Advertising in South Carolina."
[18] Vitrano, Frank A. Draft Census 2000 Dress Rehearsal Evaluation Results Memorandum No. B1, "Evaluation of Housing Unit Coverage on the Master Address File.
