These evaluations examine various modes for providing responses to the census. We will study the use of the telephone and Internet as response options. The effectiveness of mailing practices and the targeted dissemination of forms will also be assessed. These evaluations focus on respondent behavior and how that behavior impacts response rates (i.e., mailback, telephone, and Internet). Findings from these evaluations will identify methods that can be used in future censuses to improve the overall response rates.

The findings from these evaluations will answer a number of critical questions about how quickly the U.S. population responded to Census 2000. From a technical standpoint, the use of an Internet Questionnaire Assistance module will demonstrate the utility of employing the "most current" technologies and provide insight into respondent perception of using this mode for requesting information or completing a questionnaire. Likewise, an enhanced telephone questionnaire assistance program that is user-friendly and comprehensive will provide further insight into respondent needs and preferences.

Analyzing mail response/return rates (by form type, demographics, and geography) and mailing practices, such as tracking undeliverable questionnaires, will provide insight into improving overall response rates. Assessment of the Be Counted Campaign will help determine demographic groups that responded via the Be Counted Campaign. We also will examine the frequency of using language assistance guides and questionnaires in languages other than English, along with the number of returned non-English questionnaires.

The Census 2000 Telephone Questionnaire Assistance system was developed with contractor support to provide the following services to respondents: 1) helping them complete questionnaires, 2) providing questionnaires (English forms only) and foreign language guides upon request, and 3) conducting short form questionnaire telephone interviews when necessary. This operational evaluation assesses calling patterns, caller behavior, and system performance.

This evaluation focuses on customer reaction to the Census 2000 Telephone Questionnaire Assistance program. It includes analyses in the following areas: accessibility, ease of use, overall satisfaction with the assistance and appropriateness of the information provided.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

For Census 2000, respondents had the opportunity to complete the short form questionnaire on the Internet. This was the first time a decennial census had used this data collection mode. Since there is no background data, a general evaluation of the Internet data collection mode is planned on what might be expected in terms of frequency of use.

Customer satisfaction surveys are used to measure respondent satisfaction with both the Internet Questionnaire Assistance and the Internet Data Collection programs.

The Be Counted Campaign made blank questionnaires available at convenient locations for people who believed they were left out of Census 2000. This evaluation will examine person and housing unit coverage gains from the campaign along with the characteristics of those enumerated on Be Counted forms. This evaluation also will assess the impact on the Master Address File through documentation of housing unit adds resulting from this program, and it will evaluate our ability to geocode and process Be Counted forms.

This study will document how many housing units were mailed the advance letter about requesting a non-English questionnaire, by state and type of enumeration area (e.g., mailout/mailback, update/leave, etc.); how many non-English forms were requested, completed, and checked in; and the frequency of requests for non-English short and long forms. This study also will document the number of language assistance guides requested through Telephone Questionnaire Assistance, Questionnaire Assistance Centers, and the Internet, along with an analysis of which languages were most often requested, whether the requests were clustered geographically, and how many requests for a language assistance guide resulted in a mail returned form.

This evaluation will provide insight into how Spanish, Vietnamese, and Russian speaking households coped with the census questionnaire in Census 2000. Specifically, we will look at how these non-English speaking long form households were enumerated. We will assess their use of language guides, Questionnaire Assistance Centers, Telephone Questionnaire Assistance, and their experience with the English form.

For Census 2000, the questionnaire mailout/mailback system provided the primary means of enumeration. This type of enumeration was conducted mainly in urban and suburban areas, but also in some rural areas that contained city-style address (house number/street name) systems. This evaluation examines the rates at which housing units were classified by the U.S. Postal Service as "undeliverable as addressed" (UAA) for varying levels of geography; the occupancy status of those housing units; demographic characteristics for housing units that were deemed undeliverable but had a final status of occupied; the effect that undeliverable questionnaires had on nonresponse rates; and the check-in pattern of UAA questionnaires according to date of receipt.

This evaluation further examines the issue of the undeliverability of census mailout questionnaires. After the U.S. Postal Service determined that mail pieces were "undeliverable as addressed" (UAA), the Census Bureau attempted to deliver these cases at the Local Census Office level. This evaluation assesses the quantity of questionnaires designated as UAA and the distribution of the UAA questionnaires according to reason for undeliverability.

Housing units in mailout/mailback and update/leave enumeration areas were asked to return questionnaires in postage paid envelopes. Those questionnaires were received and checked in at Data Capture Centers. This evaluation examines mail response rates at varying levels of geography and quantifies information about incoming questionnaires according to form type and timing with respect to critical operational dates.

Housing units in mailout/mailback and update/leave enumeration areas were asked to return questionnaires in postage paid envelopes, and once all followup operations were complete, those housing units were assigned a final status. Only the housing units assigned to receive an update/leave or mailout/mailback questionnaire with a final status of occupied on Census Day (April 1, 2000) are factored into the mail return rates. Data on mail return rates provide more accurate measures of cooperation than mail response rates, for which the denominator also includes units that turned out to be vacant or non-existent on Census Day (April 1, 2000). This evaluation examines mail return rates at varying levels of geography, quantifies information about incoming questionnaires from occupied housing units according to form type and timing with respect to critical operational dates, and provides return rate data according to certain housing unit demographic characteristics.

For Census 2000, the Census Bureau conducted an update/leave enumeration for the first time in Puerto Rico. That is, census enumerators left a questionnaire at each housing unit with a mailing address, and residents were asked to complete the questionnaire and return it to the Census Bureau in a postage-paid envelope. The response rate was close to 50 percent. The purpose of this research is to obtain information on why nonrespondents did not return the questionnaire by mail for Census 2000 in Puerto Rico. This information will help develop strategies for improving the response rate for the 2010 Census.

For Census 2000, the public had five ways of providing census data. These modes included mailing back a questionnaire, filling out a census short form on the Internet, picking up and returning a Be Counted form, completing a short form census interview via telephone questionnaire assistance, or completing a personal visit interview with an enumerator. With this in mind, and the likelihood that the 2010 Census may offer additional options for response, studies in this category will document the hundred percent data item nonresponse by response mode. Additionally, the data quality of each mode will be assessed. This category includes a Content Reinterview Survey study that will measure response variance, and a Master Trace Sample study. The latter will create a database containing a sample of census records with information pertaining to them from the entire census process. Other research will evaluate multiple responses to the new race question.

The findings from these evaluations will answer a number of critical questions on our process to define content (i.e., what questions to ask) and the resulting quality of data for Census 2000. These findings, in turn, can help us do a better job for the 2010 Census and the American Community Survey.

We will learn about the completeness of the data by calculating item imputation rates for several data items. We also will look at hundred percent data item nonresponse by data collection mode. We will assess responses to the new race question. In particular, we will recontact a sample of households with responses of two or more races, and collect additional information, including an instruction to choose a single race category. This study is needed to meet the data requirements of other agencies that use only single race categories, and for comparison to 1990 Census race data.

We also will gain knowledge about data quality in comparison to external benchmarks by matching and/or comparing census data to data collected by Census Bureau demographic surveys including the Current Population Survey, American Community Survey, and the Survey of Income and Program Participation. The results of these matching and comparison studies will also help us to improve the design of future surveys and censuses.

Some of the reports that were developed in an expedited manner to inform the Executive Steering Committee for the Accuracy and Coverage Evaluation (ESCAP) decisions were sufficiently complete and informative to answer research questions from the planned evaluation reports.

One of these reports and its corresponding ESCAP report is noted in the following section of evaluation report descriptions. To review the ESCAP report on the Internet, go to www.census.gov and search on "ESCAP."

To deal with missing and inconsistent data, three components will comprise the imputation process for Census 2000: assignment, allocation, and substitutions. Rates for each of these components will be produced for the 100 percent characteristics and for the tenure item. This analysis will document imputation rates and will serve as a supplement to other evaluations.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. Some of the hundred percent data item nonresponse aspects have been incorporated into the B.1 evaluation.

Data by race from most federal surveys currently reflect a collection methodology of asking respondents to mark only one race category. Users of the Census 2000 data on race will need to compare the race distribution from Census 2000 to these other sources. The objective of the study is to produce a data file that will improve users' ability to make comparison between Census 2000 data on race, that allowed the reporting of one or more races, and data on race from other sources that allow single race reporting. The primary goal is to improve comparisons of 1990 and Census 2000 race distributions at national and lower geographic levels. Other goals are to facilitate comparisons between Census 2000 and Census Bureau surveys which instruct respondents to mark one race, and comparisons with data from the vital records system, which uses census data, to calculate such indicators as birth and death rates.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation is relevant to this evaluation. Refer to ESCAP I report, "A.C.E. Consistency of Post-Stratification Variables" (report B-10).

The Content Reinterview Survey utilizes a test-retest methodology, whereby a sample of households designated to receive the census long form were reinterviewed shortly after they had been enumerated by the census. These households are essentially asked the same questions posed on the long form. Then the responses to the census and reinterview survey are compared. This survey assesses response variance and error that results from data collection and capture operations.

While most evaluation studies will provide detailed information on specific Census 2000 operations, the Master Trace Sample database will provide information that can be used to study various operations, along with correlates of error across various systems, for a randomly selected group of census records. This database will contain, but is not limited to: address list information (e.g., source of address), final values for questionnaire items along with their values at various stages of processing, and enumerator information (e.g., number of enumerator attempts before completing an interview and enumerator production rates). This database also will contain information about the data capture system (from rekeying and reconciling a subset of Master Trace Sample questionnaire images), the Accuracy and Coverage Evaluation, and the Content Reinterview Survey. This evaluation report will document the process of developing the Master Trace Sample database. It will include information on the sources of data, limitations with the data, and some basic statistics from the database itself. The majority of research and analysis that will be conducted using the Master Trace Sample database will not be done as part of this evaluation.

Using the results of a person-level match of responses to the Current Population Survey (CPS) and Census 2000, this study provides a data set about differences between the Census and Survey estimates of social, demographic, or economic characteristics. Its strength is its ability to represent differences arising from non-sampling variation. The study focuses on the difference between CPS and Census estimates of poverty and labor force status (which are measured officially by the CPS).

The purpose of this evaluation is to determine to what extent Census 2000 poverty, unemployment, and income estimates are comparable with estimates from the Current Population Survey, the American Community Survey, and the Survey of Income and Program Participation. This study focuses on changes made to the Census 2000 questionnaire and forms processing systems that were designed to improve unemployment estimates. This evaluation examines whether these changes brought the Census 2000 unemployment estimates (for states, and for various demographic and socio-economic groups) closer to the official Current Population Survey estimates than they were in 1990. This analysis may be extended to compare data, definitions, and collection procedures with the Survey of Income and Program Participation.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted because the available data cannot answer the specified questions for this study.

For the first time, in Census 2000, households in Puerto Rico were asked to answer questions on race and ethnicity. These were the same questions used on the stateside questionnaire. This evaluation will explore how the census questions on race and Hispanic origin were answered by people living in Puerto Rico. Specifically, it will investigate whether there are any generalizable patterns in how people responded to these questions according to age, level of education, level of income, and response mode. In addition, it will look for patterns in responding to the Hispanic origin question by race and to the race question by Hispanic origin. Finally, it will compare the patterns of responding to these questions with those of the general U.S. population. This investigation will be conducted by preparing special data tables from the Census 2000 data for Puerto Rico. The results will be used to help in the interpretation of the tabulated results on race and Hispanic origin for the Commonwealth.

The purpose of this research is to conduct a series of focus groups across Puerto Rico to learn more about how persons in Puerto Rico view these questions. The results of the focus groups will be useful in preparing for the 2010 Census in Puerto Rico.

The focus of this evaluation is to determine the effects of disclosure prevention measures on Census 2000 data products. We will examine the limitations and effects of data swapping and our confidentiality edit - a combination of strategies used to prevent the disclosure of data that can be linked to an individual - on our data products.

In studying our data swapping techniques, we will examine rates for different geographic levels and race groups and document any issues and problems that resulted from multiple responses to the race question.

For Census 2000, the data swapping methods first used in 1990 were refined through better targeting and expanded to include sample data. This evaluation examines variations in the effects of swapping due to: 1) a region's geographic structure, 2) a region's racial diversity, and 3) the number of dimensions used in the swapping.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. Some focus groups were conducted and the results will be documented in a report, "Focus of Key Customer Segments." Additionally, "Revisiting Standard Census Products in a New Environment," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301) 457-4218.

Information about the effectiveness of the data products strategy was rolled into evaluation C.2 in 2001, which recently was cancelled. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

During Census 2000, we used new methods to promote census awareness and increase public cooperation. The Census 2000 Partnership and Marketing Program combined public awareness, promotion, and outreach activities to generate clear and consistent messages about the importance of participating in Census 2000. The plan incorporated five components: direct mail pieces; media relations; promotions and special events (including Census in Schools); partnerships with businesses, non-governmental organizations, and government entities; and paid advertising.

The primary goal of our comprehensive marketing plan, including the first ever paid advertising campaign coupled with an expanded partnership program, was to increase the mailback response rate, especially among historically undercounted populations. The advertising marketing strategy included messages delivered through print media, radio, television, and out-of-home media (billboards, bus shelters, mobile billboards). The partnership program built partnerships with state, local, and tribal governments, community-based organizations, and the private sector. Partners were asked to assist in three major areas: field data collection support, recruitment, and promotion. In addition, a major school-based public information campaign was launched to inform parents and guardians about the census through their school-age children. The planned evaluations for this category will assess the effectiveness of these activities as part of the integrated program components of the Census 2000 Partnership and Marketing Program.

These studies will help us understand how people's attitudes, knowledge, and behavior were affected by the Census 2000 Partnership and Marketing Program. We will examine which elements of the paid advertising media were reported/recalled most often by hard-to-enumerate groups, and provide data for Hispanics and for five race categories: African-American, Asian, American Indian and Alaska Native, Hawaiian/Pacific Islanders, and White. Specifically, we will look at what impact the marketing program had on the likelihood of returning a census form. We also will compare these data to the 1990 Census, which had no paid advertising campaign. The primary goals in studying the Partnership Program are to measure how well national and regional components accomplished their objectives in communicating a consistent census message of program initiatives and to determine which populations were best served by the program. Our evaluation of the Census in Schools program will tell us about the effectiveness of census educational materials and whether teachers receiving census materials incorporated them in their curricula. Overall, we will analyze how well the integrated strategy of the Census 2000 Partnership and Marketing Program met its two goals: 1) to increase the awareness of the census, and 2) to increase mailback response rates, especially among historically undercounted populations.

The Census Bureau hired the National Opinion Research Center to conduct an assessment of the marketing and advertising campaign by fielding a survey before the campaign began, during the education phase of the campaign, and after the campaign had been launched. From this evaluation, we will assess intended and self-reported response behavior and establish a baseline and pre- and post-census measures of awareness, knowledge, and attitudes of the census. We will obtain the actual response behavior for respondents to our survey. We statistically model what effect self-reported advertising exposure has on the likelihood of responding to the census. This evaluation also explores the link between raised awareness, knowledge, attitudes, and response to the census. Due to the integrated strategy of the Census 2000 Partnership and Marketing Program, the analysis will address components from the entire program, not individually for paid advertising and partnerships.

The Census Bureau hired Macro International to conduct a post-census survey of school teachers to assess the effectiveness of Census educational materials and whether teachers receiving census material incorporated them in their curricula.

We will assess the helpfulness of Census 2000 materials to partners, the types and value of services rendered, the specific partnership activities conducted, and the effectiveness of the program in reaching the hard-to-enumerate population. We also will obtain from partners the organizational costs incurred to support and promote Census 2000. The sample of partners will be selected using the Contact Profile and Usage Management System database. Westat, an independent contractor, was hired to conduct this survey. To improve response among the geographically dispersed and diverse partners in the Partnership Program, this evaluation uses as data collection modes a mailout/mailback questionnaire with a nonresponse followup telephone interview.

The vast majority of U.S. residents live as families or individually in houses, apartments, mobile homes, or other places collectively known as "housing units." However, there are millions of people in the United States who live in group situations such as college dormitories, nursing homes, convents, group homes, migrant worker dormitories, and emergency and transitional shelters. Our evaluations will analyze the effectiveness of procedures to enumerate persons living in different types of group quarters.

The Census Bureau developed a specialized operation to enumerate selected service locations that served people without conventional housing. The service-based enumeration operation was conducted from March 27 to March 29, 2000, at shelters, soup kitchens, mobile food vans, and targeted nonsheltered outdoor locations.

Some studies will focus on such things as enumeration at "service based locations" (shelters and food facilities for the homeless; outdoor locations where homeless people sleep). Major evaluations are planned for two operations designed to enhance the address list of special places: the Special Place Facility Questionnaire and the Special Place Local Update of Census Addresses.

The findings from these evaluations will answer important questions on how effective enumeration procedures were in obtaining the count for group quarters. We will compare the telephone and personal visit operations of the Facility Questionnaire. The evaluations will include distributions of the group quarters populations by type of group quarters, counts of persons at group quarters on Census Day who indicated a usual home elsewhere, and comparison of the predicted group quarters universe from the Facility Questionnaire operation with the group quarters universe as enumerated.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation used personal visit reinterviews at a sample of special places to assess the accuracy of the information collected from the Facility Questionnaire via computer assisted telephone interview or personal visit. This evaluation will address how often changes occur in the special place type code and whether classification discrepancies differ by type of special place.

This evaluation focuses on local governments' participation in the Special Place Local Update of Census Addresses. It will document changes to the address list along with operational issues that were encountered.

The purpose of this study is to evaluate the return on the efforts made to compile the inventory of Service-Based Enumerations places and addresses that were included in Census 2000. The study will look at various sources that provided names and addresses and the results from enumeration to determine which sources proved to be more reliable.

This study evaluates the content, coverage, and sources of the Decennial Frame of Group Quarters by comparing editions and records to independent sources, notably the contemporary Business Register. This evaluation examines the feasibility and constraints to enrich or integrate these frames.

This study will document various aspects of the group quarters enumeration. Some of the topics covered by this study include the total count of the group quarters population, the number of special places that were enumerated, and the number of group quarters that were enumerated. Additionally, the numerical distribution of group quarters per special place and of residents per group quarters will be documented.

The goal of the Service-Based Enumeration (SBE) was to enumerate people without housing who may have been missed in the traditional enumeration of housing units and group quarters. A complete enumeration of shelters, soup kitchens, regularly scheduled mobile food vans and targeted nonsheltered outdoor locations was conducted in March 2000. This evaluation will document data collection completeness, partial interviews, and whether the SBE unduplication process successfully identified individuals who were enumerated more than once.

These evaluations cover a broad spectrum of activities, both internal and external, involved with building address files and the related geographic database, including field operations from which address information and related map updates are gathered. The address list development category includes various evaluations of the Census Bureau's Master Address File (MAF), and the Topologically Integrated Geographic Encoding and Referencing (TIGER) database. These include examination of the completeness and accuracy of address information in the MAF, as well as of the design of the MAF and DMAF. An evaluation of the U.S. Postal Service's Delivery Sequence File used in the MAF building process is also planned. A variety of census field and local/tribal partner operations will be evaluated to measure the impact of each operation on the MAF and the TIGER database. These include, but are not limited to: Address Listing, Block Canvassing, Update/Leave, List/Enumerate, and multiple cycles of the Address List Review (also referred to as the Local Update of Census Addresses). Combined, these field operations offer comprehensive address checks in rural and urban areas and are a primary source of address information used for MAF and TIGER database enhancement. Additional evaluations focus on the geocoding accuracy of addresses in the census.

The findings from the address list development evaluations will provide insight into the most accurate methods for updating the MAF and the related TIGER database. This includes understanding the individual contribution of each operation as it is implemented. For each operation, we will look at the characteristics of addresses that were added, corrected, or flagged for deletion. We also will look at the geographic impact of each operation (i.e., we will examine how changes to the MAF are distributed geographically). Additionally, we will learn some things about the overall housing unit coverage in the census. Finally we will learn more about quality and coverage by examining addresses that are on the full MAF, but were not included in the census for various reasons. All of these evaluations will help inform continued MAF and TIGER database updating through the decade and also will provide insight for the 2010 Census and the American Community Survey.

The Delivery Sequence File (DSF) is a file of addresses produced and maintained by the U.S. Postal Service. The Census Bureau uses this file, along with the 1990 Census address list and other information, to create a permanent national address list called the Master Address File (MAF). For Census 2000, the Census Bureau used the DSF as a primary source to enhance the initial MAF for mailout/mailback areas of the country. Subsequent DSFs were used to update the address list through April of 2000, in order to maximize the inclusion of all existing addresses in the census. This evaluation will assess the impact of each of the DSFs through Census 2000 operations by profiling the number and characteristics of housing units added to and deleted from the MAF following each delivery of the DSF.

For Census 2000, an Address Listing Operation was used in update/leave areas of the country to create the initial Master Address File (MAF) and provide a comprehensive update of the streets/roads and their names in the TIGER database. In this operation, in areas where census questionnaires were subsequently hand delivered, census enumerators went door-to-door to identify the mailing address and physical location of every housing unit. They also verified and updated the location and names of geographic features such as streets. The Census Bureau used this procedure in order to create a file of good locatable addresses for Census Bureau field operations in Census 2000 as well as its future demographic surveys, including the American Community Survey. This evaluation will assess the impact of the Census 2000 Address Listing Operation on the MAF by profiling the number and characteristics of housing units added to the MAF.

The Local Update of Census Addresses (LUCA) operation (also known as Address List Review) for Census 2000 included a LUCA 98 operation that focused on mailout/ mailback areas. For this operation, local and tribal government entities were provided a Census Bureau address list containing addresses derived from the Delivery Sequence File and the 1990 Address Control File. The objective of the LUCA operations was to provide local entities the opportunity to review the Bureau's address information and related maps and then provide feedback in the form of 1) address adds, deletes and corrections and 2) street and street name adds, deletions, and corrections on the maps. The Census Bureau compared the results to the block canvassing results in mailout/mailback areas, and all discrepancies were field verified. After Census Bureau review of submissions, local and tribal entities were given the opportunity to review results and to appeal situations in which they believed the Master Address File (MAF) still was incomplete or incorrect. This evaluation will assess the number and profile of housing unit adds to the MAF, the extent of geographic clustering of these adds, and the total number and profile of housing unit deletions and corrections. The evaluation also will include information documenting the participation rates of local and tribal governments and the proportion of addresses covered by these governments.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

For the 1990 Census, the Census Bureau conducted an operation called Precanvass to improve its address list for mailout/mailback areas. For Census 2000, a similar operation, called Block Canvassing, was implemented. As with the1990 Precanvass, this operation was conducted primarily in areas where city-style addresses are used for mail delivery; however, for Census 2000, the Block Canvassing Operation covered a larger geographic area than did the 1990 Precanvass Operation, and the scope of the operation was expanded to include map (i.e. TIGER database) updates. The objective of this evaluation is to determine the overall effect of the Block Canvassing Operation on the Master Address File (MAF) by measuring the number and characteristics of housing unit adds, deletes, and corrections to the MAF.

The Local Update of Census Addresses (LUCA) operation (also known as Address List Review) for Census 2000 included a LUCA 99 operation for Update/Leave areas. For LUCA 99, local and tribal government entities were provided with census housing unit block counts that were created using addresses obtained from the Address Listing Operation. Participating entities were asked to review the counts and provide feedback when they believed the number of housing unit addresses for the block should have been higher or lower. Participating governments could challenge block counts, but could not provide specific housing unit adds, corrections, or deletes. Blocks that were challenged were sent to LUCA 99 Field Verification for relisting, then returned to participating governments for another review. This evaluation will document the participation rates of those tribal and local governments that were eligible to participate, the proportion of addresses covered by those governments, the number of blocks that were challenged and went to LUCA 99 Field Verification, and the extent to which changes occurred during the field verification.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. Some of the planned research for this study will be evident in evaluation I.4,Coverage Improvement Followup.

The Update/Leave operation was conducted in areas where mail delivery of questionnaires was problematic. Field staff dependently canvassed their assigned area, updated the address list and map, and distributed a questionnaire to each housing unit. This evaluation will document address corrections, added units, and units flagged for deletion during the operation. We also will study problem referral forms completed by enumerators for difficult listing situations (e.g., unable to obtain access, gate blocked, road washed away, no trespassing signs), to see how well these situations were followed through and how they might have contributed to coverage errors.

The Urban Update/Leave was an operation that targeted whole census blocks and was conducted in areas where the Census Bureau was not confident that the addressed questionnaires will be delivered to the corresponding housing units. For Census 2000, eight of the 12 Regional Census Centers identified blocks for this operation. The Charlotte, Kansas City, Los Angeles, and New York Regional Census Centers decided to use other special enumeration methodologies in lieu of Urban Update/Leave. This evaluation will assess the number of addresses added, deleted, corrected, and moved as a result of Urban Update/Leave. It will profile the housing unit addresses as follows: type of address, single/multi-unit; drop/nondrop delivery. Delivery Sequence File match/nonmatch. It will also look at the addresses in terms of occupancy status and will describe the persons in Urban Update/Leave addresses by sex, age, Hispanic origin, and race.

Update/Enumerate is similar to Update/Leave, except that interviewers enumerated the unit at the time of their visit rather than leaving a questionnaire to be completed and mailed back. The operation was conducted in communities with special enumeration needs and where most housing units may not have house numbers and street name addresses. These areas include some selected American Indian Reservations and the Colonias. Update/Enumerate was implemented in resort areas with high concentrations of seasonally vacant housing units. Most Update/Enumerate areas were drawn from address listed areas, but some came from block canvass areas. This evaluation will document the number and characteristics of housing units added, deleted, corrected, and moved in Update/Enumerate areas.

List/Enumerate was an all-in-one operation conducted in sparsely populated areas of the country. The address list was created and the housing units enumerated concurrently. The main objectives of this evaluation will be to profile all addresses produced by the List/Enumerate operation, as well as to specifically profile the List/Enumerate addresses that matched to the Delivery Sequence File.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. Objectives of this evaluation will be met with the Address List Development topic report, which will synthesize results from evaluations, experiments, and include some new analysis across all of the major sources and operations that contributed to the Master Address File in Census 2000.

The objective of this evaluation is to measure the quality of residential address geocoding in Census 2000 and to identify the source of the geocode (i.e., the TIGER database, one of the several field operations, LUCA/New Construction participants, etc.).

Block Split operations are conducted by the Census Bureau to provide for tabulation of data where governmental unit and statistical area boundaries do not conform to collection block boundaries. This evaluation will measure the accuracy of block splitting operations for tabulation purposes.

Prompted by the difficulties in recruiting applicants and high turnover of employees in the 1990 Census, the Census Bureau redesigned its recruitment, staffing, and compensation programs for Census 2000. Several new programs were developed to address the 1990 issues and to help the Census Bureau successfully recruit several million applicants, hire several hundred thousand employees, and retain this staff through the decennial census. Some of these programs included frontloading, higher pay rates, and paid advertising.

The purpose of this evaluation is to study the effects of these new program activities upon recruitment, staffing, and retention. A contractor, for example, determined that the 1990 District Office (now Local Census Office) pay rates were not adequately set to attract and retain staff when compared to local economic conditions of that area. The methodology to set the Census 2000 pay rates, based on this knowledge, was revised and set to a derivative of the local prevailing pay rate. The effectiveness of this higher pay rate will be evaluated, as well as other recruitment and hiring programs (such as frontloading and paid advertising).

This evaluation examines the effectiveness of the Census 2000 hiring programs during Nonresponse Followup (NRFU). Study questions will focus upon the effectiveness of the higher pay rate program, frontloading, paid advertising, and other areas. Some of the questions are: 1) was the Census Bureau able to adequately hire and attract staff to execute NRFU, Accuracy and Coverage Evaluation, and other various field operations; 2)were the pay rates effective in attracting and retaining staff needed for Census 2000 NRFU; and 3) did recruiting activities provide an adequate supply of applicants and replacements. A portion of this study will examine the effectiveness of the higher pay rates on productivity and evaluate the pay model as a predictor of local economic conditions.

This evaluation has been moved to Evaluation Category R, Automation of Census Processes; see R.2.a, Operations Control System 2000, System Requirements Study for its description.

This category includes studies of various field operations and strategies whose goals were to curb questionnaire delivery and enumeration problems, and obtain census data from individuals who did not respond to the census by a specified date. For example, the Nonresponse Followup operation consisted of sending an enumerator to collect census data from every address from which no mail, telephone, or Internet response was received. Evaluations in this category will analyze whether field operations were conducted as planned and will assess their effectiveness. Additionally, operational results will be documented for each LCO for historical purposes.

Analyses in this category also will examine our efforts to count those categorized as hard-to-enumerate. Our targeting methodologies consisted of 1990 person and housing unit census data that are indicators of nonresponse and the potential to be undercounted. This information assisted the Regional Census Centers in determining the placement of Questionnaire Assistance Centers and Be Counted Forms. The information also assisted participants of our partnership program. Studies in this category will evaluate how successful our targeting methodologies were along with the usage of Questionnaire Assistance Centers. In addition, we will evaluate our targeted enumeration methods such as blitz enumeration (use of a group of enumerators to conduct enumeration in a compressed time frame), team enumeration (two enumerators working together where safety is a concern), and the use of local facilitators (long-time neighborhood residents or church leaders who assist the enumerator in gaining entry to the neighborhood).

Because some respondents were able to provide data without a census identification number (e.g., Be Counted and Telephone Questionnaire Assistance), it was possible that respondents submitted addresses that were not on our Master Address File. We conducted a field verification of these types of addresses. If an enumerator verified that the address was a valid housing unit, then it was added to the Decennial Master Address File. We also will conduct an evaluation of the effectiveness of this operation.

The results of these evaluations will give us an indication of how successful we were at obtaining data from nonrespondents including those living in areas where we employed special enumeration methodologies, and how to better plan these types of operations for future censuses. The evaluation of Nonresponse Followup will report proxy rates, the number of partial interviews, vacancy rates, and the number of units enumerated during final attempt procedures, which will help us to assess whether the operation was conducted as planned.

Other analyses will provide information about the quality of our enumerator training program, the utility of our targeting methods, and a profile of Local Census Offices which will contain various descriptive statistics.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. "Using the Planning Data Base to Plan Data Collection for Census 2000," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301) 457-4218.

Non-ID questionnaires are those from the Be Counted and Telephone Questionnaire Assistance operations or questionnaires for which an enumerator was not able to verify that the address existed. During field verification, enumerators visited the location of these non-ID housing units and verified their existence on the ground before they were added to the Master Address File (MAF)/Decennial Master Address File (DMAF). For Census 2000, non-ID questionnaires that were geocoded to a census block, but did not match to an address already in the MAF were assigned for field verification. This operational analysis will attempt to answer questions such as how many units were added to the MAF/DMAF after verification and if operational problems were encountered during the implementation of field verification.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

The Census Bureau provided walk-in assistance centers where respondents received assistance with completing their questionnaire. Language assistance guides were available in over 40 different languages, along with Be Counted forms that were available in English and five other languages. This study will document various aspects of the Questionnaire Assistance Centers (QACs) such as location, employees, and types of assistance. In addition, the frequency of use of the QACs will be analyzed.

This operation was conducted for all housing units in the mailout/mailback and update/leave areas for which the Census Bureau did not check in a questionnaire by April 11, 2000. During Nonresponse Followup (NRFU), enumerators visited each nonresponding unit to determine the occupancy status of the unit on Census Day and to collect the appropriate data (i.e., long form or short form) for the household members. The objective of this analysis is to document various aspects of the NRFU operations. Some of the topics covered in this study include determinationof NRFU workloads, identification of the demographics of those enumerated in NRFU, and documentation of the number of NRFU Enumerator Questionnaires that were partial interviews, refusals, completed via proxy respondents, or completed during final attempt procedures. The percent of NRFU units classified as occupied, vacant, or delete will be documented. Additionally, this evaluation will determine when each Local Census Office (LCO) started and completed their NRFU operation and the cost of the operation.

Tool kit methods were special enumeration procedures (e.g., blitz enumeration, and the use of local facilitators) available for improving cooperation and enumeration in hard-to-enumerate areas. For this evaluation, the Census Bureau will assess the tool kit methods used, where they were used, and the effectiveness and feasibility of the tool kit methods.

During Census 2000, we hired over 500,000 people to fill temporary positions. The largest number of these workers were hired for the Nonresponse Followup (NRFU) operation. Adequate employee training was critical to the success of NRFU. The overall objective of this evaluation is to examine the quality of the NRFU enumerator training program as well as the enumerator's state of preparedness following training.

Census 2000 was the first time that an Update/Leave mailback methodology was used to conduct the enumeration in Puerto Rico. This evaluation will determine how many addresses were encompassed by this enumeration methodology, a profile of the addresses, and what operational problems were encountered in the field as a result of address list compilation and processing procedures. This study also will make comparisons to stateside Update/Leave data.

This operational summary will provide descriptive statistics at the Local Census Office (LCO) level for many census operations. For example, total housing units, average household size, and mail return rate will be among the statistics reported for each LCO.

The Census 2000 questionnaire stated that the respondent should report age as of April 1, 2000. This study will document the average date of reference used by census respondents and the average date of reference by method of enumeration. This study also will document various types of discrepancies between date of birth and reported age.

The coverage improvement evaluations examine various Census 2000 operations that are intended to improve the coverage of both housing units and people in the census. Following the mailback efforts to complete the census, a series of operations were conducted to ensure that people were counted at their correct Census Day address, to confirm the status of housing units that were deleted or enumerated as vacant, and to ensure the inclusion of all persons in a household when the returned form showed discrepancies in the number of persons enumerated.

From these evaluations we will learn about the effectiveness of these various operations as they attempt to improve census coverage. From the Nonresponse Followup operation, we will examine the potential coverage gain from identifying movers and checking to see if they were counted at their Census Day address. We will also analyze the situations where entire households were identified as having a "usual home elsewhere." For the Coverage Improvement Followup, we will examine the person and housing unit coverage gains from this operation, which determined the Census Day status of certain types of housing units (most of which were identified as deletes or coded as vacants in earlier census operations). The evaluation of the Coverage Edit Followup will measure coverage gains from this operation, which consisted of contacting households whose completed forms showed discrepancies regarding the number of persons enumerated, or whose completed form indicated there were more than six persons in that household. Furthermore, we will evaluate the coverage questions on the enumerator questionnaire to determine how well enumerators asked these questions and used the answers to obtain an accurate household roster.

The Coverage Edit Followup (CEFU) was designed to increase within household coverage and improve data quality in two ways. A standard questionnaire only has room for six persons, so CEFU was used to collect data on additional persons in large households. Second, it resolved discrepancies on mail return forms between the reported household size and the actual number of persons for which data were recorded on the census form. An attempt was made to resolve all households that failed edits for these situations by using a Computer Assisted Telephone Interview. This analysis will document the workload, operational aspects, and coverage gains from conducting this operation.

During the Nonresponse Followup (NRFU), List/Enumerate, and Update/Enumerate operations, enumerators asked respondents whether their address was a seasonal or vacation home and if the whole household had another place where they lived most of the time. When respondents indicated they had a usual home elsewhere on Census Day, enumerators recorded census information about this on a blank Simplified Enumerator Questionnaire (SEQ - a version of the mail return questionnaire that is easier to use for personal visit enumeration) and enumerated the current address as a vacant unit or obtained information about the people living there on Census Day. This evaluation examines how often SEQs were completed as Whole Household Usual Home Elsewhere (WHUHE); how many of these addresses were matched to an address on the Decennial Master Address File (DMAF); how often addresses could neither be matched to the DMAF or geocoded; for matched addresses, how often was the WHUHE household already included on the census form for their ususal place of residence; and how often did we find a different household on the census questionnaire.

In Census 2000, in-movers (households that moved there after Census Day) were identified during the Nonresponse Followup (NRFU), List/Enumerate, and Update/Enumerate operations and were asked if they were enumerated at their Census Day address. If a respondent did not recall completing a census form at their Census Day address, the enumerator completed a questionnaire for the in-mover household using their Census Day address. This evaluation looks at how many of these cases occurred, and how many persons were added to the census as a result of this procedure.

The Coverage Improvement Followup (CIFU) universe consisted of units classified as vacant or deleted in NRFU, adds from the new construction operation, late adds from Update/Leave, blank mail returns, and lost mail returns, if any. During CIFU, enumerators visited these units to verify the Census Day status and collect person and housing unit data as appropriate. This evaluation will document the person and housing unit coverage gain from conducting the CIFU, including the number of units that changed status from vacant to occupied or from delete to either vacant or occupied. This study also will examine the characteristics of persons and housing units added as a result of the CIFU, start/finish dates, and the cost of the operation.

In 1990, enumerators began their interview with an explanation of who should be included as residents of the household. This procedure was changed for Census 2000. Now, enumerators begin by asking how many people were living or staying in the housing unit on Census Day. After collecting appropriate person and housing unit data, the enumerator asked two coverage questions. The first asked about typical situations in which persons who should be included as residents tend to be missed - babies, foster children, persons away on business or vacation, roomers or housemates, and temporary residents with no other home. If someone had been missed, then he or she was added to the form and their census information was collected. The second question asked about typical situations in which persons who should not be included as residents tend to be included as such - persons away at college, in the armed forces, in a nursing home, or in a correctional facility. If someone was included on the form but should have been counted elsewhere, then the enumerator deleted them from the form by marking the cancel box under their name. The purpose of this analysis is to study the effectiveness of the new coverage questions in the identification of persons who would have otherwise been missed or included in error.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

These evaluations will study certain aspects of coverage for various populations and attempt to identify areas where methods of collecting census data for these populations can be improved. One study in this category will apply social network field and analysis methods to evaluate census coverage and processes. We also will conduct ethnographic research on mobile populations and Colonias - areas lacking basic infrastructure and services along the border between the United States and Mexico.

Study results will help us determine whether individuals can be better identified from their position in social networks (based on their interactions and transactions with others) than by comparing sets of address and person records. We will also learn how to improve procedures to enumerate mobile populations by tracing Census Day travel routes or stopover sites for a sample of such persons and determining undercounts or multiple enumerations of them in the census. We also will learn how to overcome barriers to enumerating Colonias in future censuses.

This evaluation was reclassified under the Coverage Improvement category as evaluation I.6, which was recently cancelled.

This study will use ethnographic and social network methods to study the following five questions. 1) What interactions in social networks influence and explain or determine the duration of individuals' stays in domiciles (i.e., households, institutions, or other places where people sleep) and their residential mobility? 2) How much more likely are people who change domicile once or more in 6 months to be omitted or erroneously enumerated in Census 2000 (and in contemporary demographic surveys) than people who remain residentially stable over a 6-month period? 3) What characteristics (of people, their networks, mobility, housing, household, occupational, or other social or economic factors) are closely associated with omission in the census? 4) Can people be more reliably identified (and re-identified) from their position in social networks and from their interactions with others, than by comparing sets of address and person records? 5) How well do Census Bureau categories fit with the socially represented characteristics that people use to form interacting social networks?

In this study, a sample of selected mobile people will be traced to identify their Census Day travel routes or stopover sites. The information will be matched and reconciled with census results. Coverage errors found in the census will be analyzed to develop recommendations for improving procedures.

Colonias are unincorporated, generally low income residential subdivisions lacking basic infrastructure and services (e.g., paved roads and public water systems) along the border between the U.S. and Mexico. In order to develop appropriate enumeration procedures and effective outreach and promotion programs for Colonias, it is necessary to better understand the unique situations and issues associated with conducting the census or other Census Bureau surveys in these areas. This research will examine the potential barriers to census enumeration in Colonias in the context of Census 2000 through participant observation, in-depth interviews, and focus groups with selected Colonia residents. Based on previous research, topics of particular interest include irregular housing, concerns regarding confidentiality, complex household structure, knowledge of English, and literacy.

The Data Capture System for Census 2000 (DCS 2000) processed more than 120 million census forms by creating a digital image of each page and interpreting the entries on each image using Optical Mark Recognition (OMR), Optical Character Recognition (OCR), or keying. These evaluations are designed to assess components of DCS 2000, the Data Capture Audit Resolution (DCAR) process, and to measure the impact of the data capture system on data quality and on subsequent data coding operations.

Findings from these evaluations will determine the level of accuracy at which the data capture system performed. Detailed information about the system will be collected, ranging from the number of forms processed by form type, date, and processing office, to measuring the accuracy of each of the three capture modes - OMR, OCR, and Key From Image. Operational problems and their resolution will be documented. Evaluation of the DCAR process will examine the system's ability to identify and resolve capture problems stemming from problems with response entries. The impact of data capture errors on our ability to correctly assign industry and occupation codes will also be assessed.

This evaluation documents the results of Data Capture Audit Resolution by failure reason, form type, and Data Capture Center. Using these same categories, it also will document the number and types of changes that can be made by Audit Review clerks and the results of the Audit Count review.

This evaluation examines how the data capture system affected data quality and whether the rules for determining where cases are routed (e.g., to key from image) were set appropriately. In addition, this evaluation will document and compare the data quality of each data capture method for every field on the questionnaire, as well as by form type, Data Capture Center, and racial and ethnic categories.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. Results from evaluations K.1.a, K.1.b, and K.1.c will be included in a topic report addressing key findings for all data capture evaluations.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census. Significant aspects of this study will be addressed in evaluations K.1.a and K.1.b.

The information provided by respondents to the industry and occupation questions on the census form were assigned (coded) to a standard set of categories. This evaluation examines how data capture errors affected the ability of the autocoding system and clerical coders to assign correct Industry and Occupation codes.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

Once census data from all sources were captured by the Data Capture System 2000, they were normalized (put in a standard format, regardless of input source) and stored in a file known as the Decennial Response File, stage 1 (DRF1). Several processes then were applied before the data were used to produce official census counts and tabulations. One process was applied to link multiple questionnaires that were used to enumerate that same household. For example, a large family could have a mail return form with data on six members of the household and an additional followup Computer Assisted Telephone Interview (CATI) with data on the rest of the household. Under these circumstances, CATI person records are appended to the original mail return person records. Another process was used for situations where multiple questionnaires involving different households were received for the same address. For example, one form could be for a household that moved out near Census Day, and the other form could be for the household that then moved in. A computer program known as the Primary Selection Algorithm (PSA) then was used to decide which person and housing unit data should be used for census tabulations. The input into PSA is the DRF1, with the resulting file being the Decennial Response File, stage 2 (DRF2). Following all these processes, the DRF2 was merged with key elements of the Decennial Master Address File (DMAF) to create the 100 Percent Census Unedited File (HCUF), which contains the response data (i.e., the 100 percent questions from both the short and long form census questionnaires) selected by PSA processing to represent a household in the census. The Sample Census Unedited File (SCUF) contains the unedited responses for households with sample (long form) data.

A variety of post-census activities were needed to prepare the data from the original responses to releasing the official counts and tabulations. These activities include editing and imputation of the HCUF and SCUF to create the 100 Percent Census Edited File (HCEF) and the Sample Census Edited File (SCEF) respectively, coding of write-in response items (such as race, language, industry and occupation, and place of work/migration), conversion to tabulation geography, tabulation recoding, and applying disclosure avoidance techniques.

The Beta Site was a software testing site for Bureau of the Census application developers and is used as an integration center for Regional Census Centers (RCC) and Local Census Offices (LCO) systems, a testing center for all systems, and a support center for RCC, LCO, and the National Processing Center systems. We will examine the effectiveness of this software testing site.

Analysis of a reinterview of multiple questionnaire addresses will determine if the PSA methodology and rules for resolving these cases accurately identified the Census Day household members. The evaluation of the DRF creation and processes will examine how well multiple forms for the same household were linked. Analysis of CUF creation will document the number of times each specific DMAF/DRF rule was applied. The Beta Site analysis will include information on whether the data collection systems were successfully integrated, and the benefits of the software testing and release process.

This evaluation was not conducted because the operation was not necessary in Census 2000.

This evaluation will document how frequently census forms were linked during the Decennial Response File processing and the types of linkages that were constructed. It will also assess the accuracy of the automated process for setting the expected household size and its effects on the census population.

The objective of this evaluation is to document the effects of using the Primary Selection Algorithm in resolving situations when multiple household questionnaires were received for the same address.

The objective of this evaluation is to determine the accuracy of Primary Selection Algorithm rules for determining the Census Day residents for an address. Comparisons were made between final Census 2000 data and data that were collected using a reinterview of a sample of addresses where the Primary Selection Algorithm was applied.

This evaluation documents the results of the process of determining the final housing unit inventory. The final housing unit inventory for the census was determined during the process of creating the Census Unedited File. The final housing unit inventory was created by merging information on the processed Decennial Response File with the information on the Decennial Master Address File.

This evaluation will answer questions about how well the Beta Site integrated the software systems supporting Census 2000 and its overall utility for software testing and release.

Census 2000 involved more than 20 major field operations and, at its peak, more than 500,000 temporary workers. Managing the quality of the deliverables produced by this large, decentralized, and transient workforce was a major challenge for the Census Bureau. The quality assurance (QA) programs were designed to minimize significant performance errors, to prevent the clustering of significant performance errors, and to promote continuous improvement.

The first evaluation will determine the effectiveness of the QA programs used in the address list development and enumeration operations and will determine if different QA approaches should be explored for the next census. For the second study, the effectiveness of variables that were used to detect discrepancies will be measured, and appropriate variables will be added and/or deleted from the detection process.

The study will determine the effectiveness of the quality assurance philosophy and activities used to manage the quality of the deliverables produced in the address list development and enumeration operations. This study will document operational experiences with this approach, measure quality levels achieved, and determine if other approaches should be explored for the 2010 Census.

The reinterview program was a quality assurance measure whose major objective was to detect enumerators whose work indicated discrepancies. This evaluation examines variables used in this model to determine if they were effective in detecting discrepancies; whether other variables should be added to the model; and to provide suggestions on other ways to improve this program.

The Census Bureau conducted the Accuracy and Coverage Evaluation (A.C.E.), a nationwide sample survey, to determine the number of people and housing units missed or incorrectly counted in the census. The basic approach was to independently relist a sample of blocks, re-enumerate them during the A.C.E. survey, and then compare the results to the census data for the same blocks. The Census Bureau may use the results of the A.C.E. to correct the census counts obtained through the preceding enumeration procedures.

The studies in this category will measure how well the Census Bureau carried out different components of the A.C.E. For instance, analysis projects and evaluations will be conducted that measure the completeness of the housing unit lists used for A.C.E. interviewing, the quality of the A.C.E. person interviewing process, and the accuracy of the procedures used to match persons counted during the A.C.E. interview to those that were enumerated in the census. The success of each A.C.E. component affects the quality of the final estimates.

The results of these A.C.E. analysis projects and evaluations will help the Census Bureau to document this coverage measurement operation and improve its procedures. For example, we will determine how well we detect discrepant results, while also looking at their effect on the A.C.E.

These operational analyses and evaluations will document the A.C.E. process and give the Census Bureau greater insight into what causes error in the measurement of coverage error. Moreover, matching errors may add to errors in the estimates of census coverage. One evaluation in this category will examine a subsample of rematched A.C.E. blocks to measure matching errors. We also will measure the effect of matching error on Dual System Estimates and undercount rates.

The evaluations in this category will help the Census Bureau to identify operational causes of error in measuring coverage and will help to minimize them when planning future censuses.

Many evaluations in this category that were planned for the A.C.E. were no longer need when the decision was made not to adjust the Census 2000 population counts. It was determined that some of the reports that were developed in an expedited manner to inform the Executive Steering Committee for A.C.E. (ESCAP) decisions were sufficiently complete and informative to answer research questions from the planned evaluation reports. Five of these reports and their corresponding ESCAP reports are noted in the following section of evaluation report descriptions. To review the ESCAP reports on the Internet, go to www.census.gov and search on "ESCAP."

This evaluation examines whether census and A.C.E. operations were kept operationally independent (a key requirement for avoiding bias in the dual-system estimates of coverage error) by comparing census results in A.C.E. and non-A.C.E. clusters and through debriefing of field staff.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation ( ESCAP ) analysis and documentation is relevant to this evaluation. Refer to the ESCAP report, "Evaluation Results for Movers and Nonresidents in the Census 2000 Accuracy and Coverage Evaluation" (report B-16).

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation is relevant to this evaluation. Refer to the ESCAP I report, A.C.E. Person Interviewing Results ( report B-5). Additionally, "Automating the Census 2000 A.C.E. Field Operations" and "Results of Quality Assurance on the Person Interview Operation of the A.C.E. of Census 2000" provide preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of these papers, contact the Census Bureau on (301) 457-4218.

This evaluation examines how well the quality assurance process identified interviewers who entered discrepant data in the A.C.E. interview and the impact of undetected discrepant data on A.C.E. estimates.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation is relevant to this evaluation. Refer to the ESCAP I report, "A.C.E. Person Matching and Followup Results" (report B-6). Additionally, "Results of Quality Assurance on the A.C.E. Matching Operations," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301) 457-4218.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation is relevant to this evaluation. Refer to the ESCAP I report, "A.C.E. Variance Estimates by Size of Geographic Area" (report B-11).

A potential source of error in the coverage estimates is the matching operation used to classify persons as missed or erroneously enumerated in the census. This evaluation will determine the relative error associated with the matching operation and how matching error affects the Dual System Estimates.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau. Significant aspects of this study will be addressed in evaluation N.17.

In 1990, the search area for matching was extended to surrounding blocks for all clusters. In 2000, this was only done for clusters deemed most likely to benefit from this additional searching. This report will document overall targeted extended search results and identify characteristics that may be related to matches and correct enumerations found in surrounding blocks due to geocoding error.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Effect of Excluding Reinstated Census People from the A.C.E Person Process."

This analysis provides an overall assessment of the quality of housing unit and person coverage in A.C.E. operations. Some of the topics addressed in the analysis are quality of A.C.E. listing, effect of housing unit followup interviewing on the enhanced list, effectiveness of housing unit and person followup quality assurance, and noninterview rates.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

The studies in this category include a group evaluating A.C.E. coverage and a group evaluating census coverage. These studies will identify person and housing unit characteristics that are related to being missed or erroneously enumerated. Analysis in this area will also study the quality of data from proxy respondents, and the frequency and patterns of geocoding error. Furthermore, census counts and dual system estimates will be compared to demographic benchmarks to evaluate accuracy and completeness.

Results from these evaluations will allow us to determine how complete our Master Address File was for Census 2000. Net coverage rates of housing units will be computed at the national and subnational levels along with gross omission and erroneous enumeration rates. Other studies will explain factors that contribute to housing unit coverage error. For example, we will learn whether type of address (city style versus noncity style) had an effect on housing unit coverage. In addition, there will be a study of housing unit duplication, to identify characteristics of duplicate units and their operational source.

Similarly, we will identify factors that contribute to person coverage error. We will acquire knowledge about erroneous enumerations by determining which demographic, housing unit type, and type of enumeration variables were associated with them. Furthermore, we will conduct an analysis of measurement error, which will help us determine why people were erroneously listed in the census and the Accuracy and Coverage Evaluation.

Many evaluations in this category that were planned for the A.C.E. were no longer need when the decision was made not to adjust the Census 2000 population counts. It was determined that some of the reports that were developed in an expedited manner to inform the Executive Steering Committee for A.C.E. (ESCAP) decisions were sufficiently complete and informative to answer research questions from the planned evaluation reports. Ten of these reports and their corresponding ESCAP reports are noted in the following section of evaluation report descriptions. To review the ESCAP reports on the Internet, go to www.census.gov and search on "ESCAP."

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation assesses 1) the net coverage rate of housing units, 2) the gross omission rate of housing units, and 3) the erroneous enumeration rate of housing units. These assessments are made at the national level, smaller geographic levels, and for each post-strata. This evaluation also examines the potential impact on housing unit coverage had we excluded specific Master Address File building operations. This study is similar to the Housing Unit Coverage Study conducted in 1990. It also has a corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation. Refer to the ESCAP II report, "Census 2000 Housing Unit Coverage Study" (report 17).

During A.C.E. housing unit matching, situations were found where the census and A.C.E. listed two entirely different families. This study will document the follow-up interviewing results for these households to determine if the census was in error, the A.C.E. was in error, if the two families both live at the address, if there was misdelivery of the census form, and so on.

Both the census and A.C.E. sometimes must collect data from proxy respondents--persons who are not members of the household where data are needed. This study will examine match rates and erroneous enumeration rates for such cases in the A.C.E.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "P-sample Nonmatch Analysis" (report 18). Additionally, "Consistency of Census 2000 Post Stratification Variables," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301)457-4218.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II reports, "E-sample Erroneous Enumeration Analysis" (report 5) and "Analysis of Nonmatches and Erroneous Enumerations Using Logistic Regression" (report 12).

Duplication in the census was one type of erroneous enumeration. This analysis will identify duplicate housing units in Census 2000 and their characteristics. The study will also determine if duplication was more likely for one group or another (e.g., owners vs. renters). The census operations most likely to produce housing unit duplication will be identified, along with the most plausible sources of duplication.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation that is relevant to this evaluation. Refer to the ESCAP II report, "E-Sample Erroneous Enumeration Analysis" (report 5). Additionally, "Census 2000 E-Sample Erroneous Enumerations," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301) 457-4218.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Logistic Regression" (report 19). Additionally, "Modeling A.C.E. Non-matches in the Census 2000," provides preliminary research findings that were presented at the 2001 American Statistical Association Conference. For a copy of the paper, contact the Census Bureau on (301) 457-4218.

This study combines the housing unit data and the person data to study coverage. Anew link between A.C.E. housing units and census housing units in the sample will be created in the combined data based on the person matching result. Then the combined data will be used to examine whether coverages are affected by variables such as address style, income, education, property value or rent, type of family, and household complexity.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II reports, "Evaluation Results for Changes in A.C.E. Enumeration Status" (report 3) and "Followup Review" (report 24).

This evaluation will not be conducted because the available data cannot answer the specified questions for this study.

People were duplicated in the census for many different reasons. This analysis will identify the number and characteristics of duplicate persons in Census 2000. The study will also determine if duplication was more likely for one group or another (e.g., owners/renters). The census operations most likely to cause duplication will be identified, along with the most plausible sources of the duplication.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

The goal of this study is to assess the completeness of housing unit coverage on the early Decennial Master Address File (DMAF). We will determine which census operations contributed to undercoverage by deleting units that should have not been deleted, and which operations improved coverage by adding units not previously accounted for. We also will identify which census operations reduced housing unit duplication.

This study uses independent demographic benchmarks to evaluate the accuracy of the Census 2000 counts and the completeness of coverage in Census 2000. While this approach cannot produce estimates for as many demographic groups and geographic areas as A.C.E., results can be compared to A.C.E. at aggregate levels.

This evaluation will not be conducted. It was cancelled before February 2001.

This evaluation, which was added after February 2001, will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Evaluation of Lack of Balance and Geographic Errors Affecting A.C.E. Person Estimates" (report 2).

This evaluation, which was added after February 2001, will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "E-Sample Erroneous Enumeration Analysis" (report 5).

This evaluation, which was added after February 2001, will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Evaluation of Lack of Balance and Geographic Errors Affecting A.C.E. Person Estimates" (report 2).

This evaluation, which was added after February 2001, will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation II (ESCAP II) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Analysis of Movers" (report 15).

This evaluation, which was added after February 2001, will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation is relevant to this evaluation. Refer to the ESCAP I report, "A.C.E. Data and Analysis to Inform the ESCAP Report" (report B-1).

The evaluations in this category were designed to examine the quality of Accuracy and Coverage Evaluation (A.C.E.) estimates. Because of resource reallocation and Executive Steering Committee for Accuracy and Coverage Evaluation (ESCAP) analyses and documentation that informs evaluations, the evaluations for this category were not conducted. Refer to specific evaluations for more information.

Some of the reports that were developed in an expedited manner to inform ESCAP decisions were sufficiently complete and informative to answer research questions from the planned evaluation reports. Two of these reports and their corresponding ESCAP reports are noted in the following section of evaluation report descriptions. To review the ESCAP reports on the Internet, go to www.census.gov and search on "ESCAP."

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation (ESCAP) analysis and documentation is relevant to this evaluation. Refer to the ESCAP II report, "Analysis of Missing Data Alternatives for the A.C.E" (report 12).

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

This evaluation will not be conducted. A corresponding Executive Steering Committee for Accuracy and Coverage Evaluation I (ESCAP I) analysis and documentation that is relevant to this evaluation. Refer to the ESCAP I report, "A.C.E.: Variance Estimates by Size of Geographic Area" ( report B-11).

This evaluation will not be conducted. A summary report of various quality measures was previously planned. However, this information will be included in a topic report assessing key findings for all A.C.E. evaluations.

This evaluation will not be conducted. In early 2002, the Census 2000 Evaluation Program was refined and priorities reassessed due to resource constraints at the Census Bureau.

Research in this category will document headquarters decision making processes and the impact of headquarters organizational structure on the decennial census. We plan to study the management approach, structure, processes, and management tools.

The findings from this study will help the Census Bureau to better manage future censuses and similar projects. This study will document how well the Management Information System worked in helping us manage Census 2000. We will compare the activities and recommendations of the Census 2000 research and development program to what was actually implemented for Census 2000 to determine which projects were most beneficial. In addition, we will examine the roles and influences of both external and internal entities on planning and implementing the census.

The purpose of this study is to evaluate the management model for Census 2000 including the organizational structure, the decision making process, and the management information tools. The study will also assess the staffing, the use and management of contracts, and the impact of external influences such as the Census Monitoring Board, the Congress, the funding history, the General Accounting Office, and other stakeholders.

For Census 2000, the Census Bureau implemented a series of automated systems to aid the conduct of the decennial census. These systems included, but were not limited to, data collection and capture, cost and progress reporting, management controls, customer reaction, quality assurance and analysis, and the Internet. Many of these systems were implemented for the first time in Census 2000. There are a total of twelve systems that we will evaluate. In general, we will assess whether the correct requirements and proper functionality were specified for each of these twelve systems, whether the systems performed adequately in terms of either impact on data quality or in providing useful management information, and whether we specified our requirements in a timely manner. We also will examine any contract management issues, as applicable. The twelve systems to be evaluated are as follows: Telephone Questionnaire Assistance; Coverage Edit Followup; Internet Questionnaire Assistance; Internet Data Collection; Operations Control System 2000; Laptop Computers for the Accuracy and Coverage Evaluation; Accuracy and Coverage Evaluation 2000 Control System; Matching and Review Coding System for the Accuracy and Coverage Evaluation; Pre-Appointment Management System/Automated Decennial Administrative Management System; American FactFinder; Management Information System 2000; and Census 2000 Data Capture.

Evaluation reports will be generated using information collected from debriefings with program managers, systems users, and others affiliated with the systems. Questionnaires will be developed for each system that will address general issues concerning the system's functionality and the correct and timely specification of its requirements along with questions that are unique to each system. We expect to gain insight, as appropriate, in areas such as maintenance and security needs, respondent acceptance, initial investment required, ease/difficulty of setup, reliability, level of training required, effects on coverage and response rates, additional costs or savings, and technology life cycle issues.

A list of key questions to be answered for each system follows.

• Did we specify the right requirements and functionality?
• Did the system do what we needed it to in terms of either its impact on data quality or in providing useful management information?
• Did we define our requirements in a timely manner?
• If a contractor was hired to work on the system, did the contractor effectively complete the required tasks?

Telephone questionnaire assistance (TQA) was a toll-free service provided by a commercial phone center to answer questions about Census 2000 or the census questionnaire. This system also included a reverse-CATI (computer-assisted telephone interview) operation. For Census 2000, TQA was operated out of 22 phone centers nationwide from March through June 2000.

Coverage Edit Followup (CEFU) was an outbound service operating out of 13 call centers to resolve count discrepancies (coverage edit failures) and to obtain missing information for large households. The CEFU was conducted from May to August 2000.

Internet questionnaire assistance (IQA) was an operation that allowed respondents to use the Census Bureau's Internet site to ask questions and receive answers about the census questionnaire, job opportunities, or general questions about the purpose of the census. This service was operative from March through June 2000.

From March through April 2000, respondents to the Census 2000 short form had the option of completing their census form electronically by accessing the Census Bureau's Internet site and providing a 22-digit ID number found on their form received in the mail.

The Operations Control System (OCS) 2000 was a decennial field interface system and was used for control, tracking, and progress reporting for all field operations conducted for the census, including production of materials used by field staff to do their work. This system was operative from October 1997 through October 2000 for the pre-census and decennial operational phases.

The Accuracy and Coverage Evaluation was a coverage measurement methodology that was used to determine the number of people and housing units missed or counted more than once in Census 2000. The laptop computers were used to conduct personal and telephone interviews. This evaluation examines and assesses the use of laptop computers in determining coverage error.

The Accuracy and Coverage Evaluation 2000 Control System was a decennial system to aid management in tracking and controlling the Accuracy and Coverage Evaluation field operations.

The Matching and Review Coding System for the Accuracy and Coverage Evaluation was also referred to as the Accuracy and Coverage Evaluation survey matching system. The system provided for a computer matching of housing units and persons followed by a clerical review of unmatched records. This system was used at the Census Bureau's National Processing Center in Jeffersonville, IN.

The Pre-Appointment Management System/Automated Decennial Administrative Management System was an integrated structure of administrative management programs that support applicant tracking and processing, background checks, selection records, recruiting reports, personnel and payroll processing, and archiving of historical data. This system was used in the hiring of temporary workers for the census.

The American FactFinder is a generalized electronic system for access and dissemination of Census Bureau data. The system is available through the Internet and offers prepackaged data products and the ability to build custom products. The system will serve as the vehicle for accessing and disseminating data from Census 2000 (as well as the 1997 Economic Censuses and the American Community Survey). The system was formerly known as the Data Access and Dissemination System. Census 2000 data products will be available through the American FactFinder began in January 2001.

The Management Information System (MIS) provides decision support functions, such as, critical-path analysis and what-if analysis. It also provides information on dates, the responsible organization, budget, cost to date, and current progress of Census 2000 operations. The MIS includes the master activity schedule, the executive information system, and the cost and progress system. Designed as a tool for Census 2000, the MIS has an ongoing function.

The data capture process is a full electronic data capture and processing system for imaging Census 2000 questionnaires. This process involves: 1) the check-in of paper forms; 2) the scanning and imaging of those forms; and 3) the use of optical mark and optical character recognitions to capture data from census questionnaire images that convert it to a computer readable format. The Census Bureau worked with private sector companies to operate four data capture centers that were operative from March through October 2000.