This report is released to inform interested parties of ongoing research and to encourage discussion of work in progress.  The views expressed on the statistical and methodological  issues in this report are those of the authors and not necessarily those of the U.S. Census Bureau.

This report compares national and state employment status estimates based on data from the 2007, 2008, and 2009 American Community Survey (ACS) with those based on data from the 2007, 2008, and 2009 Current Population Survey (CPS).  The report begins with a description of survey methodological differences between the ACS and CPS.  Next, comparisons of labor force distributions, employment-to-population ratios, and unemployment rates are presented.  We then identify variations between the two surveys that are both statistically and substantively different, and for those found, offer possible explanations.

The tables included in this report compare employment status data from the ACS and CPS.  Comparisons consist primarily of percentage-point differences between the two surveys.  Tables display the survey estimates, the margins of error from which 90 percent confidence intervals of the estimates can be derived, and the percentage-point difference between the two survey estimates.  An asterisk (*) denotes statistically significant differences.  It is important to note that while both surveys collect data on a monthly basis, ACS data is reported annually while CPS data is reported monthly.  To address this disparity, CPS estimates presented in this report are based on an average of the monthly employment status data across twelve months (January through December) of each data year.

Sample Frame

The ACS surveys a national sample of housing units, both occupied and vacant, and institutionalized and noninstitutionalized group quarters (GQ).  The sample is designed to provide single-year estimates of housing and socio-economic characteristics for the nation, all states, and areas with a population of 65,000 or more. ^[1]

The CPS surveys a national sample of housing units and noninstitutionalized group quarters from roughly 824 sample areas. ^[2] The sample is designed primarily to produce estimates of the labor force characteristics of the civilian noninstitutionalized population 16 years of age and older for the nation and all states.

One notable difference between the ACS and CPS survey universes relates to the GQ population.  The ACS includes both institutionalized (e.g., prisons, residential treatment centers, and nursing facilities) and noninstitutionalized (e.g., college/university housing and military quarters) GQ in its survey universe, while the CPS includes only noninstitutionalized GQ.  This survey sample difference affects total population counts and, likewise, employment/population ratio comparisons.  However, it does not impact employment and unemployment estimates since those in the institutionalized GQ population are not considered part of the labor force.  For comparison purposes in this paper, all individuals living in institutionalized group quarters were excluded from the ACS estimates.

Sample Size and Mode of Data Collection

ACS data were collected continuously throughout the year using a combination of mail-out/mail-back questionnaires, Computer-Assisted Telephone Interviewing (CATI), and Computer-Assisted Personal Interviewing (CAPI).  Each month a unique national sample of addresses received an ACS questionnaire.  Addresses that did not respond were telephoned during the second month of collection if a phone number for the address was available.  For a subsample of the remaining non-responding units, personal visits were conducted during the third and last month of data collection.  The sample size in ACS was, on average, roughly 1.9 million per year from 2007 through 2009.^[3]; The response rate averaged 97.9 percent across these three years. ^[4]

For the CPS, each month roughly 60,000 occupied units were eligible for interview. ^[5] All CPS data are collected via Computer-Assisted Telephone and Personal Interviews (CATI/CAPI), with interviews conducted during the calendar week that includes the 19^th of the month. ^[6] CPS had an average response rate of 92.2 percent between 2007 and 2009. Both the ACS and CPS employ experienced, permanent interviewers for CATI and CAPI data collection.^[7]

Residence Rules

The ACS and CPS utilize different residence rules to determine which individuals in a household are eligible for interview.  This distinction may contribute to variation in the universes from which labor force characteristics are measured.

The ACS collects interviews from every person in the housing unit on the day of interview that is living or staying there for more than two months, regardless of whether or not they maintain a usual residence elsewhere.  If a person who usually lives in the sample housing unit is away for more than two months at the time of the survey contact, he or she is not considered to be a current resident of that unit.  For GQ, a person is considered a resident if he or she is a resident of the GQ unit at the time of the interview. 

The CPS interviews all persons staying in the housing unit at the time of the interview who consider the housing unit to be their usual residence or who have no usual residence.  The CPS also includes temporarily absent individuals who consider the housing unit as their usual residence.

The most notable impact of the difference between ACS and CPS residence rules relates to GQ, with the clearest example being college dormitories.  As a result of the residence rules noted above, ACS interviewers consider a college student living in a dormitory at the time of interview as a resident of that particular dormitory/GQ unit.  In contrast, CPS interviewers are instructed to include as household members any college student who was temporarily absent from the household, including those who were currently residing in college dormitories.  At the national level this difference has no consequence because the population of college students living in dormitories is captured in both surveys, albeit in different ways.  However, at lower level geographies (i.e., at the state level) employment status estimates may vary as a result. 

Reference Periods

The ACS and CPS differ in their reference period.  The CPS questions refer to the week containing the 12^th day of each month, while the ACS questions ask about the full calendar week prior to the interview.  However, the difference in reference periods would not have impacted relative estimates unless employment or unemployment consistently rose and fell in monthly cycles.  For example, we assume that employment status would, on average, not be drastically different during the week of the 12^th of any given month compared to other weeks within that same month.

In the CPS, households are in the sample for four consecutive months, out of the sample for the following eight months, and then in the sample again for four months.  There is a tendency among households surveyed for the first time to report higher levels of unemployment than those who have been in the survey for several consecutive months. ^[8] In ACS, virtually every household is reporting for the first time.  Thus, any bias in unemployment estimates associated with first time respondents could potentially affect the entire ACS, but only a portion of the CPS. 

Question Wording

Differences between the ACS and CPS in presentation and wording of questions may contribute to differences in estimates. ^[9] A complete comparison of employment status questions for both the ACS and CPS is found in Appendix A of this report. 

The employment-classification concept used in both the CPS and the ACS is defined operationally in terms of a set of criteria for deciding which of the three categories – employed, unemployed, or not in labor force – best characterizes the respondent’s relationship to the labor market during a particular week.  Since, even within a week, this relationship can vary, the criteria assign priorities among categories so that each respondent is classified in one and only one category.  To apply these criteria, both the CPS and the ACS obtain employment information from a battery of questions.  Each question obtains a piece of information required by one of the criteria; the role, if any, of a given piece of information in the final classification decision depends upon the other pieces of evidence collected.

While the ACS is a general purpose demographic survey intended to collect data to produce estimates on a variety of economic, social, and housing characteristics, the CPS has the primary purpose of generating the official household-based estimates of employment and unemployment for the United States.  The CPS currently uses a more detailed battery of questions to determine a respondent’s employment status compared to the ACS (please see the Appendix for additional detail).

The ACS underwent a question change in 2008 in order to improve the measurement of employment status by addressing several limitations suggested by previous research, specifically the underestimating of employment levels and the overestimating of unemployment levels, relative to CPS and Local Area Unemployment Statistics (LAUS) benchmarks. ^[10]

One wording issue between the ACS and CPS concerns the “worked last week” question. In 2007, the CPS included four questions related to worked last week while the ACS included only one:

CPS

     a. LAST WEEK, did you do ANY work for (pay/either pay or profit?)
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

     b. LAST WEEK, did you do any unpaid work in the family business or farm?
         — Yes
         — No

     c. LAST WEEK, did you have more than one job/job or business, including part time?
         — Yes
         — No

     d. Altogether, how many jobs/jobs or businesses did you have?
         — 2 jobs
         — 3 jobs
         — 4 or more jobs

2007 ACS

     LAST WEEK, did this person do ANY work for either pay or profit?

Mark (X) the "Yes" box even if the person worked only 1 hour, or helped without pay in a family business or farm for 15 hours or more, or was on active duty in the Armed Forces.
         — Yes
         — No

This difference likely contributed to the ability of the CPS to capture more employed persons, particularly those with irregular work arrangements.  To address this problem, the 2008 ACS question change split the worked last week question into two parts to further clarify the question:

2008 and 2009 ACS

     a. LAST WEEK, did this person work for pay at a job (or business)?
         — Yes
         — No

     b. LAST WEEK, did this person do ANY work for pay, even for as little as one hour?
         — Yes
         — No

Another labor force wording difference between the ACS and CPS concerns the “looking for work” question. In 2007, the CPS included two questions related to looking for work while the ACS included only one:

CPS

     a. Have you been doing anything to find work during the last 4 weeks?
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

     b. What are all of the things you have done to find work during the last 4 weeks?
         — Contacted employer directly/interview
         — Contacted public employment agency programs/courses
         — Contacted private employment agency
         — Contacted friends or relatives
         — Contacted school/university employment center
         — Sent out resumes/filled out applications
         — Checked union/professional registers
         — Placed or answered ads
         — Other active
         — Looked at ads
         — Attended job training
         — Nothing
         — Other passive

2007 ACS

     Has this person been looking for work during the last 4 weeks?
         — Yes
         — No

Due to this difference, unlike the CPS, the ACS could not readily distinguish between a respondent who is actively looking for work (and, thus, potentially unemployed) and someone who is passively looking for work (and potentially out of the labor force). ^[11]

One of the 2008 ACS question changes attempted to address the issue of distinguishing actively versus passively looking for work.  The 2007 ACS “looking for work” question was changed to the following:

2008 and 2009 ACS

     Has this person been actively looking for work during the last 4 weeks?
         — Yes
         — No

Item Nonresponse

Item nonresponse occurs when an individual does not provide complete and usable information for a data item.  Item allocation rates are often used as a measure of the level of item nonresponse.  Allocation rates are computed as the ratio of the number of eligible people for which a value was allocated during the editing process for a specific item to the number of people eligible to have responded to that item.  For the ACS, the average allocation rate for employment status data between 2007 and 2009 was 4.1 percent. ^[12] This compares to an average allocation rate of 2.2 percent for labor force data in the CPS between 2007 and 2009. ^[13]

A difference in data collection procedures could account for some of the differences in nonresponse rates.  As noted previously in this report, the CPS survey is conducted entirely by CATI and CAPI, while ACS data are collected through a combination of mail-out/mail-back questionnaires, CATI, and CAPI.  Because mail-out/mail-back questionnaires may be more susceptible to item nonresponse than interviewer-conducted questionnaires (i.e., CATI and CAPI), the ACS has a higher risk than the CPS of having a significant proportion of cases with completely or partially missing responses to employment questions.

The ACS and CPS edit and imputation rules are designed to ensure that the final edited data are as consistent and complete as possible.  In each case where a problem is detected, consistent, pre-established edit rules govern its resolution.  Regardless, an imputation represents an educated opinion, which may be valid on average, but wrong in any particular case.  The fact that the ACS employment-status data contain slightly higher imputation rates compared to the CPS could be a factor in producing differences between estimates.

Controls and Weighting

There are a number of differences in the selection of controls and the calculation of weights between the two surveys, which may lead to differences in estimates.  The ACS and the CPS are both weighted to account for the probability of selection and for housing-unit nonresponse.  After the initial weighting, data from the ACS and the CPS are both controlled to be consistent with independent population estimates.  Data from the ACS are controlled, at the county level, to independent estimates of the household population and housing units using controls from July 1^st of each year.  Comparatively, data from a particular month in the CPS are controlled to independent national estimates of the civilian noninstitutionalized population for the first day of that month.

Tables 1, 2, and 3 present a summary of 2007, 2008, and 2009 employment status data from the ACS and CPS, including the total working-age population, the civilian labor force, the number of employed and unemployed persons, the unemployment rate, and the number of people not in the labor force. Tables 4, 5, and 6 provide unemployment rate comparisons across the 50 states and the District of Columbia. All comparisons made in this report have undergone statistical testing and are significant at the 90 percent confidence level unless otherwise noted.

The Census Bureau currently controls population estimates based on a methodology that modifies the Census 2000 base population. ^[14] Given this population control, the ACS and CPS estimates of the civilian noninstitutionalized population 16 years and over should be identical for each year.  The slight discrepancy between the two estimates is likely due to the fact that the CPS estimates in this report are based on an average of the monthly reported employment status data across 12 months. 

The ACS estimate of civilian noninstitutionalized employment was approximately 3.5 million lower than the CPS estimate in 2007.  In 2008, after the question changes, the ACS civilian noninstitutionalized employment count was higher than the CPS by roughly 665,000 persons.  The 2009 estimates followed a similar pattern, with the ACS civilian noninstitutionalized employment count being roughly 725,000 higher than the CPS estimate.

The difference between the ACS and CPS employment/population ratios, similar to employment counts, changed after the 2008 ACS question changes. ^[15] In 2007, the ACS employment/population ratio was 61.6 percent, approximately 1.4 percentage points lower than the CPS estimate of 63.0 percent.  In both 2008 and 2009, the ACS employment/population ratio was 0.4 percentage points higher than the CPS estimate.

Consistent with the employment comparisons discussed above, ACS unemployment rate estimates tended to be significantly higher than CPS estimates in 2007.  The ACS unemployment rate estimate of 6.3 percent in 2007 was 1.7 percentage points higher than the CPS estimate of 4.6 percent.  The unemployment rate gap between the two surveys narrowed considerably after the ACS question changes.  In both 2008 and 2009, the ACS unemployment rate estimate was 0.6 percentage points higher than the CPS estimate. 

Table 4 shows that, similar to the nation as a whole, ACS unemployment rate estimates for the 50 states and the District of Columbia were higher than CPS estimates in 2007.  The majority of states (47 out of 50 states) and the District of Columbia reported statistically different unemployment rate estimates in the ACS than in the CPS in 2007.  As with the national estimates, the ACS question changes improved consistency between the two surveys across states.  In 2008 (see Table 5), only 24 out of 50 states reported statistically different unemployment rates in the ACS than in the CPS.  State level unemployment rates in 2009 saw a comparable pattern (see Table 6), with 21 states reporting statistically higher unemployment rates in the ACS versus the CPS.

Although the ACS and CPS produce similar estimates for the civilian noninstitutionalized population 16 years and over, employment status estimates are different between the two surveys.  In 2007, the ACS significantly underestimated employment and overestimated unemployment compared with the CPS.  Question changes implemented in the 2008 ACS helped improve consistency between the two surveys in regard to both employment and unemployment estimates.  Forty-seven states and the District of Columbia had statistically different unemployment rates between ACS and CPS in 2007.  After the question change, the number of states with statistically different unemployment rates between the two surveys dropped to 24 in 2008 and 21 in 2009.

Differences in data-collection methods such as a comparative absence of specificity in some ACS questions may explain several of the discrepancies in estimates.  Measurement errors such as those associated with first-time and subsequent-time reporting in the CPS may explain most of the remaining differences.  Also, because the CPS is collected and weighted to generate monthly estimates and the ACS to generate annual estimates, the methodology employed to generate comparable annual estimates may influence relative values.

Topic 1: Work Last Week

2007 ACS:

     LAST WEEK, did this person do ANY work for either pay or profit?

Mark (X) the "Yes" box even if the person worked only 1 hour, or helped without pay in a family business or farm for 15 hours or more, or was on active duty in the Armed Forces.
         — Yes
         — No

2008 and 2009 ACS:

     a. LAST WEEK, did this person work for pay at a job (or business)
         — Yes
         — No

     b. LAST WEEK, did this person do ANY work for pay, even for as little as one hour?
         — Yes
         — No

CPS:

     a. LAST WEEK, did you do ANY work for (pay/either pay or profit?)
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

     b. LAST WEEK, did you do any unpaid work in the family business or farm?
         — Yes
         — No

     c. LAST WEEK, did you have more than one job/job or business, including part-time?
         — Yes
         — No

     d. Altogether, how many jobs/jobs or businesses did you have?
         — 2 jobs
         — 3 jobs
         — 4 or more jobs

Topic 2: On Layoff

2007 ACS:

     LAST WEEK, was this person on layoff from a job?
         — Yes
         — No

2008 and 2009 ACS:

     LAST WEEK, was this person on layoff from a job?
         — Yes
         — No

CPS:

     LAST WEEK, were you on layoff from a job?
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

Topic 3: Temporarily Absent

2007 ACS:

     LAST WEEK, was this person TEMPORARILY absent from a job or business?
         — Yes, on vacation, temporary illness, labor dispute, etc.
         — No

2008 and 2009 ACS:

     LAST WEEK, was this person TEMPORARILY absent from a job or business?
         — Yes, on vacation, temporary illness, maternity leave, other family/personal reasons, bad weather, etc.
         — No

CPS:

     a. LAST WEEK, did you have a job either full or part time?
     Include any job from which you were temporarily absent.
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

     b. What was the main reason you were absent from work LAST WEEK?
         — On layoff
         — Slack work/business conditions
         — Waiting for a new job to begin
         — Vacation/personal days
         — Own illness/injury/medical problems
         — Child care problems
         — Other family/personal obligation
         — Maternity/paternity leave
         — Labor dispute
         — Weather affected job
         — School/training
         — Civic/military duty
         — Does not work in the business
         — Other

     c. Are you being paid by your employer for any of the time off last week?
         — Yes
         — No

Topic 4: Recalled from Layoff

2007 ACS:

     Has this person been informed that he or she will be recalled to work within the next 6 months OR been given a date to return to work?
         — Yes
         — No

2008 and 2009 ACS:

     Has this person been informed that he or she will be recalled to work within the next 6 months OR been given a date to return to work?
         — Yes
         — No

CPS:

     a. Has your employer given you a date to return to work?
         — Yes
         — No

     b. Have you been given any indication that you will be recalled to work within the next 6 months?
         — Yes
         — No

Topic 5: Looking for Work

2007 ACS:

     Has this person been looking for work during the last 4 weeks?
         — Yes
         — No

2008 and 2009 ACS:

     During the LAST 4 WEEKS, has this person been ACTIVELY looking for work?
         — Yes
         — No

CPS:

     a. Have you been doing anything to find work during the last 4 weeks?
         — Yes
         — No
         — Retired
         — Disabled
         — Unable to Work

     b. What are all of the things you have done to find work during the last 4 weeks?
         — Contacted employer directly/interview
         — Contacted public employment agency programs/courses
         — Contacted private employment agency
         — Contacted friends or relatives
         — Contacted school/university employment center
         — Sent out resumes/filled out applications
         — Checked union/professional registers
         — Placed or answered ads
         — Other active
         — Looked at ads
         — Attended job training
         — Nothing
         — Other passive

Topic 6: Available for Work

2007 ACS:

     LAST WEEK, could this person have started a job if offered one, or returned to work if recalled?
         — Yes, could have gone to work
         — No, because of own temporary illness
         — No, because of all other reasons (in school, etc.)

2008 and 2009 ACS:

     LAST WEEK, could this person have started a job if offered one, or returned to work if recalled?
         — Yes, could have gone to work
         — No, because of own temporary illness
         — No, because of all other reasons (in school, etc.)

CPS:

     a. LAST WEEK, could you have started a job if one had been offered?
         — Yes
         — No

     b. Why is that?
         — Waiting for new job to begin
         — Own temporary illness
         — Going to school
         — Other

[1]This report compares single-year ACS data to CPS data for one year. ACS data is currently available for smaller geographies in 3-year and 5-year estimates. For more information see //www.census.gov/programs-surveys/acs/guidance/estimates.html

[2] Current Population Survey Technical Notes to Household Survey Data published in Employment and Earnings //www.bls.gov/cps/eetech_methods.pdf.

[3] The ACS sample size in this report does not include group quarters.

[4] The response rate is calculated as the initially weighted estimate of interviews divided by the initially weighted estimate of cases eligible to be interviewed. More information can be found at //www.census.gov/programs-surveys/acs/methodology/sample-size-and-data-quality/response-rates-definitions.html

[5] U.S. Department of Labor, Bureau of Labor Statistics.  Explanatory Notes and Estimates of Error, Employment and Earnings 54.12 (December 2007): pp. 175-214.

[6] U.S. Department of Labor, Bureau of Labor Statistics and U.S. Department of Commerce, U.S. Census Bureau. Design and Methodology, Current Population Survey Technical Paper 66 (October 2006).

[7] Unlike the ACS, participation in the CPS is voluntary.

[8] Comparing Employment, Income, and Poverty: Census 2000 and the Current Population Survey; Appendix I.  //www.census.gov/hhes/www/laborfor/final2_b8_nov6.pdf.

[9] American Community Survey questionnaires for each year can be found at: //www.census.gov/programs-surveys/acs/methodology/questionnaire-archive.html

[10] More information on the impact of the 2008 employment status question change can be found at //www.census.gov/hhes/www/laborfor/researchnote092209.html

[11] Active job search methods include registering at a public or private employment office, meeting with a prospective employer, investigating possibilities for starting a professional practice or opening a business, placing or answering advertisements, writing letters of application, being on a union or professional register, etc. They do not include looking at want ads or attending job training. A U.S. Census Bureau report entitled Comparing Employment, Income and Poverty: Census 2000 and the Current Population Survey analyzed passive searchers in the CPS sample.  The report found that roughly 7.5 percent of all CPS respondents between March and May of 2000 who report they searched for work indicated that they used passive methods.  For further details, see Appendix I in the report: //www.census.gov/hhes/www/laborfor/final2_b8_nov6.pdf.

[12] //www.census.gov/acs/www/methodology/sample-size-and-data-quality/item-allocation-rates/#labor_force

[13] U.S. Census Bureau, Current Population Survey Report for December 2009.

[14] For more information regarding the methodology of United States resident population estimates, please see: //www.census.gov/popest/topics/methodology/2007-nat-meth.pdf.

[15] The employment/population ratio is the percentage of all working-age civilians who are employed.