Small Area Income and Poverty Estimates

General Cautions About Comparing Estimates

Cautions about comparing direct survey estimates obtained from different surveys

Differences between direct survey estimates obtained from different sources (Census 2000, CPS ASEC, ACS) reflect both differences in the true levels of income and poverty (when comparing estimates for different places or different years), as well as differences in the methods by which the survey data were collected and the estimates were made. The following papers describe these differences between the data sources.

• Census 2000 and CPS
  Comparing Employment, Income, and Poverty: Census 2000 and the Current Population Survey [PDF - 4.4M]
• Census and ACS
  Report 5: Comparing Economic Characteristics With Census 2000. Meeting 21st Century Demographic Data Needs-Implementing the American Community Survey [PDF - 4.4M]
  Income in the American Community Survey: Comparisons to Census 2000 [PDF - 374k]
• CPS and ACS
  Evaluation of Poverty Estimates: A Comparison of the American Community Survey and the Current Population Survey [PDF - 74k]
  What Do We Know About Differences Between CPS and ACS Income and Poverty Estimates? [PDF - 713k]
  Evaluation of Median Income and Earnings Estimates: A Comparison of the American Community Survey and the Current Population Survey [DOC - 175k]

Cautions about comparing SAIPE intercensal model-based county estimates and direct survey estimates

Census 2000 and ACS direct survey estimates are used in construction of the SAIPE model-based estimates of poverty and median household income. Therefore, the intercensal model-based estimates almost certainly exhibit a positive correlation with Census 2000 and ACS direct survey estimates.

When testing the significance of differences in estimates from these sources, caution should be used in deriving the confidence intervals. Although we currently do not have estimates of the individual correlations or advice on estimating the general magnitude of the correlation, failure to incorporate the positive correlation between the estimates will produce confidence intervals that are too wide; thus, too many differences will be considered "not significant." This caution does not apply to comparisons between Census 2000 state-level data because the Census 2000 estimates have close to negligible sampling error.

Cautions about comparing SAIPE model-based estimates for different years

Statistical comparisons of SAIPE state and county estimates across years are possible; however, certain cross-year correlations must be taken into account. For all years of estimates, there is correlation of the model error, and for the years prior to 2005 there is correlation of the sampling error as well. Methodology for Testing for a Rise in Child Poverty Rate [PDF - 141k] describes a methodology for taking into account these cross-year correlations in estimating if any states have a five percent or greater significant change in child poverty rate between two years. A similar methodology for counties, as well as the modifications required for comparing year-to-year changes starting in 2005, is currently undergoing research.

Of note, there is a break in the SAIPE time-series between 2004 and 2005 due to the switch from CPS ASEC to ACS data in SAIPE modeling. Comparisons across these particular years are not advised because of this break in series See Estimation Procedure Changes for the 2005 Estimates for more details. Also, with the introduction of group quarters populations to the ACS starting with the 2006 ACS, comparability for certain age groups across 2005 and 2006 is limited. Generally residents of group quarters have higher poverty rates than residents of households, and this affects the comparison. See http://www.census.gov/acs/www/UseData/Comparison_Guidance2006.htm - povpeople for more details.

Cautions about comparing SAIPE model-based estimates for different states or different counties in the same year

All SAIPE model-based estimates are correlated because they depend on the same regression coefficients. Therefore, to make comparisons between states or between counties, it is not sufficient to take the variances (implied by the confidence intervals) for the two different places and apply the usual means difference hypothesis test. For counties, there is the additional consideration that the sampling and model errors are probably correlated across counties within a state. Methods for making such state-to-state or county-to-county statistical comparisons are currently being researched.