Lower response rates are not necessarily problematic if respondents and nonrespondents are roughly similar, such that the resulting sample of respondents remains representative of the nation. To investigate this possibility, we combined administrative records, 2010 Census data, and American Community Survey responses to determine whether households that did and did not respond to the CPS ASEC differed, and whether these differences ultimately biased the resulting income and poverty estimates. Figure 2 plots earnings differences between respondents and nonrespondents using IRS Form W-2 earnings records linked by address. It shows that in 2020, respondents began to have higher earnings than nonrespondents, whereas in earlier years — 2017 to 2019 — the gap was narrower. That phenomenon has repeated each year since 2020. Specifically, differences in median earnings between respondents and nonrespondents were statistically significant in each year, but each of the years from 2020 through 2023 had respondent-nonrespondent differences at the median that were significantly larger than the differences in each of the years 2017 to 2019, with the exception that differences in 2017 and 2022 were not significantly different from each other.²

Still, the standard nonresponse adjustments could potentially have controlled for these unconditional earnings differences. In the usual course of survey data processing, the Census Bureau creates weights designed to adjust for nonresponse and to target weighted counts to independent population estimates by age, sex, race, and Hispanic origin. To check whether these standard adjustments were sufficient to make respondents representative of the whole population, we tested whether survey response continued to vary by W-2 earnings after controlling for all the factors in the weighting adjustment, as well as some other factors observed in the linked data. Figure 3 shows that earnings have indeed remained correlated with survey response since 2020, even after controlling for these other characteristics.³

The final step is to measure whether — and by how much — these differences between respondents and nonrespondents, even after adjusting for other characteristics, ultimately bias the resulting income statistics. As we have each year since the 2020 CPS ASEC, we measure this bias by estimating an alternative nonresponse adjustment that leverages information from linked data sources that are not available in standard survey processing. This information includes linked administrative, decennial census, and survey data available for respondent and nonrespondent households. Since low-income households have been less likely to respond, this adjustment shifts weight away from higher-income respondent households toward lower-income respondent households.

Using alternative weights created from the alternative nonresponse adjustment, we calculate alternative income and poverty statistics and compare them to the official (survey-only) estimates. We find that estimates of median household income and poverty rates were not statistically different in 2017, 2018, or 2019. This indicates that in pre-pandemic years the normal nonresponse adjustment in the survey weights sufficiently addressed nonresponse bias for these statistics.

However, Figure 4 shows that since 2020, the alternative income estimates have diverged from those estimated using the CPS ASEC survey data alone. Figure 4 plots the ratios of these estimates across the household income distribution. The horizontal line at one indicates the ratio at which the alternative-weight estimate would equal the survey-only estimate at that point in the distribution. Before 2020 these ratios hovered near one, but since 2020 the alternative income estimates have been about 2% to 3% lower than the survey-only estimates.