Survey earnings data are the foremost source of available information on personal and household income in the United States. They provide a timely and complete picture of household earnings profiles to the public, policymakers and researchers that alternative sources of earnings measures (like the Social Security Administration’s Master Earnings File) cannot. Thus, the gradual increase in the share of survey respondents who do not provide their earnings in household surveys – the so-called earnings nonresponse rate – is concerning.

In a new working paper, we show that monetary incentive payments intended to increase household survey response unexpectedly mitigated the rise in earnings nonresponse rates among respondents. The money was only intended to incentivize survey participation and not necessarily encourage completion of specific questions. As the earnings items are among the most sensitive and often difficult-to-answer questions, reducing nonresponse was an unanticipated benefit. In fact, households randomly selected to receive the payments upon survey participation would remain eligible for those payments even if they refused or did not know how to answer earnings questions. Thus, beforehand, it was not clear that inducing more households to participate in a survey would increase the share of survey respondents who provide their earnings.

We further assess the unanticipated benefit of conditional monetary incentives in reducing earnings nonresponse by considering their effect on the probability of future survey dropout (attrition). We show that people who did not provide their earnings in the first round (wave) of a survey, are more likely to cease their participation by the next wave. Therefore, incentives may indirectly lower attrition by lowering earnings nonresponse. We show that incentives may also directly affect attrition, and that this effect increases with the size of the incentive. Yet, receiving payments and their amount had no qualitative effect on attrition of earnings responders.

In the 2014 panel of the Survey of Income and Program Participation (SIPP), survey administrators carried out a randomized, two-wave incentive experiment that divided households into four groups. As shown in Table 1, in Wave 1, Groups 1 and 2 received $0, while Groups 3 and 4 were offered conditional incentives of $20 and $40, respectively. In Wave 2, Group 2 was offered $40, and Group 3 no longer received a monetary incentive. Group 4 was further divided into Groups 4a and 4b receiving no incentive and $40, respectively. Group 1 continued to receive $0 in Wave 2.
 

Note: Obs. and Pct. columns indicate incentive group sample sizes as counts and percentages, respectively. Unweighted counts have been rounded according to U.S. Census Bureau disclosure avoidance policies. Dollar amounts indicate amount of incentive offered to household in each wave conditional on a completed interview. The Census Bureau reviewed this data product for unauthorized disclosure of confidential information and approved the disclosure avoidance practices applied to this release. CBDRB-FY21-POP001-0223.

Source: Authors’ calculations from the Survey of Income and Program Participation (SIPP) 2014, Waves 1 and 2.

This blog summarizes and discusses these findings on how earnings nonresponse compared across these groups.

We define earnings nonresponse as a binary variable equal to “1” if any earnings components (e.g., pay rates and pay frequencies for wages and salaries, extra earnings amounts) are imputed for an individual in a wave, and “0” otherwise. The average difference in earnings nonresponse rates between individuals in households that received a monetary incentive from those in households that did not is presented in Figure 1, for a sample that pools Waves 1 and 2 and for wave-specific samples. Figure 1 presents evidence that monetary incentives lower earnings nonresponse. In the pooled sample, the earnings nonresponse rate was 1.4 percentage points lower for individuals in households that received a monetary incentive. Earnings nonresponse rates were 1.0 and 1.4 percentage points lower for individuals in incentive-receiving households compared to the nonincentive-receiving in Waves 1 and 2, respectively.

Note: The estimates are average individual earnings nonresponse rates for the pooled SIPP sample and by wave. Earnings nonresponse is defined as having any earnings question imputed for any job in the reference period. Estimates are weighted by annual person weights. Sample is restricted to the set of individuals who reported jobs for an employer or other work arrangement. Incentive receipt is defined by a respondent's address being randomly selected prior to Wave 1 to receive a conditional incentive. Whiskers on point estimates represent 95 percent confidence intervals. The U.S. Census Bureau reviewed this data product for unauthorized disclosure of confidential information and approved the disclosure avoidance practices applied to this release. CBDRB-FY22-POP001-0046.

Source: Authors’ calculations from the Survey of Income and Program Participation (SIPP) 2014, Waves 1 and 2.

Did the amount of the monetary incentive matter for earnings nonresponse? Figure 2 shows estimates of the average differences in earnings nonresponse rates by incentive amount. We find no evidence that the earnings nonresponse rates were not statistically lower for individuals in the $20-incentive-receiving households than individuals in nonincentive-receiving households in Wave 1. The earnings nonresponse rates between individuals in $40-incentive-receiving households and $20-incentive-receiving households in Wave 1 also were not statistically different. However, the earnings nonresponse rate is lower for individuals in $40-incentive-receiving households than for individuals in nonincentive-receiving households in both Waves 1 and 2. The $40 incentive lowers the earnings nonresponse rate by 1.4 percentage points in both waves.

Note: Estimates are differences in average individual earnings nonresponse rates for $20 vs. $0, $40 vs. $0, $40 vs. $20 in Wave 1, and $40 vs. $0 in Wave 2, respectively. Earnings nonresponse is defined as having any earnings item imputed for any job in the reference period. Estimates are weighted by annual person weights. Sample is restricted to the set of individuals who reported jobs for an employer or other work arrangement. Negative estimates indicate that, (1) average earnings nonresponse rates among individuals in $20 and $40 incentive-eligible households are lower than those in incentive-ineligible households, and (2) average earnings nonresponse rates among $40 incentive-eligible households are lower than those in $20 incentive eligible households. Whiskers indicate 95 percent confidence intervals for the differences. The U.S. Census Bureau reviewed this data product for unauthorized disclosure of confidential information and approved the disclosure avoidance practices applied to this release. CBDRB-FY22-POP001-0046.

Source: Authors’ calculations from the Survey of Income and Program Participation (SIPP) 2014, Waves 1 and 2.

Figure 3 plots the probability of attrition in Wave 2 for different incentive groups and their earnings response status in Wave 1. Earnings nonrespondents were more likely to leave the survey than their respondent counterparts in every incentive group. Specifically, the probability of attrition was 8.8, 10.1, and 5.3 percentage points higher for earnings nonrespondents compared to respondents who received $0, $20, and $40 incentives, respectively, in Wave 1 (all statistically significant at the 1% level). Therefore, it stands to reason that by lowering earnings nonresponse rates, monetary incentives may indirectly lower attrition. 

Note: Estimates are weighted average attrition rates in Wave 2 by incentive amounts ($0, $20, and $40) and earnings nonresponse status. Attrition is determined based on the presence of a longitudinal weight in Wave 2. Earnings nonresponse is defined as having any earnings item imputed for any job in the reference period. Attrition estimates are weighted by annual person weights in Wave 1. Sample is restricted to the set of individuals who reported jobs for an employer or other work arrangement. The U.S. Census Bureau reviewed this data product for unauthorized disclosure of confidential information and approved the disclosure avoidance practices applied to this release. CBDRB-FY22-POP001-0046.

Source: Authors’ calculations from the Survey of Income and Program Participation (SIPP) 2014, Wave 1.

Yet, Figure 3 suggests that the incentive may also affect attrition directly, without operating through its effect on earnings nonresponse. The $40 incentive lowers the attrition rate among earnings nonrespondents by 5.0 percentage points more than $20 and 2.7 points more than $0, although the latter difference is not statistically significant at the 10-percent level. The incentive did not affect attrition rates for earnings responders, which diminishes the combined effect of incentives on attrition.

The SIPP is a nationally representative longitudinal survey administered by the Census Bureau that provides comprehensive information on the dynamics of income, employment, household composition, and government program participation. SIPP is also a leading source of data on economic well-being, family dynamics, education, wealth, health insurance, child care, and food security. SIPP interviews individuals for several years and provides monthly data about changes in household and family composition and economic circumstances over time. For more information, visit the Survey of Income and Program Participation (SIPP) page at census.gov. For technical documentation and more information about SIPP data quality, visit the  SIPP Technical Documentation page (at census.gov).