SPD Status Reports

Survey of Program Dynamics

Status Report

April 8, 2003

The following is an update of the Survey of Program Dynamics (SPD) activities for the past seven months.

Data Products

SPD data products have experienced a number of schedule delays as each subsequent year of collection adds complexity to the review process and reconciliation with previous years' reporting. The following summarizes the data released to date. Plans for later products are addressed in item V.

• The 1997 Bridge experimental cross-sectional file was released in May 1999. This is a fully edited file.
• The 1998 experimental cross-sectional file was released in April 2000. This is a minimally edited file.
• The first longitudinal public use file was released on FERRET in September 2001. The file was made available on CD-ROM in October 2001. Copies of the CD were sent to approximately 40 data users.
• The second longitudinal file was released on FERRET in July, 2002 without the eligibility module variables. The file was reissued with the eligibility module variables on FERRET in September, 2002, and on CD in November, 2002. This second longitudinal file adds data for calendar years 1998 and 1999 and replaced the first longitudinal file.

• The attachment is a summary of key findings drawn thus far from the SPD data.

2002 Data Collection

Data collection for the 2002 SPD began on April 25, 2002. Due to budget constraints, the full SPD sample that was interviewed in 2001 (core sample, 1997 Bridge attriters, 1992/1993 SIPP attriters) was not interviewed in 2002. The 2001 sample was cut by approximately one-third across-the-board (randomly) to yield about 20,000 households across all three sample types. Additionally, the field period was shortened by two weeks and ended on June 14, 2002, because anticipated efficiencies did not materialize.

• The 2002 questionnaire included the basic core modules plus the Extended Measures of Children's Well-being modules.
• Incentives in the form of debit cards were used to encourage participation in the 2002 SPD. All households that were Type A noninterviews in 1998, 1999, 2000, and 2001 and all new Type A noninterviews in 2002 received a $40 incentive in 2002. Also, all other households that were given an incentive in 1999, 2000, and 2001, and gave an interview received a $40 incentive in 2002.
• All 2,011 eligible households from the 1997 SPD noninterview sample received a $40 incentive in 2002.
• All 2,533 eligible households from the 1992 and 1993 SIPP noninterview sample received a $40 incentive in 2002.

II. Sample Loss and Recovery

Sample Loss -- The SPD inherited a 26.6 percent sample loss rate from the 1992/1993 Survey of Income and Program Participation (SIPP) sample. In 1999, after three years of interviewing, the SPD basic sample loss rate was 49.8 percent. From previous studies of SIPP sample loss, we know that sample loss is not uniform. Households in and near poverty attrit at a higher rate than other households. Since poverty households are a key target population in the study of welfare reform, there was considerable concern about nonresponse bias. Additional funds were appropriated to address this concern. We received an additional $1M in FY 2000 which enabled us to bring back Type A and Type D noninterview cases from the 1997 SPD data collection and reduced the sample loss rate to 44.3%. In FY 2001 we received an additional $5M to bring back Type A and Type D noninterview households from the 1992 and 1993 SIPP.

Table 1A below shows the sampling scheme used to select 1997 SPD noninterview cases for interviewing. Table 1B shows the sampling scheme used to select the SIPP 1992 and 1993 noninterview households. A total of 5,540 were included in the sample of households brought back for interviewing in 2001

Note that due to problems with locating the original household control cards, only 5,540 of the 6,000 cases were sent out to be interviewed.

Table 2 shows the effects of sample loss and the subsequent re-entry of previous attriters on the SPD survey from the conclusion of the 1992/93 SIPP to the present.

Table 3 shows the components of nonresponse for the SPD from its inception.

* The SPD Steering Committee set the response rate goals for the 2000, 2001, and 2002 data collection periods as follows:

1) Core (Basic) sample - stays at 85%+ for all 3 years

2) Bridge (97NI) sample - starts at 50% in 2001 and stays at 45-50% for 2002

3) SIPP 92/93 NI sample - starts at 35% and stays at 30% - 35% for SPD 2002

III. Field Cost and Progress

Data collection (data collection plus incentives) costs were 67.3 percent of the SPD budget in 2001 and 57.1 percent of the SPD budget in 2002. Data collection costs and the use of incentives vary between regional offices. Response rates vary as well, although they are not necessarily related to the use of incentives. Table 4A below shows response rates and Table 4B shows cost per case expenses for fiscal years 2000, 2001 and 2002 for data collection and incentives.

IV. Budget Distribution and Incentive Utilization

Budget Distribution -- For FY 2001, the SPD received an additional $5M from the Department of Health and Human Services to continue to interview the noninterview cases (Type As or Ds) from the 1997 SPD data collection and to bring back into the active sample 5,540 noninterview cases (Type As or Ds) from the 1992 and 1993 SIPP Panels. The $5M was a one-time funding increase, bringing the total SPD budget for fiscal year 2001 to $15M. In 2002, we had to reduce the sample assigned by about 1/3 (see Table 2 above) to stay within the $10M budget. In addition, we had to end interviewing earlier than expected because of higher than expected costs.

Incentives Utilization -- During the 1999, 2000, 2001, and 2002 data collection period, Field Division was allowed to use $40 incentives to maintain sample and to convert Type A households for the SPD sample. In 2000, Field Division used a $100 incentive to convert Type A and Type D households reintroduced from the 1997 SPD Bridge Survey; in 2001 Field .Division used a $100 incentive to convert Type A and Type D households reintroduced from the SIPP 1992 and 1993. Additionally, a $40 conditional incentive was used in 2001 for completion of the Self-administered Adolescent Questionnaire (SAQ).

Effectiveness

Since all households receiving a debit card were Type As or were potential Type As, it is interesting to see the effectiveness of incentives in obtaining an interview. Table 6 below shows that during the 2002 data collection, there was almost a 9 in 10 chance that a household who received and cashed an incentive (debit card) would complete the interview.

V. Data Production

Based on the current budget constraints, we plan to release the SPD data products in the following priority order as follows:

• Release the 2001 cross-sectional minimally edited file (only demographic variables are edited)
• Release the 2002 cross-sectional minimally edited file (only demographic variables are edited)
• Release the fully edited third longitudinal file to add data for calendar years 2000 and 2001. Also, data collected from the Bridge noninterview households and the Survey of Income and Program Participation (SIPP) 1992/1993 noninterview households will be added to the third longitudinal file
.
• The 1999 and 2000 cross-sectional files will be released after the third longitudinal file is released.

VI. General Issues

2003+ -- Depending on funding, the SPD will continue to release data products

Research and evaluation activities -- DSMD applied a cross-sectional weight to the 1997 Bridge data to compare to official CPS March income data to help assess the effects of nonresponse bias on the SPD estimates. This file available for research purposes.

SPD Website -- The SPD survey has a web site (http://www.sipp.census.gov/spd). The site contains an overview of SPD, tables created from SIPP, technical documentation, SPD working papers, SPD data collection materials, instrument documentation for the 1998, 1999 and 2000 SPD instruments, and program code for merging files. The 1997 Bridge and the second longitudinal files are available on the web site through FERRET.

SPD grant competition – On behalf of the Census Bureau, the Poverty Research Center of Northwestern University-University of Chicago holds a small grant competition for academic research using the SPD. The results of the first grants were presented at a conference at the Census Bureau in May 2001. A second round of grants has been awarded and a third round of competition is underway.

Instrument - The chart below shows the various modules included in the SPD instruments.

ATTACHMENT

Paper: How Have Former Welfare Recipients Fared in the Wake of PRWORA?
Author(s): Arthur Jones, Jr. (Census Bureau)
Date: November 2002

Key Findings: This paper examines how well former welfare recipients are faring after leaving welfare rolls and entering the labor market. It examines income received through labor force participation, and total money income (the sum of all sources of income). The results suggest that most former welfare recipients were able to find work for pay and their employment rate equaled that of the non-welfare population (about 60 percent were active in the paid labor force in 1999). However, the jobs they obtained tended to be in low-wage industries and occupations, with wages below those paid to workers at the national level.

Paper: An Empirical Analysis of Joint Decision on Food Stamp Program, Temporary Assistance for Needy Families and Labor Force Participation

Author(s): Sonya Kostova Huffman and Helen Jensen (Iowa State University)
Date: September 2002
Key Findings: This paper looks at interactions among the Food Stamp Program (FSP) and Temporary Assistance to Needy Families (TANF) program with respect to participation, provides a model of joint decisions made by households on FSP, TANF, and labor force participation, and explains why households choose different alternatives. The results show that program parameters do matter. The higher the benefits are, the higher FSP and TANF participation is. TANF participation is positively related to the earned income disregard, and negatively related to the benefit reduction rate. Participation in TANF is positively related to the probability of FSP participation and negatively related to being in the work force. Work, TANF, and FSP are significantly related. The results imply that TANF and FSP participation among households with low assets who are potentially eligible for TANF are sensitive to changes in program parameters. Although labor force participation is affected by change in program parameters to some degree, it is most affected by the (predicted) wage.

Paper: The Economic Well-Being of Children and Families After Welfare Reform
Author(s): Laura Stander Connolly and Christine Enerson Marston (University of Northern Colorado)
Date: September 2002
Key Findings: This paper examines an important question surrounding welfare reform: "What has happened to the earnings and incomes of families who were receiving public assistance when the new policies took effect?" It does so by looking at the impacts of welfare reform policies on the earnings and income of welfare recipients and nonrecipients. The authors find that welfare reform policies do not affect the family earnings of welfare recipients within the first year, but do have a positive effect in the following year. The effect on total family incomes is mixed in the first year. The overall effect is negative, but work-related reforms have a positive effect. All effects on family income are insignificant in the following year.

Paper: Parental Co-residence Transitions for Children in the Years Surrounding Welfare Reform: Evidence from the Survey of Program Dynamics
Author(s): Rose M. Kreider (Census Bureau)
Date: August 2002
Key Findings: This paper looks at transitions in living arrangements between 1993 and 1998 for low-income children under 18 (in 1993). In the five-year period under study (which spans welfare reform), most low-income children (81 percent) did not experience a change in the number of parents with whom they lived. Over half of those who did experience a transition gained a parent. Children living with one parent are over four times more likely to gain a parent if they live with dad only rather than with only their mother. Finally, living with a parent receiving aid who gets married is not associated with significantly higher odds that the household will leave aid.

Paper: Longitudinal Attrition in SIPP and SPD
Author(s): Denton Vaughan (Census Bureau) and Fritz Scheuren (NORC)
Date: August 2002
Key Findings: Longitudinal surveys like the Survey of Income and Program Participation (SIPP) and the SPD suffer from attrition as the survey process is extended in time. This paper attempts to directly examine the effect of attrition on longitudinal change measures by using Social Security Administration longitudinal earnings records. It concludes that, eventually, net change measures from SPD, with only modest adjustment, may be completely comparable to cross-sectional surveys, like Current Population Survey, for earnings and variables closely related to earnings, such as work experience. However, for SPD to be used successfully to measure gross change, the data adjustment is likely to be greater. The very success of SPD as a measure of net change creates this seeming paradox. The paradox arises because those who attrit start out with lower incomes than those who remain in the survey longer. For attriters to catch up in six years, they necessarily have to have a different time trajectory than the more stable group who remain.

Paper: Participation in Public Assistance Programs, 1993 – 1997: A First Look at the Survey of Program Dynamics
Author(s): John Hisnanick (Census Bureau)
Date: August 2002
Key Findings: This paper discusses techniques and methods used to address missing 1995 data from the SPD: 1st Longitudinal File. The limitation of missing data can be overcome, but not without a cost. For example, linking the longitudinal SPD file with the 1993 SIPP longitudinal file provides 1995 data, but for only half the sample. Other methods of supplementing missing data are discussed including nearest neighbor imputation and forward/backward imputation. The method selected for this paper was linking to the 1993 SIPP. The paper looks specifically at public assistance program participation for two points in time, 1993 and 1997 for the full sample on the SPD: 1st Longitudinal File compared with yearly transitions from the half sample resulting from the merged SPD/SIPP files. The paper shows that using the full sample at two points in time allows for formulating hypotheses regarding the effectiveness of welfare reform, but does not allow a longitudinal look at transitions in public assistance program participation. It also shows that using the merged file with half sample is an effective way of overcoming missing data and examining transitions at the aggregate level, albeit with increased sample variance that hinders research on issues requiring a more focused use of the data.

Paper: Complex Problems – Complex Data: Welfare Reform and the Survey of Program Dynamics
Author(s): Peter Fronczek, Pat Doyle, Robert Kominski, and Charles Nelson (Census Bureau)
Date: August 2002
Key Findings: This paper looks at the origins and purpose of the SPD, and provides information on the background, development, design, and content or the survey. It also explains why the SPD is likely to be an invaluable database for measuring the effects of welfare reform and describes some of the important research issues that can be addressed with SPD data. While acknowledging that the SPD can provide researchers unique opportunities, the paper also points out a significant number of technical challenges that they will face in the longitudinal use of the survey data. It suggests that researchers become as familiar as possible with the content, formats, and caveats associated with using the SPD longitudinal files. Some of the technical issues discussed are: missing data points in the SPD longitudinal files; use of different collection vehicles for gathering SPD data; incorporating topical modules, core files, and experimental files with the SPD longitudinal files; attrition; and use of different accounting periods for the same topics.

Paper: The Stickiness of Poverty and Early Welfare Reform Efforts: A First Look Using the Survey of Program Dynamics
Author(s): John J. Hisnanick (Census Bureau)
Date: July 2002
Key Findings: This paper looks at individual's participation in public assistance before and after welfare reform. Initial findings suggest a "stickiness of poverty" at the beginning of the welfare reform effort. Of the estimated 5.8 million individuals age 15 or older who reported receiving public assistance in 1993, more than one-third (2.0 million) also reported receiving public assistance in 1997. Working from a conventional utility maximization model of public assistance participation, the paper identifies characteristics that differentiate "leavers" from "stayers" on the public rolls.

Paper: Adolescent Outcomes, Poverty Status, and Welfare Reform: An Analysis Based on the Survey of Program Dynamics
Author(s): Eileen Trzcinski and Jerry Brandell (Wayne State University)
Date: September 2001
Key Findings: This paper describes research using the SPD to examine the links between outcomes for adolescents, source of income, mother's employment, and welfare reform. Specifically, the research examines how poverty status and family welfare receipt during middle childhood interact with current poverty status and welfare receipt during adolescence to influence a range of outcomes for adolescents. Results show that the explanatory variables that are most highly correlated with positive outcomes for adolescents are: the average income to needs ratio, maternal/parental education, and whether the adolescent has lived in a two-parent family; that current full-time employment is never associated with more positive outcomes for adolescents; that only in a few cases are adolescent outcomes affected by whether the mother/parent reported that welfare benefits were cut off; and that when income insufficiency is severe enough to affect the adequacy of an adolescent's diet, the adolescent is affected on a number of outcomes. Finally, the adolescent outcomes most sensitive to differences in income, patterns of program participation, and the time and extent of maternal employment are: school outcome variables, status offense/criminal behavior variables, and substance abuse variables.

  SPD Introduction

   SPD Overview

   Analytic Uses of SPD Data

Contact: (dsd.survey.of.program.dynamics@census.gov)
URL: http://www.sipp.census.gov/SPD/

| Introduction to SPD |  Survey Design & Content |  Data Editing |  Finding SPD Info |  Sampling & Weighting | 
| Linking Files | Publications |  S&A  |  News&Notes |  Users' Guide |