By Daniel H. Weinberg and Stephanie S. Shipp, U. S. Census Bureau
February 2, 2000
The Survey of Program Dynamics (SPD) is a ten-year longitudinal survey designed to provide data about families before and after the 1996 nationwide welfare reform. The SPD's value derives from three characteristics: (1) it was designed to focus its content on welfare; (2) its sample is representative of the 1992 and 1993 civilian noninstitutionalized population; and (3) its response rates are reasonably comparable to other longitudinal household surveys. Attrition of respondents is nevertheless a problem that has necessitated the use of incentives and special efforts to return nonrespondents to the survey.
Because researchers have questioned the usefulness of data from the SPD due to the attrition of respondents, the purpose of this paper is to evaluate the quality of the SPD data. The conclusions drawn from the analysis below is that the SPD data are representative of the population when compared with the Current Population Survey (CPS), and the SPD response rates are comparable with response rates from two other major longitudinal household surveys -- the Panel Study of Income Dynamics (PSID) and the National Longitudinal Survey of Youth (NLS-Y). However, attrition is still a problem for the SPD. Based on an experimental study that the Census Bureau conducted in 1998, monetary incentives were successful in gaining cooperation from panel nonrespondents, which suggests that SPD adopt the use of monetary incentives to reduce attrition.
This paper addresses the following questions:
The Survey of Program Dynamics is a national longitudinal survey that follows the same families for up to 10 years, from 1992 through 2001. In 1996, Congress mandated that the Census Bureau continue to collect data from households who participated in the 1992 and 1993 panels of the SIPP, households that had already completed survey participation by January 1995 or 1996, respectively. This additional data collection allows the Census Bureau to obtain information on changes in program participation, employment and earnings, as well as measures of adult and child well-being in the post-1996 time period. The data collected from the 1992 and 1993 SIPP panels provided us with three years of longitudinal baseline data prior to major welfare reform. Data collected in those panels included program eligibility, access and participation, transfer income and in-kind benefits, detailed economic and demographic data on employment and job transitions, earnings and other types of income, and family composition. The SIPP data (1992-1995 for half the sample, 1993-1995 for the other half) combined with the SPD data (1996-2001) will provide ten years of annual panel data capturing both pre- and post-welfare reform data. As do most longitudinal surveys, the SPD follows original sample people who move or form new households. For a more detailed description of the SPD, see Box 1.
Several other surveys will also contribute to understanding the changes that result from welfare reform. The 1996 SIPP will provide nearly four years of longitudinal data for almost 37,000 households from April 1996 through March 2000. A special welfare reform module was collected from August to November 1998. (3)
Box 1. Description of the Survey of Program Dynamics (SPD)
The Census Bureau conducts the SPD under the authority of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (Public Law 104-193). P.L. 104-193 requires (and funds) the Census Bureau to:
"continue to collect data on the 1992 and 1993 panels of the Survey of Income and Program Participation as necessary to obtain such information as will enable interested persons to evaluate the impact [of the law] on a random national sample of recipients of assistance under state programs funded under this part and (as appropriate) other low-income families, and in doing so, shall pay particular attention to the issues of out-of-wedlock birth, welfare dependency, the beginning and end of welfare spells, and the causes of repeat welfare spells, and shall obtain information about the status of children participating in such panels."
The 1997 SPD "Bridge Survey" attempted to interview all sample persons in the 38,000 households that completed all waves of the 1992 and 1993 SIPP panels (73 percent of the original sample). The field staff interviewed 82 percent of those households (approximately 30,000) using a modified version of the March 1997 Current Population Survey (CPS) in May-June 1997. This survey provides a bridge between the 1992-93 SIPP data and the 1998-2001 SPD data.
A new core SPD questionnaire was developed for 1998 with the assistance of Child Trends Inc. and funding from the U.S. Departments of Agriculture and Health and Human Services. The 1998 survey also included a self-administered adolescent questionnaire (SAQ). The SPD core instrument includes retrospective questions for all people aged 15 years and over on jobs, income, and program participation, as well as detailed questions about children under 15. Because of budget constraints, the sample for the 1998 SPD was approximately 18,500 households, including all sample households with children in or near the poverty threshold and an over-representation of other households with children or near the poverty threshold. The field staff obtained interviews from 89 percent of households eligible for the 1998 SPD. The 1999 SPD includes extended measures of children's well-being; the 2000 SPD includes a retrospective children's residential history; the 2001 SPD will repeat the 1998 SAQ, and the 2002 SPD will repeat the 1999 extended measures of children's well-being.
The 1997 SPD (data for 1996) were released in February 1999. The 1998 SPD (data for 1997) were released in February 2000. Although we released calendar year files, our main focus is developing a longitudinal processing system to create a unified data file with common formats. It is only with such a file, and an appropriate longitudinal weight, that sophisticated before-and-after analysis of the effects of welfare reform can take place. Developing such a processing system is a major undertaking.
The Census Bureau's Current Population Survey (CPS) and the Urban Institute's National Survey of American Families (NSAF) are two cross-sectional surveys that will be used to study the effect of welfare reform. The CPS has already been used to study other non-experimental welfare changes, such as those made in 1981 to the Aid to Families with Dependent Children (AFDC) program. (4) The NSAF data are being collected specifically to evaluate the 1996 changes. (5)
Researchers also hope to learn about different components of the 1996 changes by looking at pre-existing, continuing experimental studies, such as welfare waiver demonstration projects (e.g. Wemmerus and Gottlieb, 1999; Weissman, 1997; Bos and Fellerath, 1997; and Fraker, et. al., 1997). Other useful approaches include ethnographic studies, such as the Manpower Demonstration Research Corporation's Urban Change Study and the General Accounting Office's studies of welfare reform in selected states. (6) Each of these surveys and studies will provide insight into some aspects of welfare reform and should be considered part of the portfolio needed to understand that major program change.
The SPD is a unique tool for evaluating welfare reform because of its welfare reform-specific content (see Box 2) and the ability to analyze the economic and social well-being of families at two points in time as well as longitudinally over a 10-year period.
Box 2: Welfare-Reform Specific Content in the Survey of Program Dynamics
Demographic information: basic demographic characteristics, household composition, educational enrollment, work training, functional limitations and disability
Economic information: employment and earnings, income sources and amounts, assets, liabilities, and program participation and eligibility information (including reasons for leaving programs and reasons not accepted into programs), health care use, health insurance coverage, and food adequacy
Child well-being: school enrollment and enrichment activities, disability and health care use, contact with absent parent, child care arrangements, payment of child support on their behalf, and residential history
Two self-administered questionnaires: a series of questions for adults about marital relationship and conflict and a depression scale; and a questionnaire for adolescents 12 to 17 years old on issues such as household routines and chores, parental monitoring, identification with parents, contact with non-residential parent, delinquent behaviors, knowledge of welfare rules, crime-related violence, substance use, dating, sexual activity and contraceptive use
How do the SPD's response rates compare to those of the 1968 PSID and the 1979 NLS-Y?
The usefulness of data from any study that interviews the same respondents
over a period of years depends on whether the data represent the relevant
(7) Nonresponse by members of
the original sample is a potential source of bias that can undermine the
quality of estimates derived using longitudinal data. This section compares
overall response rates between the initial interview and the most recent
interview for three major national longitudinal household surveys: the
SPD, the PSID (conducted by the Survey Research Center at the University
of Michigan), and the NLS-Y (conducted by the National Opinion Research
Center for the Center for Human Resource Research at Ohio State University).
See Box 3 for a brief description of each survey and the survey universe.
How these surveys have tried to minimize attrition is also discussed.
PSID: SRC= Survey Research Center, SEO= Survey of Economic Opportunity;
Table 1 presents the current overall response rates for specified survey periods, both including and excluding deceased people from the base. (8) The current mortality-adjusted cumulative response rate for the entire survey period between initial sample selection and the most recent interview is 50 percent for the SPD, 64 percent for the NLS-Y, and 35 percent to 41 percent for the PSID (see the bottom line of Table 1). Among people designated for interview during sample selection, these rates indicate the proportion who were successfully interviewed during each round of interviews. (9)
Note also that the PSID-SRC (Survey Research Center) sample
was intended to be a representative sample of the U.S. population, while
the PSID-SEO (Survey of Economic Opportunity) subsample and the entire
NLS-Y were representative only of selected portions of the population --
low-income households in 1968 and individuals aged 14-22 in 1979, respectively.
Note: Data collection year and wave (interview) number for most recent survey at the time this paper was prepared:
SPD = 1998 (Wave 12),
PSID= 1993 (Wave 26); SRC= Survey Research Center, SEO= Survey of Economic Opportunity;
NLS-Y= 1996 (Wave 17); the label "always" mean that a respondent never missed an interview; the label "currently" means that a respondent may have missed one or more interviews but is currently in the survey.
NA = not available.
See Appendix 1 for a comparison of SPD, PSID, and NLS-Y response rates.
Response rates for other longitudinal surveys often appear higher than
for the SIPP in the literature because they often report their response
rates based on the number of households actually interviewed in Wave 1
rather than based on the original sample selected for interview. In Table
2 response rates for SPD, PSID, and NLS-Y are compared at Interview 1 and
at Interviews 11, 12, and 13 (the SPD's most recent interviews). Interview
1 response rates based on sample selected are 91 percent, 76 percent, and
89 percent for the SPD (originally SIPP), PSID, and NLS-Y respectively.
A comparison of response rates from sample selection to the same number
of interviews (11, 12, and 13) shows SPD rates comparable to PSID and somewhat
lower than NLS-Y. The Census Bureau conducted SPD interviews during the
1990s, when response rates for household surveys were generally somewhat
lower than in the 1978-80 period for the PSID and 1989-91 period for the
NLS-Y. Both the PSID and NLS-Y also used incentives throughout their field
period to encourage participation, and the SIPP used no incentives.
SPD: 1st Interview - 1992 or 1993; 11th-13th Interviews - 1997 to 1999.
PSID: 1st Interview - 1968; 11th-13th Interviews - 1978 to 1980.
NLS-Y: 1st Interview - 1979; 11th-13th Interview - 1989 to 1991.
The SPD overall response rate held steady at 50 percent between the 12th and 13th Interview because the Census Bureau made additional efforts to bring wave 12 nonrespondents back into the sample and to encourage Wave 13 nonrespondents to respond. A $40 incentive was mailed to Wave 12 nonrespondents, and during Wave 13, field representatives were allowed to give $40 incentives to encourage nonrespondents to be interviewed. (11)
Examining the 1992 SPD data shows that some differential attrition occurred
by income group, by program participation characteristics, and by family
(12) Table 3 presents differential
attrition rates for SPD sample cases, according to the income-to-poverty
ratio in the first interview month. Attrition rates are calculated for
three periods: (1) between the first and last SIPP interview months, for
the 1992 and 1993 SIPP panels that provide the sample for the subsequent
SPD Bridge interview;
(13) (2) between the
last SIPP interview month in October 1994-January 1995 and the SPD Bridge
interview in 1997; and (3) between the first SIPP interview month and the
SPD Bridge interview. Attrition rates are calculated as the percent of
the sample at the first point in time who are not successfully followed
and interviewed at the second point in time.
People with lower incomes have higher attrition rates than people with higher incomes. But the differences are not enormous, and about 50 percent of the people of most direct interest to researchers for evaluating welfare reform were interviewed both in the first SIPP interview month and in the SPD Bridge interview.
What affected SPD attrition?
Budgetary and other reasons may have exacerbated attrition. First, to capture the pre-welfare reform situation of households (including pre-waiver behavior), the 1992 and 1993 panels of the SIPP were used as the sampling frame for the SPD; thus, the SPD sample inherited a 27 percent attrition rate from the 1992 and 1993 SIPP panels. (15) Second, the budget was insufficient to interview all households in both the 1992 and 1993 SIPP panels for the length of the SPD; therefore, households that participated in both Wave 1 and Wave 9 or 10 (16) interviews were selected for the SPD sample. Third, due to budget constraints, the Census Bureau sub-sampled the 1997 SPD Bridge sample for the 1998-2002 SPD. The low income population and households with children were oversampled with certainty or near certainty to maximize the sample population most likely to receive welfare. If these types of households are also most likely to become nonrespondents, then measured attrition would be biased upward compared with a non-stratified sample.
How do data from the SPD compare with data from the CPS March Demographic Supplement?
Tables 4a (for all respondents) and 4b (for young women) compare selected measures from the SPD to the CPS March Income Supplement. (17) The tabulations of both the SPD and CPS data presented here use normalized weights (the individual weight divided by the average sample weight); the results of the normalized weighting procedure resemble unweighted counts. The normalized weights, however, preserve the weighted relationship between variables. That is, the proportional distribution is the same whether normalized or cross-sectional weights are used. These results are not national estimates.
Statistical differences between the SPD and CPS at the 90 percent significance level are asterisked. (18) SPD-CPS comparisons for women 20 to 26 years old in 1997 (21 to 27 years old in 1998) (19) are a proxy for potential young mothers. This group is useful in evaluating the potential of SPD data for examining the effect of welfare reform on young mothers.
In the comparisons for all individuals, some statistical differences
are apparent, more for the 1998 data than for 1997. For example, the percent
of people participating in programs for the SPD and CPS are quite comparable
for 1997. There are more significant differences for 1998, with the SPD
showing a slightly higher percent participating in programs; however, the
percentages are still reasonably close. In the comparisons for women age
20 to 26 in 1997 and 21 to 27 in 1998, only a few differences are statistically
significant, possibly indicating that the data compare quite well to the
CPS, but more likely because of the relatively small sample size of this
* The SPD estimate is significantly different from the CPS estimate
at the 90 percent confidence
Note: Except for educational status, the data are for the previous
year. Educational status is as of the interview date.
* The SPD estimate is significantly different from the CPS estimate
at the 90 percent confidence
Note: Except for educational status, the data are for the previous year. Educational status is as of the interview date.
What was learned from the SPD Exploratory Attrition Study and the use of incentives?
Addressing concern about the SPD response rates, the Census Bureau conducted an Exploratory Attrition Study (20) to assess the extent to which nonrespondents could be brought back into the sample. Other longitudinal surveys (such as the PSID and NLS-Y) have gone back to early panel nonrespondents and successfully brought them back into sample. A key aspect of this experiment was to test the effectiveness of paying incentives to nonrespondents to encourage people who had not responded as long as 5 or 6 years earlier to re-enter the sample and respond to a current questionnaire. The project focused on people at or below 200 percent of the poverty threshold, because they are of interest in studies of welfare reform and their attrition is much greater than that of the higher-income population.
Possible reasons why attrition is a problem for the SPD include the fact that SPD households will be followed much longer than SIPP households--10 years for SPD versus 4 years for the 1996 SIPP panel and 3 years for other SIPP panels. Further, respondents from the 1992-93 SIPP panels had been told that they had completed their eligibility for the survey, but additional interviews were required because of the clear legislative mandate for SPD. Moreover, some of these people had refused the Census Bureau many times before, thus making them "hardcore" nonrespondents.
Because of the complexities and cost of programming a computer-assisted
personal interview (CAPI) instrument for a small sample, a revised paper
questionnaire and control card from Wave 9 of the 1993 SIPP panel were
used to interview households in the experiment. Three incentive amounts
were tested ($0, $50, and $100) to see if the size of the incentive affected
response rates. Table 5 shows the results.
* Response to the $100 incentive is significantly different from the $0 rate at the 90 percent level, but not different from the $50 rate.
The Exploratory Attrition Study sample consisted of 358 randomly selected low-income (below 200 percent of poverty) cases that became nonrespondents in the 1992 and 1993 SIPP panels and 48 cases spawned (21) after SIPP but before the SPD Exploratory Study interview, for a total of 406 cases. Of these 406 cases, 373 were eligible to be interviewed. They include those that were interviewed, refused, no one home, temporarily absent, or movers who could not be found. The remaining 33 cases included vacant units, units under construction, units occupied by people whose usual residence is elsewhere, demolished units, units converted to a business, or units that have been moved (e.g. a mobile home).
All the comparisons were tested at the 90 percent significance level.
The response rate for all eligible cases was 37 percent. The response rate
for those who received the $100 incentive
Incentives have a larger effect on original Type A nonrespondents (refusals, no one home, and temporarily absent) than on Type D nonrespondents (unlocated movers). In the $100 group, the response rate for Type A nonrespondents (54 percent) was higher than for Type D nonrespondents (35 percent). Obviously, incentives could be offered only if the cases were located.
What response rates are expected if the Census Bureau receives funding to bring back in nonrespondents to the 1997 SPD and the 1992 and 1993 Survey and Income Participation Program (SIPP)?
If funding becomes available, the Census Bureau plans to interview a targeted sample of SIPP and SPD Bridge nonrespondents. In 2000-2002, we will interview a targeted sample of SPD Bridge (1997) nonrespondents, while in 2001-2002 we will interview a targeted sample of SIPP (1992-1995) nonrespondents. (22) The targeting will follow rules parallel to those used to subsample the 1998 SPD from the 1997 Bridge sample. The proposal involves paying nonrespondents an initial $100 incentive in the first year and $40 maintenance incentives in subsequent years. If this plan to re-interview nonrespondents is implemented, the Census Bureau projects response rates to increase from the current (1999) 50 percent rate to:
The Survey of Program Dynamics (SPD) is only one of many tools for evaluating welfare reform, yet it has the potential to be particularly valuable. SPD data are representative of the national population (23) based on comparisons with the CPS March Income Supplement. SPD response rates are comparable to NLS-Y and PSID response rates, although attrition remains a problem. Despite these positive signs, the experimental evidence suggests it will be worthwhile to pay incentives to current nonrespondents to bring them back and improve SPD response rates.
Appendix 1: Comparisons of SPD, PSID, and NLS-Y response rates.
Appendix 2: Table 1: Household Income Distribution: The 1997 SPD Bridge and the 1997 CPS March Income Supplement.
Table 2: Selected Household Data from the 1997 SPD Bridge and the 1997 CPS March Income Supplement..
Comparison of SPD, PSID, and NLS-Y Response Rates
Household response rates between sample selection and the first interview were calculated for the SIPP 1992 and 1993 panels combined, based on results presented in McMahon, 1995. People interviewed in the first and last waves of the SIPP 1992 and 1993 samples became the SPD Bridge sample. Individual response rates between the first SIPP 1992 and 1993 panel interviews and the SPD Bridge interview (1997) were derived by Donald J. Hernandez using the SIPP 1992 Panel Waves 1-10 Longitudinal File, the SIPP 1993 Panel Waves 1-7 Longitudinal File, and the U.S. Census Bureau internal SPD 1997 file available on the Housing and Household Economic Statistics Division server on December 4, 1998. Deceased are identified from the SIPP data for the period between interview 1 and the final SIPP interview prior to the SPD interview. The response rate between SPD 1997 and SPD 1998 is preliminary, and both deceased and newly institutionalized populations were removed from the base.
The PSID User's Guide (Hill, 1992) notes that the original PSID sample actually consisted of two independent samples, one drawn by the Survey Research Center and referred to as the SRC sample, the other selected from the Survey of Economic Opportunity (SEO) conducted in 1966 and 1967 by the Census Bureau for the Office of Economic Opportunity (OEO). The initial response rate for the SEO sample (Survey Research Center, 1972) was calculated to be 74 percent, based on the sample of households provided to SRC by the Census Bureau and the Office of Economic Opportunity. This does not include the effect of (1) attrition between sample selection by the Census Bureau and the first interview by the Census Bureau of respondents in 1967, which led to a response rate of 91.6 percent (Office of Economic Opportunity, 1970), (2) sample loss through subsequent refusals to remain in the sample that became the SEO component of the PSID, since about 25 percent refused to allow their names to be passed to SRC (Hill 1992), and (3) the failure of some sampled addresses to be transmitted from OEO to SRC (Hill, 1992). To calculate the PSID-SEO sample-selection to interview response rate of 50.8 percent, the initial response rate of 91.6 percent is multiplied by the 75 percent rate of "willingness" to have names transmitted from the Census Bureau to SRC, and then by the 74 percent response rate obtained by SRC in seeking to interview households provided by the Census Bureau and the OEO. This does not take into account the fact that address information for some sample people who were willing to participate was not transmitted from the Census Bureau and OEO to SRC. Introducing this source of sample loss into the calculations will reduce the current estimate of 50.8 percent. Of course, as in all the surveys discussed here, weighting procedures were designed to take various factors, including sample attrition, into account. The response rate for the SRC sample was 76 percent. The SRC sample constituted about 60 percent of the initial PSID sample, while the SEO sample constituted about 40 percent of the initial PSID sample.
The response rates for "interview 1 to the most recent interview" were obtained from the Survey Research Center (1972) and from Table 2a of the documentation provided by PSID via Internet entitled "A Panel Study of Income Dynamics: Procedures and Code Books, Guide to the 1993 Interviewing Year, Procedures, Wave XXVI, A Supplement," obtained December 1998. The first and most recent interview years were 1968 and 1993, respectively. Tecla Loup of the PSID staff was very helpful in identifying needed estimates and confirming the interpretation of specific estimates. Deceased are identified by PSID staff from the PSID data between the first and most recent interviews. Sandra Hofferth provided the estimated response rate for 1994 with expected deceased removed from the base.
The source for these estimates is "NLS-Y-79 Users' Guide, A Guide to
the 1979-1996 National Longitudinal Survey of Youth Data" prepared for
the U.S. Department of Labor by the Center for Human Resource Research,
the Ohio State University, August 1997. The response rate of 89.2 percent
between sample selection and first interview is obtained from Table 3.3.1
based on the cross-sectional and supplemental (sub)samples. The response
rate of 69.6% between the initial interview (1979) and the most recent
interview (1996) is obtained from Table 3.7.1. Deceased who number 224
by 1994 according to Table 3.6.1 were removed from the base. An additional
39 deaths (personal communication, from Randall J. Olsen) for years 1995
and 1996 were also removed from the base. The "always" interviewed column
includes in the numerator those people who were interviewed in each of
the 17 interviews. The "currently" interviewed column includes in the numerator
people who were interviewed in at least the first interview and the current
interview. If the base is limited to those not dropped from the survey
or deceased, the proportion of those interviewed in the first interview
who missed no more than one interview out of 17 was 83.0 percent, and combined
with the 4.6 percent who missed only two out of 17 interviews, the response
rate was 87.6 percent.
Source: US Bureau of the Census, March CPS 1997 and the 1997 SPD Bridge
Note: The distribution for both the SPD Bridge and
CPS March Income Supplement household income is very similar. The one distinction
between the two distributions is that the 1997 CPS March Income Supplement
has a higher percentage of households with total income below $25,000 when
compared to the SPD -- 35.7 percent versus 34.1 percent; this difference
is statistically significant.
Source: US Bureau of the Census, March CPS 1997 and the 1997 SPD Bridge
TANF = Temporary Assistance to Needy Families
SSI = Supplemental Security Income
* The SPD estimate is significantly different from the CPS estimate at the 90 percent confidence level.
Bos, Johannes M., and Veronica Fellerath, LEAP: Final Report on Ohio's Welfare Initiative to Improve School Attendance Among Teenage Parents, Manpower Demonstration Research Corporation, New York, NY August 1997.
Center for Human Resource Research, Ohio State University, NLS-Y Users' Guide, A Guide to the 1979-1996 National Longitudinal Survey of Youth Data. Prepared for the U.S. Department of Labor, August 1997.
Eargle, Judy, "Final 1993 Panel Type A and Type D Sample Loss Report." Internal Census Bureau memorandum for the record. January 27, 2000.
Fraker, Thomas M., Lucia A. Nixon, Jan L. Losby, Carol S. Prindle, and John F. Else, "Iowa's Limited Benefit Plan: Summary Report," prepared by Mathematica Policy Research, Inc. and the Institute for Social and Economic Development, Washington, DC, May 1997.
Hernandez, Donald J., "Comparing Response Rates for the Survey of Program Dynamics (SPD), Panel Study of Income Dynamics (PSID), and the National Longitudinal Survey of Youth (NLS-Y)," internal Census Bureau memorandum to Daniel Weinberg. January 28, 1999.
Hernandez, Donald J., "The Survey of Program Dynamics: Sample Retention, the Accuracy of Change Estimates, and Implications for Assessing Welfare Reform, Research Brief," Internal Census Bureau memorandum. November 4, 1998.
Hill, Martha S., The Panel Study of Income Dynamics: A User's Guide, Sage Publications, Newbury Park, California, 1992.
King, Karen, "Preliminary Evaluation of the Survey of Program Dynamics (SPD) Exploratory Attrition Study" Internal Census Bureau memorandum to the SPD Steering Committee. June 17, 1999.
McMahon, Michael F., "Final 1992 Panel Type A and Type D Sample Loss Report." Internal Census Bureau memorandum for the record. November 27, 1995.
Moffitt, Robert A., "Incentive Effects of the U.S. Welfare System," Journal of Economic Literature, (30), March, 1992, pp 1-61.
Manpower Demonstration Research Corporation, Biannual Report 1996/97, 1998.
Nelson, Charles, and Pat Doyle, "Recommendations for Measuring Income and Program Participation in the Post Welfare Reform Era," paper presented at the Joint Statistical Meetings, Baltimore, MD. August 1999.
PSID Internet site. "A Panel Study of Income Dynamics: Procedures, Wave XXVI, A Supplement," Obtained December 1998.
Personal Responsibility and Work Opportunity Reconciliation Act of 1996. Public Law 104-193.
U.S. General Accounting Office, Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence, GAO/HEHS-98-109, June 1998.
Weinberg, Daniel, Pat Doyle, Arthur Jones Jr., and Stephanie Shipp, "Measuring the Impact of Welfare Reform with the Survey of Program Dynamics," Social Statistics Proceedings, American Statistical Association, Dallas, Texas. August 1998.
Weissman, Evan, Changing to a Work First Strategy: Lessons from Los Angeles County's GAIN Program for Welfare Recipients, Manpower Demonstration Research Corporation., New York, NY, June 1997.
Wemmerus, Nancy and Bruce Gottlieb, Relaxing the FSP Vehicle Asset Test: Findings from the North Carolina Demonstration," Mathematica Policy Research, Inc., Washington, DC, January 22, 1999.
1. This paper reports the results of research and analysis undertaken by Census Bureau staff. It has undergone a more limited review than official Census Bureau publications. This report is released to inform interested parties of research and to encourage discussion. We would like to thank the participants in an American Enterprise Institute seminar for the opportunity to develop and present this information. We particularly thank Donald Hernandez for key work analyzing attrition, Karen King and Michael McMahon for information about the current status of the SPD, Stephen Campbell, Charita Castro, and Arthur Jones for assistance in computations, Jenny Hess, Larry Long, and Kenton Kilgore for their excellent comments, and Roberta Payne for preparing numerous drafts of this manuscript.
2. As will be explained later, this study was conducted to test the feasibility and costs of finding and interviewing nonrespondents from the SIPP sample - - the sample of households used for the SPD survey.
10. The "SEO" sample of the PSID was selected from low-income respondents to the 1967 Survey of Economic Opportunity conducted for the Office of Economic Opportunity (OEO) by the Census Bureau. Thus, these households had an extra opportunity for non-random attrition (non-response to the SEO). See Appendix 1.
14. See SIPP and SPD documentation for specific rules used to ascertain whether or not a sample person is designated as interviewed at a particular point in time. The website addresses are http://www.sipp.census.gov/sipp/ and http:/www.sipp.census.gov/spd/.
17. Stephen Campbell, Charita Castro, and Arthur Jones in the Census Bureau's Housing and Household Economic Statistics Division defined the variables and wrote the SAS programs to produce these data.
18. When multiple comparisons are made at the 90 percent confidence level, 10 percent of differences will appear to be statistically significant just due to chance. In Table 4a, about 12 of the 17 SPD-CPS comparisons (one is dependent) are significant and are thus suggestive of sample differences. On the other hand, in Table 4b, for young women, the much smaller number of significant differences does not suggest sample differences.
19. Since we have not yet constructed family and household variables for the 1998 SPD, tabulations are shown at the individual level. Household level comparisons for 1997 (1996 data) are shown in Appendix 2, Tables 1 and 2.
23. To fully use the SPD data, researchers must understand that complex modeling is needed to adjust for nonresponse and to incorporate other data sources (e.g., state-specific variables that describe state welfare programs).