Measuring the Impact of Welfare Reform With
The Survey of Program Dynamics 1
Daniel Weinberg, Pat Doyle, Arthur Jones Jr, Stephanie Shipp,
Census Bureau, Washington, DC 20233-8500
October 7, 1998
The role of social science lies not in the formation of social policy, but in the measurement of its results.
Senator Daniel Patrick Moynihan, 1969
1 This paper reports the results of research and analysis undertaken by Census Bureau staff. It has undergone a more limited review than official Census Bureau publications. This report is released to inform interested parties of research and to encourage discussion. Parts of this paper were presented at the 1998 Joint Statistical Meetings, Dallas, Texas, and the 1998 Association for Public Policy Analysis and Management (APPAM) annual meetings, New York, New York.
Measuring the Impact of Welfare Reform With The Survey of Program Dynamics
Daniel Weinberg, Patricia Doyle, Arthur Jones J, Stephanie Shipp,
Census Bureau, Washington, DC
October 7, 1998
Contact author: Stephanie Shipp, Census Bureau, Washington, DC 20233-8500, firstname.lastname@example.org
Key Words: welfare reform, panel survey, longitudinal panels
Abstract. The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 directs the Census Bureau to measure the effects of welfare reform. To carry this out, we are conducting the Survey of Program Dynamics (SPD), a large longitudinal, nationally-representative survey. It is designed to provide information on intervals of actual and potential program participation over the ten-year period 1992 to 2001 and to examine the causes of program participation and its long-term effects on the well-being of individual recipients, their families, and their children. We will describe the survey, address the Census Bureau's plans to construct a consistent, longitudinal data series from three distinct survey instruments, and then describe how the data are to be used for research purposes. Finally, a brief description of other welfare-related data sources is presented, including a discussion of how national and state data can be used to supplement each other.
Measuring the Impact of Welfare Reform With The Survey of Program Dynamics
This paper describes the origin, purpose, status, and plans of the Survey of Program Dynamics (SPD) (1). The focus of this paper is to describe the survey to researchers who will ultimately analyze the date.
WHY THE SURVEY OF PROGRAM DYNAMICS IS NEEDED
On August 22, 1996, President Clinton signed legislation passed by Congress, and the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 became Public Law 104-193. This comprehensive legislation has extensive implications for many programs. The law eliminates the open-ended federal entitlement program of Aid to Families with Dependent Children (AFDC), creates a new program called Temporary Assistance for Needy Families (TANF), which provides block grants for states to offer limited cash assistance, makes extensive changes to child care, the Food Stamp Program, Supplemental Security Income (SSI) for children, benefits for legal immigrants, (2) and the Child Support Enforcement program, modifies children's nutrition programs, reduces the Social Services Block Grant, and retains child welfare and child protection programs. The law also directs the U.S. Census Bureau to carry out a new survey to permit researchers to evaluate the impacts of the new law.
The goals of the welfare reform legislation are to end welfare dependence by promoting job preparation, work, and marriage, preventing out-of-wedlock births, encouraging the formation and maintenance of two-parent families, and providing states increased flexibility to achieve these goals. States are moving away from a welfare system based on entitlement towards ones focused on assistance in finding employment and self-sufficiency. Thus, the primary goals of the SPD are to provide information on spells of actual and potential program participation over a ten-year period, 1992 to 2001, and to examine the causes of program participation and its long-term effects on the well-being of recipients, their families, and their children. In the SPD, we are required to pay special attention to the issues of out-of-wedlock births, welfare dependency, the beginning and end of welfare spells, and the causes of repeat welfare spells. We must also obtain information about the status of children. The SPD will provide a convincing set of baseline data, assuming as we must that differential attrition will not vitiate the usefulness of the data collected. Researchers analyzing these data will use pre-reform characteristics of the population to control for preexisting differences among households in order to evaluate post-reform outcomes for the same people.
To address the issue of having sufficient pre-reform data, the legislation directed the Census Bureau to continue to collect the data necessary to evaluate the impact of the law from households previously interviewed in the 1992 and 1993 SIPP panels. (3) This survey, the Survey of Program Dynamics (SPD), will simultaneously describe the full range of state welfare programs along with social, economic, demographic and family changes that will help or limit the effectiveness of the reforms. Responding households will be followed annually from 1997 to 2002.
The SPD instrument has a core that essentially remains the same over the 1998-2002 period. Core data are collected on employment, income, program participation, health insurance and utilization, child well-being, marital relationships, and parents' depression. The SPD also has topical modules that vary. The self administered adolescent questionnaire is asked in 1998 and 2001. Additional child-related questions are asked in 1999 and 2002. Residential histories of children are asked once in 2000.
BACKGROUND OF THE SURVEY OF PROGRAM DYNAMICS
Several years before the passage of the actual legislation, the U.S. Department of Health and Human Services and the U.S. Department of Agriculture (USDA is responsible for the food stamps program) invested substantial resources in having the Census Bureau develop a survey to provide the data necessary to understand the effects of anticipated public policy changes on the population. In these planning activities, several design features emerged as essential. The survey should
o measure program eligibility and participation for the full range of welfare programs; money income, in-kind benefits, and services received from programs; employment, earned income, and income from other economic sources; family composition; and child outcomes, including key features of the environments of children (because reforms may have positive or negative consequences for children through these intervening mechanisms);
o be a large longitudinal, nationally representative study that measures changes in each of these areas and allows the identification of interrelationships linking these changes;
o include baseline data for a period before the initiation of reforms;
o continue to collect data throughout the period of reform to monitor the process of change; and
o collect data for the period after the states implement the reforms.
These requirements mean that we need a survey that casts a wide net, one that simultaneously measures important features of (1) both reformed and unchanged welfare programs, and (2) other important social, economic, demographic and family changes that will either help or limit the effectiveness of the reforms. Further, ideally, such a survey should be in place before reforms are effective to allow adequate assessment of baseline circumstances.
The data already collected in the 1992 and 1993 SIPP panels provide extensive baseline (background) information from which to determine the effects of welfare reform. SIPP is a longitudinal survey of households, interviewed at least nine times at four-month intervals, and followed if they have moved. The SIPP collects more detailed data than any other national survey on program eligibility, access and participation, transfer income, and in-kind benefits. Coupled with an extensive array of economic and demographic data (e.g., employment and job transitions, income, and family composition), the 1992 and 1993 SIPP panels serve to characterize the pre-reform situation of households quite well.
Further, the Census Bureau worked closely with policy agencies to develop and field topical modules that enhance the value of the basic SIPP data. Modules of special interest here include those on (1) education and training, (2) marital, fertility, migration, and program participation histories, (3) family relationships within the home, (4) work schedules, child care, child support, support for non-household members, (5) medical expenses and use of health care services, and (6) child well-being.
By interviewing the same households in the SPD, analysts will then have data for the baseline pre-reform period, the reform implementation period, and the medium-term post-reform period. Researchers require these data to assess short-term and medium-term consequences and outcomes for families and individuals. The use of both panels doubles the size of certain groups of interest, subject of course to the Census Bureau's ability to recontact households in the two panels and their willingness to participate. (Because the funding provided is not sufficient to interview all households in both panels past 1997, the Census Bureau is subsampling after 1997.)
The topics that the retrospective SPD covers are an extension of those covered by the SIPP, but placed in an annual survey questionnaire, using guidance from such annual surveys as the March supplement to the Current Population Survey, the Panel Study of Income Dynamics, and the National Longitudinal Surveys.
SPD STATUS AND PLANS
Current plans. We are planning to collect data six calendar years from 1996 through 2001. This will provide panel data for ten years (1992-2001) when combined with the 1992 SIPP panel data (and nine when combined with the 1993 SIPP panel data). (4) The Census Bureau's original plans were to have an instrument ready to field concurrently with welfare reform. Because President Clinton vetoed the legislation twice during 1995, the Census Bureau put its plans on hold. However, we were able to revive the project with funding rreceived late in 19XX. In its revived form SPD has three main components:
(1) the 1992 and 1993 SIPP panels that cover the 1992-95 time period;
(2) the 1997 Bridge survey provides the link between the 1992 and 1993 panels of the SIPP and the SPD and includes 'gap' questions to collect program participation data for 1995 for the 1992 SIPP panel;
(3) the core survey uses the bridge instrument already developed to collect annual retrospective data starting in 1998 plus:
* a teen self administered questionnaire in 1998 and 2001;
* extended child well-being questions in 1999 and 2001;
* a children's residential history module in 2000.
SPD "Bridge" Survey. Because the Census Bureau received funding for the SPD late in 1996, there was little time to develop a full-scale instrument. However, it was critical to collect income and program participation data in spring 1997 for calendar year 1996 from as many of the 1992 and 1993 SIPP households as the Census Bureau could find. Waiting any longer ran the risk of losing too many households (or not finding them) and missing the opportunity to collect data for 1996. Data for 1996 were collected in April-June 1997 by administering a modified version of the annual March 1997 Current Population Survey (CPS) demographic supplement, with a few new questions designed to collect summary 1995 data for the 1992 SIPP panel (who were last interviewed in January 1995).
All SIPP persons interviewed in the first wave of the 1992 and 1993 SIPP panels and still being interviewed at the end of the panel (9 waves) were eligible for the SPD bridge sample. The Census Bureau decided not to try to find all persons in the 1992 and 1993 panels who left the survey (attrited) before the end of the SIPP because of the difficulty and cost involved in trying to find those people (who had already declined to participate even after repeated attempts to interview them) and because they felt that most analysts would need as much baseline data as possible. We will reweight the remaining sample to compensate for this loss.
Finding people who move is critical to the success of any longitudinal survey, particularly one as focused on the low-income population as SPD. Naturally, SIPP has developed many procedures that help find movers. Luckily, the CPS implemented a "mover module" in January 1997 to track people leaving formerly-interviewed CPS households. This was crucial to the SPD, especially given the time that had elapsed since the last interview, though the CPS approach did not fully meet the needs of the SPD. The Census Bureau tracked most movers and we will attempt to interview in 1998 those that we know about but could not interview in 1997.
1998 SPD. During the first half of 1997, the University of California at Berkeley translated the SPD instrument (5) into computer code. They are the developer of the CASES authoring language used for computer-assisted Census Bureau surveys. The Census Bureau carried out a pretest in October 1997 using 400 retired 1996 CPS households in four of its regional offices. After this test, the Census Bureau modified the questionnaire to correct problems, and separated the self-administered, audio cassette adolescent questionnaire into two separate supplements-one for those youths 12 to 14 years old and the other for those 15 to 17.
Using the fully-developed computer-assisted personal interview instrument, annual data collection will occur once each year in May and June (6), with annual recall for the preceding calendar year. The 1998 survey will include a set of retrospective questions covering 1997 for all persons aged 15 and older in the household. The topics covered are
o Basic demographic characteristics, including educational enrollment and work training, functional limitations and disability, and health care use and health insurance;
o Basic economic characteristics, including employment and earnings, income sources and amounts, assets, liabilities, and program eligibility information, and food security;
o Information about children, including their school enrollment and enrichment activities, disability and health care use, contact with absent parent, care arrangements, and payment on child support of their behalf; and
o Two self-administered questionnaires - a short question sequence for adults focusing on marital relationship and conflict and a depression scale, and a relatively lengthy questionnaire for adolescents aged 12 to 17 focusing on such issues as family conflict, vocational goals, educational aspirations, crime-related violence, substance abuse, and sexual activity (developed in collaboration with the Child and Family Research Network).
1999 SPD and later. Work is finishing up to identify the topics for extended child well-being questions to be asked in 1999 and 2001. The Census Bureau plans to focus on elements that allow analysts to measure changes from pre-reform periods or that illuminate other mechanisms affecting outcomes. We are currently planning to collect data on where the children have lived and the reasons for any absence from the parent in the 2000 SPD. Also under investigation are question variants to address the changing nature of welfare programs in the 50 states and the District of Columbia. We are also considering extending the core questions and adding other topical modules.
SPD CONTENT (7)
The SPD has two components and topical modules. The two components consist of adult questions and child-related questions. The topical modules include the teen self-administered questionnaire, the childrens' residential history, and extended childrens questions. The Adult Questions cover:
Employment and earnings, income, and eligibility. One of the primary goals of SPD is to chart the economic well-being of families over time. For each person age 15 or over in the household, we collect whether they are currently working and a detailed account of work-related activities in the past calendar year, including weeks they worked, weeks on layoff, and weeks spent looking for work. We collect information for up to four jobs in the previous calendar year including annual earnings from each job.
Next we collect income data, including data on cash and non-cash transfer programs. In addition to measuring economic well-being, these questions help examine changes in the characteristics of transfer programs resulting from welfare reform. These questions are similar to SIPP. We first do an inventory of all the types of income received in the previous calendar year for all household members age 15 or over at the person level. We ask, for example, "Did anyone in this household receive unemployment compensation?" If they answer "yes," we ask who received it. After collecting this inventory, we move on to the "Amounts" section. In this section we will collect the months during the previous calendar year that the person received the income as well as the amount the person received. It is important to track the number of months the person received the income so that researchers can examine the impact of time limits associated with welfare reform.
In addition to income, we ask about assets and debts. These include items such as the value of homes, cars, stocks, bonds, mutual funds, and payments made to support children not living in this household. These questions are included because assets and debts are a key component to economic well-being and are also critical to measuring program eligibility.
Educational enrollment and work training. Questions on educational enrollment are asked of persons 18 and over about the current academic year. Persons 15 to 17 are included in the children's school enrollment questions since the children's series of questions are more appropriate for that age group. The adult questions on educational enrollment track the progress of adults toward receiving high school or high school equivalency degrees as well as college and graduate degrees. The work training questions focus on whether adults received any work training designed to help them look for a job or to train them for a new job. Both educational enrollment and work training are important activities to monitor since participation in these activities is tied to eligibility for receiving welfare benefits in some states.
Disability, health care utilization, and health insurance. Questions on disability, health care utilization and health insurance are condensed versions of similar series included as topical modules in SIPP. These questions are included to measure changes in the health care system and how the changes affect accessibility to government health insurance such as Medicaid and Medicare as well as private and employer-provided insurance, utilization of health services, and outcomes.
Food security. This series of questions is a shortened version of the USDA-sponsored Food Security Supplement to the CPS and is intended to measure the subjective experience of hunger. The questions are used as a scale to measure the severity of hunger in a household. Direct changes in the Food Stamp program account for nearly half the total Federal cost savings under the legislation. Additionally, the food stamp benefits of legal immigrants and able-bodied persons ages 18-50 years old without dependents are affected by changes in the legislation. Measuring the food security status of household provides a direct indicator of well-being.
Child-related questions are described in more detail in the paper by Kominski and Bass (1998) so are only summarized here. We attempt to interview the mother or designated parent when asking questions about children. We ask questions about school enrollment and enrichment activities, including extracurricular enrichment activities which add to the overall portrait of the child's development. The extent to which parents have the financial resources or the time to devote to such activities may be strongly influenced by their program participation and employment in the labor market. These questions provide the basis to study how welfare reforms affect key child outcomes by influencing children's exposure to enriching activities.
We also ask about disability and health care utilization, similar to those asked about adults. A key objective of welfare reform is to encourage single mothers to enter or re-enter the labor force, so child care is an important issue to measure. The amount of time children are in child care and the type of care they receive, as well as the stability of care arrangements has been linked with child well-being.
Enforcement of child support and contact with absent parent has been highlighted as a cornerstone of welfare reform. These questions will allow researchers to examine the nature of the awards and whether the awards are being followed or enforced. Child support payments are also an important factor in determining the economic status of children living in single-parent families.
Another objective of welfare reform is to encourage marital and family stability. Answers to questions on these subjects provide indicators of marital happiness and of overall stress that can contribute to marital and family harmony or instability. Changes in program participation and employment can have fairly large and immediate consequences for marital and family stability.
The adolescent self administered questionnaire (SAQ) is modeled on the Youth Behavior Survey conducted by the Census Bureau in 1992. The SAQ asks about housework and chores; family routines, parent-child relationships, participation in the labor force, parental monitoring, and contact with absent parents, similar to the series administered to the mother or designated parent. TANF prohibits states from giving benefits to unwed teenage parents not living with an adult and attending school, which may reduce drop-out rates. Or parental employment may reduce parental monitoring and support and might undermine education. Other topics include questions on substance abuse, early sexual initiation, contraception, and childbearing. TANF allows family cap provisions that prohibit women on welfare from receiving additional benefits when additional children are born. It also establishes an abstinence education program, gives states bonuses for reducing illegitimate births and permits states to spend money on family planning. It also seeks to reduce the involvement of teen girls with older males.
The teen SAQ will be asked in the 1998 and 2001 SPD. In 1999 and 2002, we plan to ask more extensive questions on adult substance abuse and extended questions on child well being. We are also considering expanding the core instrument to included questions on child care
employee fringe benefits, health care utilization while uninsured consumer expenditures (8) for food, transportation, school, and health, months receiving rental assistance, use of food banks, (9)coping mechanisms when losing benefits, and noncach child support.
This section deals with technical issues that affect the administration and usefulness of the survey.
Subsampling. The response rate to the 1997 bridge survey, 81.7 percent, was good, given the time that had elapsed since the prior interview. This yielded a sample of 30,125 interviewed households. The budget for the survey, $10 million per year, will allow us to interview about 18,500 households in 1998, given the projected length of the interview. Thus we subsampled the interviewed households, based upon household characteristics as of the bridge survey, as follows:
o all households with income less than 150 percent of poverty, and all households with incomes between 150 percent and 200 percent of poverty with children are selected for the sample with certainty;
o households with incomes above 200 percent of poverty with children are subsampled at a 90 percent rate;
o households, with incomes between 150 percent and 200 percent of poverty without children are subsampled at an 82 percent rate;
o households with incomes above 200 percent of poverty without children are subsampled at a 27 percent rate. (10)
Weighting and database development. The SPD is designed exclusively to support longitudinal analysis of the impact of welfare reform. It represents the behavior and characteristics of persons in two fixed cohorts over a period up to 10 years. One cohort represents the population as it existed in March 1992 from the 1992 panel of SIPP and the other the population as of March 1993 from the 1993 panel. This is not a traditional longitudinal survey in that it does not repeat the same measure through out the period. Furthermore, the sample is divided into two parts, each with a different reference period. Finally, the individual rounds of interviewing beginning with the bridge are not and cannot be made to look like cross-section snapshots of the U.S. population.
Given the design and our budget constraints, we have designed a series of data products to focus scarce resources on just the data products the users need most. (11) These are:
Longitudinal Data. The core information common throughout the data collection (although with varying reference periods and question phrasing) consists of basic demographics, labor force activity, income, and program participation. Therefore, we will produce a series of longitudinal files reflecting these data, and reformatting them to be consistent across time. We have chosen the variable names reference period based on the March Current Population Survey (CPS) public use files because of the CPS's popularity and its use as the basis for designing the SPD questions.
Files containing these longitudinal variables can be supplemented with information collected in the SPD and in SIPP that did not appear in the bridge or SIPP core surveys. The added SPD information will be processed longitudinally if it was repeated across SPD interviews, as was true for the self-administered adolescent questionnaire collected in 1998 and to be repeated in 2001. Other supplemental information, like the residential history that only appears in the 2000 survey, will only be processed cross-sectionally.
The resulting longitudinal products to be issued will resemble the SIPP longitudinal files in format in that they will each be a person file, they will reflect the original cohort plus persons with whom these people residedlongitudinal variable will be replicated ten times reflecting the ten potential years of repetitions of the questions. The information repeated across several SPD surveys will appear following the data common across the SIPP panels, the Bridge Survey, and the SPD. The additional SPD data will appear at the end of the persons record. Depending on size constraints we may need to subdivide the person-level information across two or more files that can be easily linked. Additional SIPP information will be accessible from other files that can easily be linked to the SPD longitudinal files. (12)
For the SPD longitudinal data, person weights will be developed for persons belonging to the initial cohorts who continued to be interviewed for as long as they remained with the SIPP/SPD universe as defined by the 1992/1993 SIPP panels (Source and Reliability Statement, forthcoming). We refer to this group of people as the longitudinal sample. Other people to be included in the data file will have zero weights (with the exception of persons born to a person in the longitudinal sample.) Their presence in the data file is to facilitate development of household and family characteristics of the persons in the longitudinal sample. This will permit the user to construct contextual information on the cohort sample members' household and economic circumstances.
Persons born to original sample members after the two cohorts were defined, will be included in the longitudinal file and assigned a nonzero person-based longitudinal weight and so they will appear to be members of the longitudinal sample (even though technically they are not). Thus, the data files will reflect a dynamic population over the 10-year period rather than two fixed cohorts of the population.
The first longitudinal file to be produced will have data covering the 1992/1993 through 1998 time frame. Data from the 1992 and 1993 SIPP panels, the 1997 bridge survey, and the 1998 and 1999 SPD surveys will be used for this first longitudinal file. The second longitudinal file will repeat the first and extend the reference period to 1999 (adding in the 2000 SPD) and the last file will add in data for 2000 and 2001 collected in the 2001 and 2002 SPD.
Interim Calendar Year Files. So that the research community can assess the quality of the SPD sample and make some preliminary assessment of the immediate impact of welfare reform, we are issuing an interim file from the 1997 bridge survey and will issue calendar year files for each collection year. These interim files will be identical to the March CPS in format. It will repeat the March CPS content and include additional information from the SPD. This additional information includes special weights corresponding to the cohort, link-variables needed to merge SPD and earlier SIPP data for analysis, and new SPD data starting in 1998.
A note of caution about these interim files. They will appear to be a cross-section snapshot of the population, but it is important to emphasize that they are not. They are simply one segment in the array of longitudinal data being produced for SPD. The weights are longitudinal and only valid to decipher 1996 characteristics, for example, of the 1992/1993 cohorts. Newborns do not receive longitudinal weights in this first file so the distribution population by age will be skewed toward the older population. However, users can construct an approximate weight for them by assigning the weights of the mother to each one.
We recognize that users may have some difficulty using these file so we have taken some steps to make it easier. First, we will be producing benchmark results for comparison to March CPS, as discussed below. We will compare the information on program participation and income across the March 1997 CPS and the Bridge survey. Since the longitudinal weights on the Bridge survey file will not be comparable to the March CPS cross-section weights, the comparison will be based on normalized weights and weights assigned to newborns from their mother. We will present these results in the context of a quality profile along with other information on sampling and nonsampling errors in the SPD.
SPD Bridge Data Tabulations
Using data from the 1992 and 1993 SIPP panels and the 1997 SPD Bridge file, we will compute simple tabulations on income recipiency for employment income, income from government transfer programs (non-means tested and means-tested) and retirement income. In addition to these tabulations, the tables will include a section on health insurance coverage by type of coverage. SIPP and SPD follow people over time and thus a person based analysis is appropriate and will be used for our initial analysis. Since SIPP is a person (rather than household) based survey, it is computationally more convenient to analyze these data at the person level.
There are four major household characteristics by which persons will be tallied: all households, married couple households, other-family households, and non-family households. Major categories are refined further by appropriate sub-groups. For married-couple and other-family households, we will present tabulations for all households as well as for those households with children and those without children. Non-family households are displayed by total households and by gender of the head. The universe for these tabulations is limited to those people who form the SPD sample as described above. (13)
Since the SPD weight is a longitudinal weight, using simple weighted counts would not provide meaningful results. As an alternative, tabulations are based on normalized weighted counts (14) for each income source and health insurance coverage concept. This is accomplished by dividing each persons SPD weight by the average SPD weight for all persons. The use of normalized weights is to allow some comparison to CPS. This procedure provides analysts with an adequate distribution with which to examine and compare data from the SIPP and SPD calendar year distributions.
Supplementary data (15)
Data on welfare reform can be divided into three broad groups: databases based on state administrative data, in-depth state studies, and national data. Each source of data is important in helping us to understand the impact of welfare reform. For example we know from information in state databases that the number of families receiving cash assistance nationwide decreased 30 percent between January 1994 and September 1997 and two-thirds of this decrease occurred since January 1996 (GAO, 1998). From in-depth state studies, we know that states are reducing their caseloads by increasing their job placement rates. However, little is still known about the impacts of welfare reform on families leaving welfare with respect to economic stability and child and family well-being (GAO, 1998). National surveys, such as the SPD, will provide this type of information. Information from state studies and state databases can be used to supplement national data. Information in state databases that are defined using common dimensions or descriptive factors are possible explanatory variables in investigations of outcomes using national survey data.
There are several sources of supplementary information that can be used to examine state-by-state changes as well to enhance analysis of national data. Information that states must report about their programs to the U.S. Department of Health and Human Services (DHHS) are likely to have only the minimum necessary to satisfy the requirements of the legislation and therefore will probably be insufficient on their own for research purposes. Nevertheless, one can use this basic "tracking" information to tell some basic stories. HHS is also responsible for developing a formula to reward 'high performing' states. For 1999, high performing states are defined by job placements, job retention and wage progression. This information, in conjunction with national survey data, will be useful for evaluating the relationship between high performance states and family well-being. (See reference list for DHHS studies.)
The General Accounting Office has completed a report that focuses on how states are restructuring their welfare programs to meet the objectives of TANF specified in the law. The report provides a 50-state overview and an in-depth review of seven states--California, Connecticut, Louisiana, Maryland, Oregon, Texas, and Wisconsin (GAO, 1998).
The Urban Institute, in collaboration with Child Trends, Inc, and Westat, Inc, is conducting a multi-faceted study "Assessing the New Federalism (ANF)."
The project includes a National Survey of American Families (NSAF), a survey of 50,000 households that monitors the well-being of low income children and adults during and after welfare reform using several waves of interviewing. The first wave was collected in 1997 and additional waves are planned for 1999, and if funding is available, for 2001 and beyond. The ANF project also includes a database with information on all states and in-depth studies of 13 states. The in-depth studies include interviews with program managers to determine how they are implementing the new law (Gallagher et al, 1998).
Mathematica Policy Research (MPR) is using existing administrative data and SIPP data to create a microsimulation model capable of projecting the new welfare law's impact on costs, caseloads, distributional effects, and other outcomes. The Manpower Demonstration and Research Corporation (MDRC) is conducting the "Devolution and Urban Change Project." They are examining the impact of welfare reform on families living in economically depressed neighborhoods in four cities (Cleveland, Miami, Philadelphia, and Los Angeles). They are linking changes in the 'safety net' to agency practices to study the outcomes on low income families. The study is using surveys, ethnographic research, and administrative records (Besharov et al, 1997).
Several universities are conducting their own studies. The Northwestern University/University of Chicago Joint Center for Poverty Research (called the Poverty Center) has formed a national advisory panel to pursue development of research-ready data from administrative sources to be used for poverty research. It is reviewing administrative data to examine ways of improving its quality for research. Their goal is to guide welfare reform research to become more policy friendly and thus facilitate the use of data in decision making (Besharov et al, 1997). Columbia University's School of Public Health, the University of Wisconsin Institute for Research on Poverty, the University of Maryland, and other universities are conducting their own research. Some studies are at the national level while others are state-specific. The University of Maryland, for example, is tracking a random sample of families that exited welfare during the first 12 months of the state's welfare reform program (October 1996 to September 1997). Columbia University is planning a large scale evaluation of an intervention for substance abusers, one of the most difficult to serve groups with respect to welfare (Columbia, Spring 1998).
The Survey of Program Dynamics will provide national data and trends on welfare reform and provide both a pre-reform baseline and extensive post-reform information. Other national surveys that will be also used to study the impact of welfare reform include the 1996 Survey of Income and Program Participation (SIPP), the Panel Study of Income Dynamics(PSID), started in 1998, and the National Longitudinal Survey of Youth (NLSY)), begun in 1979. These national surveys provide data on income and program participation. The latter two are useful for studying intergenerational program participation.
The Survey of Program Dynamics is an important source of national data on welfare reform. Results from the SPD will help researchers understand the impacts of welfare reform on the well being of families and children. We will know if welfare recipients find jobs, the quality of the jobs, and the types of support they need to make the transitions from welfare to work. We will also be able to study those families and individuals who have left the welfare rolls by choice or due to time limits but are not working. Finally, we will be able to study the effect of moving from welfare-to-work on the well-being of families and children.
Besharov, Douglas, Peter Germanis, and Peter Rossi, Evaluating Welfare Reform, A Guide for Scholars and Practitioners, University of Maryland, School of Public Affairs, 1997 (The quote from Moynihan is from this book.)
Brick, Michael et al. "Analysis of Nonresponse and Coverage Adjustment Procedures for a Dual Frame Survey," Proceedings of Statistics Canada Symposium XIV, Ottawa, 1998.
Columbia School of Public Health, National Center for Children in Poverty, Research Forum on Children, Families, and the New Federalism, Vol. 1, No. 2, May 1998 and News & Issues, Vol. 8, No. 1, Spring 1998.
Gallagher, Jerome L., Megan Gallagher, Kevin Perese, Susan Schreiber, Keith Watson, One Year After Welfare Reform: A Description of State Temporary Assistance for Needy Families (TANF) Decisions as of October 1997, The Urban Institute, Assessing the New Federalism, Occasional Paper, No. 6
Hess, Jennifer and Jennifer Rothgeb. "Measuring the Impact of Welfare Reform: Issues in Designing the Survey of Program Dynamics Questionnaire," Proceedings of Statistics Canada Symposium XIV, Ottawa, 1998.
James, Tracy M. "Results of the Wave 1 Incentive Experiment in the 1996 Survey of Income and Program Participation," 1997 Proceedings of the Survey Research Methods Section of the American Statistical Association, 1998.
King, Karen, et al., "SPD Source and Reliability Statement," forthcoming.
Kominski, Robert, and Loretta Bass, "Developing Children's Data for the Survey of Program Dynamics," presented at American Statistical Association meetings, Dallas, TX, August 1998.
Manpower Demonstration Research Corporation. Biannual Report 1996/97. 1998.
U.S. Census Bureau, Survey of Income and Program Participation Users' Guide, Second Edition, Washington DC, 1991.
U.S. Department of Health and Human Services, Office of the Assistant Secretary fro Planning and Evaluation, Building and Employment Focused Welfare System: Work First and Other Work-Oriented Strategies in Five States, prepared by the Urban Institute under contract number HHS-100-95-0021, June 1998
U.S. Department of Health and Human Services, Office of the Assistant Secretary and Planning and Office of Human Services Policy, Aid to Families with Dependent Children, the Baseline, June 1998.
U.S. Department of Health and Human Services, Office of the Assistant Secretary and Planning and Administration on Families, National Evaluation of Welfare-to-Work Strategies, Evaluating Two Welfare-to-Work Program Approaches, May 1998.
U.S. Department of Health and Human Services, Indicators of Welfare Dependence, Annual Report to Congress, October 1997.
U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Building an Employment Focused Welfare System: Work First and Other Work-Oriented Strategies in Five States, conducted by the Urban Institute, June 1998
U.S. General Accounting Office (GAO), Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence, GAO/HHES-98-109, June 1998
Weinberg, Daniel, Huggins, Vicki, Kominski, Robert, and Nelson, Charles,. "A Survey of Program Dynamics for Evaluating Welfare Reform," Proceedings of Statistics Canada Symposium XIV, Ottawa, 1998.
1. This paper draws from the paper by Weinberg, Huggins, Kominski, and Nelson (1998).
2. Since changed by the Balanced Budget Act of 1997.
3. Some might ask why we are not using the 1996 SIPP panel to examine the impact of welfare reform. Only some of the interesting questions can be answered using the 1996 panel of the Survey of Income and Program Participation (SIPP). The SIPP will provide nearly four years of longitudinal data for almost 37,000 households beginning in April 1996 . Welfare reform legislation took effect on October 1, 1996, only six months after the 1996 SIPP panel started. One could argue that this pre-reform information is sufficient for many analyses, and the SIPP does try to collect retrospective program participation information. However, other analyses need a longer pre-reform period to accurately measure some initial condition; this need is particularly acute for this research because many (if not most) states had already begun to make changes under federal program waivers well before the beginning of the SIPP panel. A welfare reform module is underway for wave 8 of the 1996 SIPP panel. Data are being collected between August and November 1998. More details about the SIPP can be found in U.S. Census Bureau (1991); a third edition is currently being prepared. See also <http://www.sipp.census.gov/sipp/>.
4. The SPD longitudinal files will have ten years of data for the 1992 SIPP panel and nine years of data for the 1993 SIPP panel. We are considering whether to impute 1992 data to 1993 observations or to analyze the 1992 and 1993 cohorts separately.
5. For more information on the design process for the SPD, see Hess and Rothgeb (1998).
6. Data collection was delayed by one month in 1998 and therefore is occurring in June and July.
7. Much of this section is from Hess and Rothgeb (1998).
8. The USDA food and nutrition assistance programs need more information on the family management processes that link income and food assistance programs to food access. SIPP and SPD provide information on income and program participation, and, recently, on food security. But to understand how these relate, they need to understand better the relationship of spending on food to spending for other household needs.
9. Food banks and soup kitchens are the mainstays of the emergency food system, complementing the Federal food and nutrition assistance programs. There is very little understanding of the role of these enterprises vis-a-vis USDA food programs. USDA wants to understand better their role and whom they serve - especially the overlap or complementarity with the Food Stamp Program and other USDA programs.
10. Also sampled at 27 percent were individuals who had been institutionalized at some time after the beginning of the SIPP. If they are out of the institution, they will be interviewed; if they are not, they will be tracked.
11. See also <http://www.sipp.census.gov/spd/>.
12. SPD data products will contain the SIPP public use version of the person identifiers in the 1992 and 1993 panels of SIPP (sample unit identifier, entry address identifier, and person number). Furthermore, we are supporting efforts at the Social Security Administration to develop reformatted SIPP data from the 1992 and 1993 panels. This reformatting will consist of recoding information on the SIPP longitudinal files so that it corresponds to an annual reference period for income and program participation. The reformatted files will closely resemble the March CPS data files.
13. In order to properly select
the appropriate sample, we will select individuals based on their
14. Sometimes referred to as weighted-deweighted counts.
15. See reference list for sources of information on this supplementary data.