U.S. Census Bureau Homepage

Survey of Program Dynamics


 

 Tracking Devolution: The Survey of Program Dynamics
 

 Robert A. Kominski
 Population Division
 U.S. Bureau of the Census

 Daniel H. Weinberg
 Housing & Household Economics Division
 U.S. Bureau of the Census

 Paper presented at the
 Population Association of America
 Annual Meeting

 March 29, 1997
 Washington, D.C.
 

 The views and conclusions expressd in this
 paper are those of the authors, and do not
 necessarily reflect those of the Census Bureau.

Tracking Devolution: The Survey of Program Dynamics

 Robert A. Kominski
 Daniel H. Weinberg


Introduction

The Census Bureau is currently in the field collecting data for a new study, the Survey of Program Dynamics.  The project has been brought to life as a result of the passage of welfare reform legislation developed during the 1996 Congress.  The goal of the survey is to track some of the long-lasting effects of the decentralization of the welfare system in the United States, and the social, economic and demographic impacts these changes will  have on families, adults and children over time.  The initial impetus for the survey did not arise from the welfare reform act, but in a desire to develop a vehicle for monitoring long-term changes in families.  Preparatory work done during the past few years laid the groundwork for the Bureau to capitalize on the opportunity created by the passage of the welfare reform bill.  In this paper, we discuss some of the background leading up to the development of the survey, the basic design as currently formulated, the main components of the survey design, some of the technical issues we have had to resolve thus far, and our current status.

Background

The beginning of the Survey of Program Dynamics was located in a small group of Census Bureau researchers who felt that there was a need for a long-term survey vehicle to monitor social, demographic and economic change in the U.S. population.  During the early 1990's  the Census Bureau was actively engaged in the redesign of the Survey of Income and Program Participation (SIPP), the agency's primary vehicle for measuring longitudinal economic conditions in U.S. households.  Among the issues being seriously discussed at that time was the appropriate length of the new longitudinal panel.  Panels conducted since 1984 (the start of the SIPP program) had lasted, for the most part, about 32 months each, with data collected for each month of observation, and a new panel gegun at the start of each calendar year.  Some researchers (both within and outside the Bureau) had argued that the 32-month period was not fully effective in monitoring the kinds of phenomena the survey was supposed to focus on, for example, job activities, spells of unemployment and program participation.  The National Academy of Sciences, in its review of the SIPP program, had urged the Bureau to consider lengthening the longitudinal observation period.  After much deliberation and research, the Bureau ultimately decided on a new non-overlapping panel design of 4 years observation, with the first new panel slated to begin in 1996.

Despite this decision, there was still a feeling among some researchers that even a 4-year period of observation would not be highly useful for studying many of the social and demographic changes associated with changing economic conditions households experience over time.   In early 1993 a group of Bureau scientists began discussions about a possible ‘extended' SIPP panel which would follow respondents from a conventional SIPP panel for some additional amount of time.  The result of the work of these individuals and their discussions with colleagues at other agencies, especially Health and Human Services, was that there was interest in the development of an ‘extended SIPP panel'.    Interest was particularly strong regarding the long-term effects of economic conditions and changes on the children in these households.  During 1993 and 1994, several federal agencies, Health and Human Services, and Food and Consumer Services in the Department of Agriculture, began to reinforce their interest with the appropriation of small amounts of  money to the Census Bureau to try to develop a new long-term survey instrument.  Other sources of interest, focused through NICHD's Family & Child Well-Being Research Network, a national collaboration of academic and other researchers working on children's issues,  began providing background material for a wide array of issues that might be addressed in such a survey.

Throughout the development and planning process, a number of substantive topics emerged as important for the data collection activity:

    o  The survey should measure program eligibility and participation for the full range of welfare programs.
    o  The survey should measure the money income, in-kind benefits, and services received from programs.
    o  The survey should measure employment, earned income, and income from other economic sources.
    o  The survey should measure family composition.
    o  The survey should be a large, longitudinal, nationally-representative study that measures changes in each of these areas
        and allows the inter-relationships linking these changes to be identified.
    o  The survey should include baseline data for a period prior to the initiation of welfare reforms, and continue to collect data
        throughout the period of reform to monitor the process of change, and for the period after the reforms are implemented to
        assess short-term to medium-term consequences or outcomes.
    o  The survey should measure child outcomes including measurement of key features of the environments of children,
        because it is through these intervening mechanisms that reforms will have positive or negative consequences for child
        outcomes.

By the fall of 1994, discussions had proceeded far enough that the rough goals of the Survey were becoming clear.  Essentially, the design would focus on two primary elements:

 1) Providing information on actual and potential program participants over the period
     1993-2002    (Thus, assuming the extension of the 1993 SIPP panel)

 2) Examining the consequences of program participation on the well-being of recipients ,
      their families, and their children

At the beginning of 1995, the working team at the Bureau had assembled a large amount of questionnaire material for consideration as the content of the SPD.  One large portion of this was essentially derivative of the content and topic coverage of the SIPP - since the SPD panel was to be an extension of a retired SIPP panel, this made obvious sense.  The second assortment of materials were a series of questions submitted by various experts on children's research issues.  The primary task before the team now was to try to bring this varied collection of materials into a single coherent document that met the survey goals.  Throughout the early months of 1995, many decisions were made about content scope and design, working toward the tentative goal of a pretest in the spring of 1996 and implementation in the spring of 1997.   During this time, it also became apparent to the steering committee for the SPD that the full proposed scope of question content for the survey could not be realistically administered.  For this reason, the survey was broken into two parts.  The ‘track 1' portion would be the annual portion of about an hour that would focus on issues such as employment, financial conditions and program activities, with some questions on children's statuses.  The ‘track 2' questionnaire would be administered annually also, but would begin one year after ‘track 1' (because of needed additional development time).  This part of the study would focus specifically on children, using more detailed psychological and other assessment measures.  A meeting held with a group of external reviewers in mid-1995 confirmed the decision to separate and refine the two distinct questionnaires.

Throughout 1995, development work proceeded somewhat cautiously, because the study still had no real sponsor.  During 1994 it had become apparent that possible welfare reform legislation might be the vehicle which would create the funding opportunity for the SPD, but with no clear legislation in sight, it was possible only to prepare contingency plans for conducting the survey.  In 1995 it seemed possible that the Congress might develop some welfare reform legislation that the President would sign.  We continued to use the small resources available to move forward and prepare.  However, by late 1995, with no bill passed, and no additional source of funding available, the study was reduced to finalization of the track 1 CAPI instrument specifications and some cognitive pretesting of the adolescent self-administered questionnaire for track 1.  The SPD began a ‘hiatus' that would last until mid-1996, when we received word that a new welfare reform bill had been introduced, one which had greater chances of becoming law than the previous incarnation.  We began focussing on the problems of operationally taking on the survey, since the likelihood of funding had become much more elevated.  In the fall of 1996 we received notification that the President had signed legislation passed by the Congress, and the Personal responsibility and Work Opportunity Reconciliation Act of 1996 was enacted as Public Law 104-193.  This bill directs the Census Bureau to:

        "continue to collect data on the 1992 and 1993 panels of the Survey of Income and Program Participation [SIPP] as
        necessary to obtain such information as will enable interested persons to evaluate the impact of the amendments made by
        Title I of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 on a random national sample of
        recipients of assistance under State programs funded under this part and (as appropriate) other low income families, and
        in doing so, shall pay particular attention to the issues of out-of-wedlock birth, welfare dependency, the beginning and
        end of welfare spells, and shall obtain information about the status of children participating in such panels." [Section 414]

With passage of the public law, ten million dollars were annually appropriated to the Census Bureau for each of the fiscal years 1996-2001, and the Survey of Program Dynamics moved from being an idea to an authorized and funded program.

Survey Design

The basic design of the SPD uses the retired 1992 and 1993 SIPP longitudinal panels as the base sample.  All households which were in the initial (wave 1) interview of these surveys, and which were still in sample at the panel's conclusion, are eligible households for inclusion in the first interview of the SPD.   The use of both retired samples (approximately 20,000 households each) gives us a large initial sample for the SPD.  Given likely difficulties in locating many households, and possible nonresponse from some households who felt their initial 2 ½ years in the SIPP are all they are willing to give, this very large sample should provide us a good resultant sample with which to continue in subsequent years.  This should also allow us latitude to conduct optimized subsampling in the later years, particularly since funding levels will likely not allow us to continue with a full 40,000 household design,

The plan to continue data collection for both the 1992 and 1993 SIPP panels  provides extensive baseline (background) information from which to determine the effects of welfare reform.  Regarding program data, the 1992 and 1993 panels collected more detailed data than any other national survey with regard to program eligibility, access and participation, transfer income, and in-kind benefits.  For economic and demographic data, the 1992 and 1993 panels collected very detailed data on employment and job transitions, income, and family composition.

From the very start of the SIPP, the Census Bureau also has worked closely with policy agencies to develop and field topical modules that enhance the value of these SIPP core data.  Modules fielded in both the 1992 and 1993 panels of special interest here include those on (1) education and training histories, (2) marital, fertility, and migration histories, (3) migration history, allowing the identification of immigrants and citizenship status, (4) detailed family relationships within the home, (5) work schedules, child care, child support activities, and financial support for non-household members, (6) medical expenses and utilization of health care services, (7) work disability histories, (8) extended measures of child well-being, and, (9) measures of social support networks.  Thus, the data collected as part of the original SIPP samples provides an incredibly rich soil in which to plant the seed of a new survey designed to track welfare reform as it takes effect.  By interviewing the same households in the SPD, analysts will have data for the baseline pre-reform period, the reform implementation period, and for the medium-term post-reform period.  All of these data are required to assess short-term and medium-term consequences and outcomes for families and individuals.

Survey Components

Current plans are for data to be collected for each of the seven years for 1996 through 2001, providing an ultimate panel of data for nine years (1992-2000) when combined with the 1992 SIPP panel, and eight years (1993-2000) when combined with the 1993 SIPP panel.  Due to the delay in the start of the SPD, we have had to develop a mechanism for filling in the space between the end of SIPP observation, and the start of routine SPD observation.  Thus, as currently configured, the survey has three fundamental pieces; (1) the ‘bridge' survey to be fielded in the spring of 1997 (April-June) only; and providing the link between SIPP and SPD; (2) the ‘track 1' survey, to be administered yearly beginning 1998 (for reference year 1997)  and, (3)  the ‘track 2' survey to be administered yearly starting 1999.

Creation of the ‘bridge' survey was forced by the hiatus of the program during 1996, when funding did not emerge, and virtually all planning and development work had to stop, and necessitated by the need to locate the retired SIPP panel households and collect information for them before too much time had passed.  When it became clear in late 1995 that we would not have funds to continue the project, most work on the project, including CAPI development of the instrument, stopped.  By mid-1996, when it appeared that passage of the welfare reform bill was more likely, we had to find a way to bring the project back on line quickly.  The SIPP panels of 1992 and 1993 had ended their survey lives in late 1994 and 1995, respectively.  We would not have enough development time to fully author and test the instrument to get it in the field for early 1997; however, in order to provide data continuity with the expired panel, we needed a mechanism to get data for 1996.  We were also concerned that the longer we waited to reestablish contact with households, the more problems we would have with movers and tracking them down (about 17% of all persons in the U.S. move each year).  This could be particularly a problem in SPD, since SIPP rules follow all members of a household, even if different members go different places.  Consequently, it became important that we get some instrument into the field sometime during 1997.

The solution we have implemented is to use a modified version of the annual March Current Population Survey demographic supplement to collect information for 1996.  By making only a relatively small set of changes to the CPS instrument (already in CAPI form), we are able to bring the SPD online in the field this spring.  (The major change is the addition of a few questions to get some summary calendar-year data for 1995 for the 1992 SIPP panel- who were last interviewed in January 1995.)  While the use of the CPS supplement has not been effortless, it has required far fewer resources than attempting to field a brand-new survey would have needed.  Use of the CPS supplement should mean less intensive content training for the field staff, and allow them to focus on the more critical problem (at this point) of tracking the respondents down.  We anticipate fielding the bridge survey in the period of April-June of this year.  The  Office of Management and Budget has approved an incentive payment of $20 for some households, as part of a test of incentive payments to reduce nonresponse.
 
The second ‘piece' of the SPD is the basic yearly survey we anticipate using for the years 1998-2001, which we refer to as ‘track 1'.  Remember that early on in the development process, an organizational decision was reached to separate the routine yearly collection of basic job, income and program information, as well as some basic child and social condition data, from the more detailed developmental information regarding children (‘track 2').  In fact, the  ‘track 1' survey  itself has several distinct sections.  The first of these consists of a set of retrospective questions (referenced to the previous calendar year), for all persons ages 15 and older,  that focus on a variety of issues.  Primarily, these topics will include jobs, income, and program participation (both for the individual, and he household as well).  The next section of the track 1 survey will consist of a series of topics focusing on children in the household, including schooling status, activities at home, diability, child care and health care, child support, and ontact with missing parent(s).  The third part of this questionaire will use a very small set of questions to collect information on topics such as neighborhood safety, martial conflict, parental depression, and parental attitudes about self-direction.  The final part of the ‘track 1' instrument will be a small survey adminsitered to adolesecents ages 12-17.  This questionnaire will be self-adminsitered, perhaps using a device like a cassette tape player to lead the child through the questions.  Included in this instrument are scales on educational aspirations, school internship programs,  violence the child has been exposed to, family conflict items, and personal evaluations of likelihood of life events.

While the basic paper questionnaire for track 1 was developed before the hiatus period, it has undergone relatively little work since that time.  We are using 1997 to conmplete CAPI development (authoring is being done at the University of California-Berekely, developers of the CASES language the Bureau uses)  and to field an extensive field pretest of the instrument  in September 1997.  From this, we expect to have a good idea of the how well the instrument will perform in a real operations context, and what kind of problems need to be worked our before routine field implementation for the period of April-June 1998.

The second component of the survey -- ‘track 2' -- will be brought on-line during the 1999 interviews, and will be a part of the survey for the years 1999-2001.  In this component we hope  to address more child-oriented topics, with particular focus on elements of child assessment, quite likely using clincially-tested assessment scale devices.  These may include tests (such as math or reading ability), or evaluative scales (such as psychological adjustment scales).  Given the short duration of time (3 years), it is unlikely that we will be able to administer many of the measures more than one time; however, we are considering the possibility of having some recurring items in this part of the survey.

Technical Issues

A variety of technical issues have occupied part of our time in preparing for SPD, and are likely to be of continuing interest to the project.

Locating Households

One of the first important tasks we will be faced with will be bringing the households of the 1992 and 1993 SIPP panels back into sample for SPD.  We are assembling a set of procedures to gather as much information as possible to assist in this activity.  We will begin, of course, with the final addresses used at the close of the 1992 and 1995 panels.  Given our knowldge of mover rates from other surveys, we believe that at least 20% of the 1993 panel and at least 30% of the 1992 panel, will have moved one or more times.  It is this last group which, of course, will be most troublesome.  Generally, short-term multiple movers tend to be persons with less stable household situations and living conditions - in short, the kind of people we most want in sample for the SPD are also those to likely be most difficult to locate.

Just because individuals move, however, does not mean we will not be able to track and find them. SIPP currently uses tracking procedures on the shorter 4-month interview-to-interview cycle, and manages to keep sample loss relatively low.   Many of the field representatives working on the SPD will be persons who currently work on SIPP.  In some cases, we expect them also to be the same people who worked on the 1992 and 1993 panels.  While the interview period for the 1997 bridge survey will officially be the period from April through June, it is quite likley that we will extend observation beyond this period in order to locate as many of the households as possible.  While we have to expect that we will lose some households, we are optimistic that we will find and interview a sizable proportion of them.

Subsampling

As noted earlier, it is clear that we will not be able to interview all households in the 1992 and 1993 SIPP panels beyond the 1997 interview.  How much we will have to subsample depends on the ultimate response rate to the 1997 SPD ‘bridge' survey, and what focus we wish to make on specific subpopulations.  Further, no decisions have yet been made on which groups should be overrepresented, but we expect to use the welfare reform law as a guide.  For example, our initial thoughts focus on keeping all low-income households (e.g. households with incomes 150 percent of poverty or less) and households with children. Since SIPP uses a clustered interview design, we would probbaly also maintain the sample clusters that these designated households are in.  If,  after this strategy, sample sizes of high interest populations are too small, it has been suggested that we consider adding "retired" March 1997 CPS households (most of whom also have a March 1996 interview) to the SPD sample.  We will need to see the results of our 1997 ‘bridge' efforts before moving onto a final sample designation for 1998 and beyond.
 
Weighting

In this area we have also not made final decisions.  Our current thinking involves weighting the interviewed population to represent the April 1997 U.S. population, rather than the 1992 or 1993 population (basis for the SIPP weights).  We are using as the initial SPD sample all households that were in the first interview of their respective panels (100-level households, in SIPP parlance) and who were also in sample at the conclusion of the survey.  This is done to insure a full set of ‘baseline' information for the SPD households.  We will interview all persons in those households, including those who entered after the first SIPP interview.  However, if a non-original member of a SIPP household leaves an SPD household, we will not follow them, because the base weights for longitudinal purposes will need to be derived from the original SIPP panel weights.
 
Of course, differential attrition of selected subpopulations is a serious concern in developing appropriate weights.  One possibility is to use the 1997 sample (or perhaps the ultimate subsampled group for 1998) as the basis for developing  weights for the entire SPD panel, using the April 1997 U.S. population as the weighting base.  Doing this will require us to to assume the subsample we will follow in SPD is representative of the U.S. population.

Data Products

A variety of data products will ultimately come to fruition from the SPD effort.   We recognize that users of the longitudinal data will have a hard time figuring out how to use data from three separate surveys (SIPP, CPS, and SPD) simultaneously in a longitudinal analysis.  The microdata from the SIPP surveys are already available on-line at the Census Bureau's web site (www.census.gov) through the "Surveys On-Call" program; plans are underway to provide the data as well through FERRET (the Census Bureau's Federal Electronic Research and Retrieval  system, which already provides the CPS microdata to users).  As soon as the bridge survey data are processed, they will be made available in a CPS-compatible format on FERRET.  The next challenge is to create a longitudinal data set with annual data from the SIPP and the CPS in a format consistent with the way data will be provided from the 1998 SPD, so that users can develop familiarity with the data and be ready for the first wave of SPD.

Beyond the basic longitudinal file that will grow over over time, the various topical module supplements of the orignal SIPP panels will provide added substantive detail.  In addition, we are attempting to include some some items to be used in the 1996 SIPP panel in the SPD as well.  This would provide a possible ‘enhanced' cross-sectional sample for some topics using the combined SIPP-96/SPD data files. Another possibility that has been discussed is the possible creation of a combined bridge-March 1997 CPS file.  Since therse two surveys will have highly overlapping content, this file could provide a very large national household sample (approx 100,000 households) for detailed one-time cross-sectional analyses.
 
Other possibilites exist with the introduction of other data sets to complement the SPD as it develops.   We have contracted with the University of Wisconsin to create a complementary data base of state and county welfare program characteristics that could then be matched to the SPD data.  (County-level matches would have to remain confidential and researchers would have to work on that matched data set at the Census Bureau.)  Other administrative data provided to us electronically (e.g. tax returns, welfare program records) could also be matched to the survey data.

Conclusion

Like many projects, the current SPD is derived from an idea somewhat different from where it now stands poised.  The dramatic changes posited for the social welfare system deem it necessary that tools be on hand to attempt to measure the possible large scale social, economic and demographic changes that could follow in the near future.  The merger of the orignal vision of the SPD with the practical data needs and collection realities (especially timeliness and cost) we now face have produced a program that we believe will provide useful data and information for many years to come.

end of content rule

SPD Working Papers

SPD Publications


Skip bottom navigation groups

Contact: (dsd.survey.program.dyanmics@census.gov)
URL: http://www.census.gov/spd

 

Introduction to SPD |  Survey Design & Content |  Data Editing |  Finding SPD Info |  Sampling & Weighting | 
Linking Files | Publications |  S&A  |  News&Notes |  Users' Guide 

 


Census 2000  |  Subjects A to Z  |  Search  |  Product Catalog  |  Data Access Tools  |  FOIA  |  Privacy · Policies  |  Contact Us  |  Home
separator rule
U.S. Census Bureau: Helping You Make Informed Decisions