by
Barbara A. Downs
Family Statistics Branch
Population Division
US Bureau of the Census
Washington, DC 20233-8800
(301)457-2465
barbara.a.downs@ccmail.census.gov

This paper reports differences in respondent characteristics between the families of adolescents completing a supplemental self-administered questionnaire in a national survey versus the families of adolescents who did not fill out the self-administered questionnaire. The survey in question is the Census Bureau's Survey of Program Dynamics (SPD), designed specifically to monitor the effects of the 1996 welfare reform legislation. In the 1998 wave of the SPD, a supplement, the Adolescent Self-Administered Questionnaire (SAQ), was administered to the children of survey respondents in order to gather information on adolescents' home, school, and social lives. Of the 5,579 SAQ-eligible adolescents, 58.4 percent, or 3,259, completed the questionnaire, resulting in a very high non-response rate of 42 percent.

If completion of the SAQ is related to respondent characteristics, as measured in the CORE section of the SPD, then the next administration of the supplement, in 2001, should receive special attention designed to reduce the likelihood of differential non-response. This paper therefore reports any significant differences among respondents and non-respondents regarding the SAQ. Furthermore, the findings presented here will provide users of SPD data, and in particular, SAQ data, with information which will be of use in interpreting their analysis results. Specifically, these findings will show which variables may be unreliable indicators because they are related to whether the survey was completed or not.

The data collected in the SPD span 10 years; they are a unique source of information with which to study the consequences of changes in public assistance laws and patterns. Losing cases may result in less reliable estimates of the statistics attempted to be measured by the survey. Therefore, high non-response is of great concern in this longitudinal survey. This paper explores what factors may be related to failure to complete the Adolescent SAQ. The results will tell us if child and parent characteristics are related to the low level of response to the SAQ, or if the high non-response is random in nature.

Survey non-response affects not only item-level biases, but survey-level biases. For instance, item- or question-specific non-response can seriously affect a scale or index created from a series of questions. Non-response over several waves of a longitudinal survey can affect overall representativeness of data, yielding poor quality data that may be of limited use to researchers.

Respondent and household characteristics are related to the likelihood a person will complete a survey. Persons with higher education levels, and those with higher incomes are less likely to complete surveys, as are those persons who are more likely to fear being victims of a crime (Groves and Couper 1998).

However, the SPD Adolescent SAQ non-response is unique, in that non-response to the SAQ occurs despite completion of the SPD CORE survey itself. Procedures for administering the SAQ to adolescents require approval of parents, the SPD respondents. Thus non-response to the SAQ could be due to either adolescent or parental refusals. The majority, 83 percent, of the SAQ refusals were due to parents acting as gatekeepers. This may imply that parental, or household, characteristics are related to SAQ completion. Conversely, a parent's feelings about a given adolescent's characteristics may also influence whether the youth is allowed to complete the SAQ. Thus in this case, both parental and youth characteristics may be related to survey response.

This paper examines whether previously found determinants of non-response are the same as the determinants of gatekeeper refusals. Variables examined across response conditions were also selected in order to gauge potentially disruptive household situations from both the parent and adolescent perspectives. These sorts of situations may make respondents less likely to give permission for interviewers to interact with their children.

The SPD is a longitudinal national survey of the non-institutionalized US population, and is based on a subsample (n=18,500) of two retired Survey of Income and Program Participation (SIPP) panels. These panels, begun in 1992 and 1993, were re-contacted in 1997, three years after they completed the SIPP, and asked to take part in the SPD survey. The reason for using the SIPP panels as the SPD sample was to comply with Federal legislation mandating the Census Bureau to collect data which could be used to study the effects on families of the 1996 Personal Responsibility and Work Opportunities Reconciliation Act (PRWORA). The SIPP data were collected prior to the PRWORA, and the SPD data post-PRWORA.

The SPD content includes the core economic and household measures collected in all waves of the SIPP. Additionally, new sections that measure child and family well-being are included in the SPD. These new sections are included in order to determine how families are affected when parents leave the welfare rolls and go to work. The consequences of such a change could be positive, with newly working parents experiencing increased self-esteem, resulting in better parenting skills. Conversely, newly working parents may feel more stress, and that stress may spill over into their family relationships, causing children to suffer emotionally or developmentally.

The 1998 Adolescent Self-Administered Questionnaire (SAQ) asks youths 107 questions about their family routines and relationships, academic involvement, social activities, alcohol and drug use, and romantic relationships. The intent of the SAQ module is to collect information regarding the activities and well-being of adolescents post-welfare reform, in order to gauge the effect of changes in parental or household economic status on adolescent functioning. The module will be re-administered in 2001, to provide researchers and policymakers with more information on the same topic. The combination of the 1998 and 2001 data should allow for an examination of trends in adolescent well-being at the end of the 20^th century.

The SPD is a rich data set, containing an abundance of information about family economic functioning and interpersonal relationships, and social life. The SAQ data can be linked to the SPD household data, giving researchers even more information with which to examine family well-being. The family and household data provide a context within which adolescent behavior and feelings can be interpreted.

In order to test for differences between the households of adolescents who did versus did not complete the Self-Administered Questionnaire, bivariate analytic techniques were employed. Using the Chi-square test to test for significant differences, we compare the following nine child and family/household characteristics across SAQ interview status: adolescent school expulsion, adolescent school type, adolescent enrollment in gifted classes, adolescent sports participation, adolescent extracurricular lesson involvement, adolescent behavior problems, family housing tenure (rent versus own), household public assistance receipt, and mother's education. Each of these variables is taken from the SPD CORE instrument.

These variables were chosen for study because they may reflect family turbulence, which could affect whether a parent allows a child to participate in the survey, or whether a child is inclined to fill out a survey also completed by his or her parent(s). For instance, if a child is experiencing emotional, school, or behavioral problems, he or she may not want to complete a survey which asks questions about emotions, school, and behaviors. Similarly, a parent may not want the child to answer questions about those topics for fear of presenting a less than ideal picture of the family life.

Housing tenure, family public assistance receipt, and mother's education were included in this study in order to gauge whether household characteristics, which are known to affect survey participation in general, also affect the parental gatekeeper role, once the parent has agreed to do the CORE SPD survey.

The dependent variable, SAQ interview status, is coded yes for those adolescents who completed the SAQ, or no for those adolescents who did not complete the SAQ. Reasons for not completing the SAQ include parental refusals, adolescent refusals, no contact between the interviewer and the adolescent, and other, unspecified reasons. Some parents refused the SAQ because the adolescent was living temporarily away from home to attend summer camp or live with other relatives for the summer.

The independent variables are coded categorically. School expulsion, gifted class enrollment, sports participation, extracurricular lesson involvement, behavior problems, and household public assistance receipt are all coded yes or no. School type is coded public versus private. Family housing tenure is a three-category variable: family owns or mortgages the home; family rents the home; or, family resides in the home without paying. Mother's education is a five-category variable measuring less than high school completion, high school graduation (or its equivalent), some college, college graduation (Bachelor's degree), and post-college schooling.

The results presented in Table 1 show the unweighted frequency distribution of the independent and dependent variables. With an overall response rate of 58.4 percent, an eyebrow may be raised as to the representativeness of the SAQ data. That, of course, is the reason for this paper. Other research on this topic has demonstrated that the adolescents completing the SAQ are similar to the national population in terms of exposure to sex, smoking, drinking, and marijuana usage (see Bass and Downs 1999). But still, the question remains, are there subtle differences in the family lives of the children who completed the SAQ versus those who did not?

Continuing with Table 1, only a small portion of the youths had been expelled from school (7.9%), or had behavior problems (8.1%). The vast majority were attending public school (91.5%), and about half were involved in extracurricular sports activities (46.8%). The same proportion, 24.6 percent, of children were enrolled in extracurricular lessons and gifted education classes. Most of the youths lived in homes that were owned or mortgaged (76.9%), and very few were in families that received welfare benefits in the past year (7.7%). Just over half the mothers had an education beyond high school graduation (52.6%), with thirty two percent earning a Bachelor's degree or more. ⁽¹⁾ One can also see that significant proportions of children have basic information on their activities and school behavior missing from the CORE sections of the survey (about 1200 respondents are missing on these variables).

Table 2 contains cross-tabulations between the independent variables and the dependent variable, interview status. Also included in Table 2 are results of Chi-square tests of the bivariate relationships between each independent variable and the dependent variable. These results provide preliminary answers to the question of whether and how the families of adolescents who completed the Self-Administered Questionnaire differ from the families of adolescents who did not complete it. We use a p<.05 level to gauge statistical significance of differences.

A quick glance through Table 2 shows that there are only two characteristics which significantly differ between the group of adolescents completing the SAQ and those who did not complete it. These characteristics are sports involvement, and enrollment in gifted education classes.

There are no significant differences in the likelihood of the adolescent completing the SAQ between families of children who were expelled from school in the past year versus families of children who were not. This suggests that parents of children with school problems are just as likely to allow their children to participate in surveys as parents of children who do not experience school difficulties. Similarly, adolescents are no more likely to refuse to complete the SAQ if they have been expelled from school than if they have not been expelled.

Additionally, children who attend private schools are not significantly more or less likely to complete the survey than children who attend public schools. Children who attend private schools may be from families with higher incomes, as private schools generally require tuition payments. Yet despite prior evidence that higher income households are less likely to complete surveys, it appears that parents who are able to send their children to private school are not more likely to restrict their children's participation in a survey, once they themselves are survey participants.

Children who engage in extracurricular sports are significantly more likely to complete the SAQ. The parents of children on sports teams may be particularly proud of their children and want them to participate in the survey so that their behaviors and views are counted. Children who participate in extracurricular enriching activities such as sports or clubs are viewed by adults as more responsible and mature than their peers (Smith and Casper 1999). Involvement in activities such as these require children and teens to be organized and committed to meeting the responsibilities set before them. Perhaps these youths are more likely to be perceived (by themselves and their parents) as mature enough to participate in a national survey.

Taking extracurricular lessons does not, apparently, make a parent more likely to allow a child to complete the SAQ, nor does it make a child more likely to complete the SAQ, given parental approval. However, children taking gifted education classes are significantly more likely to complete the SAQ. Enrollment in gifted education classes and extracurricular lessons are positively and significantly related to each other (p=.001; data not shown), so one might expect these two variables to affect response similarly. Perhaps, though, some of the extracurricular lessons are non-school related, such as dance or art classes. Consequently, each measure may reflect a different motivator for responding or not responding to the SAQ.

The youths enrolled in gifted education classes may come from more stable homes, with more parental involvement, and thus may be more available for the interview, resulting in their higher completion rate. One way to measure stability is to determine whether these youths moved more or less than others. A cross-tabulation demonstrates that youths in gifted classes moved significantly fewer times than their counterparts, providing some support for the explanation that these children come from more stable homes (p=.022; data not shown).

Gifted education enrollment is also strongly associated with sports participation (p=.001; data not shown). The children who participate in these socially endorsed activities may be perceived by their parents as "model children" and thus are prime candidates for completing a national survey.

Children who have behavior problems are not statistically more or less likely to have completed a SAQ. Like school expulsion, which is positively related to behavior problems, (data not shown), the existence of behavior problems in general does not appear to be a factor in determining whether parents give permission for their children to participate in the SPD, or whether the youths actually complete the SAQ.

Housing tenure, welfare receipt in the past year, and mother's education are also not significantly related to adolescent completion of the SAQ. It appears, then, that household characteristics do not significantly affect the gatekeeper role and prevent or encourage the completion of the adolescent questionnaire.

However, we can see that household which have children who have missing characteristics from the CORE sections are about 20 percentage points more likely to not complete the SAQ. It is possible that parents who are uneasy about answering items about their children's behavior may be even more uneasy about permitting children to answer the SAQ, which probes more deeply for "embarrassing" or "negative" behavior patterns.

Despite our concerns, we found that for the most part, response to the SAQ is not dependent on family or household characteristics among those households who provide answers for these characteristics. The factors most related to adolescent participation in the survey are factors which could be described as indicators of "exceptional" or "model" children - participation on sports teams and classes for gifted students.

It appears that the parents who are unwilling to provide basic answers about their children in the CORE part of the survey are also those most non-responsive to the administration of the SAQ. Perhaps once the non-respondents are weeded out of the sample, it is a matter of parent-child dynamics, or unmeasured attitudes about children's participation in surveys, that determines whether a child is allowed to or actually does complete the SAQ.

The Self-Administered Questionnaire non-response appears to be random regarding the variables explored in this paper, at least among those parents providing basic answers to questions administered at the front end of the survey. Other research has also documented a random nature to the non-response. For instance, neither living in a two-parent household, or parent's age are significantly related to completion of the SAQ. There are, however, significant differences in response between Black- and White-headed households, and across geographic region, although these differences are small (Bass and Downs 1999). Consequently, we can conclude that these data are of suitable quality to be included in analyses of adolescent and family well-being, but with a strong caution that a significant proportion of children missing key background indicators may bias the survey in an unknown direction.

Future studies should be conducted to verify further the representativeness or quality of the data. These studies could examine SAQ completion with respect to parental marital or employment status, or household income level. A comparison along official Census Bureau race classifications would also provide insight into the quality of the SAQ data.

Bass, L. and B. Downs. 1999. "What Can the SPD Adolescent SAQ Tell Us About the Well-being of Adolescents in the Aftermath of the 1996 Welfare Reform Act?" Paper presented at the annual meetings of the Population Association of America, March 25-27, 1999, New York, NY.

Groves, R.M. and M.P. Couper. 1998. Nonresponse in Household Interview Surveys. John

Wiley & Sons, Inc. New York, NY.

Smith, K. and L. Casper. 1999. "Home Alone: Reasons Parents Leave Their Children

Unsupervised." Paper presented at the annual meetings of the Population Association of

America, March 25-27, 1999, New York, NY.

1. These data are unweighted, thus comparisons to other national data are unwarranted. Once appropriate weighting schemes for the SPD have been developed, the data will be compared to other sources in order to examine its representativeness.