This paper reports the results of research and analysis undertaken
by Census Bureau staff. It has undergone a more limited review than
official Census Bureau publications. This report is released to
inform interested parties of research and to encourage discussion.
This paper reports differences in respondent characteristics between
the families of adolescents completing a supplemental self-administered
questionnaire in a national survey versus the families of adolescents
who did not fill out the self-administered questionnaire. The survey
in question is the Census Bureau's Survey of Program Dynamics (SPD),
designed specifically to monitor the effects of the 1996 welfare
reform legislation. In the 1998 wave of the SPD, a supplement, the
Adolescent Self-Administered Questionnaire (SAQ), was administered
to the children of survey respondents in order to gather information
on adolescents' home, school, and social lives. Of the 5,579 SAQ-eligible
adolescents, 58.4 percent, or 3,259, completed the questionnaire,
resulting in a very high non-response rate of 42 percent.
If completion of the SAQ is related to respondent characteristics,
as measured in the CORE section of the SPD, then the next administration
of the supplement, in 2001, should receive special attention designed
to reduce the likelihood of differential non-response. This paper
therefore reports any significant differences among respondents
and non-respondents regarding the SAQ. Furthermore, the findings
presented here will provide users of SPD data, and in particular,
SAQ data, with information which will be of use in interpreting
their analysis results. Specifically, these findings will show which
variables may be unreliable indicators because they are related
to whether the survey was completed or not.
The data collected in the SPD span 10 years; they are a unique
source of information with which to study the consequences of changes
in public assistance laws and patterns. Losing cases may result
in less reliable estimates of the statistics attempted to be measured
by the survey. Therefore, high non-response is of great concern
in this longitudinal survey. This paper explores what factors may
be related to failure to complete the Adolescent SAQ. The results
will tell us if child and parent characteristics are related to
the low level of response to the SAQ, or if the high non-response
is random in nature.
Survey non-response affects not only item-level biases, but survey-level
biases. For instance, item- or question-specific non-response can
seriously affect a scale or index created from a series of questions.
Non-response over several waves of a longitudinal survey can affect
overall representativeness of data, yielding poor quality data that
may be of limited use to researchers.
Respondent and household characteristics are related to the likelihood
a person will complete a survey. Persons with higher education levels,
and those with higher incomes are less likely to complete surveys,
as are those persons who are more likely to fear being victims of
a crime (Groves and Couper 1998).
However, the SPD Adolescent SAQ non-response is unique, in that
non-response to the SAQ occurs despite completion of the
SPD CORE survey itself. Procedures for administering the SAQ to
adolescents require approval of parents, the SPD respondents. Thus
non-response to the SAQ could be due to either adolescent or parental
refusals. The majority, 83 percent, of the SAQ refusals were due
to parents acting as gatekeepers. This may imply that parental,
or household, characteristics are related to SAQ completion. Conversely,
a parent's feelings about a given adolescent's characteristics may
also influence whether the youth is allowed to complete the SAQ.
Thus in this case, both parental and youth characteristics may be
related to survey response.
This paper examines whether previously found determinants of non-response
are the same as the determinants of gatekeeper refusals. Variables
examined across response conditions were also selected in order
to gauge potentially disruptive household situations from both the
parent and adolescent perspectives. These sorts of situations may
make respondents less likely to give permission for interviewers
to interact with their children.
The Survey of Program Dynamics
The SPD is a longitudinal national survey of the non-institutionalized
US population, and is based on a subsample (n=18,500) of two retired
Survey of Income and Program Participation (SIPP) panels. These
panels, begun in 1992 and 1993, were re-contacted in 1997, three
years after they completed the SIPP, and asked to take part in the
SPD survey. The reason for using the SIPP panels as the SPD sample
was to comply with Federal legislation mandating the Census Bureau
to collect data which could be used to study the effects on families
of the 1996 Personal Responsibility and Work Opportunities Reconciliation
Act (PRWORA). The SIPP data were collected prior to the PRWORA,
and the SPD data post-PRWORA.
The SPD content includes the core economic and household measures
collected in all waves of the SIPP. Additionally, new sections that
measure child and family well-being are included in the SPD. These
new sections are included in order to determine how families are
affected when parents leave the welfare rolls and go to work. The
consequences of such a change could be positive, with newly working
parents experiencing increased self-esteem, resulting in better
parenting skills. Conversely, newly working parents may feel more
stress, and that stress may spill over into their family relationships,
causing children to suffer emotionally or developmentally.
The 1998 Adolescent Self-Administered Questionnaire (SAQ) asks
youths 107 questions about their family routines and relationships,
academic involvement, social activities, alcohol and drug use, and
romantic relationships. The intent of the SAQ module is to collect
information regarding the activities and well-being of adolescents
post-welfare reform, in order to gauge the effect of changes in
parental or household economic status on adolescent functioning.
The module will be re-administered in 2001, to provide researchers
and policymakers with more information on the same topic. The combination
of the 1998 and 2001 data should allow for an examination of trends
in adolescent well-being at the end of the 20th century.
The SPD is a rich data set, containing an abundance of information
about family economic functioning and interpersonal relationships,
and social life. The SAQ data can be linked to the SPD household
data, giving researchers even more information with which to examine
family well-being. The family and household data provide a context
within which adolescent behavior and feelings can be interpreted.
Methods of Analysis
In order to test for differences between the households of adolescents
who did versus did not complete the Self-Administered Questionnaire,
bivariate analytic techniques were employed. Using the Chi-square
test to test for significant differences, we compare the following
nine child and family/household characteristics across SAQ interview
status: adolescent school expulsion, adolescent school type, adolescent
enrollment in gifted classes, adolescent sports participation, adolescent
extracurricular lesson involvement, adolescent behavior problems,
family housing tenure (rent versus own), household public assistance
receipt, and mother's education. Each of these variables is taken
from the SPD CORE instrument.
These variables were chosen for study because they may reflect
family turbulence, which could affect whether a parent allows a
child to participate in the survey, or whether a child is inclined
to fill out a survey also completed by his or her parent(s). For
instance, if a child is experiencing emotional, school, or behavioral
problems, he or she may not want to complete a survey which asks
questions about emotions, school, and behaviors. Similarly, a parent
may not want the child to answer questions about those topics for
fear of presenting a less than ideal picture of the family life.
Housing tenure, family public assistance receipt, and mother's
education were included in this study in order to gauge whether
household characteristics, which are known to affect survey participation
in general, also affect the parental gatekeeper role, once the parent
has agreed to do the CORE SPD survey.
The dependent variable, SAQ interview status, is coded yes for
those adolescents who completed the SAQ, or no for those adolescents
who did not complete the SAQ. Reasons for not completing the SAQ
include parental refusals, adolescent refusals, no contact between
the interviewer and the adolescent, and other, unspecified reasons.
Some parents refused the SAQ because the adolescent was living temporarily
away from home to attend summer camp or live with other relatives
for the summer.
The independent variables are coded categorically. School expulsion,
gifted class enrollment, sports participation, extracurricular lesson
involvement, behavior problems, and household public assistance
receipt are all coded yes or no. School type is coded public versus
private. Family housing tenure is a three-category variable: family
owns or mortgages the home; family rents the home; or, family resides
in the home without paying. Mother's education is a five-category
variable measuring less than high school completion, high school
graduation (or its equivalent), some college, college graduation
(Bachelor's degree), and post-college schooling.
Results: SPD Adolescent Self-Administered Questionnaire Non-response
The results presented in Table 1 show the unweighted frequency
distribution of the independent and dependent variables. With an
overall response rate of 58.4 percent, an eyebrow may be raised
as to the representativeness of the SAQ data. That, of course, is
the reason for this paper. Other research on this topic has demonstrated
that the adolescents completing the SAQ are similar to the national
population in terms of exposure to sex, smoking, drinking, and marijuana
usage (see Bass and Downs 1999). But still, the question remains,
are there subtle differences in the family lives of the children
who completed the SAQ versus those who did not?
Continuing with Table 1, only a small portion of the youths had
been expelled from school (7.9%), or had behavior problems (8.1%).
The vast majority were attending public school (91.5%), and about
half were involved in extracurricular sports activities (46.8%).
The same proportion, 24.6 percent, of children were enrolled in
extracurricular lessons and gifted education classes. Most of the
youths lived in homes that were owned or mortgaged (76.9%), and
very few were in families that received welfare benefits in the
past year (7.7%). Just over half the mothers had an education beyond
high school graduation (52.6%), with thirty two percent earning
a Bachelor's degree or more. (1)
One can also see that significant proportions of children have basic
information on their activities and school behavior missing from
the CORE sections of the survey (about 1200 respondents are missing
on these variables).
Table 2 contains cross-tabulations between the independent variables
and the dependent variable, interview status. Also included in Table
2 are results of Chi-square tests of the bivariate relationships
between each independent variable and the dependent variable. These
results provide preliminary answers to the question of whether and
how the families of adolescents who completed the Self-Administered
Questionnaire differ from the families of adolescents who did not
complete it. We use a p<.05 level to gauge statistical significance
A quick glance through Table 2 shows that there are only two characteristics
which significantly differ between the group of adolescents completing
the SAQ and those who did not complete it. These characteristics
are sports involvement, and enrollment in gifted education classes.
There are no significant differences in the likelihood of the adolescent
completing the SAQ between families of children who were expelled
from school in the past year versus families of children who were
not. This suggests that parents of children with school problems
are just as likely to allow their children to participate in surveys
as parents of children who do not experience school difficulties.
Similarly, adolescents are no more likely to refuse to complete
the SAQ if they have been expelled from school than if they have
not been expelled.
Additionally, children who attend private schools are not significantly
more or less likely to complete the survey than children who attend
public schools. Children who attend private schools may be from
families with higher incomes, as private schools generally require
tuition payments. Yet despite prior evidence that higher income
households are less likely to complete surveys, it appears that
parents who are able to send their children to private school are
not more likely to restrict their children's participation in a
survey, once they themselves are survey participants.
Children who engage in extracurricular sports are significantly
more likely to complete the SAQ. The parents of children on sports
teams may be particularly proud of their children and want them
to participate in the survey so that their behaviors and views are
counted. Children who participate in extracurricular enriching activities
such as sports or clubs are viewed by adults as more responsible
and mature than their peers (Smith and Casper 1999). Involvement
in activities such as these require children and teens to be organized
and committed to meeting the responsibilities set before them. Perhaps
these youths are more likely to be perceived (by themselves and
their parents) as mature enough to participate in a national survey.
Taking extracurricular lessons does not, apparently, make a parent
more likely to allow a child to complete the SAQ, nor does it make
a child more likely to complete the SAQ, given parental approval.
However, children taking gifted education classes are significantly
more likely to complete the SAQ. Enrollment in gifted education
classes and extracurricular lessons are positively and significantly
related to each other (p=.001; data not shown), so one might expect
these two variables to affect response similarly. Perhaps, though,
some of the extracurricular lessons are non-school related, such
as dance or art classes. Consequently, each measure may reflect
a different motivator for responding or not responding to the SAQ.
The youths enrolled in gifted education classes may come from more
stable homes, with more parental involvement, and thus may be more
available for the interview, resulting in their higher completion
rate. One way to measure stability is to determine whether these
youths moved more or less than others. A cross-tabulation demonstrates
that youths in gifted classes moved significantly fewer times than
their counterparts, providing some support for the explanation that
these children come from more stable homes (p=.022; data not shown).
Gifted education enrollment is also strongly associated with sports
participation (p=.001; data not shown). The children who participate
in these socially endorsed activities may be perceived by their
parents as "model children" and thus are prime candidates for completing
a national survey.
Children who have behavior problems are not statistically more
or less likely to have completed a SAQ. Like school expulsion, which
is positively related to behavior problems, (data not shown), the
existence of behavior problems in general does not appear to be
a factor in determining whether parents give permission for their
children to participate in the SPD, or whether the youths actually
complete the SAQ.
Housing tenure, welfare receipt in the past year, and mother's
education are also not significantly related to adolescent completion
of the SAQ. It appears, then, that household characteristics do
not significantly affect the gatekeeper role and prevent or encourage
the completion of the adolescent questionnaire.
However, we can see that household which have children who have
missing characteristics from the CORE sections are about 20 percentage
points more likely to not complete the SAQ. It is possible that
parents who are uneasy about answering items about their children's
behavior may be even more uneasy about permitting children to answer
the SAQ, which probes more deeply for "embarrassing" or "negative"
Despite our concerns, we found that for the most part, response
to the SAQ is not dependent on family or household characteristics
among those households who provide answers for these characteristics.
The factors most related to adolescent participation in the survey
are factors which could be described as indicators of "exceptional"
or "model" children - participation on sports teams and classes
for gifted students.
It appears that the parents who are unwilling to provide basic
answers about their children in the CORE part of the survey are
also those most non-responsive to the administration of the SAQ.
Perhaps once the non-respondents are weeded out of the sample, it
is a matter of parent-child dynamics, or unmeasured attitudes about
children's participation in surveys, that determines whether a child
is allowed to or actually does complete the SAQ.
The Self-Administered Questionnaire non-response appears to be
random regarding the variables explored in this paper, at least
among those parents providing basic answers to questions administered
at the front end of the survey. Other research has also documented
a random nature to the non-response. For instance, neither living
in a two-parent household, or parent's age are significantly related
to completion of the SAQ. There are, however, significant differences
in response between Black- and White-headed households, and across
geographic region, although these differences are small (Bass and
Downs 1999). Consequently, we can conclude that these data are of
suitable quality to be included in analyses of adolescent and family
well-being, but with a strong caution that a significant proportion
of children missing key background indicators may bias the survey
in an unknown direction.
Future studies should be conducted to verify further the representativeness
or quality of the data. These studies could examine SAQ completion
with respect to parental marital or employment status, or household
income level. A comparison along official Census Bureau race classifications
would also provide insight into the quality of the SAQ data.
Bass, L. and B. Downs. 1999. "What Can the SPD Adolescent SAQ Tell
Us About the Well-being of Adolescents in the Aftermath of the 1996
Welfare Reform Act?" Paper presented at the annual meetings of the
Population Association of America, March 25-27, 1999, New York,
Groves, R.M. and M.P. Couper. 1998. Nonresponse in Household
Interview Surveys. John
Wiley & Sons, Inc. New York, NY.
Smith, K. and L. Casper. 1999. "Home Alone: Reasons Parents Leave
Unsupervised." Paper presented at the annual meetings of the Population
America, March 25-27, 1999, New York, NY.
1. These data are unweighted,
thus comparisons to other national data are unwarranted. Once appropriate
weighting schemes for the SPD have been developed, the data will
be compared to other sources in order to examine its representativeness.