for the
1999 Survey of Program Dynamics
Jennifer C. Hess
Jennifer M. Rothgeb
Elizabeth M. Nichols
Center for Survey Methods Research
Statistical Research Division
Bureau of the Census
June 17, 1998
Table of Contents
I. Background
II. Methodology
III. Cognitive Test Results: Child Well-being Module
A. Positive Interaction
B. Cognitive Stimulation
C. Enrichment Activities
D. Positive Behaviors/Social Competence
E. Grades/Achievement
F. Household Resources
G. Conflict Between Parents Who Co-reside
H. Conflict Between Parents Who Do Not Co-reside
I. Family Conflict
J. Attitude Toward Welfare/Work
K. Knowledge About Welfare Regulations
IV. Cognitive Test Results: Expanded Content for the SPD Core Interview
A. Child Care Subsidies
B. Child Care Problems
C. Quality of Child Care
D. Employee Fringe Benefits
E. Health Care Utilization While Uninsured
F. Consumer Expenditures
1.Food Expenditures
2. Transportation Expenditures
3. School Expenditures
4. Health Insurance Expenditures
G. Months Receiving Rental Assistance
V. Cognitive Test Results: Children's Residential History
A. General comments
B. Results of Round 1 Testing
C. Results of Round 2 Testing
Attachment A Development Plan (time line) for SPD99
Attachment B Objectives for the Child and Family Well-Being Questions
Attachment C Content Areas Not Cognitively Tested by CSMR
Report on Cognitive Interview Results - SPD99
I. BACKGROUND
Early in the development of the Survey of Program Dynamics (SPD), the SPD Steering Committee envisioned the survey as composed of two parts: a 60 minute "core"interview administered annually starting in 1998, and a 30 minute child well-being module to be added in 1999. In summer 1997, scarcity of resources (staff and monetary) and the need to focus efforts on ensuring the success of the core SPD in order to meet legislative goals, prevented moving forward with the original plans for a child well-being module. The SPD Steering Committee reexamined resources and determined that a 5-10 minute child focused module was a realistic alternative.
In Fall 1997, the Bureau of the Census solicited input from the SPD Interagency Committee for a 5-10 minute child focused module. A development plan (Attachment A) was circulated, soliciting content and identifying dates for subject matter to be proposed, tested, and evaluated. Solicitation for content was eventually expanded to cover not just the child well being module, but the core SPD instrument as well. In addition to government agencies, input was solicited from data users including those from academia and the private sector. Proposed content was received from the Family and Child Well Being Network of the National Institute of Child Health and Human Development (NICHD), Child Trends, Inc. (CTI), the Department of Health and Human Services (HHS), the Department of Agriculture, the Office of Management and Budget (OMB), Manpower Demonstration Research Project (MDRC), the Department of Health Care Policy at Harvard University, and the Population Division (POP), and the Housing and Household Economic Statistics Division (HHES) of the Bureau of the Census. The SPD Steering Committee decided that all proposed topics be considered for the module and, in accordance with the Census Bureau's pretesting policy, required testing be conducted for appropriate question series.
Proposed content includes the following:
Child Well Being
Positive interaction
Cognitive stimulation
Enrichment activities
Positive behaviors/social competence
Grades/achievement
Household resources Family conflict
Conflict between biological parents who co-reside
Conflict between biological parents who do not co-reside
Attitudes towards welfare/work
Knowledge about welfare regulations
Children's residential history
Family routines
School engagement
Behavior problem index
Aggravation in parenting
Children's routines
Substance abuse
In addition to the child well-being module, proposed expanded content for the SPD core instrument includes the following:
SPD Core expansion
Knowledge of child care subsidies
Child care problems
Quality of child care
Employee fringe benefits
Health care utilization while uninsured
Consumer expenditures (food, transportation, school, health insurance)
Months receiving rental assistance
Coping mechanisms when losing welfare benefits
We requested that persons proposing content for SPD99 provide specific question wording and the universe for whom the questions applied, along with the objective of the question or question series, background on results of prior use (e.g. item nonresponse), and, if available, the results of prior testing. This information was needed to determine whether results of prior testing or prior use demonstrate that proposed question series "work "and don't require further testing. Additionally, information on the objectives of the questions aid in development of appropriate probes for the test protocol to indicate whether respondents understand the question as intended and ensure that the objective of the question is being met. Objectives provided for some of the proposed content are contained in Attachment B.
Based on review of limited documentation and discussions with subject matter experts and survey methodologists, CSMR determined that the following proposed SPD99 content did not require testing because there was enough evidence from prior use and/or testing that the items "worked."
Proposed content areas NOT cognitively tested by CSMR
(Questions for these series are contained in Attachment C.)
Aggravation in parenting
School engagement
Family routines
Children's routines
Mental Health Indicator (Behavior problems index)
Food Bank
Coping mechanisms when losing welfare benefits
Substance abuse
The SPD Interagency Committee agreed that, once testing was completed and evaluated, content areas would be prioritized and selected for inclusion in the SPD99 instrument based on discussions and decisions reached within the SPD Interagency Group. This report documents the results of the cognitive laboratory testing conducted with the proposed SPD99 content.
II. METHODOLOGY
We conducted thirty-four concurrent cognitive think-aloud laboratory interviews to test respondent comprehension, task difficulty, and item sensitivity of proposed questions. Three researchers from the Center for Survey Methods Research (CSMR) conducted the cognitive interviews and summarized results. We tried to keep cognitive interviews to no more than one hour to avoid respondent fatigue. Respondents received $30 for their participation. Interviews were conducted in CSMR's Response Research Laboratory, as well as at off-site locations more convenient for respondents. All interviews were audio taped (with written consent from respondents). Due to the magnitude of material proposed for testing and late delivery of required information for some proposed topics (e.g. question wording, objectives, prior use, prior testing), we conducted two rounds of interviews, referred to as Rounds 1 and 2 below.
Due to the quantity of content tested, we minimized the use of direct probes and maximized use of general probes. General probes used included: "How did you arrive at your answer? " "Tell me more about that. " "In your own words, what is this question asking? " During most interviews there was only time to administer the child-focused questions for one child within the household. Efforts were made to balance the ages of the children for whom information was collected.
In Round 1, we included twelve content areas (enrichment, cognitive stimulation, positive interaction, family conflict, parental conflict (co-reside), parental conflict (don't co-reside), household resources, quality of child care, attitudes towards welfare/work, knowledge about welfare regulations, consumer expenditures, and children's residential history). Respondents were recruited primarily from job training programs, GED programs, and other social service programs. Typically, the CSMR recruiter contacted a program director, who then identified potential respondents and scheduled interviews. Round 1 respondents were primarily low-income single mothers, many of whom were current or former welfare program recipients. (None of the respondents was male.) Eighteen Round 1 interviews were conducted March 9 - March 13 with adults with children under 18 living in their households. Children in Round 1 were in the following age groups: 3 children 0-2 years old; 5 children 3-5 years old, 5 children 6-11 years old; and 5 children 12-17 years old. Most of the Round 1 interviews were conducted off-site.
In Round 2, we included seven new content areas (positive behaviors/social competence, grades/achievement, knowledge of child care subsidies, child care problems, employee fringe benefits, utilization of health care while uninsured, rental assistance) as well as revised versions (or subsets) of eight content areas previously included in Round 1 (family conflict, parental conflict (don't co-reside), positive interaction, cognitive stimulation, enrichment activities, attitudes toward welfare/work, food expenditures, children's residential history.) Respondents were recruited through newspaper ads and from CSMR's respondent pool. Sixteen Round 2 interviews were conducted between April 7 and May 5 with adults with children under 18 in their household. Children in Round 2 were in the following age groups: 1 child 0-2 years old; 3 children 3-5 years old, 5 children 6-11 years old; and 7 children 12-17 years old. Most of the respondents were lower-middle income and married. In Round 2, five respondents were males. All of the Round 2 interviews were conducted on-site in CSMR's laboratory. Population Division (POP) staff observed (through a one-way mirror in the observation room) some of these interviews.
III. COGNITIVE TESTING RESULTS - CHILD WELL-BEING MODULE
Provided below are the results of cognitive testing, grouped by content area. In general, for each content area, the tested question wording is provided followed by a description of problems identified through testing, and recommendations for revised question wording or response options. The majority of the time, problems identified are immediately below the tested wording for each specific question, followed by the recommended revisions, as applicable. In some cases, problems identified pertain to a whole series of questions rather than a specific question. In such instances, the question series is provided and "problems (series)"are located at the end of the series. If content was tested in Round 1 and Round 2, then the tested wording and problems identified are labeled specific to the round of testing to which they apply. If there is no recommendation provided after the "problems"section, then it should be assumed that no question wording revision is recommended. We provide the universe for each content area at the beginning of each section. If the universe changes, we note that in the specific item. If a flashcard is intended to be used we also note that at the beginning of the section.
In some series there are not any recommended revisions from Round 2 testing. This is especially true for the enrichment questions and the cognitive stimulation series. Midway through testing the sponsoring division (POP) decided that, in order to retain time series (back to SIPP 92/93 Child well-being module), revisions to these series in 1999 are unacceptable. For these sections, we document whatever problems we detected, but do not provide recommended revisions. This is also true for the residential history section. In late May the Census Bureau decided that the residential history module will not be ready in time for the SPD99, and will be administered instead in the SPD 2000. This allows more time for thorough development. Census and external experts are currently negotiating objectives and design issues for this series. Therefore, we defer recommendations for revisions to that group. We do identify various issues that we think should be considered during development of the module.
A. Positive Interaction
(Universe : 0-11; Flashcard)
Round 1:
Tested wording:
How often do you praise (child) by saying something like "Good for you! "or "What a nice thing you did! "or "Good going! "? (Same response options for all items during Round 1.)
<1> Never
<2> About once a week or less
<3> A few times a week
<4> One or two times a day
<5> Many times each dayProblems:
Nearly all respondents selected the response option "many times each day. " We expressed concern to CTI about the utility of this item if there is no variance in response. We recommended deleting this item (and series). CTI recommended using NLSY HOME wording and retesting.
Recommendation:
Revise question wording: How often do you praise (child) for doing something worthwhile?
Round 2:
Tested wording:
<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
<5> One or two times a day
<6> Many times a day
Problems:
In Round 2, only 5 of the 10 respondents chose "many times a day ". The rest chose response options <4> and <5>. It is unknown whether the question revisions, response revisions, or differences in respondents led to result differences between Round 1 and Round 2.
Recommendation:
Round 1
Tested wording:
Problems:
Recommendation:
Round 2
Tested wording:
<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
<5> One or two times a day
<6> Many times a day
Problems:
As with Round 1, respondents did not mention five or more minutes in their responses when asked why they picked a particular category. And when probed, many said they had not considered the timeframe. We think that this timeframe should be retained because it may provide a subtle clue that shorter durations are important. Many of the respondents might have used five minutes or more as a benchmark, even though they didn't articulate that is what they did. A number of respondents gave off the cuff answers of "everyday, "which was not part of the response set. Field Representatives (FRs) need to be trained to probe further if this response occurs. Only two respondents chose "a few times a week. " The other respondents chose response options 5 or 6.
Recommendation:
Round 1
Tested wording:
Problems:
Recommendation:
Round 1
Tested wording:
Problems:
We also think the response scale is inappropriate and possibly misleading. The frequencies in the response options seemed to imply that "something special "is an everyday event, given that the low frequency was "about once a week or less, "and the high frequency was "many times a day. " CTI recommended rewording the question.
Recommendation:
How often do you do something special with (child)?
<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
Round 2
Tested wording:
<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
Problems:
Two respondents were probed about previous wording, "... that she enjoys. "These respondents both thought that the former wording would require more thought, since children don't enjoy everything. One respondent said the former wording left it more open to interpretation since "special "for children might be candy, and that is not necessarily what this parent would consider special.
There were much more varied responses to this question in Round 2, evenly spread among response options 2, 3 and 4. It is unknown whether the question wording revision, response option revisions or differences in characteristics of respondents led to results so different than in Round 1.
Recommendation:
Round 1:
Tested wording
(Universe: Ages <2 years old)
Tested wording
(Universe: Ages 2-11)
Problems:
Recommendation:
Series Comment:
B. Cognitive Stimulation
(Flashcard)
Round 1
Tested wording:
(Read/Show pictures) books to (child)?
(Universe: read, ages 1-11; show pictures, ages <1)
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Regarding the distribution of responses, CTI informed us that the experience with the National Evaluation of Welfare-to-Work Strategies Child Outcomes Study, which obtained similar information, showed that there was variation in the survey responses obtained.
Recommendation:
_______number
Round 2, Version 1:
Tested wording:
How many times in the past week did you or any family member read stories to (child)?
(Universe: Ages 0-11)
_________ number
Problems:
Round 2, Version 2:
Tested wording:
( Universe: Ages 0-11)
<1> Never
<2> Once this week
<3> Several times this week
<4> Everyday or almost everyday
<5> More than once a day
Problems:
Recommendation:
Round 1
Tested wording:
(Universe tell, ages 1-11; sing, ages <1)
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Recommendation:
_________ number
Round 2
Tested wording:
(Universe: Ages 0-11)
_______number
Problems:
Recommendation:
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Everyday
Round 1
Tested wording:
(Universe: Ages 3-11)
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Recommendation:
Item was not retested in Round 2.
Round 1
Tested wording:
(Universe: Ages 3-11)
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Round 1
Tested wording:
(Universe: Ages 3-11)
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Recommendation:
Round 1
Tested wording:
<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day
Problems:
Round 1
Tested wording:
(Universe: Ages 3-11)
<1> Never
<2> A few times during the past year
<3> About once a month
<4> About once a week
Problems:
Recommendation:
Round 1
Tested wording:
(Universe: Ages 3-11)
___ number of books
Problems:
Recommendation:
None
1 or 2 books
3 to 9 books
10 to 19
20 or more books.
Round 1
Tested wording:
(Universe: Ages 9-17; Flashcard)
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
Problems:
Round 1
Tested wording:
(Universe: Ages 9-17; Flashcard)
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
Problems:
Round 1
Tested wording:
(Universe: Ages 9-17; Flashcard)
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
Problems:
Recommendation:
Round 1
Tested wording:
(Universe: Ages 9-17; Flashcard)
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
Problems:
Recommendation:
Round 1
Tested wording:
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
Problems:
Round 1
Tested wording:
<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday
During the past year, did (child) attend an overnight or day camp for a week or more? (Universe : Ages 9-17)
<1> Yes
<2> No
During the past year, did (child) attend an overnight trip or retreat sponsored by a school, religious, or community group? (Universe : Ages 9-17)
<1> Yes
<2> No
During the past year, did (child) go on an overnight trip with family or friends? (Universe : Ages 9-17)
<1> Yes
<2> No
Problems:
Recommendation:
During the past year, how often has (child) gone on a trip at least an hour or more away from home?
During the past year, did (child) go on an overnight trip or retreat with family or friends, or attend an overnight camp?
CTI recommended that FRs be trained that this could include a trip or camp sponsored by a school, religious, or community group.
Round 2
Tested wording:
<1> Never
<2> Several times a year
<3> About once a month
<4> About once a week
<5> Everyday or almost everyday
Problems:
Recommendation:
Round 2
Tested wording:
Problem:
Recommendation:
Series Comment:
C. Enrichment Activities
(Universe: Ages 6-17)
We tested the enrichment questions during both rounds of testing. We reordered the series in Round 2 and made a few minor wording changes. Part way through the Round 2 testing, we received word from CTI and Population Division that the wording for these items was under discussion. CTI preferred we use wording from the 1992 SIPP Wave 9 Child Well-being module, instead of SPD 98 wording or revised wording. Based upon instructions from the Population Division, the Round 2 wording was modified and is labeled as Round 2, Version 2 below. The following discussion is organized by the question topic. We will discuss the results from both rounds of testing for each topic included in the enrichment series (e.g. participation in sports teams, lessons, clubs or organizations, etc.). This is followed by the results of the questions on frequency of participation in these activities during the previous year.
1. Sports teams
Round 1
Tested wording:
Problems:
Round 2 Version 1
Tested wording:
Since March 1997, was [child] on any kind of organized sports team?
Problems:
Round 2 Version 2
Tested wording:
Problems:
2. Lessons
Round 1
Tested wording:
Problems:
Recommendation:
Round 2, Version 1
Tested wording:
Problems:
Round 2, Version 2
Tested wording:
Problems:
3. Clubs/Organizations
We tested several different versions of questions on participation in clubs and organizations. During all of Round 1 and during Version 1 of Round 2, we asked separate questions on participation in 1) religiously-affiliated groups, 2) school-based clubs, and 3) civic or charitable clubs. Based on discussions between the Census Bureau's Population Division and CTI regarding time series concerns, we used the same question wording used in the 1998 SPD core questionnaire during Version 2 of Round 2. This is a single question asking about participation in any club or organization after school or on weekends. To capture detailed information on the specific type of club or organization, we asked three follow-up questions for persons responding positively to the 1998 SPD item, in order to determine if the activity was religiously-affiliated, school-based, or civic/charitable in nature.
In order to understand the differences among the three versions tested, we present the questions on clubs/organizations for the three questionnaire versions tested below. For ease of presentation, the frequency questions that would follow each specific club/organization question have been omitted as are the other enrichment questions that were interspersed between the "club/organization"questions in the questionnaires tested.
Round 1
- The next few questions are about activities (child) may have participated in outside of the regular school day.
- Did (child) participate in any school-based clubs or organizations after school or on weekends, such as band, chorus, drama club, or student government since March 1997?
- Did (child) participate in any other civic or charitable organizations such as Scouts, Big Brothers or Sisters, 4H, or any similar organizations since March 1997?
Since March 1997, did [child] participate in any religiously-affiliated youth groups or organizations, not including attendance at regular religious services?
Round 2, Version 1
- Since March 1997, did (child) participate in any civic or charitable organization, such as Scouts, Big Brothers or Sisters, 4H, or any similar organization?
- Since March 1997, did (child) participate in any school-based clubs or organizations such as band, chorus, drama club, or student government?
- Since March 1997, did (child) participate in any religiously-affiliated activities, youth groups, or organizations, including attendance at religious services?
Round 2, Version 2
- Did (child) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, Boys/Girls club, or a religious group at any time since April 1997? (This is a "screener"item. If "yes,"then we ask items 2 through 4 below.)
- Were any of them school-based clubs or organizations such as band, chorus, drama club, or student government?
- Of these clubs or organizations that (child) participated in, were any of them civic or charitable organizations such as Scouts, Big Brothers or Sisters, 4H, or any similar organization?
- Of the clubs and organizations that (child) participated in, were any of them religiously-affiliated activities, youth groups or organizations, including attendance at religious services?
Below are the item-by-item results for each round of testing. Shown first are the results for the "screener"item from Version 2 of Round 2. Following that are the results for each round of testing by the type of club/organization.
Round 2, Version 2
Tested wording:
Problems:
There also needs to be a clearer definition of religious clubs or organizations. Two respondents mentioned religious education classes. One of these respondents included religious education classes by answering affirmatively to this question; the other respondent did not. That respondent did not include it because s/he did not consider it to be a group. She said the child didn't sign up for "Sunday school like you would sign up for Boy Scouts; we just participate."
Round 1
Tested wording:
Since March 1997, did [child] participate in any religiously-affiliated youth groups or organizations, not including attendance at regular religious services?
Problems:
Based on cognitive testing, we think that the order of the Round 1 enrichment questions is awkward and that it would be better not to start the series with a religiously-oriented question. In addition, there is a separate question on attendance at religious services, social events and classes in the cognitive stimulation section, and we wondered whether the two questions could be combined into one item on religious activity. In Round 2, we reordered the series and combined the two items.
Round 2, Version 1
Tested wording:
Problems:
Round 2, Version 2
Tested wording:
Problems:
Problems similar to those discussed in the Version 1 wording were encountered when this question was asked. It confirmed the need for two separate questions to determine participation in religiously-affiliated activities such as youth groups and organizations and to determine participation in religious services.
Round 1
Tested wording:
Problems:
Round 2, Version 1
Tested wording:
Problems:
Round 2, Version 2
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 2, Version 1
Tested wording:
Problems:
Round 2, Version 2
Tested wording:
Problems:
These questions were asked if the respondent answered affirmatively to an enrichment question.
Tested wording:
<1> 1-2 months
<2> 3-6 months
<3> 7-10 months
<4> 11-12 months
Problems:
Recommendation:
Tested wording:
<1> Once or twice a month
<2> About once a week
<3< Several times a week
Problems:
Recommendation:
These questions were asked if the respondent answered affirmatively to an enrichment question.
Tested wording:
<1> 1-2 months
<2> 3-6 months
<3> 7-10 months
<4> 11-12 months
Problems:
Tested wording:
__April 1997 __
October 1997
__May 1997 __
November 1997
__June 1997 __
December 1997
__July 1997 __
January 1998
__August 1997
__ February 1998
__ September 1997 __ March 1998
Problems:
Tested wording:
<1> Once or twice a month
<2> About once or twice a week
<3< Several times a week
Problems:
4. Gang-related activity
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problem:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
D. Positive Behaviors/Social Competence
(Universe : Ages 3-11)
Round 2
Tested wording:
a. Is warm and loving. Is that often true, sometimes true, or not true at all of (fill name)? (Same response options used for all items)
<1> Often true
<2> Sometimes true
<3> Not true at all
b. Gets along well with other children. Is that often true, sometimes true, or not true at all of (fill name)?
c. Is admired and well liked by other children. Is that often true, sometimes true, or not true at all of (fill name)?
d. Shows concern for other people's feelings. Is that often true, sometimes true, or not true at all of (fill name)?
e. Is helpful and cooperative. Is that often true, sometimes true, or not true at all of (fill name)?
f. Is considerate and thoughtful of other children. Is that often true, sometimes true, or not true at all of (fill name)?
g. Tends to give, lend, and share. Is that often true, sometimes true, or not true at all of (fill name)?
Problems Series:
During our initial review of this series, prior to testing, we modified the last response option (within the question) to read "not true at all "instead of "not true." This was done so the response options would be consistent with the wording of the introduction. During cognitive testing, none of the respondents chose the "not true at all" response option. We wonder if that may be due, in part, to the modified wording which, in retrospect, sounds so definitive that it is unlikely any parent would select that option.
Also, during the first three interviews, the question wording did not specify the name of the reference child. For interviews 4-16, we used a slightly revised wording that included the child's name so the parent could stay focused on which child they were providing information.
Recommendation:
Here are some statements which may or may not describe your child. As I read each statement, decide whether it is often true, sometimes true, or not true of your child over the last three months. (Fill name of child):
Is warm and loving. Is that often true, sometimes true, or not
true of (fill name)?
<1> Often true
<2> Sometimes true
<3> Not true
(Universe : Ages 12-17)
Tested wording:
a. Plans ahead. Is that often true, sometimes true, or not true at all of (fill name)?
b. Is admired and well liked by kids the same age. Is that often true, sometimes true, or not true at all of (fill name)?
c. Looks for ways to help other people. Is that often true, sometimes true, or not true of at all of (fill name)?
d. Is considerate and thoughtful of other people. Is that often true, sometimes true, or not true at all of (fill name)?
e. Goes out of his or her way to make the family feel proud. Is that often true, sometimes true, or not true at all of (fill name)?
f. Is able to express his/her thoughts and feelings to other people. Is that often true, sometimes true, or not true at all of (fill name)?
g. Is warm and affectionate toward members of our family. Is that often true, sometimes true, or not true at all of (fill name)?
h. Spends his or her own time helping other people. Is that often true, sometimes true, or not true at all of (fill name)?
Problems (Series):
E. Grades/Achievement
(Universe : ages 6-17)
Tested Wording:
<1> Very well
<2> Well
<3> Average
<4> Below average
<5> Not well at all
<6> Child not attending school
Problems:
F. Household Resources (aka Cognitive Stimulation)
(Universe: Ages 0-17 )
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Recommendation:
Series Comment:
G. Conflict Between "Parents" Who Co-reside
(Universe is two-parent households, which includes biological, step, adoptive,
and foster parents, and households in which one parent is living with
an unmarried partner, ages 0-17, flashcard)
Round 1
Tested wording:
a. Household tasks (same response options used in all items)
<1> Never
<2> A few times a year
<3> Once a month
<4> Several times a month
<5> About once a week
<6> Several times a week
<7> Almost every day
b. Money
c. Spending time together
d. Sex
e. In-laws
f. The children
Problems:
Recommendation:
a. Household tasks (same response options used in all items)
<1> Never
<2> A few times a year
<3> Once a month
<4> Several times a month
<5> About once a week
<6> Several times a week
<7> Almost every day
b. Money
c. Spending time together
d. Sex
e. In-laws
f. The children
Round 1
Tested wording:
Problems:
H. Conflict Between Biological Parents Who Do Not
Co-reside
(Universe : Households where there is an absent biological or adoptive
parent of one of the resident children, ages 0-17, flashcard)
Round 1
Tested Wording:
a. where child lives? (Same response options used for other items)
<1> No conflict
<2> A little conflict
<3> Some conflict
<4> Pretty much conflict
<5> Great deal of conflict
<6> I have no contact with my child's biological (mother/father)
b. how (he/she) is raised?
c. how you spend money on (child)?
d. how (he/she) spends money on (child)?
e. the time (she/he) spends with (child)?
f. (her/his) financial contribution to (child's) support?
Problems:
Recommendation:
Revised wording follows:
In what month and year did (designated parent) last have contact of any kind, including phone calls, letters, or face-to-face contact with (child's) (mother/father)?
____Month ______Year (If prior to March 1997, skip to next series.)
I am going to read you a list of issues that you and (child's) (mother/father) may have conflict over. By conflict we mean, arguments, disagreements, or fighting. For each one, please tell me if you have no conflict, a little, some, pretty much, or a great deal of conflict.
This series was tested in Round 2.
Round 1
Tested wording:
Problems:
Round 2
Tested wording:
____Month ______Year (If prior to March 1997, skip to next series.)
I am going to read you a list of issues that you and (child's) (mother/father) may have conflict over. By conflict we mean, arguments, disagreements, or fighting. For each one, please tell me if you have no conflict, a little, some, pretty much , or a great deal of conflict.
During the past year, how much conflict did you and (child's father/mother) have about....
a. where child lives?
<1> No conflict
<2> A little conflict
<3> Some conflict
<4> Pretty much conflict
<5> Great deal of conflict
<6> I have no contact with my child's biological (mother/father)
b. how (he/she) is raised?
c. how you spend money on (child)?
d. how (he/she) spends money on (child)?
e. the time (she/he) spends with (child)?
f. (her/his) financial contribution to (child's) support?
Problems (Series):
In this series, there are two items (d and f) related to financial spending on/for the child by the absent parent. In some cases, respondents interpreted item d as asking about child support payments, which we don't think is the intent of the question. It seems unnecessary to include two such closely related items.
Recommendation (Series):
Series Comment:
I. FAMILY CONFLICT
(Universe: Ages 0-17)
Round 1
Tested wording:
Problems:
We requested additional information from CTI on the issues identified above. CTI informed us that for this set of questions "family" is defined as the family that lives in the household with the respondent. CTI preferred that we not modify the lead-in to define what was meant by "family."
Recommendation:
Next are some statements about how families get along and settle arguments. For each one please indicate how strongly you agree or disagree that the statement is like the family you currently live with.
Round 2
Tested wording:
Problems:
Recommendation:
Round 1
Tested wording:
<1> Strongly agree
<2> Somewhat agree
<3> Somewhat disagree
<4> Strongly disagree
Problems:
Recommendation:
Round 2
Tested wording
Problems:
Round 1
Tested wording
Problem:
Recommendation:
Round 2
Tested wording
Problems:
Recommendation:
1. Family members lose their tempers. "This eliminates the modifier of "hardly ever." (Eliminating the modifier on all the questions might prove to provide more consistent answers across the series. This recommendation is based more on an expert review, than on results of cognitive interviewing.)
2. "Family members often lose their tempers." This replaces "hardly ever" with "often." This changes the direction of the statement from a negative direction to a positive direction. Since the other statements are this direction, this maintains consistency among the questions.
3. "Family members seldom lose their tempers. This replaces "hardly ever" with "seldom." This maintains a similar direction to the question and response in the original version. "Seldom" was used by two respondents when paraphrasing the original question wording.
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Recommendation:
Round 2:
Tested wording:
Problems:
Recommendation:
Problems (Series):
Series Comment:
J. Attitudes Towards Welfare/work
(Universe : All respondents)
Round 1:
Tested wording:
Welfare makes people work less than they would if there wasn't a welfare system. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?
<1> strongly agree
<2> agree
<3> in the middle
<4> disagree
<5> strongly disagree
<d> don't know
Problems:
Recommendation:
Round 2:
Tested wording:
Problems:
Recommendation:
Round 1:
Tested wording:
Problems:
Recommendation:
Round 2:
Tested wording:
Problems:
Recommendation:
Round 1:
Tested wording:
Problems:
Recommendation:
Round 2:
Tested wording:
Problems:
Recommendation:
Round 1:
Tested wording:
Problems:
Round 1:
Tested wording:
Problems;
Recommendation:
Round 2:
Tested wording:
Problems:
Recommendation:
Round 1:
Tested wording:
Problems:
Round 1:
Tested wording:
Problems:
Series Comment:
Series comments:
CTI recommended that we delete the "in the middle" response option so the response scales between the SPD and the National Survey of American Families (NSAF) will be consistent. The NSAF does not have the "in the middle" response option. We recommend keeping this category. Many respondents used that response option since they didn't want to categorize people one way or the other. Other respondents reasoned that for some people the statement was true and for some people the statement wasn't true. Without the inclusion of this response option, respondents may be forced into selecting an response option which may not represent their true attitude. The other response categories worked well. No one used the "don't know" response option.
We observed differences in interpretation of the statements based on the two rounds of testing. Round 1 respondents, who were more likely to have been current welfare or former welfare recipients were more likely to interpret the statements more literally. Round 2 respondents, who were, in general, not welfare recipients, understood the implied meaning of the statements. Thus, in our opinion, the statements seemed to be more accurately interpreted during Round 2. We feel this is primarily due to the education level and life experiences of the two groups. If the statements are included in the SPD99, analysts should keep in mind that information collected from the cognitive interviews suggest that the interpretation of the statements may be quite different among various subgroups of the population.
K. Knowledge about welfare regulations
Round 1
Tested wording:
Problems:
Round
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems (Series)
Round 1
Tested wording:
a. Where must a teenage who has a baby live in order to receive welfare benefits? (Read categories)
<1> She can get her own apartment
<2> She must live with her parents
<3> She must live with her parents unless the State makes an exception that she can live with another adults
<D> Don't know
b. In order to receive welfare in your State, what is the rule for school attendance for a teenage who has a baby? (Read categories)
<1> She can stay at home with her baby until the time limit for welfare runs out
<2> She is required to return to school when the baby is three months old
<D> Don't know
Problems:
We posed questions to CTI regarding the intent of the question. CTI informed us that they do not have a strong preference to retain these items in the parent interview, but if they are retained, CTI prefers the wording to be identical to that in the adolescent self-administered questionnaire.
Recommendation:
Recommended question wordings (from the SPD98 adolescent self-administered questionnaire) are provided below.
The next two question are about welfare rules in your State for teenagers who are age 17 or younger.
Can a teenager who has had a baby get her own apartment without any adult supervision and still receive welfare benefits?
In order to receive welfare, is there a rule requiring a teenager who has had a baby to attend school?
IV. COGNITIVE TEST RESULTS - EXPANDED CONTENT FOR THE SPD CORE INTERVIEW
A. Knowledge of Child Care Subsidies
(Universe: Households with a regular child care arrangement, ages 0-14.)
Round 2
Tested Wording:
Problems:
Recommendation:
Tested Wording:
Problems:
The universe of persons for whom this question is asked should be carefully thought through. It seems inappropriate to ask of middle and upper income households. If a nonworking parent has a regular arrangement for child care a few times a week, so they can go to the gym, or do volunteer work, etc., it seems like this question would not be applicable. Consideration should be given to including an income screener prior to this item.
Recommendation:
B. Child Care Problems
(Universe: Ages 0-14 with regular child care arrangement)
Tested wording:
a. At any time between January and December 1997, did (you/designated parent) have to quit a job, school, a job search, or a training activity, because (you/designated parent) had problems arranging child care or keeping a child care arrangement?
b. At any time between January and December 1997, were (you/designated parent) unable to take a job, or start school, a job search, or a training activity because (you/designated parent) had problems arranging child care or keeping a child care arrangement?
Tested Wording:
a. At any time between January and December 1997, did (you/designated parent) have to quit a job, a job search, a training activity, or school, because (you/designated parent) had problems arranging child care or keeping a child care arrangement?
b. At any time between January and December 1997, were (you/designated parent) unable to take a job, start a job search, start a training activity or school, because (you/designated parent) had problems arranging child care or keeping a child care arrangement?
Problems:
Upon the initial reading of the two questions, many respondents thought that two questions were asking the same thing. It was only AFTER both questions were reread slowly and emphasis was put on the words "quit" and "unable," that most respondents told us there was a difference between the two questions..
Note: This item is also a household-level question, not a child specific question and therefore should appear only after the last child has cycled through the series.
Recommendation:
a. At any time between January and December 1997, were (you/designated parent) unable to TAKE a job, START a job search, START a training activity or START school, because of child care problems.
b. At any time between January and December 1997, did (you/designated parent) QUIT a job, QUIT a job search, QUIT a training activity, or QUIT school, because of child care problems.
Alternative recommendation:
a. At any time between January and December 1998, did you have to do any of the following because of child care problems:
Quit a job Yes No
Quit a job search Yes No
Quit a training activity or quit school Yes No
b. At any time between January and December 1998, were you unable to do any of the following because of child care problems:
Unable to take or start a job? Yes No
Unable to start a job search? Yes No
Unable to start a training activity or start school? Yes No
C. Quality of Child Care
(Universe: Ages 0-14 that are in a regular day care arrangement of family
day care, nursery school/preschool, Head Start programs, childcare/daycare
programs or before or after school care)
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
Round 1
Tested wording:
Problems:
D. Employee Fringe Benefits
(Universe: Persons 15+ who worked during the past calendar year)
Round 2
Tested wording:
a. During 1997, did (employer name) give you paid vacation?
b. How many days per year did this employer provide?
c. During 1997, did (employer name) provide paid sick leave?
d. How many days of paid sick leave per year did this employer provide?
Problems:
Another issue pertains to part-year employment where an employer might provide x number of days per year, but the employee wasn't there long enough to be eligible for the full amount. Some respondents tended to report the full amount even though the employee would not have been able to take that amount. In another instance, the respondent had started a new job in October, was accruing leave from the beginning, but could not access any of her leave until after she had been there 90 days, which would have been in January 1998. Do we want the amount of time a person would be eligible to take? If so, we should word the question to reflect that. (Note: the first question in this series uses the word "give," whereas the similar question in the sick leave series uses the word "provide." The word "give" may have contributed to some of the confusion.)
There did not seem to be as many problems with items c and d (sick leave series) as with the vacation series. (That is most likely due to the probing of the responses to the vacation series.) The issue of whether we are asking for sick leave provided versus sick leave taken exists. One respondent reported that there is not a set amount of sick leave. The first day a person is sick they do not get paid, but they get paid for the second, third, etc. days that they are out sick.
We might also want to consider allowing the FR to record the amount of leave in whatever periodicity the respondent provides. Several respondents reported their leave as X number of hours every pay day, or every two weeks.
Recommendation:
b. During 1998, how many paid vacation days was (name) eligible to take?
c. During 1998, did (employer name) provide paid sick leave?
d. During 1998, how many paid sick leave days was (name) eligible to take?
The sponsoring divisions may also want to consider using ranges as response options for items "b" and "d" above (e.g. 0 days, 1-5 days, 6-10 days, 11-15 days, 16 or more days). Some respondents who don't know the exact number of days may be able to provide a range.
Round 2
Tested wording:
Problems:
E. Health Care Utilization While Uninsured
(Universe: Persons who were uninsured for part or all of the previous
calendar year.)
There is very limited information for this series of questions. Only 4 of the 16 Round 2 respondents had no health insurance at some time during 1997. Of those 4, only 1 had sought medical care during 1997. One other respondent had sought medical care early in 1998 and the questions were administered for this person as well. Both respondents had used emergency rooms, and both respondents had to pay full price for the services. Neither had been asked their income by the emergency room staff.
Round 2
Tested wording:
Did (fill all names of uninsured) receive treatment for an illness or injury while uninsured in 1997?
Did (fill all names of uninsured) receive any routine or preventive care, such as a checkup, prenatal care, immunizations, or family planning while uninsured in 1997?
Did (fill all names of uninsured) receive treatment for a drug or alcohol problem while uninsured in 1997?
Where did (fill all names of uninsured) go to get those health care services?
MARK ALL THAT APPLY
<1> Clinic or Public Health Department
<2> Emergency room
<3> Doctor's office or an office at the hospital
<4> Someplace else
Were these services (fill all names of uninsured) received while uninsured, free, or did (you/name) have to pay something for them?
<1> Free
<2> Paid something
<3> Both
Problems:
Recommendation:
Round 2
Tested wording:
<1> Full price
<2> Reduced price
<3> Both
Did the (clinic/hospital/doctor) ask what your income was before they set a price for the services?
<1> Yes
<2> No
<3> Sometimes
F. Consumer Expenditures
(Universe: All households.)
1. Food Expenditures
Round 1
Tested wording:
Thinking about all the places you shop for food -- for example, the supermarket, warehouse clubs, convenience stores, bakeries, produce stands -- about how much does your household usually spend per week or per month on groceries? (Please include any purchases made with food stamps.)
Problems:
Round 2
Tested wording:
Thinking about all the places you shop for food -- for example, the supermarket, warehouse clubs, convenience stores, bakeries, produce stands -- about how much does your household usually spend per week on groceries? (Please include any purchases made with food stamps.)
READ IF NECESSARY:
Is that per week, every two weeks or every month?
<1> week
<2> every two weeks
<3> every month
Problems:
The son of one respondent is away from home at a boarding school. The respondent is paying his room and board and was not sure whether expenditures for food while he is at school should be included. The respondent decided to estimate the portion of room and board that she thought was for food. Whether this should be included or not needs to be addressed in the training manual and on a help screen.
Some respondents were more comfortable providing a range for this item (e.g. $75-$100 per week) rather than giving an exact dollar amount. The CPS Food Security Supplement (FSS) allows respondents to report a range if they cannot give a single dollar amount. We may want to consider this as well.
Recommendations:
Thinking about all the places (you shop/your household shops) for food -- for example, the supermarket, warehouse clubs, convenience stores, bakeries, produce stands -- about how much does your household usually spend per week on groceries? (Please include any purchases made with food stamps.)
(X) Enter "X" if range given (enter range on separate screen)
$_________
Is that per week, every two weeks or every month?
<1> week
<2> every two weeks
<3> every month
Round 2
Tested wording:
(This item was not included in Round 1. It was tested in Round 2 to see if respondents are including non-food items when reporting grocery expenditures.)
Problems:
Round 1
Tested wording:
Problems:
Round 2
Tested wording:
READ IF NECESSARY:
Is that per week or per month?
<1> week
<2> month
Problems:
Recommendation:
Transportation Expenditures
Round 1
Tested wording:
(Universe: Persons 15 or over who are currently working.)
<1> car, truck, van
<2> bus or street car
<3> subway or elevated
<4> railroad
<5> taxicab
<6> motorcycle
<7> bicycle
<8> other vehicle
<9> walked only
<10> works at home
Problems:
Round 1
Tested wording:
Problems:
Tested wording:
Problems:
Recommendation:
Tested wording:
Problems:
3. School Expenditures
Tested wording:
.....rental or purchase of school books, equipment, or supplies;
or
.....tuition (excluding daycare/nursery school expenses); or
.....room and board?
Problems:
Recommendation:
Including adults and children, did anyone in this household have
any school-related expenses during the past 12 months, such as:
.....school books, supplies, or trips;
.....tuition, room or board (excluding daycare/nursery school expenses)
Tested wording:
Problem:
Recommendation:
4. Health Insurance Expenditures
(Universe: Persons who had health insurance for all or part of the preceding
calendar year.)
Tested wording:
$_________ __Weekly
__Biweekly
__Twice monthly
__Monthly
__Quarterly
__Annually
Problems:
G. Months Receiving Rental Assistance
(Universe: Persons who reported receiving rental assistance during the
preceding calendar year.)
Round 2
Tested wording:
Problems:
V. COGNITIVE TEST RESULTS - CHILDREN'S RESIDENTIAL HISTORY
We tested two completely different versions of the residential history module. Presented below are general comments regarding the series we tested followed by the results from Round 1 and then the results from Round 2. The Steering Committee has tentatively decided to field this module in 2000, rather than 1999. Recommendations for revising question wording are not included in this report since discussions are currently taking place between Census and external experts regarding the objectives of the series as well as deciding questionnaire design issues. Problems identified during cognitive testing, however, are included.
A. General comments
1. What is the universe for the question on children under 18 who live outside the household? Which adults do we want to ask this about? Should it be asked of 100-level people age 15 or over? Should it be asked of all adults in the household? Questions on residential history are asked for all children added to the roster at this point. The concern is that we will collect residential history data for children who have no attachment whatsoever to this household and for whom we have no data other than residential history.
2. In Round 1, the questions were framed in terms of separations of the child from the respondent. Thus, the responses depended on who answered the questions and didn't provide consistent data for all children. For example, if the child is living with a grandparent, the separation data would be separations from the grandparent. No data were collected on separations from the biological mother or father. Moreover, even if the respondent was a biological parent, these questions did not obtain data on separations from the other biological parent. In Round 2 we addressed this issue by asking about separations from the biological mother and biological father, as well as the person the child is currently living with.
3. In Round 1, the presumption was that the child went to live with someone else during the separation. In some cases, the respondent went to live elsewhere (e.g. due to work) and the child stayed at home with the other parent. We addressed this by modifying the question wording in Round 2.
4. The length of time of the separation and the number of separations for which we collected data differed between Rounds 1 and 2 . In Round 1 we collected separations of one month or more and only collected data on the first and last separation during the child's lifetime. In Round 2 we collected separations of four months or more and collected data on up to four separations during the child's lifetime. We need to determine what length of separation we want to collect data for and the number of separations. Do we need data for the child's lifetime or just since the panel began?
5. In Round 2, we added questions on additional people who may have provided "care for the child." The question about living with a grandparent caused problems for some respondents particularly when the biological parent and child lived with the child's grandparent and the biological parent continued to care for the child. In such cases, the child lived in the same house with the grandparent, but the grandparent wasn't in charge of caring for the child. The question about living with a step-parent was awkward for respondents who are currently living with both biological parents. This could be corrected by skipping these people over the stepparent questions.
Item by item results are shown below.
B. Results of Round 1 Testing
(Universe: All households.)
Tested Universe:
Problems:
Tested wording:
Name of Child Sex Age Relation of child to this person
Do you have any (other) biological children who are currently under 18 and living somewhere else?
Problem:
Tested wording:
What is that child's sex?
What is that child's age?
Problems:
Display roster of all children younger than 18 for whom this person is the biological, adoptive, or step parent or guardian.
Tested wording:
Name of Child Sex Age Relation of child to this person
Is that correct?
Problems:
Tested wording:
Problems:
For the most part, the results indicated that respondents understood question and were able to answer it. Some people knew answer immediately. Others needed to think whether period lasted 30 days or more.
Tested wording:
Problems:
Tested wording:
____ months
____ years
Problems:
Tested wording:
Problems:
Tested wording:
____ months
Problems:
Tested wording:
MARK ALL THAT APPLY
<1> Another person or family
<2> Institution (hospital, orphanage, etc.)
<3> Living alone
<4> Other arrangement
<D> Don't know
<R> Refused
Problem:
Tested wording:
<1> Child's other parent
<2> Child's grandparent
<3> Child's aunt or uncle
<4> Other relative of child
<5> Friend
<6> Foster parent
<7> Other (specify ______________)
Problems:
Tested wording:
MARK AS MANY REASONS AS APPLY.
<1> Child on vacation with other relatives or friends
<2> Child in summer camp
<3> Child in boarding school
<4> Marital conflict between parents
<5> Parent/guardian on vacation or business trip
<6> Child living with other biological parent
<7> Physical health problem of parent
<8> Mental health problem of parent
<9> Substance abuse problem of parent
<10> Incarceration of parent
<11> Physical health problem of child
<12> Mental health problem of child
<13> Substance abuse problem of child
<14> Behavior problem of child
<15> Financial hardship
<16> Child removed by government agency (court, child protective
services)
<17> Other reason (specify____________________)
Problems:
C. Results of Round 2 Testing
Tested wording:
Name of Child Sex Age Relation of child to this person
Do you have any other biological children who are currently under 18 and living somewhere else?
Note: If appropriate, ask:
Does (spouse's name) have any biological children who are currently under 18 and living somewhere else?
Problems:
Tested wording:
What is that child's sex?
What is that child's age?
Problems:
Tested wording:
Problems:
Tested wording:
Problems:
Tested wording:
Problems:
Tested wording:
____ years ____ months old
Problems:
Tested wording:
_____ months
_____ years
Problems:
Tested wording (Note: Two versions of the question were tested during Round 2):
a. How was the person that (child's name) lived with (the first/second...time) related to (child's name)?
b. Who was the person that (child's name) was living with when he/she was living apart from (you) (the first/second...time)?
SHOW FLASHCARD
MARK ALL THAT APPLY
<1> Child's biological/adoptive father
<2> Child's stepparent
<3> Child's grandparent
<4> Other relative
<5> Friend
<6> Foster parent
<7> Away at school
<8> Hospital or other health-related institution
<9> Other institution (e.g. orphanage, juvenile detention
center)
<10> Living alone
<11> Other (specify _____________)
Problems:
Tested wording:
MARK AS MANY REASONS AS APPLY.
<1> Child went to live with biological or adoptive parent
<2> Child adopted
<3> Child's illness
<4> Child in boarding school
<5> Child placed in foster care
<6> Child jailed/institutionalized
<7> Child cared for by friend/relative while mother looked for work
<8> Child got married
<9> Child out on own/ got a job
<10> Mother's illness
<11> Mother's work schedule
<12> Mother jailed/institutionalized
<13> Mother moved elsewhere
<14> Mother and father separated or divorced
<15> Mother died
<16> Agency/court took child away from mother
<17> Financial hardship
<18> Other reason (specify________________________________)
Problems:
Tested wording:
Problems:
Tested wording:
Problems:
One respondent, whose husband is in the military, said that her husband was usually gone for 105 days at a time, which was less than the four months asked about in the question. This same respondent reported two extended absences from the father for one of her children, but almost failed to report the same absences for the other child. It was later, when asked the question about living with a grandparent that she caught her mistake. This is an instance where a calendar aid would likely have helped to improve accuracy of reporting across children in this family.
One respondent had trouble reporting this information about the biological children of her husband who live outside the household. Her husband has two biological children, by two different women. The children have never lived in the respondent's household.
Tested wording:
____ years ____ months old
Problems:
Tested wording:
____ _months
_years
Problems:
Tested wording (Note Two different versions of the question was tested in Round 2):
a. How was the person that (child's name) lived with (the first/second...time)
related to (child's name)?
b. Who was the person that (child's name) was living with when he/she
was living apart from (you) the (the first/second...time)?
SHOW FLASHCARD;
MARK ALL THAT APPLY
<1> Child's biological/adoptive mother
<2> Child's stepparent
<3> Child's grandparent
<4> Other relative
<5> Friend
<6> Foster parent
<7> Away at school
<8> Hospital or other health-related institution
<9> Other institution (e.g. orphanage, juvenile detention
center)
<10> Living alone
<11> Other (specify _____________________)
Problems:
The third respondent, who also lived with his biological mother while the father was away, thought that the question might mean "who else does he live with." He explained that his wife lives with her sister and her family.
There were no problems noted with version "b."
Tested wording:
MARK AS MANY REASONS AS APPLY
<1> Child went to live with biological/adoptive parent
<2> Child adopted
<3> Child's illness
<4> Child in boarding school
<5> Child placed in foster care
<6> Child jailed/institutionalized
<7> Child cared for by friend/relative while father looked
for work
<8> Child got married
<9> Child out on own/ got a job
<10> Father's illness
<11> Father's work schedule
<12> Father jailed/institutionalized
<13> Father moved elsewhere
<14> Father and mother separated or divorced
<15> Father died
<16> Agency/court took child away from father
<17> Financial hardship
<18> Other reason (specify____________________________________)
Problems:
Tested wording:
Problems:
Tested wording:
Problems:
Tested wording:
____ years ____ months old
Problems:
Tested wording:
________ years ____ months
Problems:
Tested wording:
SHOW FLASHCARD
MARK ALL THAT APPLY
<1> Child's biological/adoptive father
<2> Child's stepparent
<3> Child's grandparent
<4> Other relative
<5> Friend
<6> Foster parent
<7> Away at school
<8> Hospital or other health-related institution
<9> Other institution (e.g. orphanage, juvenile detention
center)
<10> Living alone
<11> Other (specify __________________________________________)
Problems:
Tested wording:
MARK AS MANY REASONS AS APPLY
<1> Child living with biological/adoptive parent
<2> Child's illness
<3> Child in boarding school
<4> Child placed in foster care
<5> Child jailed/institutionalized
<6> Child cared for by friend/relative while (you/designated
parent/guardian) looked for work
<7> Child got married
<8> Child out on own/ got a job
<9> (Your/designated parent's/guardian's) illness
<10> (Your/designated parent's/guardian's) work schedule
<11> (You/designated parent/guardian) jailed/institutionalized
<12> (You/designated parent/guardian) moved elsewhere
<13> Marital conflict between guardians
<14> Financial hardship
<15> Other reason (specify________________________)
Problems:
Tested wording:
____ years ____ months old
Problems:
Tested wording:
_________ months ______ years
Problems:
Tested wording:
SHOW FLASHCARD
MARK ALL THAT APPLY
<1> Child's biological/adoptive father
<2> Child's stepparent
<3> Child's grandparent
<4> Other relative
<5> Friend
<6> Foster parent
<7> Away at school
<8> Hospital or other health-related institution
<9> Other institution (e.g. orphanage, juvenile detention
center)
<10> Living alone
<11> Other (specify ________________________________________)
Problems:
Tested wording:
MARK AS MANY REASONS AS APPLY
<1> Child living with biological/adoptive parent
<2> Child's illness
<3> Child in boarding school
<4> Child placed in foster care
<5> Child jailed/institutionalized
<6> Child cared for by friend/relative while (you/designated
parent/guardian) looked for work
<7> Child got married
<8> Child out on own/got job
<9> (You/designated parent's/guardian's) illness
<10> (You/designated parent's/guardian's) work schedule
<11> (You/designated parent/guardian) jailed/institutionalized
<12> (You/designated parent/guardian) moved elsewhere
<13> Marital conflict between guardians
<14> Financial hardship
<15> Other reason (specify_________________________)
Problems:
Tested wording:
Did (child's name) ever live with a stepmother for a period of 4 months or more from the time (he/she) was born until now?
Problems:
Tested wording:
From age ____ years ____months
To age ____ years ____months
Were there any other times? If "yes", enter ages.
From age ____ years ____months
To age ____ years ____months
Problems:
Tested wording:
Problems:
Tested wording:
From age ____ years ____months
To age ____ years ____months
Were there any other times? If "yes", enter ages.
From age ____ years ____months
To age ____ years ____months
Problems:
Tested wording:
Problems:
Tested wording:
From age ____ years ____months
To age ____ years ____months
Problems:
Tested wording:
Problems:
Tested wording:
Problems:
Tested wording:
From age ____ years ____months
To age ____ years ____months
Problems:
Tested wording:
Problems:
Tested wording:
Problems
Tested wording:
From age ____ years ____months
To age ____ years ____months
Probe: Any other times? If yes, enter ages.
From age ____ years ____months
To age ____ years ____months
Problems:
10/97 Attachment A
The Proposed Time Line for Developing and Testing
a
Five to Ten Minute Module
on
The 1999 SPD Survey
|
November 1 ,1997 thru January 31, 1998 |
Identify Module Content |
|
February 1, 1998 thru February 28, 1998 |
Develop Test Plans and Protocols
|
|
March 1, 1998 thru April 30, 1998 |
Cognitive Testing of Questions
|
|
May 1, 1998 thru June 15, 1998 |
Analysis of Interviews and Preparations of an Analysis Report
|
|
June 16, 1998 |
Issue Cognitive Analysis Report |
|
June 17, 1998 thru July 31, 1998 |
Finalize Module Questions and Skip Patterns |
|
August 1, 1998 thru December, 31, 1998 |
Specify, Author and Test Instrument |
|
January 4, 1999 |
OMB receives SPD 1999 Questionnaire Request |
|
January 4, 1999 thru March 15, 1999 |
Systems Test of the SPD Instrument |
|
April 30, 1999 thru June 30, 1999 |
Data Collection |
Attachment B
OBJECTIVES FOR THE 1999 SURVEY OF PROGRAM DYNAMICS
CHILD AND FAMILY WELL-BEING QUESTIONS
(Prepared by Child Trends, Inc.)
POSITIVE INTERACTION
Stress due to mandated employment, sanctions, or time limits, may affect how parents interact with their children. Among the most consistent findings in child development research is that parental warmth exhibited in the first few years of life is one of the strongest predictors of positive developmental outcomes. Warmth, sensitivity, and nurturance are global constructs that include numerous affective and behavioral components. Supportive parenting is associated with positive child outcomes, even in the presence of extreme socioeconomic disadvantage. Sampson and Laub also suggest that positive parenting provides a buffering effect for children in poverty.
COGNITIVE STIMULATION/HOUSEHOLD RESOURCES
Welfare reform provisions have several implications for parents' provision of cognitively stimulating resources and children's participation in cognitively stimulating activities. For instance, by altering labor-force behavior, the size and duration of cash grants, and family composition, numerous waiver provisions may produce important changes in the level of family income. Family income can affect children because money can be used to buy things that are essential for growth and development. Money may provide such resources as books, toys, and musical instruments, that facilitate learning. Money can also enable participation in child care settings, activities and lessons that support positive development. Lack of resources to purchase toys or good quality child care may mean that low-income children do not have as many stimulating experiences (such as reading, playing learning games or playing musical instruments) as do children in more affluent families.
An environment that is stimulus-rich without being overwhelming and chaotic is important for children's healthy cognitive and emotional development. High levels of appropriate stimulation in multiple aspects of the environment, as measured by the Home Observation for Measurement of the Environment scales (HOME) have been found to be positively associated with social and mental development, including measures of cognitive development and IQ in preschool and later, and school achievement.
For younger children, we have included a cognitive stimulation scale used in the Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K) Parent survey. For older children and adolescents, we have included a set of items adapted from the HOME scale and developed by Child Trends, Inc.
CONFLICT BETWEEN PARENTS
One of the goals of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWOR) is to reduce dependence on public assistance through marriage and to encourage the formation of two-parent families. At the present time, there is not much that is known about the impacts of prior welfare reforms on family formation and dissolution. However, the legislation potentially has several implications for the quality of the marital relationship. First, sanctions and time limits may encourage the formation of families for recipients who are at risk of losing some of their income. These marriages may be harmonious or conflictual, depending on the nature of the relationship prior to the marriage. Other incentives may force adults to stay in high-conflict relationships when they might have otherwise separated. The quality of the marital relationship is also associated with economic instability. Thus, families that move into poverty because of welfare reforms may find that their lives are more stressful, conflictual, and unpredictable. On the other hand, improvements in a family's financial circumstances through employment may lessen marital conflict.
There is substantial evidence that marital conflict, like marital status, is strongly linked to children's adjustment. There is also evidence that the children of married parents in high conflict relationships have adjustment problems that are similar to those experienced by children of single parents. Welfare reform program incentives that lead adults to stay in high-conflict relationships may not result in benefits to children.
For biological parents who live in the same household, we have included a brief set of items used previously in the National Survey of Families and Households. We have included a set of seven items, also from the National Survey of Families and Households, for those biological parents who do not reside together.
FAMILY CONFLICT
Several welfare reform provisions, such as sanctions, time limits, and mandated participation in employment, may serve to increase family stress and conflict. It is not only interparental conflict, but other forms of family conflict that appear to have negative effects on children. Exposure to physically violent conflict, in particular, is associated with serious behavioral and emotional disorders in children. In contrast to family conflict and violence, family cohesion is associated with positive outcomes for children.
As a measure of family conflict, we recommend a 6-item family conflict scale used in the Panel Study of Income Dynamics Child Supplement 1997. This is an adapted version of the Family Environment Rating Scale (FERS) which is also being used in the Early Head Start 14-month parent interview (except for one item about calmly discussing problems).
CHILD CARE QUALITY
Children's activities and time use are very likely to be affected by changes in maternal employment that may arise due to welfare reform provisions. Younger children are likely to experience an increase in nonparental child care. Extent and quality of child care in turn have the potential to affect children's development. The research literature suggests that high quality early childhood programs can play a significant, positive role in the lives of disadvantaged children. Quality of care and stability of care over time have been found to be related to children's cognitive and socioemotional development.
We have included a series of questions from the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies (formerly the JOBS Child Outcomes Study) which tap two aspects of child care quality: group size, and the ratio of providers to children.
AGGRAVATION IN PARENTING
Parents who are mandated to participate in employment and educational activities might experience an increase in their subjective sense of stress, which can lead to more impatient, more aggravated parenting practices. For example, questionable feelings of self-efficacy, time stress, or difficulties securing transportation or child care might increase parents' feelings of aggravation in the parenting role. Alternatively, involvement in activities beyond the family may diminish negative feelings toward the parenting role. Greater parenting aggravation, in turn, is associated with poorer child outcomes. The 5-item Aggravation in Parenting Scale was created for the Child Outcomes Study of the National Evaluation of Welfare-to-Work Strategies (formerly the JOBS Child Outcomes Study) and has been used previously in the National Survey of America's Families, as well as the PSID 1997.
FAMILY ROUTINES
In the debate about the TANF legislation, there was much discussion about how the movement from welfare to work would cause parents to establish regular routines, such as regular bedtimes and mealtimes. Through this mechanism, child well-being would be enhanced. The actual effect of the welfare reform legislation on family routines is not known; it is plausible that the effect could be positive or negative. Employment may improve family routines, as mentioned above. Alternatively, the home environment may become more disorganized and less routinized as a result of increased stress or because employment occurs sporadically or at varying hours. Parents who work long hours could have less time to spend with children and less time to participate in family activities. Prior research studies have demonstrated that children benefit from organized and predictable family routines. The provision of regularly scheduled routines or rituals such as meals and bedtime activities can also be beneficial to parents because they "...help to define roles and responsibilities within the family and may be particularly important in the early stages of parenthood as roles and responsibilities become more clearly defined in family organization.
We have included a family routines scale used previously in the ECLS-K Parent Survey.
BEHAVIOR PROBLEMS
Welfare reform provisions may affect children's behavior through several different conduits. For instance, it is possible that maternal stress or depression rates could increase due to reduced grants, because of sanctions or time limits, or due to the mandatory nature of the welfare programs with regard to participation. Maternal depression, in turn, is related to an increased incidence of behavior problems in children. If the welfare program affects the family's living arrangements or socioeconomic status, changes in behavior problems may also result. Yet if welfare programs bring about enhanced maternal psychological well-being (for example, because of satisfaction with new employment roles), improved family economic well-being, and/or residence in a better neighborhood, behavior problems could diminish. Children's behavior problems, in turn, are related to later negative outcomes during adolescence.
We have included an adapted version of a mental health indicator measure developed by Achenbach for the National Health Interview Survey that was also included in the National Survey of America's Families.
SCHOOL ENGAGEMENT
PL 104-193 has several provisions that are targeted toward increasing a behavioral measure of school engagement (i.e., school attendance) among children. For instance, states have the option to sanction families in which an adult fails to ensure that his or her children attend school. School engagement is important because of strong evidence that school absences are associated with poor academic achievement and school grades, school dropout, disruptive classroom behavior, and juvenile delinquency. Engagement, in turn, has been found to predict better academic performance over time. For example, James Connell and his colleagues report that parent, child, and teacher reports of school engagement are predictive of school outcomes, such as attendance, grade point average, test scores, suspensions, and grade retention, measured two years later.
A short four-item parent report engagement scale has recently been developed by Connell and Lisa Bridges from the Institute for Research and Reform in Education for use in the Urban Institute/Child Trends' National Survey of America's Families. We are recommending this scale for inclusion in the 1999 SPD.
Attachment C
Proposed Content NOT cognitively tested by CSMR
Aggravation in Parenting
Please tell me how often the following statement are true for you.
(Response options: all of the time, most of the time, some of the time,
none of the time.)
My child/children does/do things that really bother me a lot.
I find myself giving up more of my life to meet my child's (children's)
needs than I ever expected.
I feel angry with my child (children).
I feel trapped by my responsibilities as my child's (children's) parent.
I felt my child (children) is/are much harder to care for than most children
School Engagement
For each of the following statements, please tell me if you think it is
often true, sometimes true, or not true. Would you say (child)..
Cares about doing well in school?
Only works on schoolwork when forced to?
Does just enough schoolwork to get by?
Always does homework?
Family Routines
I'm going to read some statements about things that may occur in your
family.
In a typical week, please tell me the number of days...
At least some of the family eats breakfast together.
Your family eats dinner together.
The evening meal is served at a regular time.
Children's Routines
(children ages 2-5)
I'm going to read some statements about things that may occur in your
family.
In a typical week, please tell me the number of days..
(Child's name)'s bedtime routine is the same.
(children ages 2-11)
(Child's name)'s has breakfast at a regular time.
(Child's name)'s goes to bed at a regular time.
(children ages 6-17)
During the school year, how many times a week does (child) usually get
his/her homework done on time?
During the school year, how often is (child) usually late for school?
During the school year, how often is (child) usually late for a class?
Mental Health Indicator
Ages 2-3
Now I am going to read some statements that describe behavior problems
many children have. Please tell me whether each statement has been OFTEN
true, SOMETIMES true of NOT true of (child) during the past 3 months.
Has been uncooperative. Has that been OFTEN true, SOMETIMES true, or NOT true of (child) in the past 3 months?
Has trouble getting to sleep.
Has speech problems.
Is unhappy, sad, or depressed.
Has temper tantrums or a hot temper.
Has been nervous or high strung.
Ages 4-11
Has been nervous, high strung, or tense.
Can't concentrate or pay attention for long.
Doesn't get along with other kids.
Feels worthless or inferior.
Has been unhappy, sad, or depressed.
Acts too young for his/her age.
Ages 12-17
Lies or cheats.
Doesn't get along with other kids.
Has been unhappy, sad, or depressed.
Can't concentrate or pay attention for long.
Does poorly at schoolwork.
Has trouble sleeping.
Food Bank
In the last 12 months did (you/you or other adults in your household)
ever get emergency food from a church, a food pantry, or food bank?
How often did this happen - almost every month, some months but not every month, or in only 1 or 2 months?
In the last 12 months, did (you/you or other adults in your household) ever eat any meals at a soup kitchen?
Coping Mechanisms When Losing Benefits
What did you do to get by when your family lost benefits? (Code all that
apply)
01 Cut back on expenses for necessities (food, doctor's bills, etc.)
02 Cut back on discretionary expenses (e.g., toys, movies, etc.)
03 Moved to cheaper housing
04 Moved in with others /doubled up
05 Stopped paying bills/paid bills late
06 Got a job
07 Did something illegal (sold drugs, shoplifted, prostitution)
08 Borrowed money from friends/family
09 Got my children's father to give me child support
10 Applied for benefits in another program
11 Placed my child/children in someone else's care
12 Got married
13 Other:______________
Substance Abuse
CIDI-SF 12 Month Dsm-IV Version
Alcohol Dependence
G1. The next questions are about how frequently you drink alcoholic beverages. By a "drink" we mean either a bottle of beer, a wine cooler, a glass of wine, a shot of liquor, or a mixed drink.. With these definitions in mind, what is the largest number of drinks you had in any single day during the past 12 months--none, between one and three, four to ten, eleven to twenty, or more than twenty drinks in a single day?
Interviewer: If R Volunteers "I never drink", Accept the answer and check "NONE" in the response options
1. None, go to P. 25, H1
2. 1-3, go to P. 25, H1
3. 4-10,
4. 11-20
5. MORE THAN 20
Interviewer: The next questions are awkwardly worded. Read slowly.
G2. In the past 12 months, was there ever a time when your drinking or being hung over interfered with your work at school, or a job, or at home?
1. YES, go to G2a
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), go to P. 25, H1
G2a. How often--once or twice, between 3 and 5 times, 6 to 10 times, 11 to 20 times, or more than 20 times in the past year?
1. Once or Twice
2. Between 3 and 5 times
3. Between 6 and 10 times
4. Between 11 and 20 times
5. More than 20 times
G3. During the past 12 months, were you ever under the influence of alcohol in a situation where you could get hurt--like when driving a car or boat, using knives or guns or machinery, or anything else?
1. YES
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
G4. During the past 12 months, did you have any emotional or psychological problems from using alcohol--such as feeling uninterested in things, feeling depressed, suspicious of people, paranoid, or having strange ideas?
1. YES
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
G5. During the past 12 months, did you have such a strong desire or urge to drink that you could not keep from drinking?
1. YES
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
G6. During the past 12 months, did you have a period of a month or more when you spent a great deal of time drinking of getting over the effects of alcohol?
1. YES
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
G7. During the past 12 months, did you ever have more to drink than you intended to, or did you drink much longer than you intended to?
1. YES, Go to P. 24, G7a
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
G7a. How often--once or twice, between 3 and 5 times, 6 to 10 times, 11 to 20 times, or
more than 20 times in the past year?
1. Once or Twice
2. Between 3 and 5 times
3. Between 6 and 10 times
4. Between 11 and 20 times
5. More than 20 times
G8. During the past 12 months, was there ever a time when you had to drink much more than you used to to get the same effect you wanted?
1. YES
5. NO
9. I AM A CASUAL/SOCIAL DRINKER (VOLUNTEERED), Go to P. 25, H1
SECTION H: DRUG DEPENDENCE
H1. The next questions are about your use of drugs on your own. By "on your own" we mean either without a doctor's prescription, in larger amounts than prescribed, or for a longer period than prescribed. With this definition in mind, did you ever use any of the following drugs on your own during the past 12 months?
(How about/during the past 12 months, did you use...) YES NO
A. sedatives, including either barbiturates (bar-BIT-chew-its) or sleeping pills on your own? (e.g. Seconal, Halcion, Methaqualone)
B. tranquilizers or "nerve pills" on your own? (e.g. Librium, Valium, Ativan, Meprobamate, Xanax)
C. amphetamines (am-FET-ah-means) or other stimulants on your own? (e.g. Methamphetamine, Preludin, Dexedrine, Ritalin, "Speed")
D. analgesics (an-uhl-JEEZ-icks) or other prescription painkillers on your own? (NOTE: this does not include normal use of aspirin, tylenol without codeine, etc, but does include use of tylenol with codeine and other Rx painkillers like Demerol, Darvon, Percodan, Codeine, Morphine, and Methadone)
E. inhalants that you sniff or breathe to get high or to feel good? (e.g. Amylnitrate, Freon, Nitrous Oxide ( "Whippets"), Gasoline, Spray paint)
F. marijuana (mare-ih-WAH-nah) or hashish (HASH-eesh)?
G. cocaine (KO-kane) or crack or free base?
H. LSD or other hallucinogens (ha-LOOSE-en-oh-jens)? (e.g. PCP, angel dust, peyote, ecstasy (MDMA), mescaline)
I. heroin (HAIR-oh-in)
Interviewer: If necessary, clarify: "By 'on your own' we mean either without a doctor's prescription, in larger amounts than prescribed, or for a longer period than prescribed."
H2. INTERVIEWER CHECKPOINT
1. AT LEAST ONE YES RESPONSE IN A THROUGH I, Go to H3
2. ZERO YES RESPONSES IN A THROUGH I, End
Interviewer: The next questions are awkwardly worded. Read slowly.
H3. In the past 12 months, did your use of (NAME OF DRUG/any of these substances) ever interfere with your work at school, or a job, or at home?
1. YES
5. NO
H3a. How often--once or twice, between 3 and 5 times, 6 to 10 times, 11 to 20 times, or more than 20 times in the past year?
1. Once or Twice
2. Between 3 and 5 times
3. Between 6 and 10 times
4. Between 11 and 20 times
5. More than 20 times
H4. During the past 12 months, were you ever under the influence of (NAME OF DRUG/any of these substances) in a situation where you could get hurt - like when driving a car or boar, using knives or guns or machinery, or anything else?
1. YES
5. NO
H5. During the past 12 months, did you have any emotional or psychological problems from using (NAME OF DRUG/any of these substances) -- such as feeling uninterested in things, feeling depressed, suspicious of people, paranoid, or having strange ideas?
1. YES
5. NO
H6. During the past 12 months, did you have such a strong desire or urge to use (NAME OF DRUG/any of these substances) that you could not keep from using it?
1. YES
5. NO
H7. During the past 12 months, did you have a period of a month or more when you spent a great deal of time using (NAME OF DRUG/any of these substances) or getting over (its/their) effects?
1. YES
5. NO
H8. During the past 12 months, did you ever use much larger amounts of (NAME OF DRUG/any of these substances) than you intended to or did you use (it/them) for a longer period of time than you intended to?
1. YES, go to H8a
5. NO
H8a. How often--once or twice, between 3 and 5 times, 6 to 10 times, 11 to 20 times, or more than 20 times in the past year?
1. Once or Twice
2. Between 3 and 5 times
3. Between 6 and 10 times
4. Between 11 and 20 times
5. More than 20 times
H9. During the past 12 months, was there ever a time when you had to use more (NAME OF DRUG/any of these substances) than you used to get the same effect you wanted?
1. YES
5. NO
Contact: (dsd.survey.program.dyanmics@census.gov)
URL: http://www.census.gov/spd
| Introduction to SPD |
Survey Design & Content |
Data Editing | Finding
SPD Info | Sampling &
Weighting |
| Linking Files | Publications |
S&A | News&Notes |
Users' Guide
Census
2000 | Subjects
A to Z | Search
| Product
Catalog | Data
Access Tools | FOIA
| Privacy
· Policies | Contact
Us | Home
