U.S. Census Bureau Homepage

Survey of Program Dynamics


Survey of Program Dynamics

Pretest Evaluation Report

Prepared by: Jennifer Hess

Jennifer Rothgeb
Andy Zukerberg

Center for Survey Methods Research

19 December 1997

SURVEY OF PROGRAM DYNAMICS

Pretest Report
Executive Summary

In August 1996, welfare reform legislation was passed and the Census Bureau was mandated to conduct a survey to evaluate welfare reform and its impact on the nation. The Survey of Program Dynamics (SPD), a longitudinal demographic survey, is designed to accomplish that goal. The SPD uses a sample of original respondents from the 1992 and 1993 panels from the Survey of Income and Program Participation (SIPP). The SPD is conducted each Spring from 1997 through 2001.

The first implementation of the questionnaire designed specifically for the SPD is scheduled for Spring 1998. There are two components to the SPD. The "core" survey instrument obtains information on such topics as employment, earnings, income sources and amounts, program participation, eligibility, educational enrollment, job training, disability, health care utilization, health insurance, children's enrichment activities, child care, child support, contact with absent parents, food security, marital relationship and conflict, and adult depression. Most questions in the survey are about the past calendar year, with some exceptions. The second component of the SPD is a self-administered questionnaire (SAQ) designed for persons 12-17 years of age. The SAQ is designed for administration through a personal audio-cassette player. Topics covered in the SAQ include family routines, housework and chores, relationship with parents, parental monitoring, contact with nonresidential parent, minor problem behaviors, substance use, knowledge of and attitudes towards welfare programs, marriage and childbearing, sexual initiation and contraception. (Adolescents 12-13 years of age are not asked the sexual initiation and contraception questions.)

The primary purpose of the SPD pretest was to evaluate the SPD survey instrument, the adolescent SAQ, and some of the logistical, operational, and procedural aspects of the survey. Another purpose of the pretest was to obtain timer data so we could determine whether the length of the survey necessitated cuts in content prior to the 1998 SPD. Field pretesting of the SPD was conducted between October 6-22, 1997 in four regional office areas: Boston, Kansas City, Los Angeles, and Atlanta. Personal visit interviews were conducted using computer-assisted personal interviewing (CAPI). Pretest evaluation sources include debriefings of Field Representatives, interviewing observation reports, and review of taped interviews in which FR/respondent interactions were analyzed to detect problematic items. In addition, response analysis and respondent debriefing analysis was conducted (by Child Trends, Inc.) on the data obtained from the adolescent SAQ.

This report details recommendations for changes in question wording and sequencing. We also provide suggestions for enhancements to FR training and the FR manual. Some procedural issues, such as, the use of respondent flashcards and the labor force activity worksheet are also addressed.

The majority of suggested revisions are minor question wording changes. Nearly all of these recommendations were agreed to by the sponsoring divisions (POP/HHES) and can be implemented in time for the 1998 SPD. In some cases, our suggested revisions include more complex changes to the sequencing of questions within a series or between series. We also requested changing a series currently designed at the person level to a household level design, in an effort to reduce both FR and respondent burden. Many of these structural suggestions were agreed to by the sponsoring divisions, however, most of the more complex revisions cannot be implemented for the 1998 SPD due to resource and time constraints. They will be held for 1999 implementation.

Given the size of the SPD instrument and the quantity of suggested revisions, it is not possible to summarize all the recommendations in the executive summary. Among the more significant recommendations resulting from the pretest are the following:

  • Eliminating the Labor Force Activity Worksheet, replacing it with a calendar similiar to that used in SIPP;
  • Resequencing the labor force series, so data for last year are collected prior to data for "last week";
  • Providing a response option of "retired" in selected labor force items and skipping such persons out of inappropriate items to reduce interviewer and respondent burden;
  • Reordering the employment series so the name of employer is obtained prior to the "employer loop" (which obtains detail information for each employer);
  • Including household-level income screeners for the income source module to reduce the number of irrelevant questions asked of specific income groups;
  • Including a household-level income screener for high income households for the food security module to eliminate unnecessary respondent burden;
  • Screening out inappropriate subgroups (men, never divorced adults, etc.) from questions regarding WIC, alimony, disability income, etc;
  • Revising the item that identifies type of health insurance coverage to be asked at a person level instead of a household level to reduce interviewer and respondent burden;
  • Revising question wording in almost all modules to clarify concepts and terms to increase respondent comprehension and reduce confusion, to reduce task difficulty and decrease burden, to clarify reference periods, etc.
The report also details the successful administration of the adolescent supplement. Very few revisions are recommended for the 1998 SAQ.

Pretest timer obtained from DSD indicates that no cuts to the core SPD are needed to meet the targeted 60 minute per household interview time for the 1998 SPD. Timer data revealed the average core pretest interview took 55.5 minutes, once outliers on both ends were removed. (The average when outliers were not removed was 62.19 minutes.)

Table of Contents

I. Background

A. Purpose
B. Pretest methodologies and procedures
II. Question evaluation methodologies A.Interviewing observation reports
B. Field Representative debriefings
C. FR and respondent interaction analysis
III. Recommendations A. Core questionnaire
  1. Employment
  2. Income sources
  3. Income amounts
  4. Eligibility
  5. Educational enrollment
  6. Work training
  7. Functional limitations and disability
  8. Health care utilization
  9. Health insurance
  10. Food Security
  11. Children's school enrollment
  12. Children's enrichment activities
  13. Children's disability
  14. Children's health care utilization
  15. Designated parent's work schedule
  16. Child care
  17. Child support
  18. Contact with absent parents
  19. Marital Relationship and Conflict
Adult Depression

B. Adolescent questionnaire

C. Respondent Flashcards

D. Timer Data

IV. Appendices
    A. Interviewing Observation Summary Report
    B. Field Representative Debriefing Summary
    C. Adolescent SAQ Pretest Results - Executive Summary
I. BACKGROUND

In August 1996, welfare reform legislation was passed and the Census Bureau was mandated to conduct a survey to evaluate welfare reform and its impact on the nation. The Survey of Program Dynamics (SPD) is an omnibus data collection vehicle that provides the basis for an overall evaluation of welfare reforms. The SPD is a longitudinal demographic survey designed to collect data on the economic, household, and social characteristics of the U.S. using a sample of original respondents from the 1992 and 1993 panels from the Survey of Income and Program Participation (SIPP). The SPD will be conducted each Spring from 1997 through 2001. (In Spring 1997, a modified CPS March Income supplement survey instrument was administered to the SPD sample to collect income and program participation data for calendar year 1996, thereby "bridging" the gap between years in which SIPP data were collected (1992 -1995) and SPD data will be collected (1997-2001.) The first implementation of the questionnaire designed specifically for the SPD is scheduled for Spring 1998.

Prior to conducting a field pretest, cognitive interviews, using the paper version of the questionnaire, were conducted for the following modules of the SPD survey instrument: educational enrollment, work training, functional limitations and disability, health care utilization, health insurance, children's enrichment activities, child care, child support, and contact with absent parent. Respondents with targeted characteristics, appropriate to the related modules, were recruited for one-on-one, in-depth think-aloud interviews. Researchers asked respondents probing questions to determine their understanding of specific questions and related concepts, item sensitivity, and task difficulty. Based on results of cognitive interviewing, recommendations for question revisions were proposed. Prior to the pretest, cognitive interviews were also conducted using the SAQ. Agreed upon revisions were incorporated into the SPD pretest instrument.

We were unable to conduct cognitive interviews for the "economic" modules of the SPD instrument (employment and earnings, income sources and amounts, and eligibility) since these modules are much too complex to administer using the paper version of the questionnaire. The automated instrument was not available within the timeframe needed for cognitive testing of these modules. Over half of the survey instrument is covered by these modules so it is not surprising that the pretest data revealed a disproportionate number of problems with these sections relative to the sections that had been cognitively tested and revised prior to the pretest.

Field pretesting of the SPD was conducted between October 6-22, 1997 in four regional office areas: Boston, Kansas City, Los Angeles, and Atlanta. Personal visit interviews were conducted using computer-assisted personal interviewing (CAPI). Eligible household respondents were household members 15 years of age and older. Census Bureau Field Representatives (FRs) completed 262 household interviews of the SPD core CAPI survey instrument and 60 adolescent self-administered questionnaires (SAQ). The pretest sample was selected from expired March 1996 Current Population Survey (CPS) interviewed households. Low income households were over sampled to increase the likelihood of covering more paths through the SPD questionnaire. Attempts were made to exclude non-English speaking households in the pretest sample, since only an English version of the survey instrument existed.

A. Purpose

The primary purpose of the SPD pretest was to evaluate the SPD survey instrument, the adolescent SAQ, and some of the logistical, operational, and procedural aspects of the survey. Another purpose of the pretest was to obtain timer data so we could determine whether the length of the survey necessitated cuts in content prior to the 1998 SPD. The results of the pretest will be used to finalize materials for the Spring 1998 SPD data collection.

B. Pretest methodology and procedures

The 1997 SPD pretest included questions on the following topics for household members age 15 and over: employment, earnings, income sources and amounts, program participation, eligibility, educational enrollment, job training, disability, health care utilization, health insurance (all household members), food security (all household members), marital relationship and conflict (respondent only), and adult depression (respondent only). The pretest questionnaire also included the following topics about children: school enrollment, enrichment activities, disability, health care utilization, child care, child support, and contact with absent parents. Most questions in the survey are about the past calendar year, with some exceptions. This part of the questionnaire was administered using CAPI and is referred to as the "core" questionnaire. (The paper version of the SPD core questionnaire, dated August 28, 1997 is the document to refer to when reading this report.)

The 1997 pretest also included an adolescent SAQ for persons age 12-17. The SAQ was administered through a personal audio-cassette recorder if the adolescent was at home when the FR administered the core questionnaire. If the adolescent was not at home, the FR collected the data by phone. Census Bureau staff developed two answer booklets: one for the adolescent to use with the audio cassette recorder, and a second for FRs to use during telephone administration. The former included the response categories but did not include the questions, since these were on the tape. This was done to protect adolescents' privacy. The latter included both the questions and the answer categories. Prior to administering the questionnaire, FRs obtained verbal consent from parents to allow the adolescent to participate in the survey. Parents who were interested were given a copy of the survey questions to review. To protect adolescent's privacy, the parent booklet contained the questions only and in a slightly different order than the adolescent answer booklet.

Reports from regional offices, Field Representatives, and staff who observed interviews were used to evaluate survey procedures, forms, manuals, and data collection instruments. Timers were included in the instrument to determine the interview length.

II. QUESTIONNAIRE EVALUATION METHODOLOGIES

We used three methods to evaluate the SPD pretest questionnaire: interviewing observation reports, debriefings, and interviewer and respondent interaction analysis (from taped interviews). Each method is described more fully below.

A. Interviewing observation reports

Eight Census Bureau staff observed pretest interviews (this includes staff from Demographic Surveys Division (DSD), Population Division (POP), Housing and Household Economic Statistics Division (HHES), Demographic Statistical Methods Division (DSMD), and the Center for Survey Methods Research (CSMR) in the Statistical Research Division. A total of 25 interviews were observed: 7 in Boston; 9 in Los Angeles; 7 in rural areas of Minnesota, Missouri, and Iowa; and 2 in Miami. Each observer was requested to complete an "Interviewing Observation Form". Observers were requested to note major problems observed with the instrument, question wording or sequencing, procedures, respondents willingness to be tape recorded (for use in interviewer and respondent interaction analysis, see below), and other aspects of the survey instrument and procedures. Appendix A contains the Summary Report of Interviewing Observations, prepared from the individual Interviewing Observation Forms.

B. Field Representative debriefing sessions

Census Bureau staff conducted five debriefing sessions with FRs who participated in the SPD pretest: one each in Boston, Kansas City, and Miami, and two in Los Angeles. CSMR and DSD staff facilitated the first half of the debriefing regarding questionnaire design issues, and Field Division staff facilitated the second half regarding field procedures, training, manuals, instrument layout, function keys, and other field-related issues. Nearly all FRs who conducted interviews during the SPD pretest participated in a debriefing session; a total of 38 FRs were at the debriefings. Appendix B contains the Field Representative Debriefing Summary Report for the CSMR/DSD portions of the debriefings. In a separate document, FLD provided a summary of their sections of the debriefings.

C. Interviewer and respondent interaction analysis

FRs were requested to tape record two completed core SPD interviews for use in subsequent interviewer and respondent interaction analysis. We received 61 tapes that contained either completed or partial interviews. Of those, CSMR staff analyzed interviews in 28 households, which included 46 household members ages 15 and over, and approximately 10 household members under 15. The analysis consisted of listening to the interview and noting any problems FRs or respondents had with question wording, sequencing, unclear concepts or terms, difficult to answer questions or series of questions, and other behaviors or indicators of how well a question is measuring the concept of interest.

The data from the interaction analysis are qualitative. They indicate the item number, what the problems with the question were, and, usually, verbatim transcripts of the FR/respondent exchanges for the problem areas. For example, if a respondent requested clarification of a concept or question, we recorded the question number and the verbatim question the respondent asked. If an FR made a major change to question wording, we recorded the verbatim question reading. Due to time and resource constraints, we coded only those items that indicated a potential problem existed based on the FR/respondent interaction. (Typically, when we "behavior code" taped interviews, we code EVERY question administration, even if there is no problem indicated. This allows for discussion of some "quantitative-type" results.) We decided to use the more qualitative approach so we would have more depth of understanding of what the problems were, not just that a problem existed. (Traditional behavior coding does not usually identify the specific source of the problems, which makes it less useful for developing solutions.) Because there was little time to develop recommendations for revisions, we decided this modified method would better serve our needs. (Obviously, due to the small number of cases reviewed and some of the more rare paths taken through the instrument, there were several items for which no FR/respondent interaction information was obtained.)

III. RECOMMENDATIONS

This report details recommendations for changes in question wording and sequencing. The format is such that typically, the original question wording is provided, along with the suggested revision. The justification for the revision citing information obtained from various evaluation sources is provided along with the final decision resulting from meetings between CSMR, POP, HHES, and DSD. Instrument problems identified during review of taped interviews are also noted in the report. We also provide suggestions for enhancements to FR training and the FR manual. Some procedural issues such as the use of respondent flashcards and the labor force activity worksheet are also addressed.

CSMR staff reviewed data from the three evaluation methods described above for each question in the core SPD survey. In some cases, we only had data from one source, such as the taped interviews. In other cases, we had data from all three sources. The source of the data is indicated in the justification that accompanies each suggested revision.

A. Core Questionnaire

1. Employment section

a. Items 3, 4, 5, 6, and 7 (labor force status "last week"

Original wording: LAST WEEK, did you do ANY work for pay?

Suggested revision: Delete "last week" series.

Justification: Shifting reference periods in the employment section from last week to the past calendar year is confusing, particularly for households with more than one person age 15+. Problems with the shift in reference period within the series were noted by some FRs, interviewing observation reports, and in the interviewer/respondent interactions. Respondents sometimes needed clarification on what reference period they were reporting. Deleting the current labor force status would allow the core SPD interview to focus from the very beginning on the reference period of "last year," providing a smoother flow for the interviewer and respondent at the outset of the interview.

Taped interviews and interview observation reports indicated that persons working for pay-in-kind and/or with atypical work situations had problems responding to item 3 and FRs had difficulty knowing how to classify such cases. The data indicate that the wording of the current work question is too narrow in scope and leads to misclassification of atypical work situations. This then affects classification of labor force activity for the previous calendar year. For example, if a person is working for pay-in-kind and is marked as a "no" in the current work question, it is probably more likely that the FR and the respondent will assume the answer to item 9 is also "no", even though item 9 does not specify work for pay. We suspect that with increased movement from welfare to work, programs will be established in which people may not be performing work for pay during the transition, but may be receiving other types of compensation.

Final decision: To address these problems, we recommend moving the "current work" series AFTER the entire employment series for last year has been collected for all persons age 15 or over in the household. This would allow the interview to focus on the previous calendar year, at the beginning of the core interview, which is the intended frame of reference for the survey. In addition to re-sequencing, we also recommend revising the wording of item 3, as follows:

"Did you do any work at all LAST WEEK, including work for pay or any other type of compensation?" This wording was included in a large-scale test during the CPS questionnaire redesign research. This question wording produced more "correct" responses to vignettes of marginal and atypical work situations posed to respondents, than the "work for pay" question contain in the pretest questionnaire. Results from other evaluation sources (behavior coding, response distribution, item nonresponse, FR debriefings) produced positive results regarding this wording.

Training Issues: FRs need to be provided with exercises on how to classify atypical work situations: e.g., persons working for pay-in-kind, persons working on commissions who did not receive any commissions during the reference period, etc. In these cases, respondents frequently "tell their story" and assume the FR knows how the situation should be classified. The taped interviews collected during the pretest for behavior coding could be a useful resource for developing training materials.

b. Items 4 (with a job, not at work), 5 (layoff), 10 (seasonal/part time work last year)

Original wording: Current response options include only "yes," "no," "don't know," and "refused."

Suggested revision: For persons 50+, include a response option of "retired" below the "no" option. For persons who voluntarily report being retired in item 4, skip them to item 9. For persons who voluntarily report being retired in item 5, skip them to item 9. For persons who voluntarily report being "retired" in item 9, ask item 10 to ensure they didn't do any part-time, temporary or seasonal work. If these same persons report "no or retired" in item 10, plug item 18 "retired" and go to CK64.

Justification: All three evaluation sources indicate that questions on employment are extremely burdensome for retired persons. Based on the research conducted during the redesign of the CPS, we recommend adding a "retired" response option (for persons 50 years of age or older) to items 3, 4, 5, and 10. This revision would reduce both interviewer and respondent burden.

Final decision: Suggested revision accepted. Note that HHES agreed to move items 3-7 to the end of the employment and earnings series and to ask these items after the calendar year information has been collected for everyone age 15 or over in the household. Therefore, answers of "retired" in items 4 or 5 would result in asking the current labor force status for the next eligible person in the household. Once this is complete, the instrument should go to income sources.

c. Labor force activity worksheet

We recommend that the labor force activity worksheet be eliminated from the SPD survey. There is strong evidence from FRs, observers, and taped interviews that the worksheet is not useful in the majority of situations. Based on FRs suggestions, we recommend that the worksheet be replaced with a calendar similar to that used in SIPP.

Final decision: Recommendation to eliminate labor force activity worksheet was accepted. It was decided to use the SIPP calendar with the numbered weeks.

d. Item 18 (main reason not working)

Original wording: What was the MAIN reason you did not work in 1996?

Suggested revision: Add a response option of "never worked" above the "other" response option. Add a flashcard for this item.

Justification: FRs reported that this item was problematic, particularly for persons who have not been in the labor force (specifically, elderly persons and adolescents who have never worked.) Other respondents are unclear what kind of answer we are looking for in response to item 18. To resolve these issues, we recommend adding a response option of "never worked." We also recommend adding a flashcard so respondents will have a frame of reference when answering the question.

Final decision: Accept suggested revisions; however, the use of a flashcard at this item is a low priority. Per HHES's request, revise the order of the response options in Item 18 so the more common reasons are at the top of the list, as indicated below:

<1> Retired

<2> Taking care of home or family

<3> Going to school

<4> Ill or disabled

<5> Could not find work/no work available

<6> Did not want to work

<7> On layoff

<8> Never worked

<9> Other (specify)

<D,R>

e. Items 19 and 32

Instrument problem: FRs indicated that these items need more pairs of answer spaces for workers with more than 7 different spells of employment/job search last year.

f. Item CK32

Instrument problem: The first condition of this check item (If 29 eq 1, go to 35) did not seem to work. Single jobholders were erroneously being asked item 32.

g. Item 32 (Weeks worked/main job)

The concept of main job is currently defined in SPD as it is in CPS (the job at which the person worked the most hours). The concept of main job is relevant in CPS because we collect industry and occupation data for only one job even if the person holds more than one job. In SPD, we ask about the weeks worked, usual hours worked, industry and occupation (I/O), and earnings for up to four employers during the previous calendar year. Therefore, the concept of main job is not relevant in the SPD. We recommend that FRs ask about the jobs in the order in which the person worked the most weeks, second most weeks, third most weeks, etc.

Final decision: Recommendation accepted

h. Item CK39

Instrument problem: The condition that "if 52 not marked in question 32 for this job, go to 39" needs to be revised. For single job holders, the instrument should look at information from both item 19 and item 21 to see if week 52 was entered in either of those items. (They might create out variables that sum the information from these items.) For multiple job holders, use information from item 32.

i. Items 41-43 (name of company/business) and E-review screen

Original wording: (41) What was the name of the (company/nonprofit agency/government agency) for which you worked?

Suggested decision: (29a) What is the name of the employer for which you worked (the most weeks) in (fill LAST YEAR)?

Justification: We suggest collecting the name of up to four employers immediately after item 29 (item 29a above). If this recommendation is adopted, items 41-43 will be deleted. This revision allows the employer's name to be included as a fill in the loop of questions asked about each job for which the respondent is reporting. This includes hours worked, weeks worked, class of worker, and industry and occupation for each job. This revision was considered prior to the pretest, and the need for the revision was confirmed by the comments obtained from FRs during debriefings.

It should be noted that the current design allows the company name questions (items 41-43) to be filled based on responses to the class of worker question (item 35). The suggested revision requires that a generic company name question be used, since class of worker data will not be collected until later. However, the benefit of having the company name fill in all the questions within the employer loop series seems to offset the issue of not having the company name questions tailored according to class of worker status.

This revision will allow the E-review screen to list employers' names as opposed to "employer 1", "employer 2", etc. which is the current design. (This is the screen that allows FRs to delete an employer that was entered erroneously.) During the debriefings, FRs commented that the E-review screen, in its current form, is useless.

Final Decision: Modify item 29a to read: "What is the name of the employer or company for which you worked (the most weeks, second most weeks, etc.) in (fill LAST YEAR)?"

Allow a precode (with a specifiy) at Item 29a to indicate that "self employed, no company name" exists. For subsequent questions where name of employer is used as a fill, use "at this job" (in items 35, 46, 47 and 50) and "for yourself" (in items 32, 33, and 38), as appropriate. (You may need to look more closely at the instrument to determine if certain items will need additional fill instructions due to a lack of employer name.)

Training Issues: FRs need more and better training on how name of employer should be filled for persons working for various employers such as substitute teachers, domestic workers, odd-job handypersons, persons that are self employed, etc. HHES also needs to provide DSD/FLD with consistent criteria that can be applied (for item 29) to make a determination whether persons in such situations are multiple jobholders or single jobholders.

j. Item 43A (address of employer)

Original wording: What was the address?

(Currently, the instrument requires street address, city, state and zip code)

Suggested revision: In what city (town) and state is this employer located?

(Ask AFTER industry type (item 44) is obtained)

Justification: Information from the three evaluation sources strongly indicate that respondents are unable to provide the exact address of employers, including street address and zip code. As suggested by FRs, we recommend obtaining only the city (town) and state of employers. We recommend that this item be placed immediately AFTER item 44 and labeled item 44a.

Final Decision: Use original wording and if a "don't know" response is provided, then follow up with a probe for the city and state, such as:
"In what city(town) and state is this employer located?"

Training Issues: FRs need additional training on the purpose of this item.

k. Item 45

Instrument problem: The last fill of this item should reflect the last month in the previous calendar year that the person worked. Instead, the first month was appearing. (e.g. for persons working all year the fill should be December 1996; instead it was January 1996.)

l. Item CK49

Instrument problem: This check item might need to be revised. If the instrument goes through the employment/earnings series for more than one person in the household, this check item is suppose to skip over the lead-in and the request to refer to records which is contained in item 49. However, this item was appearing when there was more than one loop through the series.

m. Item 59 (pensions)

Original wording: Did this employer offer a pension or other type of retirement plan to any of its employees during 1996?

Suggested revision: During 1997, did this employer offer a pension or other type of retirement plan to ANY of its employees?

Justification: Respondents frequently responded "no" to the question, but it is unclear if they thought the question was asking about ANY of its employees or only about benefits offered to the sample person. Sometimes FRs were not reading the phrase "to any of its employees during 1996." Additionally, it is unclear if respondents always understood that the reference period was last year. For clarification, we recommend revising the question wording so the reference period is at the beginning of the question and the word "any" is emphasized.

Final Decision: Suggested wording accepted.

n. Item 60 (pension participation)

Original wording: Did you participate in that plan?

Suggested revision: During 1997 did you participate in that plan?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

o. Item 61 (health insurance)

Original wording: Were you eligible for health insurance coverage through this employer?

Suggested revision: During 1997 were you eligible for health insurance coverage through this employer?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

p. Item 62 (health insurance participation)

Original wording: Did you participate in that plan?

Suggested revision: During 1997 did you participate in that plan?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

q. Farm-related income

Problem identified: FRs participating in the pretest in the rural sample areas in Minnesota, Missouri, and Iowa indicated that they didn't think farm income was adequately captured in the SPD. They suggested we develop a flashcard with farm-related income to improve reporting. Two different approaches could be taken. First, a separate question could be included at the end of the earnings series, just before item 59 (fringe benefits items). The March CPS includes the following item: "Other than farm income we have already talked about, did you receive any income from agricultural work done for others, recreational services, or government farm programs other than loans?" The question is given as an example only. CSMR does not like the wording of this question, but we are not familiar enough with the different types of farm income to suggest our own question. If such a question is included, we would need to develop additional follow-up questions to capture the income received. A second approach is that a flashcard (with information listing different types of farm income) could be developed for use in the earnings items (51 and 53). FRs could be instructed to use the flashcard when item 35 (class of worker) equals "working in a family business or farm" and information obtained in industry/occupation indicates the person works in farming.

Final Decision: Do not add a separate question. Do not add a flashcard. The population to which this problem applies will be EXTREMELY small in the SPD sample.

r. Use of previously reported data (dependent interviewing)

FRs and other Bureau staff have suggested including dependent interviewing for employment information to improve the efficiency of the instrument. Since the input for the 1998 SPD is the Bridge Survey, which is the Current Population Survey and its March Income Supplement, there is limited employment information to include in dependent interviewing. Additionally, any specifications prepared for the 1998 SPD instrument would have to be re-written for 1999 since the 1998 SPD will be used as input for the 1999 instrument.

The Bridge Survey included basic CPS, which asked about labor force activity "last week," (i.e. the week before the interview). Industry and occupation data is collected for the "main" job, that is, the job at which the person worked the most hours last week. We could use those data for dependent interviewing. However, we need to consider whether the effort to include dependent interviewing is an effective use of resources considering the limited amount of time available to modify the pretest instrument for 1998 SPD.

Final Decision: Do not use dependent interviewing in 1998 for employment information. For 1999, explore use of previously recorded data from the employment series.

s. Other Items in the Employment and Earnings Series - Training Issues

Item 50 (Earnings last year).

Review of taped interviews indicated that respondents who only worked part year at a job sometimes report the "annual salary" instead of "annual earnings," even though they did not work the whole year. This results in an overestimate of earnings received. FRs need additional training for such cases so they know to probe to determine if the earnings being reported are earnings the person actually received or if what was reported was the earnings had the person stayed the entire year at the job.

For earnings items, training needs to be enhanced to emphasize to FRs that earnings BEFORE taxes is what's being requested. Also, FRs might need to be reminded of the definitions of Gross and Net earnings.

2. Income sources

FRs strongly requested use of income screeners to reduce the liklihood of inappropriate questions being asked of high income households (e.g. program participation) and low income households (e.g. income from shares of stocks, royalties, trusts, etc.) . The need for these screeners was echoed in the interviewing observation reports, as well as the taped interviews. Using household level income screeners (with criteria determined by HHES) will reduce both FR and respondent burden. Such a revision will most likely reduce requests for clarification for terms unfamiliar to respondents, thereby improving the interview flow.

a. Household level income screener

We suggest that low income households (to be defined by HHES) be screened out from individual questions regarding the following income sources:

    • Royalties (item 254)
    • Estates or trusts (item 256)
    • Mutual fund shares or shares of stock (item 249)
    • Properties that were rented to others such as houses, apartments, business properties, or land (item 251)
We recommend that high income households (to be defined by HHES) be screened out from individual questions for the following income/program participation sources:
    • Supplemental Security Income (items 211 and 214)
    • AFDC (item 220)
    • WIC (item 220)
    • General Assistance (item 220)
    • Other welfare (Item 220
    • Free or reduced-price meals at school (item 223)
    • Any financial assistance on a regular basis from friends or relatives not living here (item 262)
    • Income assistance from a charitable group (item 264)
    • Food stamps (item 218)
    • Energy assistance (item 228)
Final decision: Accept recommendation to adopt household level income screener. Specifications for the income screener will be provided to DSD/CSMR by HHES. Most likely the screener will be from the administration of the "family income" question, developed for use in SIPP.

b. Catchall Questions

It is necessary to administer two "catchall" questions at the end of the income sources series: one for low income households, and one for high income households. Provided below are suggestions for such questions. Low-income households Suggested wording: Item 270: FLASHCARD X. This is a list of income sources persons sometimes have. Please tell me if anyone in this household received income from any of these sources during 1997.

FR: Read if necessary: Did anyone own mutual funds or shares of stock, own property that was rented to others, receive rental income from boarders, receive income from estates or trusts, or from royalties?

(If yes, ask:)

Item 271: What was the source of income?

(Mark all that apply)

a. Mutual funds or shares of stock..................0
b Property that was rented to others...............0
c. Rental income from boarders......................0
d. Income from estates or trusts......................0
e. Income from royalties.................................0
Final Decision: Adopt suggested wording, however remove response option "Rental income from boarders" out of the catchall question for low-income households. Inquiries about receiving income from this source will be asked in all households regardless of income level.

High income households

Suggested revision: Item 272. FLASHCARD X.

This is a list of benefits or income sources people sometimes receive. Please tell me if anyone in this household received benefits during 1997 from any of these sources. (If yes, ask:)

Item 273. What was the source of the benefits?

(Mark all that apply)

a. Supplemental Security Income (SSI)
b. AFDC, ADC, or TANF
c. WIC
d. General assistance
e. Free or reduced price lunches at school through the Federal School Lunch or Breakfast Programs
f. Food Stamps
g. Energy assistance
h. Income assistance from a charitable group
i. Other welfare
j. Financial assistance on a regular basis from friends or relatives not living here
Final Decision: Suggested wording adopted. In response option "j", bold the words "on a regular basis" in the instrument and on the flashcard.
c. Item level screeners for inappropriate items. For a few items in the income source series, the questions are inappropriate for certain subgroups. Such items are identified below.

Item 258 (Receipt of alimony): This question should not be asked if all household members 15+ reported earlier (item in the demographics) that they have never been divorced.

Item 220 (WIC): This question should not be asked if the household contains no children under age 6 and no women of childbearing age (15-45).

Item 240 (Disability income): This question should not be asked if it was reported in item 18 that the sample person never worked.

Final Decision: Suggestion to screen out inappropriate subgroups from these items is accepted.

d. Item 228

Original wording: Has this household received any energy assistance in the past 12 months, that is since (month, year)?

Suggested revision: The government has an energy assistance program that helps pay heating costs. This assistance can be received directly by the household or paid directly to the electric company, gas company, or fuel dealer. During the past 12 months, has this household received any energy assistance of this type?

Justification: Information from the interviewing observation reports and from the taped interviews demonstrate that several respondents had difficulty understanding what was meant by the term "energy assistance." To improve respondent understanding, we recommend that the lead-in explanation used in the March CPS Income Supplement be added to the SPD question and the question wording revised slightly.

Final decision: Adopt suggested revision, but delete the second sentence to make the question less wordy, as shown below.
"The government has an energy assistance program that helps pay heating costs. During the past 12 months, has this household received any energy assistance of this type?"

In an FR note or a help screen add the following information:
"This assistance can be received directly by the household or paid directly to the electric company, gas company, or fuel dealer."

Training Issues: Revise FR training to include the definition of energy assistance and what should and should not be included as energy assistance. This information is available from HHES.

e. Items 236a and 236c Original wording: Item 236a (Do you/does anyone in this household) have a physical, mental, or other health condition that prevents (you/him/her) from working?

Item 236c (Do you/does anyone in this household) have a physical, mental, or other health condition that limits the kinds or amounts of work (name/you) can do?

Suggested revision: Delete item 236c

Problem identified: Since question 236c is broader than 236a, we recommend using this single question to capture the information currently obtained through 236a and 236c, thereby deleting question 236c from the survey. While 236a asks whether there is a condition that prevents (name) from working, we think situations prompting reports of yes to item 236a will produce reports of yes to item 236c. We have no evidence on which to base this opinion. But we do know that the current series seemed extremely redundant and unnecessary.

Final decision: Retain both items.

f. Item 236C Instrument problem: The first fill for this item (Other than you/names,) did not seem to work correctly. It seemed that if the person's line number was entered in 236B, the lead-in fill was left blank and consequently the same person reported in 236a would be reported in 236c. g. Item 238 Original wording: Did (you/anyone in this household) ever retire or leave a job for health reasons?

Suggested revision: Did (you/anyone in this household) ever retire or permanently leave a job for health reasons?

Review of taped interviews indicated that some respondents reported "yes" to this question when they temporarily left a job due to health reasons such as maternity leave or hospitalization. We assume the intent is to capture whether persons have permanently left a job for health reasons. If our assumption is correct, we recommend including the word "permanently" in the question so it is clear to respondents what is being asked.

Final decision: Suggested revision accepted.

h. Item 240 Instrument problem: During review of taped interviews it was observed several times that this item did not always appear when it should have. Sometimes persons for whom a yes was entered in 238 were never asked this question i. New item for types of assistance not covered elsewhere Research conducted by CSMR (Bogen, Lee, and Griffiths 1997) on the CPS March Income Supplement indicates that there are some types of assistance not detected through the usual battery of program participant/income sources questions, particularly transportation assistance and child care assistance. Based on the results of the March Income Supplement research, we recommend adding a new item to the core SPD questionnaire immediately after item 220. CSMR cognitively tested these items for the March CPS Supplement.

New item: At any time during 1997, did anyone in this household receive any of the following types of GOVERNMENT assistance because your income was low:

  • Transportation assistance, such as gas vouchers, bus passes, or help registering, repairing, or insuring a car?
  • Any child care services or assistance last year so you could go to work or school or training?
  • Any other assistance from the government last year, that we have not already talked about, because your income was too low? (If yes, specify)
For each type of assistance reported above, an additional question is needed to determine who received such assistance.

If the recommendation for including this new item is adopted, we recommend deleting the question concerning "other welfare" from item 220. Related items pertaining to who received the assistance will need to be modified. Additionally, this question would not be asked of high-income households if the income screen is adopted, as previously suggested.

Final decision: Do not delete "other welfare" from item 220.

Arthur Jones (HHES) provided specifications to DSD regarding additional types of assistance including assistance with finding a job. These specs include information regarding the above types of assistance, who received it, and quantity or frequency of assistance received. j. Item 244 (response category modifications) Original wording: <6> Regular payments from annuities or paid up insurance policies

<7> Regular payments from IRA, KEOGH, or 401(k)

Suggested wording: <6> Regular income from annuities or paid up insurance policies

<7> Regular income from IRA, KEOGH, or 401(k)

Justification: HHES requested this change to clarify retirement income.

Final decision: Suggested wording accepted.

k. Other Income Source Items - Training Issues Item 246

Enhance training to address savings bonds people may have.

Item 249

Enhance training so it is clear to FRs what should and should not be included for this item. Specifically, instruct them on how to handle retirement funds. The rewording of the question should help, but various scenarios will probably still be presented to FRs and they will need to know how to classify the response.

Item 260

Some FRs confused child support with foster child care. Enhance training and scenarios to include such situations so distinction is clear to FRs.

Item 261

Enhance training so FRs are aware that 261 (who receives child support) is about receipt of support by a parent, not the child for whom receipt is provided.

Items 206 and 212

Respondents are OFTEN confused as to whether they receive Social Security benefits or Supplemental Security Income (SSI) benefits. Review of the taped interviews indicates that respondents frequently give the FR information expecting the FR to know how to classify the response. Enhance training so FRs are more familiar with the differences between Social Security and SSI.

  1. Income amounts

  2. a. Item 300 (Verification of income sources)

    Original wording: I have recorded that you received (READ LIST)

    [LIST INCOME SOURCES IDENTIFIED FOR THIS PERSON}

    Have I listed anything that should not be there?

    <1> Yes

    <2> No

    Suggested revision: I have recorded that you received (READ LIST)

    [LIST INCOME SOURCES IDENTIFIED FOR THIS PERSON]

    Is that correct?

    <1> Yes

    <2> No

    Justification: This item was counter to FR expectations, which resulted in erroneous entries. With the original wording, if everything was correct, the FR needed to enter a <2> for no (that nothing needed to be changed.) However, FRs were rewording the question to verify that everything was correct and then entering a <1>, which, with the current wording, indicated that something was listed that needed to be changed. FRs and respondents were confused when item 301 appeared asking for what should be deleted. FRs had to back up and change the response to item 300. Due to this problem, we recommend that item 300 be reworded.

    Final decision: Recommendation accepted.

    b. Item 302 (Lead-in to income amounts)

    Original wording: Now I am going to ask you how much you received from each of these sources during 1996 and which months you received it. The worksheet you filled out regarding your work-related activities may be helpful in answering these questions as well as any records you have.

    Suggested decision: [Fill "this" if only one income source is listed. Otherwise fill "these."]

    Now I am going to ask you how much you received from (each of these/this) source(s) during 1997.

    Justification: The Labor Force Activity Worksheet is not recommended for use in 1998, therefore, all references to the worksheet should be deleted from the instrument. Taped interviews indicated that the lead-in was awkward in cases where persons only had one income source, since it referred to multiple income sources. We recommend deleting the reference to which months the income was received since the months a particular type of income was received is only asked for selected income sources.

    Final decision: Recommendation accepted.

    c. Item 302

    Instrument problem: This lead-in is only suppose to appear the first time through the amounts series. However, sometimes it appeared for the second and third persons through the series.

    d. Item 445 and 449

    Instrument problem: The text for the fill regarding type of interest-earning accounts being asked about should be based on the responses to item 246. If 246a is yes, then the fill should read "Savings, or interest-earning checking account, or money market fund. If 246b is yes, then the fill should read "bonds, treasurey notes, certificates of deposit). If both 246a and 246b are yes, then both sets of text should be included in the fill.

    e. Item 449 and 450 (Amount of interest earned; Average amount in account)

    Original wording: (449) How much did you receive IN INTEREST from these sources during 1996, including even small amounts credited to accounts?

    (450) What is your best estimate of the AVERAGE AMOUNT that you had in these accounts during 1996?

    Suggested revision: (450) What is your best estimate of the AVERAGE AMOUNT that you had in these accounts during 1997?

    (449) How much did you receive IN INTEREST from these sources during 1997, including even small amounts credited to the accounts?

    Justification: From the taped interviews, it was evident that respondents were frequently confused when trying to respond to item 450, particularly when trying to estimate an average amount in checking accounts. Some respondents reported the amount of savings, instead of interest, when asked item 449. To reduce erroneous reports of interest earned, we suggest reversing the items, so that respondents first report the average amount in the accounts and then report the interest. These same comments apply to items 445/446, 458/459. Item 450 presents a very difficult task for respondents.

    Final decision: Suggested wording adopted with a minor modification. In item 450 ask for the AVERAGE BALANCE, instead of the AVERAGE AMOUNT, as provided below:

    (450) What is your best estimate of the AVERAGE BALANCE that you had in these accounts during 1997?

    (Note: The reordering revision and wording revision also apply to items 445/446 and 458/459.)

    f. Items 454,455, 456, 457, 458, and 459

    Problem identified: The text for these items should be revised to reflect whether the person owns mutual funds or shares of stock, depending on the response to items 249a and 249b.

    g. Other Income Amounts Items - Training Issues

    Item 315

    Sometimes an adult receives payments on behalf of a child. Other times the child receives benefits directly. Provide FRs with information about the different types of arrangements so FRs can determine how to classify the response.

    Item 449

    Include in training more information about how to fill item 449. Respondents who have savings bonds for their children don't always know the interest that has accumulated to date and they report the face value of the bond instead of the interest.

    Item 484

    Review of taped interviews revealed that child support income was sometimes reported as the child's income rather than the parent's income. Enhance training so FRs recognize when this happens and appropriately probe to obtain the correct information.

    Item 506

    Provide FRs with information of types of income that are considered "lump sum' payments and those considered "capital gains.".

    4. Eligibility/Assets Questions

    a. Item 604

    Original wording: About how much would this (house/apartment) sell for if you were to put it on the market today?

    Suggested revision: How much do you estimate this (house/apartment) would sell for if you were to put it on the market today?

    Justification: Review of taped interviews indicate that many respondents had difficulty providing an answer to this question. Respondents often said they didn't know, then said they could only guess or provide an estimate. In the original wording, we attempted to communicate to respondents that an estimate was acceptable by including the phrase "about how much." Given the difficulty encountered by respondents and confirmed by the FRs in the debriefing report, we recommend revising the question to ask for an estimate.

    Final decision: Use suggested wording.

    b. Items 605, 606, 607, 608, 609, 617, and 618a

    Original wording: (605) How much were your total property taxes, including city, county, and school taxes on this (house/apartment) in 1996?

    (606) How much did you pay for homeowner's insurance, that is, what was your premium in 1996?

    (607) Do you have a mortgage or home equity loan on this property?

    (608) Including any second mortgage or home equity loan, about how much is the remaining principal on this mortgage?

    (609) How much are your monthly mortgage payments?

    (617) Do your mortgage payments include property taxes?

    (618a) Do your mortgage payments include insurance premiums?

    Suggested revision: (607) Do you have a mortgage or home equity loan on this property?

    Lead-in: The next few questions are about your property taxes, homeowners insurance and current mortgage payments on this home. It will be much easier to provide this information if you refer to your mortgage statement or mortgage payment coupons. I'd be glad to wait while you get those records.

    (609) How much are your monthly mortgage payments?

    (617) Do your mortgage payments include property taxes?

    (605) How much are your total property taxes, including city, county, and school taxes?

    (618a) Does your mortgage payment include insurance premiums?

    (606) How much do you pay for homeowner's insurance, that is, what is your annual premium?

    (608) Including any second mortgage or home equity loan, about how much is the remaining principal owed on your current mortgage?

    Problems identified: From the taped interviews, there was strong evidence that many respondents were unable to respond to these questions; the primary reason being that the taxes and insurance payments are included as part of the monthly mortgage payment. There was evidence that many respondents do not know the remaining principal on their mortgage and some respondents don't understand the term "principal." Unless respondents have their mortgage statements, payment coupons, etc. readily available, there will be high nonresponse for several of these items.

    To facilitate use of records when reporting these data, we recommend reordering of the series and including a lead-in asking respondents to refer to their mortgage records. We suggest that the reference period of the question on how much is paid in taxes and in homeowners insurance be changed to current year instead of last year. We suspect that taxes and homeowners insurance don't change that much from year to year, so the effect on the data will be minimal. FRs indicated that there was some confusion with regard to item 608. Some respondents who have refinanced their homes several times thought the phrase "including any second mortgage" referred to the second one in a series of refinanced mortgages. With the frequency with which homes are refinanced due to shifting interest rates, we recommend revising question wording to eliminate potential respondent confusion.

    Another issue regarding these items was raised at the FR debriefing in Minnesota. The FRs reported that many farmers don't know the property tax paid on their home because their tax bill includes both the house and the land. Often the land is more valuable. Also, some farmers don't know the amount paid for homeowners insurance because they have umbrella policies that include their home, cars, and other liabilities. Neither of these problems are unique to farmers, other than their property may be much larger than non-farm houses. To minimize this problem, we recommend revising the property tax question to delete the direct reference to "this house."

    Final decision: Modify question wording, sequencing and delete some questions.

    HHES determined that some of the original questions contained in this series were not needed to determine program eligibility, but are instead used to determine wealth/net worth. Since the wealth/net worth information is not critical for the purposes of SPD, some of the items can be deleted including: the question pertaining to remaining principal on the mortgage and the separate questions asking for the amount of property taxes and homeowners insurance. (For persons whose mortgage payments don't include taxes and insurance, separate questions on property taxes and homeowners insurance will continue to be asked.)

    The agreed upon sequencing and wording of this series is provided below.

  3. Do you have a mortgage on this property?

  4. <1> Yes

    <2> No

  5. Do you have a home equity loan on this property?

  6. <1> Yes

    <2> No

    CK609. If 607= "1", ask 609.

    Otherwise go to 611.

  7. How much are your monthly mortgage payments (including any condo or association fees)?

  8. ________.00

  9. Do your mortgage payments include property taxes?

  10. <1> Yes (GO TO 612)

    <2> No

    611. How much are your total property taxes, including city, county, and school taxes?

    ________.00

    CK612. If 607= "1", ask 612.

    Otherwise go to 613.

  11. Does your mortgage payment include insurance premiums?

  12. <1> Yes (GO TO CK614)

    <2> No

  13. How much do you pay for homeowner's insurance, that is, what is your annual premium?

  14. ________.00

    CK614. If 608= "1", ask 614.

    Otherwise to 618C.

  15. What is the balance remaining on your home equity loan?

  16. Training Issues: Item 605 (Property Taxes)

    Train FRs that taxes on the housing unit and associated property is the information being requested. In some areas with farmland, respondents were unsure if they should include the acreage associated with their home.

    c. Item 618C (utilities)

    Original wording: How much did this household pay for electricity, gas, and other utilities last month?

    Suggested revision: The next few questions are about amounts paid LAST MONTH for utilities such as electricity, water, and telephone service.

    Last month, how much did this household pay for electricity?

    And how much for gas or other types of heating fuel?

    Last month, how much did this household pay for BASIC telephone service?

    And how much for water and sewer and other utilities?

    Justification: There was evidence from all three evaluation sources that the question asking for amount of utilities was problematic and time consuming. Respondents tend to report amounts paid for each utility separately. FRs had to sum the individual amounts (without a calculator) to obtain a total. Respondents also tended to report an average amount paid, rather than the amount paid last month. To reduce the interviewer burden for this item, we recommend splitting the item into separate questions for each relevant utility service and having an internal calculation of the total amount. Also, we suggest a lead-in be added to emphasize that we are asking about the amount paid last month.

    Final decision: Ask for the "usual monthly" amount instead of the amount "last month." The revised wording is provided below.

    The next few questions are about your usual monthly utility bills.

    How much (do you/does this household) usually pay for electricity per month?

    _________.00

    How much for gas or other types of heating fuel per month?

    _________.00

    How much (do you/does this household) pay for BASIC telephone service per month?

    _________.00

    And how much (do you/does your household) usually pay for water and sewer per month?

    _________.00

    Training Issues: Emphasize to FRs in training that long distance charges are NOT to be included in the amount of BASIC telephone service.

    Train FRs that the time frame covered by bills for water and sewer vary widely by area. Some bills are for a monthly period, but many areas bill every 3 months, 6 months, etc. FRs should be aware of this and trained to probe, when appropriate, to make sure respondents are reporting a monthly amount. This same issue applies to the question about "other types of fuel." It may be that a person only gets fuel once or twice during the winter. The FR and respondent will need to work together to estimate a monthly amount.

    d (1) Item 631 (see item 635 for problem with this item)

    Original wording: Does anyone in this household own a car, van, or truck, excluding recreational vehicles and motorcycles?

    Final decision: Does anyone in this household own a car, van, or truck? Do not include leased vehicles, recreational vehicles, or motorcycles?

    d(2) Item 635

    Original wording: Is this vehicle owned free and clear or is there still money owed on it?

    Problem identified: In the Interviewing Observation Report Summary, an observer noted that this question was difficult to answer for persons leasing vehicles.

    Final decision: Use original wording. (Change made to item 631 to accommodate issue related to leased vehicles. HHES does not want leased vehicles to be counted as "owned".)

    Training issue: Train FRs that leased cars are not considered "owned" and should be marked as "no" in item 631.

    e. Item 637

    Original wording: Is this vehicle used primarily for either business purposes or for the transportation of a disabled person?

    Suggested decision: Not counting routine use to get to and from work, is this vehicle used primarily for either business purposes or for the transportation of a disabled person.

    Justification: Information obtained during the FR debriefings and from the taped interviews strongly indicates that many respondents misinterpreted this question. Respondents were including routine use of their vehicle to go to and from work as using it for "business purposes." To remedy this misunderstanding, we recommend revising the question to clarify that routine use is not to be counted as business purposes.

    .

    Final decision: Use suggested wording.

    f. Item 645

    Instrument problem: The first fill for this item did not seem to work right. The fill is suppose to say "Aside from mortgages/Aside from car loans, etc." However, review of taped interviews demonstrate that frequently when the person had just reported still owing money on a car (in item 635), the fill did not appear. Also, when the person did not report having a car loan, the fill DID appear.

    5. Educational enrollment

    a. Item 701

    Original Wording: At any time since September 1996 (were you/was name) enrolled in school, either full or part time?

    Suggested revision: At any time since September 1997, were you enrolled in a school such as high school, college or vocational school?

    Justification: The suggested revision is designed to focus respondents on enrollment in regular schools and reduce the likelihood of respondents answering 'yes' to school enrollment when they were in a job training program or other skill enrichment program not associated with regular school. The suggested wording is based on information gained through SPD cognitive interviews and was agreed upon in a meeting between POP and CSMR. However, this was not the wording used in the pretest instrument.

    Additionally, information obtained from the FR debriefing indicates that some respondents reported they were in school when enrolled in English as a Second Language (ESL) class. Revising the question to include examples of the types of schools may reduce this problem. With increased transitions from welfare to work, it is likely that the number of persons enrolled in programs such as ESL will grow. If ESL classes are located at a local college or high school, it is not unreasonable that the respondent may report that he/she is enrolled in a high school or college.

    Final decision: At any time between September 1997 and April 1998 (were you/was name) enrolled in school, either full or part time?

    Training issues: Include better description in training and the manual regarding the types of educational activities that are counted as "school." For analytic purposes we want to learn about people taking credited classes that could lead to a degree. We are interested in knowing about these credited classes even if the respondent does not intend to complete the course work necessary for a degree. Frs need clarification of the difference between work training and vocational school.

    b. Item 703

    Original wording: What was the highest level at which you were enrolled?

    Suggested decision: Was it a high school, college, vocational school or something else?

    High school (Ask A)

    College (Ask B)

    Vocational, technical, business school beyond high school level

    Something else (specify) (Go to Work Training)

    A. In what grade are you enrolled?

    High school grade 9-10

    High school grade 11

    High school grade 12

    High school equivalency/GED program

    B. At what level were you enrolled?

    College year 1 (Freshman)

    College year 2 (Sophomore)

    College year 3 (Junior)

    College year 4 (Senior)

    College year 5 (first year graduate or professional school)

    College year 6 (second year or higher graduate or professional school)

    Enrolled in college, but not working towards degree

    Justification: FRs indicated during the debriefing that some respondents had difficulty with the concept of "highest level at which you were enrolled." This was confirmed by confusion ("What do you mean highest level?"), long pauses, inaccurate responses, and requests for clarification identified in the taped interviews. This is consistent with the findings from the earlier cognitive interviews in which respondents had difficulty with the wording "highest level or grade at which you were enrolled."

    To eliminate the confusion associated with obtaining highest level of school, we recommend asking a direct question with the types/broad levels of schools (high school, college, vocational school, or something else) included as the response categories. Then, the specific level of high school and college can be obtained with the same level of detail as the response categories in the original question.

    Final decision: Use suggested wording. A statement needs to be included on a help screen (or item screen) that if a person switched levels during the reference period, the higher of the two levels should be marked in this item.

    Training issues: A person taking classes to improve his/her work skills will be identified in the work training section and should not be included here. If a respondent reports work training in this section, it should be coded as >something else' and the instrument will jump into the work training section. However, choosing something else, will prevent FRs from asking the school enrollment questions. We also recommend that FRs be trained to mark persons enrolled in ESL classes as "something else" in the suggested revision for item 703. FRs should receive some training on how to determine level for college students. For example, does a respondent need a certain number of credits to be considered college year two?

    c. Other educational enrollment training issues

    Item 704

    It is not clear from the current question how a respondent who indicates he/she receives assistance from other relatives (grandparents, uncle, aunt, sibling) should be coded in this question.

  17. Work training

  18. a. Item 805

    Original wording: Where did you receive this (most recent) training?

    Business, technical, or vocational school

    High school

    Two-year or community college

    Four-year college or university

    At current or previous employer's place of work

    Correspondence course

    Sheltered workshop

    Vocational rehabilitation center

    Other

    Suggested revision: Same as above, but add "unemployment office" to list of response options.

    Justification: In one taped interview, the respondent indicated receiving training from an unemployment office. If HHES thinks it will be useful analytically to capture this information, we recommend including a specific response category; otherwise, such cases can be recorded in the "other" category.

    Final decision: Add "unemployment office" to list of response options.

    b. Resequencing work training/educational enrollment sections

    Problem identified: FRs indicated that some respondents reported work training activities earlier in the educational enrollment section. The error in reporting was usually uncovered within the work training series of questions. FRs then had to back up in the instrument to the beginning of the school enrollment questions and change all of the answers.

    Suggested revision: We propose re-sequencing the instrument so the work training questions (Items 801-810) come prior to the educational enrollment questions (Items 700-705).

    Final decision: DSD indicated this request is a low priority since TMO does not want to implement this unless they are certain it can be done without jeopardizing the total functionality of the instrument.

    7. Functional Limitation and Disability

    Problem identified: Information from all three evaluation sources indicated that the current design (person level) of this series is extremely burdensome for FRs and respondents. Taped interviews indicated that respondents frequently interrupted the FRs with answers of "no" before the question reading was finished. Additionally, FRs frequently paraphrased questions, sometimes leaving off key words or phrases, to avoid irritating respondents with multiple administrations of the series. Both FRs and interviewing observers suggested asking this series at the household level since respondents tend to >tune out' after they go through the series one time.

    We suggest asking one series of questions for persons 15+ and one series of questions for children ages 0 to 14. The impact on data quality should be moderate. Reducing repetition may do a better job of holding respondent and interviewer attention than repeating the same series over and over again. This could have positive implications for the data collected later in the survey as respondents are less likely to be tired or bored. The suggested series for persons ages 15 and over is provided below. If our suggestion is accepted, item 900 regarding each person's health in general could be moved to the beginning of the health care utilization section, which is still asked at the person level.

    Suggested revision:

    For persons age 15-21, ask:

    901. Because of a physical, learning, or mental health condition, (do you/does (fill names of persons 15 to 21 years old)) currently have any limitation in (your/his/her/their) ability to do regular school work?

    <1> Yes ==> Ask "who" if more than one person 15+

    <2> No

    For persons age 15-21, ask:

    902. (Did you/Did (fill names of persons 15 to 21 years old)) receive any special education services during the past 12 months, that is since (month, year)?

    <1> Yes ==> Ask "who" if more than one person 15+

    <2> No

    For all persons age 15+, ask:

    904. (Do you/Does anyone in this household 15 years of age or older) have difficulty seeing the words and letters in ordinary newspaper print even when wearing glasses or contact lenses?

    <1> Yes ==> Ask "who" if more than one person 15+, and ask 905 for each person identified

    <2> No (GO TO 906)

    For each person identified from 904, ask:

  19. (Are/ Is) (you / name) able to see the words and letters in ordinary newsprint at all?

  20. <1> Yes

    <2> No

  21. (Do you/ Does anyone in this household 15 years of age and older) use any special aids such as a hearing aid, cane, wheelchair, or some other aid?

  22. <1> Yes ==> Ask "who" if more than one person 15+, and ask 907 for each person identified

    <2> No (GO TO 908)

    For each person identified in 906, ask:

  23. Which type of aid (do you/does name) use?

  24. <1> Hearing aid

    <2> Cane

    <3> Wheelchair

    <4> Walker

    <5> Crutches

    <6> Leg brace

    <7> Other

  25. (Do you/Does anyone in this household 15 years of age and older) have any difficulty hearing what is said in a normal conversation with another person, (even when using a hearing aid)?

  26. (Note: last parenthetical is filled if 907=1 for anyone in the household)

    <1> Yes ==> Ask "who" if more than one person 15+, and ask 909 for each person identified

    <2> No (GO TO 910)

    For each person identified in 908, ask:

  27. (Are you/Is name) able to hear what is said in a normal conversation at all?

  28. <1> Yes

    <2> No

  29. (Do you/ Does anyone in this household 15 year of age and older) have any difficulty lifting and carrying something as heavy as 10 pounds, such as a full bag of groceries?

  30. <1> Yes ==> Ask "who" if more than one person 15+, and ask 911 for each person identified

    <2> No (GO TO 912)

    For each person identified in 910, ask:

  31. (Are you/Is name) able to lift and carry this much weight at all?

  32. <1> Yes

    <2> No

  33. (Do you/does anyone in this household 15 and older) have difficulty walking a quarter of a mile -- about 3 city blocks?

  34. <1> Yes ==> Ask "who" if more than one person 15+, and ask 913 for each person identified

    <2> No (GO TO 914)

    For each person identified in 912, ask:

  35. (Are you/Is name) able to walk a quarter of a mile at all?

  36. <1> Yes

    <2> No

  37. Because of a chronic condition (do you/does anyone in this household 15 and older) need the help of another person with any of the following activities:
Getting in or out of a bed OR a chair? <1> Yes <2> No

Taking a bath OR a shower? <1> Yes <2> No

Doing any household chores such as preparing meals,

OR washing dishes, OR sweeping the floor? <1> Yes <2> No

Going outside the home to shop or

visit the doctor's office? <1> Yes <2> No

[If "yes" to any of the above items, ask "who" and whether they need the help "usually" or "occasionally."]

Final Decision: The recommendation to restructure series to a household-level design was accepted by HHES, with caveats that it depends on resource availability for revising instrument specifications and TMO's/Berkeley's resource availability for revising instrument code. HHES raised concern about using an untested series.

After consulting with TMO staff, DSD decided that the redesigned series cannot be accommodated for 1998. This revision will be considered for 1999, (possibly using a split ballot design). Postponing the revision until 1999 also allows time for cognitive testing of the proposed revised series.

The revision to item 914 is accepted for the 1998 design

8. Health Care Utilization (Adults 15+)

We suggest reordering the series by putting the question on visits to a psychiatric hospital (item 918) after item 915 and asking this of all persons age 15 or over. Responses to that question could be used to fill subsequent questions. (This recommendation was accepted.)

The recommendations below pertain specifically to those questions asked of adults 15+. If these recommendations are accepted, the comparable children's series would need to be modified as well.

  1. Item 915.

  2. Original wording: During the past 12 months, that is, since (date), (was/were) (name/you) a patient in a hospital overnight or longer?

    Suggested decision: During the past 12 months, that is, since (date), (was/were) (name/you) admitted to a hospital for an overnight stay or longer?

    Justification: Taped interviews indicated that some respondents reported "yes" to item 915, but then reported zero nights in item 919. These people went to the hospital for care but were not admitted for an overnight stay.

    Final decision: Suggested wording accepted.

    b. Item 916.

    Original wording: How many different times (were/was) (you/name) admitted to a hospital for an overnight stay or longer during the past 12 months?

    Suggested revision: How many different times (were/was) (you/name) admitted to a (medical/psychiatric/medical or psychiatric) hospital for an overnight stay or longer during the past 12 months?

    Justification: With the proposed reordering of the series, this question could be filled based on answers to item 915 and the reordered item 918. See item 918 for additional justification regarding reordering of series.

    Final decision: Suggested wording accepted.

    c. Item 918.

    Original wording: (Was/Were) (name/you) a patient in a psychiatric hospital or a psychiatric unit of a hospital during (this hospital stay/any of your hospital stays in the past 12 months)?

    Suggested revision: (Was/Were) (name/you) admitted to a psychiatric hospital or a psychiatric unit of a hospital during the past 12 months?

    Note: We recommend moving this item after item 915 and asking it of everyone.

    Justification: Interviewing observation reports indicated that the question order in the health care utilization series caused some confusion. One respondent thought the question on number of doctor's visits was only referring to psychiatric visits. She reported "0" to item 921 when, in fact, she'd been to the doctor 6 times. We suggest reordering the series by moving this question on psychiatric hospital stays right after item 915 and use the response to that question to fill subsequent items. This item would be asked of all respondents. We believe that item 915 does not adequately prompt respondents to consider stays in psychiatric hospitals and only those persons who report staying overnight in a hospital are asked item 918 in the current questionnaire.

    Final decision: Suggested question wording and item reordering accepted.

    d. Item 919.

    Original wording: How many total nights did (name/you) spend in a hospital of any type in the past 12 months?

    Suggested revision: How many total nights did (name/you) spend in a (medical/psychiatric/medical or psychiatric) hospital in the past 12 months?

    Justification: See justification for item 918.

    Final decision: Suggested wording accepted

    e. Item 921.

    Original wording: (Excluding hospital stays,) How many times did (you/name) see or talk to a medical doctor or assistant about (your/his/her) health in the past 12 months?

    Instrument problem: Taped interviews indicated the FRs did not read "or talk to" in this question. This is an instrument problem.

    Final decision: Instrument will be corrected.

    f. Item 923.

    Original wording: Is there a place that (name/you) (go/goes) if (you/he/she) (are/is) sick or need advice about (your/his/her) health?

    Problems identified: Taped interviews indicated that both respondents and FRs had difficulty with this question. In some cases, respondents would report "no" because the sample person hadn't been to the doctor. In other cases, the FRs are probing inappropriately, for example, "If you were sick , is there a place you could go to for them to take a look at you?"

    Final decision: The intention of this question is to measure whether someone has a regular place to go for medical care, not whether they have access to some type of care.

    Training issues: The intention of the question needs to be clarified in training and the Fr manual.
     
     

    g. Item 924.

    Original wording: To what kind of place did (you/name) usually go?

    <1> Clinic or health center

    <2> Doctor's office (or HMO)

    <3> Hospital emergency room

    <4> Hospital outpatient department

    <5> Some other place

    Suggested revision: What kind of place is it - a clinic, a doctor's office, an emergency room, or some other place?

    Justification: Taped interviews indicated that FRs often read the response categories to the respondents in order to clarify the intent of the question. The suggested wording was agreed upon with HHES but was never incorporated into the pretest instrument. Note that the suggested wording does not include "hospital emergency room" or "hospital outpatient department."

    Final decision: Use original wording with the following modifications:
    "To what kind of place did (you/name) usually go: a clinic, a doctor's office, an emergency room, or some other place?

    FR: IF NECESSARY, READ RESPONSE CATEGORIES

    <1> Clinic or health center

    <2> Doctor's office (or HMO)

    <3> Hospital emergency room

    <4> Hospital outpatient department

    <5> Some other place (Specify)

    h. Item 924A and 924B

    Original wording:

    924A. During (fill last month) did (you/or anyone in this household) pay any doctor, dentist, or hospital bills, for prescription medicines for (Name)?

    (Note: "for" prescription medicines was a typo in the instrument. It should have been "or.")

    924B: Not counting amounts reported by another family member or amounts that will be reimbursed by insurance, how much was paid last month for (your/Name's) medical expenses?

    Suggested revision:

    924A. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) have any out-of-pocket expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

    (NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES)

    <1> Yes

    <2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

    924B. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) doctor, dentist, or hospital bills last month?

    924C. Did (you/anyone in this household) have any out-of-pocket expenses for prescription medicines for (yourself/name) last month?

    <1> Yes

    <2> No (SKIP TO CK925)

    924D. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) prescription medicines last month?

    Justification: FRs indicated that item 924A was too long and that some respondents thought the question was only asking about money spent on prescription medicines (due, in part, to a typo in the instrument). They suggested breaking the question into two separate items. The interviewing observation report indicated that FRs and respondents were unsure whether the amount paid for insurance premiums should be counted. The FR debriefing summary and the taped interviews both highlight the need for a transition statement to indicate that the question was only about last month. Several respondents thought the question was about last year. Taped interviews also show that respondents who had expenses that were covered by their insurance company were unclear how if they should be included.

    Final decision: Recommendation accepted with slight modification, as shown below.

    924A. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) pay any expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

    NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES.

    NOTE: "PAY" REFERS TO "OUT-OF-POCKET " EXPENSES.

    <1> Yes

    <2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

    924B. Not counting amounts that will be reimbursed by insurance, how much was paid for (your/name's) doctor, dentist, or hospital bills last month?

    924C. Did (you/anyone in this household) pay any expenses for prescription medicines for (yourself/name) last month?

    <1> Yes

    <2> No (SKIP TO CK925)

    924D. Not counting amounts that will be reimbursed by insurance, how much was paid for (your/name's) prescription medicines last month?

    9. Health Insurance

    a. Item 950.

    Original wording: The next few questions are about health insurance coverage in 1996.

    This is a list of different types of health insurance coverage. I'd like to know if (you/anyone in this household) was covered by the following types of health insurance at ANY TIME from January through December 1996:

    a. Medicare, the government medical plan for persons 65 and over and for persons with disabilities

    b. Medicaid or (state program name), the government medical plan for persons with low incomes

    c. CHAMPUS/TRICARE, CHAMPVA, Military health, Indian Health Service, or any other government- provided health insurance plan including (fill local name.)

    d. A plan provided (by a person in this household) through a current or past employer or union

    e. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer

    f. A plan of someone not living in this household

    Suggested revision: FLASHCARD R.

    [Ask for each person in the household. Fill "The next...1997" for first person in the household only. ]

    (The next few questions are about health insurance coverage in 1997.) Which type of health insurance, if any, (was/were) (you/name) covered by at any time between January and December 1997?

    MARK ALL THAT APPLY

    a. Medicare, or other health insurance paid for by Medicare (for persons over 65 or persons with disabilities)

    b. Medicaid or (state name) (for persons with low incomes)

    c. CHAMPUS/TRICARE, CHAMPVA, Military health, Indian Health Service, or any other government- provided health insurance plan (including (fill local name))

    d. A plan provided (by a person in this household) through a current or past employer or union
    e. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer

    f. A plan of someone not living in this household

    g. Not covered by health insurance at all during (year)

    Justification: Taped interviews indicated that both respondents and FRs lost sight of the fact that this question is asking about the past calendar year. Asking the question about each person in the household individually and including the reference period at the end of the question should reduce the likelihood of respondents erroneously reporting current coverage. (If this recommendation is implemented, questions asking who was covered by the different types of insurance marked in item 950 can be deleted. This includes items 951A, 953A, 955A, 957A, 959A, 961A.)

    Taped interviews also indicated that Medicare recipients who are enrolled in an HMO that is paid for by Medicare are unclear how to respond to this question. We suggest modifying category "a" as indicated above to alleviate the confusion. Additional coverage carried by some Medicare recipients, often referred to as "Medi-gap," is considered a plan purchased directly from the insurance company. Rather than modifying the flashcard to include this term, we suggest that FRs receive additional training regarding this type of health insurance and how to adequately probe Medicare recipients for this information.

    The flashcard and instrument screen should contain the lettered response category. The pretest flashcard contained lettered response options and the instrument screen contained a yes/no format for each response category without the preceding letter. The taped interviews indicated that respondents would say "d" and the FR wouldn't know to which category they were referring since "d" did not appear on their screen.

    Final decision: Although HHES accepted this recommendation, it cannot be implemented in 1998 due to resource constraints within TMO, but will be held for implementation in 1999.

    It was also agreed that the response categories should be reordered as shown below, so the more common types of insurance are towards the top of the list. (Along with the reordering, some revisions have also been made in the use of bold type.) It is unclear at this time if the reordering of the insurance types is possible for 1998, or if that also needs to be held until 1999.

    The letters used on the flashcard should be added to the FRs screen.

    a. A plan provided (by a person in this household) through a current or past employer or union

    b. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer (e.g. "Medi-gap")

    c. Medicare, or other health plan paid for by Medicare (for persons over 65 or persons with disabilities)

    d. Medicaid (or state name) (for persons with low incomes)

    e. CHAMPUS/TRICARE, CHAMPVA, Military Health

    f. Indian Health Service

    g. Other government-provided health insurance plan (including state name)

    h. A plan of someone not living in this household

    i. Not covered by any kind of health insurance for the entire year

    Note: FRs will need to write in the state names on their flashcards, if appropriate, since printing flashcards tailored to each state/local program is not possible. Also, not that the parenthetical "(e.g. Medi-gap)" for option b is to be shown on the FR screen but not on the flashcard.

    Training Issue: We suggest FRs receive additional training regarding Medi-gap coverage and how to adequately probe Medicare receipents for this information.

    b. Item 951B (and other items asking for months covered)

    Original wording: During which months in 1996 (was/were) (name/you) covered by Medicare?

    Suggested revision: Were you covered by (type of plan) for all of 1997 or for only part of 1997?

    <1> All year

    <2> Part of year (ASK WHICH MONTHS)

    Which months were you covered by (type of plan) in 1997?

    <1> Jan <7> July

    <2> Feb <8> Aug

    <3> Mar <9> Sep

    <4> Apr <10> Oct

    <5> May <11> Nov

    <6> Jun <12> Dec

    [blind] <D,R>

    Note: This suggestion also applies to 953B, 955C, 957D, 959C, 961B, 966.

    Justification: Taped interviews indicate that FRs will read the question as worded and then probe using wording similar to that included in the suggested wording above. Additionally, taped interviews indicated that most respondents are covered by the same plan all year.

    Final decision: Use suggested wording. Change this item as well as 953B, 955C, 957D, 959C, 961B, and 966.

    c. Item 955C.

    Original wording: During which months in 1996 (was/were) (name/you) covered by this type of health insurance?

    Suggested revision: During which months in 1997 (was/were) (name/you) covered by (fill from 955B)? [Fills from 955B are: CHAMPUS, CHAMPVA, Military health, Indian Health Service, a government-provided plan (including (state name))]

    Note: This change also applies to items 957D and 961B.

    Justification: The interviewing observation report suggested including the name of the plan rather than using the generic phrase "this type of health insurance."

    Final decision: Use suggested wording. Change this item as well as 957D and 961B.

    d. Item 957A.

    Original wording: Who was covered by an employer or union provided plan?

    Suggested revision: Who was covered by an employer or union provided plan in 1997?

    Justification: Taped interviews indicated that some respondents reported persons currently covered rather than those covered during the past calendar year. Similar questions about other types of health insurance include the reference period, suggesting that this was an oversight when the questionnaire was developed.

    Final decision: Suggestion is accepted.

    e. Item 957B.

    Original wording: Which person in this household is the policy holder of (name)'s plan?

    Suggested revision: Which person in this household was the policy holder of (name's) plan during 1997?

    Justification: The current wording sounds like we are asking about current coverage, but the question is intended for coverage during the past calendar year. Taped interviews indicated that respondents, and in some cases FRs, didn't know the question was about last year. We suggest adding the reference period to the end of the question to clarify this.

    Final decision: Use suggested wording.

    Instrument problem: Taped interviews indicated that items 957B, 957C, and 957D were not asked about children under 15. These questions should be asked about ALL household members as appropriate.

    f. Item 957C.

    Original wording: Does the employer or union pay for all, part, or none of the cost of the plan?

    Suggested revision: Did the employer or union pay for all, part, or none of the cost of the plan in 1997?

    Justification: Original wording asks about current coverage. Suggested wording asks about past calendar year, which is the concept of interest.

    Final decision: Use suggested wording.

    g. Item 963.

    Instrument problem: Responses of "don't know" or "refused" should skip to 968, not 967.

    Final decision: Instrument will be corrected

    h. Item 965.

    Original wording: FLASHCARD R.

    What type of health insurance (was/were) (name/you) covered by in 1996?

    {SHOW FLASH CARD; MARK ALL THAT APPLY}

    <1> Medicare - the government plan for persons 65 and over and for persons with disabilities

    <2> Medicaid or (fill local name) -- the government medical plan for persons with low incomes

    <3> CHAMPUS/TRICARE/CHAMPVA, Military health, Indian health service, or any other government-provided health insurance plan, including (fill local name)

    <4> A plan provided (by a person in this household ) through a current or past employer or union.

    <5> A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer.

    <6> A plan of someone not living in this household.

    Suggested revision: Keep question wording the same. Use flashcard for item 970 shown below in item 968. Add wording to medicare response category on flashcard, as suggested above in item 950.

    Justification: This item is asked of persons who are erroneously reported as not having health insurance in the previous calendar year. The suggested wording for item 950 would alleviate the need for this question since it collects health insurance at the person level rather than the household level. If the suggested change for item 950 is not accepted, we suggest using the same flashcard for this item and item 970. FRs noticed that the flashcards for these two items were similar but not identical. They suggested making them the same and eliminating one of the flashcards. We suggest using Flashcard T because it is easier to read and includes the distinction between policyholder and dependents. The FRs also requested that the local name for government-provided health insurance be included on the screen and on the flashcard. In the pretest instrument, the FRs were required to specify the name of the local government provided insurance (e.g. MinnesotaCare). The fill for the local name appears in number 3 on Flashcard R. If the recommendation to substitute Flashcard T for this item is accepted, the local name should be included on Flashcard T and on the instrument screen (the specs shows no fill for the local name) so that FRs don't have to type this information into the "specify" screen.

    Final recommendation: In 1999 this item will be deleted. In 1998, keep the question wording the same, but replace the response options with those from Flashcard T (shown below under item 968.)

    i. Item 967

    Original wording: Which answer on this card best describes the reason why (name/you) (wasn't/weren't) covered by health insurance in 1996?

    MARK ALL THAT APPLY

    <1> Job layoff, job loss, or any reasons related to unemployment

    <2> Employer does not offer health insurance

    <3> Can't obtain health insurance because of poor health, illness, or age

    <4> Too expensive; can't afford health insurance

    <5> Don't believe in health insurance

    <6> Have been healthy; not much sickness in the family; haven't needed health insurance

    <7> Able to go to VA or military hospital for medical care

    <8> Person outside this household did not provide health insurance

    <9> Other (specify)

    Suggested revision: Keep wording the same. Add a response category "Not eligible because haven't worked at job long enough yet."

    Justification: An FR suggested this category be added to accommodate people who are working but haven't been at a job long enough to be eligible for health insurance. If this is a group of people that are of interest to analysts, then such a category should be added. Otherwise, existing categories can remain as they are.

    Instrument problem: The question asks for one answer, while the instructions to the FR asks for as many as apply. HHES wants this to be a "mark all that apply" question.

    Final decision: Since the question is intended to be a "mark all that apply", the question needs to be slightly modified to indicate that multiple

    responses are acceptable. Also, HHES requested that the same response categories used in SIPP be included in SPD, as shown below.

    What answers on this card best describe the reasons why (name wasn't/you weren't) covered by health insurance in 1997?

    Probe: Any other reasons?

    <1> Too expensive; can't afford health insurance.

    <2> No health insurance offered by employer of self, spouse, or parent

    <3> Not working at a job long enough to qualify

    <4> Job layoff, job loss, or any reason related to unemployment

    <5> Not eligible because working part time or temporary job

    <6> Can't obtain insurance because of poor health, illness, age, or pre-existing condition

    <7> Dissatisfied with previous insurance OR don't believe in insurance

    <8> Have been healthy; not much sickness in family; haven't needed health insurance

    <9> Able to go to VA or military hospital for medical care

    <10> Covered by some other health plan, such as Medicaid

    <11> No longer covered by parents' policy

    <12> Other

    j. Item 968

    Original wording: These next few questions are about CURRENT health insurance coverage. (Are/Is) (you/anyone in this household) CURRENTLY covered by any type of health insurance, including Medicare and Medicaid?

    Suggested revision:

    [For persons covered either all year or at the end of 1997, ask the following question:]

    968a. (Is name/Are you) CURRENTLY covered by (fill types of insurance this person reported having in Devember 1997)?

    <1> Yes (Ask 968b)

    <2> No (ASK FOR CURRENT COVERAGE)

    FLASHCARD T

    968b. What type of health insurance, if any, (are you/is name) currently covered by?

    <1> Employer/union provided (policy holder)

    <2> Employer union provided (dependent)

    <3> Purchased plan directly--not employer/union, (e.g. medi-gap) (policy holder)

    <4> Purchased plan directly--not employer/union (e.g. medi-gap) (dependent)

    <5> Medicare, or other health plan paid for by Medicare

    <6> Medicaid, or (fill state name)

    <7> CHAMPUS/TRICARE/CHAMPVA

    <8> Military Health

    <9> Indian Health Service

    <10> Other government health insurance plan or (fill local name) (specify)

    <11> Covered by someone outside this household

    <12> Not currently covered by health insurance

    [blind] <D> Don't know

    [blind] <R> Refused

    [For persons not covered at all during previous calendar year or for persons not covered in December of previous calendar year, ask the following question(s):]

    969a. (Is/Are) (name/you) currently covered by health insurance?

    <1> Yes (Ask 969b)

    <2> No (Skip out of series)

    FLASHCARD T

    969b. What type of health insurance (are/is) (you/name) currently covered by?

    (Same categories as above)

    Justification: Taped interviews, interviewing observation reports, and the FR debriefings indicated that there was confusion caused by the shifting reference periods in the health insurance series. The questions about current coverage sounded very similar to the questions about coverage during the past calendar year. In addition, many respondents and FRs thought the questions on the past calendar year were actually about current coverage (see previous comments). Taped interviews indicate that for many people health insurance does not change from one year to the next. Asking if the health insurance is the same as that reported for last year simplifies this series for the vast majority of people. Additionally, taped interviews and observation reports indicated that the original wording mislead respondents into thinking we were asking if they were covered by Medicare or Medicaid. They would report "no" when, in fact, they were currently covered. This required FRs to probe extensively to obtain the correct information. The suggested wording is similar to that contained in the March CPS.

    Final decision: The suggestion was accepted by HHES but cannot be implemented for 1998 due to TMO resource constraints. This revision will be held for 1999 implementation. As an alternative option for 1998 implementation, we propose the following strategy be adopted for collecting current coverage. This has not yet been discussed with HHES or DSD, so it is unknown whether the suggested revision can be implemented for the 1998 SPD.

    Alternative suggestion for 1998:

    [For persons covered either all year or at the end of 1997, ask the following question:

    968a. (Is everyone in this household/Are you) CURRENTLY covered by the same type(s) of insurance (you/they) had in December 1997?

    [Display insurance type(s) covered by in December 1997. If possible, when persons in hhld covered by different types, display type by persons covered]]

    <1> Yes (end series)

    <2> No (Ask 968b)

    968b. Who is CURRENTLY not covered by the same type of insurance they had in December 1997?

    For each person identified in 968b, ask:

    968c. What type of health insurance, if any, (are you/is name) currently covered by?

    <1> Employer/union provided (policy holder)

    <2> Employer union provided (dependent)

    <3> Purchased plan directly--not employer/union, (e.g. medi-gap) (policy holder)

    <4> Purchased plan directly--not employer/union, (e.g. medi-gap) (dependent)

    <5> Medicare, or other health plan paid for by Medicare

    <6> Medicaid, or (fill state name)

    <7> CHAMPUS/TRICARE/CHAMPVA

    <8> Military Health

    <9> Indian Health Service

    <10> Other government health insurance plan or (fill local name) (specify)

    <11> Covered by someone outside this household

    <12> Not currently covered by health insurance

    [blind] <D> Don't know

    [blind] <R> Refused

    10. Food security

    Problem identified: During the FR debriefings, many FR's indicated that they felt embarrassed asking these questions in higher income households. The observation reports also indicated that these questions seemed unnecessary to ask in upper income households. The CPS Food Security Supplement from which these questions were taken uses an income screener. Households are still asked item 1000 (quantity and quality of food eaten in the household) and one other item (not included in this set). Depending on the answer to these two items, persons in households showing no signs of food insecurity are skipped over the remaining series of questions. The income screener used in CPS is based on a categorical variable for family income in the previous 12 months. We suggest considering the use of an income screener with these items.

    Final decision: FNS has agreed to use an income screener for the food security questions. If the family income exceeds 185 percent poverty and the respondent indicates that the household has "enough food to eat and the kinds of food they want" (response <1> in item 1000), then the remainder of the food security questions can be skipped. The CPS Food Security Supplement contains a similar economic screener. Specifications are available from Jenny Hess in CSMR or the CPS Branch in DSD. POP division has expressed concerns regarding whether this change can be made in the required time frame. POP, TMO, and DSD need to discuss this issue and determine what priority should be given to this requested change.

    a. Item 1000.

    Original wording: These next questions are about the food eaten in your household. Which of these statements best describes the food eaten in your household in the last 12 months -- (I/we) have enough to eat and the kinds of food (I/we) want, (I/we) have enough to eat but not always the kinds of food (I/we) want, sometimes (I/we) don't have enough to eat, or often (I/we) don't have enough to eat?

    Suggested revision: Keep wording the same. Include a flashcard for this item.

    Final decision: The Food and Nutrition Service (FNS) of the US Department of Agriculture (USDA) requested that the lead-in to this question be changed as follows:

    Flashcard X.

    These next questions are about the food eaten in your household in the last 12 months, since May 1997, and whether you were able to get the food you need. Which of these statements best describes the food eaten in your household in the last 12 months -- (I/we) have enough to eat and the kinds of food (I/we) want, (I/we) have enough to eat but not always the kinds of food (I/we) want, sometimes (I/we) don't have enough to eat, or often (I/we) don't have enough to eat?

    Replace the current lead-in with this new one and include a flashcard with the answer categories.

    Training issues: FRs need a better understanding of why the food security series is included in the SPD. This could reduce the FRs' discomfort asking these questions and better enable the FRs to respond to concerns that respondents raise. The FR manual and training materials from the CPS Food Security Supplement should provide some of this information. FNS may be able to advise us on additional materials to include in training and the manual.

    11. Children's school enrollment

    a. Item 1103 (For children 3-6)

    Original wording: Since September 1996 was (Name) enrolled in preschool? (Note: Include pre-kindergarten as well as preschool.)

    Suggested revision: At any time between September 1997 and April 1998 was (Name) enrolled in preschool?

    Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

    Final decision: Use suggested wording.

    b. Item 1108 (For children 5-6)

    Original Wording: At anytime since September 1996 was (name) enrolled in school?

    Suggested revision: At anytime between September 1997 and April 1998 was (name) enrolled in school?

    Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

    Final decision: Use suggested wording.

    c. Item 1108A

    Original wording: Since September 1996 which months was (name) enrolled in school?

    Suggested revision: Between September 1997 and April 1998 which months was (name) enrolled in school?

    (Note: Add a category for >all' months).

    Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

    Final decision: Use suggested wording.

    d. Item 1109

    Original wording: Since September 1996 what was the highest grade in which (child name) was enrolled?

    Suggested revision: In what grade was (name) enrolled in April 1998?

    Justification: The suggested wording was previously agreed to based on cognitive interviewing and should help to alleviate confusion generated by the original question. In the FR debriefings, FRs noted that respondents had problems knowing what was meant by the 'highest grade in which ... was enrolled'. This was confirmed by the review of the taped interviews which indicated that respondents had a difficult time with the highest grade concept. These problems seemed to stem in part from the reference period.

    Final decision: Use suggested wording. For persons who dropped out of school before April 1998, fill with the last month they were enrolled in school.

    Training issues: FRs should be trained how to handle situations in which a high school student simultaneously takes college classes.

    e. Item 1116

    Original wording: Did (name) attend special classes for gifted students or do advanced work in any subjects since September 1996?

    Suggested revision: Did (name) attend classes for gifted students or do advanced work in any subjects between September 1997 and April 1998?

    Justification: Review of taped interviews indicate that some respondents seemed to focus on 'special' rather than 'gifted or advanced work'. It is possible that respondents are accustom to hearing the term special used in the context of special or remedial education and do not think of "special" as describing gifted and talented or advanced course work. Eliminating the word "special" should alleviate this problem.

    Final decision: Use suggested wording.

    Training issues: FRs and respondents need clarification regarding what constitutes "classes for gifted students" and "advanced work." Kurt Bauman (POP) provided the following clarification.
    "The central idea of this question is to identify students who are taking ACADEMICALLY-ORIENTED courses covering subject matter that goes ABOVE AND BEYOND THE STANDARD CURRICULUM for the level of school they are in. Advanced placement (AP) courses are an obvious example of this. Students who are in "high track" courses qualify, so long as the track involves advanced academic subjects or leads directly to courses that do. DO NOT COUNT special "enrichment" courses (e.g. nature and the environment, art workshops, music, performance or internships) unless they are designed for gifted students and prepare them for advanced ACADEMIC course work later on. Any class that is remedial in nature, is designed to help students catch up in certain subject areas or deal with learning or behavioral challenges should not be included."

    f. Item 1117

    Original wording: Was (Name) suspended or expelled from school at any time since September 1996?

    Suggested revision: Between September 1997 and April 1998 was (Name) suspended or expelled from school?

    Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

    Final decision: Use suggested wording.

    g. Item 1119

    Original wording: Since September 1996 did (Name) change schools?

    Suggested revision: Between September 1997 and April 1998 did (Name) change schools?

    Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

    Final decision: Use suggested wording.

    h. Item 1120

    Original wording: How many times did (Name) change schools Since September 1996?

    Suggested revision: Since September 1997 how many times did (Name) change schools?

    Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

    Final decision: How many times did (name) change schools?

    1. Children's enrichment activities
    The primary problem with this series, identified through review of the taped interviews, was a failure by FRs to read the questions as worded for this module. In some cases, major parts of the questions were not read to respondents. (These problems can be handled through FR training.) Other problems identified through the taped interviews were related to reference periods. a. Item 1124

    Original wording: Since September 1996 was (Name) on any kind of a sports team?

    Suggested revision: Between September 1997 and April 1998 was (Name) on any kind of sports team?

    Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in the reference period used for the school enrollment series.

    Final decision: Use suggested wording.

    b. Item 1125

    Original wording: Did (name) take lessons after school or on weekends in activities such as music, dance, language, or karate at any time since September 1997?

    Suggested revision: Did (name) take any kind of lessons after school or on weekends in activities such as music, dance, language, or karate at any time between September 1997 and April 1998?

    Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

    Final decision: Use suggested wording.

    c. Item 1126

    Original wording: Did (name) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, (Boys/Girls) club, or a religious group at any time since September 1996?

    Suggested revision Did (name) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, (Boys/Girls) club, or a religious group at any time between September 1997 and April 1998?

    Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in the reference period used for the school enrollment series.

    Final decision: Use suggested wording.

    d. Item 1127

    Original wording: The next few questions are about television viewing.

    Are there family rules about what programs (name) can watch?

    Suggested revision: Are there family rules about how much television or what programs (name) can watch?

    Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

    Final decision: Use suggested wording.

    e. Item 1128

    Original wording: How many hours per week does (name) usually watch television?

    INCLUDE BOTH VIDEOS AND TV VIEWING

    Suggested revision: Including weekends, how many hours per week does (name) usually watch television?

    INCLUDE BOTH VIDEOS AND TV VIEWING

    Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

    Final decision: Use suggested wording.

    f. Item 1132

    Original wording: During the past month, did you (or any family member) take (name) on any kind of outing such as to a park, library, zoo, church, playground, or shopping center (Read categories)

    Suggested revision: How often in the past month did you (or any family member) take (name) on any kind of outing such as to a park, library, zoo, church, playground, or shopping center - never, once in the past month, about once a week, several times a week, every day or almost every day?

    Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument. In some cases respondents asked the FR to repeat the reference period. Implementing this wording should reduce reference period problems identified through taped interviews.

    Final decision: Use suggested wording.

    13. Children's disability

    We suggest having one set of children's disability questions to be asked of all children age 0 to 14 as appropriate. HHES accepts this recommendation; however, due to resource and time constraints, the recommendation will not be included in the 1998 SPD instrument, but will be considered for the 1999 SPD. This set of questions will be developed at a later date and are not included here due to time constraints. When developing these questions, we need to consider how this will work in households in which there is more than one designated parent, since the children's questions are supposed to be asked of the designated parent and this proposal would violate that principle.

    14. Children's health care utilization

    a. Item 1210.

    Original wording: During the past 12 months, that is since (date), was (name) a patient in a hospital overnight or longer?

    Suggested revision: During the past 12 months, that is, since (date), was (name) admitted to a hospital for an overnight stay or longer?

    Justification: This change was made to the comparable item in the adult section. See justification under item 915 in adult health care utilization above.

    Final decision: Suggested wording accepted.

    b. Item 1220.

    Original wording: To what kind of place did (name) usually go?

    <1> Clinic or health center

    <2> Doctor's office (or HMO)

    <3> Hospital emergency room

    <4> Hospital outpatient department

    <5> Some other place

    Note: We are not sure what wording was actually used in the pretest instrument. To make the adult and children's items comparable, we suggest using the agreed upon wording for the adult item, which is included below under "final decision."

    Suggested revision: What kind of place is it - a clinic, a doctor's office, an emergency room, or some other place?

    Justification: Same change was suggested to comparable adult item. See justification for item 924 in adult health care utilization.

    Final decision: Use original wording with the following modifications:
    "To what kind of place did (you/name) usually go: a clinic, a doctor's office, an emergency room, or some other place?

    FR: IF NECESSARY, READ RESPONSE CATEGORIES

    <1> Clinic or health center

    <2> Doctor's office (or HMO)

    <3> Hospital emergency room

    <4> Hospital outpatient department

    <5> Some other place (Specify)

    c. Item 1221a and 1221b.

    Original wording 1221a: During (fill last month) did (you/or anyone in this household) pay any medical expenses for (name)--include any doctor, dentist, or hospital bills, or any prescription medicines?

Note: This wording is different than the comparable question in the adult section. We recommend using the agreed upon wording for the adult question as shown below under "final decisions."

Original wording 1221b: Not counting amounts that will be reimbursed by insurance, how much was paid last month for (name's) medical expenses?

Suggested revision 1221a. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) have any out-of-pocket expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

(NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES)

<1> Yes

<2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

1221b. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) doctor, dentist, or hospital bills last month?

1221c. Did (you/anyone in this household) have any out-of-pocket expenses for prescription medicines for (yourself/name) last month?

<1> Yes

<2> No (SKIP TO CK1280)

1221d. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) prescription medicines last month?

Justification: This change was suggested for the comparable adult items. See the justification for items 924A and 924B in the adult health care utilization section.

Final decision: Recommendation accepted with slight modification. Revised wording provided below:

1221a. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) pay any expenses for doctor, dentist, or hospital bills for (name) last month?

NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES.

NOTE: "PAY" REFERS TO "OUT-OF-POCKET " EXPENSES.

<1> Yes

<2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

1221b. Not counting amounts that will be reimbursed by insurance, how much was paid for (name's) doctor, dentist, or hospital bills last month?

1221c. Did (you/anyone in this household) pay any expenses for prescription medicines for (name) last month?

<1> Yes

<2> No (SKIP TO CK1280)

1221d. Not counting amounts that will be reimbursed by insurance, how much was paid for (name's) prescription medicines last month?

  1. Designated parent's work schedule
a. Items 1294A.

Original wording: FLASHCARD X; READ TO RESPONDENT, IF NECESSARY.

DURING SEPTEMBER 1997, did (you/designated parent) do any of these things to look for (a/another) job?

Problem identified: Taped interviews indicated that this question plus the questions on school attendance (1295A) and job training (1296A) seemed awkward when asked of people who work full time (as indicated in item 1291).

A taped interview indicated that this question and subsequent ones using the fill for looking for work confused the respondent. This person worked full time but reported that she asked a friend/relative about a job. When she got to the questions on how many hours per week she usually spends looking for a job, she didn't know how to answer because she didn't usually spend any time looking for work per week, she just happened to ask a friend/relative about a job last month. She finally responded "less than 1 hour" in order to get past the screen. The concepts addressed in items 1294A and 1294B are inconsistent. The first asks about any time spent at this activity and the second about usual hours per week.

Final decision: POP division noted that full time workers may also be looking for work, attending job training, or going to school in order to try to get a better job, particularly persons who are in low wage jobs. They recommend continuing to ask these questions of all mothers/designated parents. Some of the problems identified above will be addressed through modifications to items 1294B, 1295B, and 1296B noted below.

Training issues: Based on the FR debriefings, FRs need more information on who the designated parent is and the reason it is defined the way it is. They also need to understand why we are asking questions about work, school, training, and looking for work for last month for the designated parent.

b. Items 1294B, 1295B, 1296B.

Problem identified: See problem discussed above for item 1294A.

Final decision: Keep wording the same. Change the range of accepted answers from 1-99 to 0-99 so that persons who did not do the activity on a regular basis could enter "0." If "0" is entered, the activity will not be used in later fills in the child care series. Include a note to the FR on the screen, "Enter "0" if activity is done less than 1 hour per week."

16. Child care

POP division indicated that they do not think it is advisable to make major structural changes to this series due to time and resource constraints. Because of this, we did not make specific recommendations for modifying this series. Instead, we include problems identified by the various pretest evaluation methods. Below is a discussion of the problems identified and the decision for minor question revisions reached jointly with POP.
  1. Item 1301.
Original wording: (The next few questions are about child care.)

(In addition to school,) Please tell me which of these you used for (child's name) on a regular basis BETWEEN JANUARY 1996 AND SEPTEMBER 1997.

<1> Child's other parent/stepparent cared for child while designated parent was (fill)

<2> Designated parent cared for child while (he/she) was (fill)

<3> Child's brother or sister age 15 or older

<4> Child's brother or sister under age 15

<5> Child's grandparent

<6> Any other relative

<7> Family day care home (caring for 2 or more kids

in provider's home)

<8> A non-relative such as a friend, neighbor, sitter or nanny

<9> Nursery school, preschool

<10> Federally-funded Head Start program

<11> Kindergarten/School (grades 1-12)

<12> Child care or day care center

<13> Before or after school care

<14> Child cares for (himself/herself)

<15> Any other type of arrangement (specify)

Suggested revision: Include the definition of "regular" in the question. Modify question wording to indicate that no regular arrangement is an option and include a response category for "no regular arrangement."

Problems identified: FRs recommended reordering the series from person level questions to household level questions (by type of arrangement) and ask if any of the children were cared for in a particular arrangement. FR's thought this approach was preferable because they think children in the same family are often cared for in the same arrangement. Martin O'Connell indicated that the type of arrangement varies by the age of the child. Additionally, without testing, we do not know if this approach would be an improvement.

FR debriefings and the taped interviews indicated that respondents missed the concept of arrangements used on a "regular" basis. The subsequent questions assume the arrangement is used on a regular basis and respondents who used something once in a while are thoroughly confused by questions on the months the arrangement was used, hours, costs, etc. This creates major problems as the series progresses. For example, we had a substitute teacher who substituted twice during September for two different school districts. The first time through the series (she has three children under 15), she chose categories 2 (designated parent cares for child while working) and 14 (child cares for himself/herself). She said her husband picks up the kid when she substitutes (she thought "designated parent" meant the parent designated to pick up the kids). She couldn't answer the subsequent questions well because she usually cares for her kids herself except when she gets a substitute job and they need to piece something together. The second time through the series, the respondent said she doesn't really use anything on a regular basis. In another interview, the respondent chose the arrangement "non-relative such as a friend, neighbor, sitter or nanny." When she was asked which months the child was cared for on a regular basis by a non-relative, the respondent said "None, really. It was only occasionally." The FR told her she had to mark at least one month in order to proceed, so the respondent told her to mark "April."

In an earlier version of these questions we included a definition of the term regular: "By regular, I mean at least once a week for a month or more." Perhaps we should consider including the definition again. We may also want to indicate that no regular arrangement is an option. FR debriefing and the taped interviews indicated a need for a category "none" or "no regular arrangement used." This category is needed for mothers /designated parents who do not work out of the home and do not use any regular arrangement, as well as for working mothers/designated parents who don't use a regular arrangement, such as the substitute teacher. Using the category "any other type of arrangement" is not a good solution because all the subsequent questions on months the arrangement was used, hours, costs, etc. will be asked and these won't make sense to the respondent. The way the question is worded now, it forces the respondent to choose an option. Adding something like, "Please tell me which of these, if any, you used for child..." might allow those people who don't use any regular arrangement to correctly opt for the none category more easily.

One taped interview indicated that the question failed to solicit the child care arrangement used by a single parent who works a night shift (11 pm - 7am) and had three children, two age 15 and over and 1 under age 15. The mother reported that she used categories 3 (child's brother or sister age 15 or over) and 14 (child cares for self). However, she reported that these arrangements were only a few hours per week while she was running errands. She did not report any child care for the time she was at work. Most likely the children are asleep and she didn't consider the children ages 15 or over as caring for the younger child during this time.

The FR debriefings and the taped interviews indicated that category 2 (designated parent cared for child while (he/she) was (working, attending job training, attending school, looking for work) was problematic. FR's and respondents didn't know who the designated parent was. As in the case of the substitute teacher, she thought the designated parent was her husband, since he was designated to pick up their children from school if she was working. FR's reported similar problems during the debriefing. From the tapes, it is clear that some FR's didn't know who the designated parent is either.

Final decision: [Fill definition of "regular" for first child through series only.]

The next few questions are about child care. (In addition to school,) Please tell me which of these, if any, you used for (name) on a regular basis BETWEEN JANUARY 1997 AND APRIL 1998. (By "regular," I mean at least once a week for a month or more.)

<1> Child's other parent/stepparent cared for child while

(fill name of designated parent) was (fill)

<2> (Fill name of designated parent) cared for child while (she/he)

was (fill)

<3> Child's brother or sister

<4> Child's grandparent

<5> Any other relative

<6> Family day care (caring for 2+ kids in provider's home)

<7> A non-relative such as a friend, neighbor, sitter or nanny

<8> Child care/day care center or nursery school/preschool

<9> Federally-funded Head Start program

<10> Kindergarten or school (grades 1-12)

<11> Before or after school care

<12> Child cares for (himself/herself)

<13> Any other type of arrangement (specify)

__________

<14> No regular arrangement used

Instrument problem: One taped interview indicated that the fill for "In addition to school" didn't come up.

Training issues: FR's need better training regarding the universe of this question, that is, we are collecting child care arrangements for both working and non-working mothers/designated parents. Additionally, we are only collecting arrangements used on a regular basis, not ones used only once in a while or on an ad hoc basis. It is possible that both working and non-working mothers/designated parents won't use any regular child care arrangement. If respondents report using an arrangement and then in subsequent questions indicate that they don't use the arrangement regularly, the FR should back up to this screen and delete the arrangement. Otherwise the respondent will be asked a series of questions that he/she can't answer and don't make sense.

FR's need to know who the designated parent is and how it is defined. They also need a better understanding of response categories <1> and <2> and the types of situations that apply to each category.

FR's reported that they didn't know the difference between some of the response categories such as family day care home, nursery school, preschool and child care or day care center. They requested that the terms be defined in the glossary of the manual and covered during training. POP division indicated that the terms are to be respondent defined and that they are ambiguous. (Note that some of these terms were combined into one category, which should alleviate some confusion.)

b. Item 1305.

Original wording: Of those (fill with hours in 1304A-D), how many of them were while (you/designated parent) (were/was) at (work/school/work training/looking for work)?

Problem identified: Taped interviews indicated that one respondent who worked full time but had reported that she had asked a friend/relative about a job during September 1997 thought this question was asking whether she was at work or looking for work during those hours. She responded, "I was working."

FR debriefings indicated that respondents didn't know whether they were to include travel time to and from work as time spent while working. POP has suggested adding a note to the instrument screen to include this time.

Final decision: Use original wording. Include note to FR on screen that travel time to and from work should be included.

c. Item 1307.

Original wording: How much do you pay for this (arrangement/program)?

Problem identified: Taped interviews indicated that one respondent was confused regarding the reference period for this question and asked, "Since 1997, or before?" The amount she paid changed once the child entered school in September 1997. Prior to that, she had used the arrangement full time and was now only using it after school. The FR instructed her to give the amount during 1996.

Final decision: How much did you pay for this (arrangement/program) for April 1998?

d. Item 1312A.

Original wording: Last year, did (you/designated parent) lose any time from work because (your/his/her) usual child care provider was unavailable to care for (child)? This DOES NOT INCLUDE times when (child) was sick and couldn't be cared for by the usual child care provider.

Instrument problem: This question was never asked in any of the taped interviews. A problem in the instrument caused this question and 1312B, 1313A, and 1313B to be skipped.

Final decision: The instrument will be fixes and these questions included in the 1998 SPD instrument.

e. Item 1315.

Original wording: How many times SINCE JANUARY 1996 has (child) changed from one child care provider to another?

THIS INCLUDES: Changes in the person who cared for (child)

Changes in the program (he/she) attended

Changes in the place where (he/she) was cared for

Changes in the number of different child care providers for (child)

Problem identified: One taped interview indicated that the child was always cared for by the grandmother. During the school year, he was cared for at the grandmother's house. During the summer, he was cared for at his own house. The respondent reported one change in child care providers because of the statement to include "changes in the place where child was cared for." Should this be counted as a change? From previous cognitive testing, respondents had a clear understanding regarding what a change in child care providers was. However, it doesn't match the list of "inclusions" in this question. We suggest deleting the inclusion statement.

Final decision: Keep original wording but delete the "THIS INCLUDES:" statements. Put those statements in a help screen. Change the last category to "Adding different child care provider for (child)"

17. Child Support

a. Item 1401C.

Original wording: Is there any kind of legal arrangement that says that (name's) (father/mother) should provide any kind of financial support for (him/her)?

Problem identified: In one taped interview, the respondent said the absent parent was supposed to contribute to the mortgage payment. She didn't know if this counted as financial support for the child. We suggest addressing this issue in training and the FR manual.

Final decision: Address this issue in FR training and the FR manual. Also, develop a help screen.

Training issues See issue included under "problem identified" above.

b. Item 1402.

Original wording: Has there ever been any other kind of agreement or understanding that says that (name's) (father/mother) should help support (him/her)?

Problem identified: In one taped interview, the respondent said in response to this question that if her son lives with her, then the absent parent is supposed to make payments. If her son lives away from her, then the father is not supposed to make payments. We suggest addressing this issue in training and in the FR manual.

Final decision: Address this issue in FR training and the FR manual.

Training issues: See issue included under "problem identified" above. In addition, FRs need better training regarding the two questions on agreement included in the child support section. Item 1401C asks if there has ever been a legal agreement. Item 1402 is about informal agreements and is asked only if the person does not have a legal agreement. The distinction between these two types of agreement -- formal and informal -- needs to be made clear for the FRs so that they can better navigate this series of questions.

c. Item 1411B.

Original wording: Which months were payments supposed to be made?

Suggested revision: (Was/Were) (you/name) supposed to receive payments every month during 1997 or for only some months?

<1> Every month in 1997

<2> Only some months (ASK WHICH MONTHS)

Which months were you supposed to receive payments in 1997?

<1> Jan <7> July

<2> Feb <8> Aug

<3> Mar <9> Sep

<4> Apr <10> Oct

<5> May <11> Nov

<6> Jun <12> Dec

[blind] <D,R>

Final decision: Suggested wording accepted.

d. Item 1412A.

Original wording: How much was (the weekly/every other week's/the twice monthly/the monthly/the quarterly/the yearly) payment supposed to be during 1996?

Problem identified: In one taped interview, the respondent received payments twice a month, but she received two different amounts. The instrument would only accept one amount. The FR entered the higher of the two amounts reported. When she confirmed the annual amount in item 1412B, the respondent said it was too much. The FR then backed up and chose "monthly" in item 1411A and added the two twice weekly figures together. The instrument will only allow one amount to be entered in item 1412A. The FR correctly addressed this problem by backing up to item 1411A and changing the entry to "monthly." We suggest addressing this issue in training and the FR manual.

Final decision: Address this issue in FR training and the FR manual.

Training issues: See issue included under "problem identified" above.

e. Item 1414.

Original wording: During 1996, were the payments sent to you by the welfare or child support agency, by a court, or did the payments come directly from (child's name with "'s") (father/mother) or (his/her) place of employment?

MARK ALL THAT APPLY

<1> Welfare or child support agency

<2> Court

<3> Directly from other parent or through (his/her) place

of employment

<4> Other (specify)

<D,R>

Suggested revision: During 1997, were the payments sent to you by the welfare or child support agency, by a court, directly from (child's name with "'s") (father/mother), from (his/her) place of employment, or were they sent some other way?

MARK ALL THAT APPLY

<1> Welfare or child support agency

<2> Court

<3> Directly from other parent

<4> Other parent's place of employment

<5> Other (specify)

Justification: In one taped interview, the respondent answered "no" to all the categories in the question and then explained that the child's father hand delivered the payments. The FR chose the "other" category and typed in the information.

Final decision: Suggested wording and revision of response options were accepted. Additionally, it was noted that FR training and the FR manual need to be improved in this area.

Training issue: FR training and the FR manual should be enhanced so FRs understand how to handle responses such as "It comes from the State of Iowa" or the child support recovery unit from the state. FRs need to know if HHES wants such responses marked as "welfare or child support agency" or "other". If there are specific criteria for which FRs need to probe to make a determination of the correct response category, HHES needs to provide DSD with those criteria.

f. Item 1415B.

Original wording: Why is there no legal agreement to help support (name)?

[MARK ALL THAT APPLY]

<1> Legal paternity not established

<2> Unable to locate parent

<3> Do not want child support

<4> Did not pursue agreement

<5> Other (specify)

<D,R>

Problem identified: In one taped interview, the respondent says, "He loves him (his son) so much he doesn't need a legal agreement." FR marked "5" and typed in the response. Perhaps we should include a flashcard for this item so that respondents will know what type of answers we are looking for.

Final decision: Suggestion to add a flashcard for this item was accepted.

g. Item 1415B2.

Original wording: (Have/Has) (you/name) ever asked a public agency such as the child support enforcement office or welfare agency for help in obtaining child support under this (legal agreement/agreement)?

Instrument problem : The intended universe for this question is all children with an outside parent including those both with and without legal agreements (and items 1415C and 1415D, as appropriate). Taped interviews indicated that there is an instrument problem and that the question was not asked, but should have been, for two respondents. In both cases, the respondents reported having agreements but didn't receive any payments. (After answering 1415B, they were not asked this item, but instead went directly to the questions on custody (item 1415C.)

Final decision: Instrument problem is being corrected.

h. Item 1415E.

Original wording: What child custody arrangements does this legal agreement specify?

<1> Joint legal and physical custody

<2> Joint legal with mother physical custody

<3> Joint legal with father physical custody

<4> Mother legal and physical custody

<5> Father legal and physical custody

<6> Split custody

<7> Other (specify)

Suggested revision: Keep wording the same. Add a flashcard.

Justification: Taped interviews indicated that FRs read categories to respondents. A flashcard would eliminate the need to read the categories.

Final decision: Suggestion to add a flashcard was accepted.

Problem identified: Taped interviews indicate that there is a skip problem in the instrument. Respondents with legal agreements should be asked item 1415E Respondents with informal agreements should be asked 1415D (What child custody arrangements does this agreement specify?). In two cases, respondents were asked 1415D even though they had legal agreements.

Final decision: Instrument will be examined by DSD to determine where the problem exists.

  1. Contact with Absent Parent
a. Item 1417A.

Original wording: In a TYPICAL MONTH, about how many times does (name) talk to (his/her) (mother/father) on the phone?

Suggested revision: FLASHCARD X

How often does (name) talk to (his/her) (father/mother) on the phone?

<1> Never

<2> Once or twice a year

<3> Several times a year, but less than once a month

<4> Once or twice a month

<5> Once a week

<6> Several times a week

<7> Everyday or almost everyday

Note: This suggestion also applies to items 1418A, 1419A, 1421A. If this suggestion is accepted, we would delete all the items referring to the past 12 months (1417B, 1418B, 1419B, 1421B).

Justification: Taped interviews and cognitive interviews conducted previously indicated that respondents and FR's had difficulty with the concepts of a "typical month" in this item and the others listed under the "note" above. Respondents gave responses that couldn't be coded and FR's did not do a good job probing to get adequate answers. Particularly when an event such as getting a card or letter (item 1418A) occurs infrequently, respondents report how often it occurs (e.g. birthdays and some holidays), but are unable to translate this into a "typical month" amount. We suggest using the same series of questions included in the adolescent questionnaire. These items include categorical response categories as shown above and cover a 12-month period. This eliminates the need for the 12-month questions.

Final decision: Use suggested wording for this item, 1418A, 1419A, and 1421A. Delete all the items referring to the past 12 months (1417B, 1418B, 1419B, 1421B).

b. Item 1418A.

Original wording: In a TYPICAL MONTH, about how many times does (name) get a letter or card from (his/her) (father/mother)?

Suggested revision: In a TYPICAL MONTH, how often does (name) get a letter, card, or e-mail from (his/her) (father/mother)?

Justification: CSMR suggests adding "e-mail" to the list since this form of communication is increasingly common. This suggestion was not based on the pretest evaluation methodologies.

Final decision: Use suggested wording.

c. Item 1420.

Original wording: In a TYPICAL MONTH, about how many hours per week does (name's) (mother/father) usually spend with (him/her)?

Suggested revision: Delete question.

Justification: POP division had agreed to delete this question previously, but the pretest instrument was never changed to reflect this.

Final decision: Delete question.

19. Marital Relationship and Conflict

Adult Depression

At the end of the interview there were two self-administered sections of the instrument; one on marital conflict and one on depression. FRs were instructed to turn the laptop towards the respondent, asking the respondent to enter the appropriate answer. Self-administration of these questions was determined by the potential sensitivity of the topics. Since the questions were self-administered there is not much information from which to determine whether the questions were understood as intended.

While many FRs did comment that they felt awkward or embarassed with these topic areas, they did not indicate that the topics or the self-administration posed any significant obtacles to collection of these data. There were isolated cases where respondents had literacy, language, or visual problems which resulted in FRs needing to read the questions to the respondents. In some instances respondents were uncomfortable using the computer and preferred for the FRs to administer the questions. However, these were the exceptions. Some FRs reported that the respondents liked being able to use the laptop. During the debriefings, several FRs mentioned that they had administered these questions on other surveys, such as the HIS and didn't encounter problems with respondent sensitivity to the topics. The other FRs who had not previously administered such questions seemed relieved to hear about those experiences.

Suggested revision: FRs suggested that it would be helpful to have the questions printed elsewhere in English and Spanish so that FRs can read the question for respondents when respondents have difficulty, yet the respondent could still enter the numbered responses without the FR seeing the answer. We recommend that this suggestion be adopted for the 1998 SPD.

There are no recommended revisons to the question wording or procedures for these questions.

B. Adolescent Questionnaire

The SPD self-administered adolescent questionnaire (SAQ) included questions on family routines, housework and chores, relationship with parents, parental monitoring, contact with nonresidential parent, minor problem behaviors, substance use, knowledge of and attitudes towards welfare programs, marriage and childbearing, sexual initiation and contraception. (Adolescents 12-13 years of age were not asked the sexual initiation and contraception questions.) Prior to the pretest, cognitive interviews were conducted using the SAQ to determine if there were problems with respondent comprehension, task difficulty and/or topic sensitivity. Revisions were made to the SAQ based on the results of that research. The revised SAQ was the instrument used for the pretest.

As previously mentioned, this component of the SPD was administered using an audiocassette personal recorder and an answer booklet. (For households in which adolescents were not available at the time of the core SPD interview, FRs administered the survey by telephone.) For the pretest, two different audiotapes were produced. On the first tape, all response categories for all questions were read. In the second tape, all response categories were read the first time a response set was used, but not repeated for all consecutive questions using the same response options. The first version was used by FRs from the Boston and Kansas City ROs and the second version was used by FRs from the Atlanta and Los Angeles ROs.

In order to obtain data on respondent's reactions to the speed of the tape, reading of response categories on the tape, sensitivity to privacy issues, etc. respondent debriefing questions were added to the end of the answer booklets. (These questions were only completed by those adolescents completing the survey using the audiocassette recorders.) Additional respondent debriefing questions were included for all respondents (regardless of mode of interview) to obtain information on their interest in the survey, their level of comfort answering questions on the various topics, and their ability to concentrate for the 30 minutes interview. Information from the FR debriefings also served as input to evaluating the SAQ. Child Trends, Inc. prepared a report analyzing the SAQ pretest data and the executive summary is provided in Attachment C. The information that follows is mostly extracted from that report.

Sixty-six adolescent cases were transmitted to headquarters. Sixty of the 66 cases were completed interviews. Of the remaining 6 cases, 3 were noninterviews due to parental refusals, 1 was due to an adolescent refusal and 2 were noninterviews because the adolescents were disabled and unable to perform the task. Approximately 57 percent of the cases were self-administered and 43 percent were FR-administered by telephone.

Information obtained from the FR debriefings indicate that the procedures for the SAQ worked much better than expected. There was very little reluctance on the part of parents or adolescents. Adolescents rarely, if ever, needed to consult with the FRs regarding the task or about terms or concepts used in the survey. FRs reported, and the analysis by Child Trends confirmed, that the questions on knowledge and attitudes towards welfare reform were difficult for adolescents. According to the analysis conducted by Child Trends, the majority of adolescents (60-80 percent) reported "don't know" to the welfare knowledge questions.

Responses to the debriefing questions completed only by those respondents using the audiocassette players, indicated that the pace of the tape was fine and adolescents had enough time to mark the answer boxes. Some respondents (40 percent) indicated that they would be "extremely" or "very" concerned about their privacy, if the questions were written in the answer booklet. Preference for the reading of response categories was mixed.

Responses to the additional debriefing questions regarding interest in the interview, comfort level with topic areas and ability concentrating did not indicate that any substantial problems exist. Level of interest was basically evenly distributed across response options ranging from "very interesting" to "not at all interesting." Adolescents did not tend to be uncomfortable answering questions on the topics areas presented. Fewer than 9 percent indicated they were very uncomfortable with any topic. Child Trends found that even if an adolescent felt uncomfortable with a specific topic area, they were NOT more likely to leave those questions blank.

Recommendations for revisions to the SAQ can be found in the Child Trends' report (Attachment C.)

Training Issues: We suggest that FR training be enhanced to provide a brief history of other adolescent surveys conducted by the Census Bureau in the past, such as the Youth Risk Behavior Survey (YBS) and the Teenage Attitudes and Practices Survey (TAPS). The YBS, in particular contained content similar to that in the SPD and used the same mode of administration. The YBS was successfully implemented. Summary reports of that survey and the associated FR debriefing have been provided to FLD as a resource for training information.

C. Respondent Flashcards

1. General

During the debriefing sessions, FRs were asked which flashcards they thought were useful and which were not. Opinions varied considerably. Provided below are suggestions identifiying flashcards that can be deleted, that need to be modified, and that are recommended for adding to the 1998 SPD.

Interviewing observation reports indicated that frequently FRs did not use the flashcards. When flashcards were used, the respondent was the person handling the flashcard booklet the majority of the time. During review of taped interviews, some of the problematic FR/respondent interactions occurred when FRs did not use the flashcard and respondents were unclear what type of answer was being sought. We recommend that the benefits of flashcards be emphasized in training. One way to accomplish this is by using flashcards during read-through interviews and practice interviews. Some of the pretest interviews that were taped could be utilized for development of training. Examples from the tapes demonstrating the confusion encountered when flashcards are not used may help FRs understand the benefits of using flashcards.

2. Flashcards that can be deleted

We recommend deleting several of the flashcards that FRs thought were unnecessary (Cards J, O, U, V and the Labor Force Activity Worksheet). We also recommend deleting cards if the response options were contained within the question (i.e. Cards H, I, U, and V). Additionally, we recommend deleting Card R and instead using the response option from Card T. Flashcards that can be deleted include the following:

1. Card H: Unemployment compensation

2. Card I: Workers' compensation

3. Card J: Veteran's payments

4. Card O: Reporting unemployment compensation payments

5. Card R: Health insurance

6. Card U: Frequency of activity -- weekly

7. Card V: Frequency of activity -- monthly

8. Labor Force Activity Worksheet

3. Flashcards requiring modification

Based on item-level questionnaire revisions agreed to for the 1998 SPD, we have recommended that the following flashcards (S,T,Y) be modified to be consistent with changes in response categories. The reason for the modification is contained within the body of the report.

Card S: Reasons Not Covered By Health Insurance

1 Too expensive; can't afford health insurance.

2 No health insurance offered by employer of self, spouse, or parent

3 Not working at a job long enough to qualify

4 Job layoff, job loss, or any reason related to unemployment

5 Not eligible because working part time or temporary job

6 Can't obtain insurance because of poor health, illness, age, or pre-existing condition

7 Dissatisfied with previous insurance OR don't believe in insurance

8 Have been healthy; not much sickness in family; haven't needed health insurance

9 Able to go to VA or military hospital for medical care

10 Covered by some other health plan, such as Medicaid

11 No longer covered by parents' policy

12 Other

Card T: Current Health Insurance

1 Employer/union provided (policy holder)

2 Employer union provided (dependent)

3 Purchased directly plan--not employer/union, e.g. "Medi-gap" (policy holder)

4 Purchased directly plan--not employer/union (dependent)

5 Medicare, or other health plan paid for by Medicare

6 Medicaid (or state name)

7 CHAMPUS/TRICARE/CHAMPVA

8 Military Health

9 Indian Health Service

10 Other government health insurance plan (or state name)

11 Covered by someone outside this household

12 Not currently covered by health insurance

Card Y: Child Care Arrangements

Child's other parent/stepparent

Designated parent/guardian cared for child while working, attending school or job training, or looking for work

Child's brother or sister

Child's grandparent

Any other relative

Family day care (caring for 2+ kids in provider's home)

A non-relative such as a friend, neighbor, sitter or nanny

Child care/day care center or nursery school/preschool

Federally-funded Head Start program

Before or after school care

Child cares for (himself/herself)

Any other type of arrangement

No regular arrangement used

Note: We have not included numbers for the response options because four categories on this card were collapsed into two categories and one category was added. We do not know what number these categories will be assigned in the instrument.

4. Additional flashcards recommended for 1998 SPD

Based on review of taped interviews, interviewing observation reports, and subsequent revisions agreed to for the questionnaire, we recommend including some new flashcards for the 1998 SPD. The reason for adding these flashcards is contained in the body of the report.

1. 12-month calendar with week numbers beside each week (similar to SIPP calendars)

2. Item 18. Main Reason Did Not Work Last Year

1 Retired

2 Taking care of home or family

3 Going to school

4 Ill or disabled

5 Could not find work/no work available

6 Did not want to work

7 On layoff

8 Never worked

9 Other

3. Items 270, 271: Other Sources of Income

1 Mutual funds or shares of stock

2 Property that was rented to others

3 Income from estates or trusts

4 Income from royalties

4. Item 272, 273: Other Sources of Income

1 Supplemental Security Income (SSI)

2 AFDC, ADC, or TANF

3 WIC

4 General assistance

5 Free or reduced price lunches at school through the Federal School Lunch or Breakfast Programs

6 Food Stamps

7 Energy assistance

8 Income assistance from a charitable group

9 Other welfare

10 Financial assistance on a regular basis from

friends or relatives not living here

5. Item 1000: Food Eaten in Household

1 We have enough to eat and the kinds of food we want

2 We have enough to eat but not always the kinds of food (I/we) want

3 Sometimes we don't have enough to eat

4 Often we don't have enough to eat

6. Item 1415b: Reason No Legal Agreement for Child

1 Legal paternity not established

2 Unable to locate parent

3 Do not want child support

4 Did not pursue agreement

5 Other (specify)

. 7. Item 1415e: Legal Custody Arrangement

1 Joint legal and physical custody

2 Joint legal with mother physical custody

3 Joint legal with father physical custody

4 Mother legal and physical custody

5 Father legal and physical custody

6 Split custody

7 Other

8. Items 1417a, 1418A, 1419A, 1421A:

Frequency of Contact with Absent Parent

1 Never

2 Once or twice a year

3 Several times a year, but less than once a month

4 Once or twice a month

5 Once a week

6 Several times a week

7 Everyday or almost everyday

D. Timer Data

Prior to the pretest there were concerns that the core SPD may exceed the targeted 60 minute household average. In Winter 1996, modules were identified that could be administered biennually, instead of annually, if it was determined during the pretest that some modules needed to be deleted, due to respondent burden constraints. Fortunately, the pretest data indicate that no such deletions are necessary for 1998. The average core pretest interview took only 55.5 minutes, once outliers were removed. The average with outliers included was 62.19 minutes. Therefore, no topical modules are being deleted for 1998.

Questions regarding this report should be referred to Jenny Hess on 457-4968 or through email.

Attachment A

SPD PRETEST

SUMMARY REPORT OF INTERVIEWING OBSERVATION REPORTS

Prepared by: Jennifer M. Rothgeb

Center for Survey Methods Research /SRD

November 14, 1997

November 14, 1997

SUMMARY OF INTERVIEWING OBSERVATION REPORTS OF SPD PRETEST

I. General

Field pretesting of the Survey of Program Dynamics (SPD) was conducted between Oct. 6-22, 1997. Reports were obtained from 8 headquarters' staff in DSD, POP, HHES, DSMD, and CSMR that observed pretest interviewing. A total of 25 interviews were observed: 7 in the Boston area, 9 in the Los Angeles area, 7 in rural areas of Minnesota, Missouri, and Iowa, and 2 in the Miami area. Each observer was requested to complete an "Interviewing Observation Form" (See attachment A.) (Note: There is missing data for some observations, therefore counts of responses to all items may not sum to the total universe for that item.)

Observers noted that 21 of the 24 respondents were not reluctant to be interviewed. The length of observed interviews ranged from 21 minutes to 95 minutes with a mean of 53 minutes. Twenty one of 25 households contained only 1 or 2 persons age 15 years of age or older. Only 5 of the observed households contained children: those households had on average 2.6 children. Only one household had an adolescent available for interview. No difficulties with procedures were reported.

The majority (17 of 25) of observed interviews were ones in which the FR requested that the interview be taped. Cooperation for taping was high with 15 respondents consenting to taping.

II. Labor Force Activity Worksheet

A Labor Force Activity Worksheet (Attachment B) was developed for use in the employment section of the questionnaire. During the observed interviews only 6 of 24 respondents used the worksheet. Explanations for not using the worksheet included reasons such as: working all year at same job, retired, not working during the calendar year, and sporadically working. Some respondents were reported to have used the worksheet as a reference calendar.

Of the 6 respondents reported to have used the Worksheet for their own labor force activity, 4 had no difficulty using it. One person had a "little difficulty," and one found it "somewhat difficult" to use. (One reason given for difficulty was that the respondents used the same worksheet for all persons in the household.) Of the respondents reported to have used the worksheet, it was observed that it seem to help the respondent recall his/her own labor force activity for the previous calendar year in 4 of 5 interviews.

Observers noted that there were 4 cases when respondents used the worksheet for other household members. In 3 of the cases it was reported that it was "not at all difficult" for respondents to use it for other household members. The 1 case for which it was "a little difficult" was due to the sporadic babysitting work done by their adolescent during 1996. The respondent did not know in which weeks the work occurred, so the respondent guessed. It was reported that

the worksheet helped respondents recall other household members' labor force activity in 3 cases. Observers also commented that the worksheet does not have any place to put the weeks during which someone might have been retired.

III. Record Usage

Observers reported that 11 out of 24 respondents used some type of records to report earnings or income amounts. The types of records used included checkbooks, tax returns, bank statements, W2 forms, Social Security payments, dividends, insurance premiums, and mortgage records. One respondent was even reported to have used the "form for the 1996 March CPS survey."

IV. Flash Card Usage

As noted by observers, flash cards were not used at all for 7 of 25 interviews and used "almost none of the time for 3 interviews. They were used "some of the time" in 5 interviews, "most of the time" in 5 interviews and "all the time" in 5 interviews. Observers noted that the flash cards seemed to help the respondent remember sources of income they may have otherwise forgotten in 5 of 16 interviews. During the interview, it was observed that the respondents held the flash card booklet 11 of 16 times. In the remainder of cases, the booklet was held by a combination of the FR and the respondent.

V. Shift in question types/reference periods

In only 2 of 20 observer reports was it noted that shifting between person-level questions and household-level questions seem to cause confusion. Areas in which confusion did occur include the amounts section, educational enrollment and health insurance sections. Shifting reference periods within topic area seemed to cause confusion for 7 out of 24 repondents. Sections reported for which the reference period were reported to be problematic include health care utilization, health insurance, and employment.

VI. Misunderstood Terms

Observers indicated that in 8 of 22 interviews there were some terms not understood by respondents, including:

  • energy assistance;
  • distinguishing between doctor's visits and rehabilitation visits;
  • "covered" by insurance;
  • inability to do work because of a disability was confusing for elderly disabled retired persons;
  • difference between interest and dividends; and,
  • trusts and estates.

  • VII. Items for which Respondents didn't know the answer or refused to answer

    Observer reports indicate that there were 13 of 22 interviews in which there were questions to which the respondent did not know the answer or refused to answer. Of all the items mentioned, "income amounts" was the most frequently reported series which posed problems in this area. Other problem areas in which resondents did not know the answer or refused to answer are provided below.

    • Respondents were unable or unwilling to provide income amounts;
    • Proxy respondent did not know other persons' work schedule or financial information;
    • Proxy respondent had difficulty distinguishing between social security income and supplemental security income;
    • Respondent and FR were unsure how to report irregular financial help from family members (Respondent also wasn't sure if he/she wanted to report it.)
    • Proxy respondent didn't know weeks worked in 1996 for casual type employment (e.g., babysitting)
    • FRs and respondents did not know how to answer Item 3 ( "work last week question) for atypical work situations: pay-in-kind work; working on commissions;
    • Proxy respondents did not know the answer for the question about the number of employees.
    • Respondent was unwilling to provide answer to "debt" question. Other respondent and FR did not know if the debt question included "IRS debt."
    • Respondent and FR had some difficulty providing/entering response to "class of worker" question. Respondent was an independent contractor and it was not clear to respondent or FR which response option was appropriate.
    • Respondent did not know the amount of energy assistance she received.
    VIII. Items for which respondents didn't understand or for which FR had to probe extensively.

    In 7 of 23 observer reports, it was noted that there were some questions that respondents did not understand or that FRs had to probe extensively. These items included:

    • Energy Assistance - need definition of program;
    • Respondent did not understand the question concerning the place people usually go for health care. The respondent reported "Medicare ;
    • Question on Social Security vs Supplemental Security Income - distinction not clear to all respondents;
    • Health insurance series required probing in some cases;
    • Food security series required probing in some cases.
    • Respondent had difficulty understanding food security items. (This was partly due to the FR not reading the items correctly.)
    IX. Transitional Statements

    None of the observers reported that there were any places where additional transitional statements are needed. There were a couple of comments regarding transitional statements being too lengthy and wordy, but the comments were not specific enough to identify the specific statement.

    X. Unnecessary Questions

    Nine of 23 observers reports indicate there were questions that were lengthy or unnecessary. Primarily, these comments pertained to questions that were unnecessary for specific subgroups of the population. Such questions or questions series included:

    • The employment series being extremely burdensome and unnecessary for elderly and retired persons. (Many, many comments on this issue.);
    • Asking program participant and food security questions in upper-income households;
    • Asking assets questions (mutual funds, royalties, estates, etc.) of impoverished households; and,
    • Asking never divorced persons with no kids the alimony and child support questions.
    • Items EVWID, AFEVER, and EDUC were unnecessary given that information had already been verified in earlier questions.
    XI. Miscellaneous Comments.

    Observers provided several miscellaneous comments. Some of these comments have been incorporated in the above sections. Others comments included questions FRs had or observers had regarding specific situations which need to be addressed through possible questionnaire revisions, incorporation into training, or inclusion in the FR manual. Provided below are the specific areas of concern.

    • Should last year's tax refund be reported in questions in the income series?
    • Should ambulance expenses be counted in with hospital bills?
    • In terms of the "30 days living away," what should be done when the respondent has a summer home?
    • Observer commented that the health care utilization section might be revised so the ordering of the questions regarding going to a doctor, going to a psychiatrist, and number of visits to the doctor is less confusing. A respondent thought the question concerning the number of visits to a doctor referred to visits to a psychiatrist and responded "zero" when in fact she had been to the doctor's six times.
    • FRs commented that they have difficulty finding the country code for the U.S.
    • How should interest income from mutual funds be reported when the interest is automatically reinvested in the mutual funds?
    • FRS need additional training on how to handle atypical work situations such as casual babysitting, work for pay-in-kind, work paid through commissions, etc.
    • Should basic telephone service or total telephone bill be included in response to item 618? Also, should cable service (television) be included in utilities paid that are reported in item 618? It was also suggested that this item be split so the amounts of individual utilities are requested and then summed (internally) for a total amount.
    • In item PAYDREXP, should insurance premium payments be included?
    • In item CURCOV, the respondent heard "Medicare/Medicaid" at the end of the question and reported "no", when his situation fit the earlier part of the question and the response should have been "yes."
    • For persons who are leasing a vehicle, there is not an appropriate category in item 635.
    • Should IRS debt be included in the amount reported in item 645?
    • Item MODIR@B1 - It was suggested by an FR that wording be changed to reflect the specific type of insurance the respondent is reporting on, instead of reading as ..."this type of insurance."
    • A person initially responded to having received VA payments, but then changed response to survivor's benefits upon seeing a different flash card. Her deceased husband was retired military and she wasn't absolutely sure what type of benefit she was receiving.
    • Item W3ABEG1 and screen preceding it - No one in the household was away for 30 days or more, but the question asking which months a person was away came up and the FR couldn't get pass the screen without entering false data. Observer noted that the problem was probably with preceding screen. The FR may have entered "N" instead of "0".
    • When a person has only 1 employer in previous year, there is no need to reask how many weeks the person worked for that employer. That information is already provided earlier.
    • Item 464 should allow a monthly amount to be entered.
    • FR wants ability to bring up total household roster at very beginning of case so all sample persons are known before the FR goes to the door. FR wants more training on function keys.
    • FR wants more training on taping equipment.
    • FR commented on need for consistency in instrument for entries of "No more".
    • Item TYPEUCR@0 -- A specify line appeared but didn't allow an entry. Probably shouldn't come up for entry of "9".
    • A respondent who is on an SSI check with someone not in the household and takes the other person shopping reported the total amount of the SSI check but not the amount which only the respondent gets.
    • FR reported preferring the CPS wording to the question asking how an adult male in the household is related to a child in the household. FR felt tension when she asked the SPD question "Who is the biological father of Brandon?"
    The information described above, obtained through the Inteviewing Observation Reports, will be used along with information from other sources (FR debriefings, behavior coding of taped interviews) to determine what questionnaire revisions will be recommended. The above information should also be useful for persons developing FR training, FR manuals, and instrument "Help" screens for the 1998 SPD. If there are any questions regarding the content of this report, please call (457-4968) or email (jrothgeb@census.gov) Jennifer Rothgeb.

    Attachment B

    SPD PRETEST

    FIELD REPRESENTATIVE DEBRIEFING SUMMARY

    Prepared by Jennifer Hess

    Center for Survey Methods Research/SRD

    November 13, 1997

    Background

    Following is a summary of the five debriefing sessions with Field Representatives (FR's) from the Boston, Kansas City, Los Angeles, and Atlanta Regional Offices. The sessions lasted approximately six hours. CSMR/DSD staff facilitated the first half of the debriefing regarding questionnaire design issues and Field Division staff facilitated the second half regarding field procedures, training, manuals, instrument layout, function keys, and other field-related issues. Nearly all FR's that conducted interviews during the SPD pretest participated in a debriefing session; a total of 38 FR's were at the debriefings. With the exception of the Atlanta RO, no SPD supervisors were present at the debriefings.

    This summary includes only the CSMR/DSD portion of the debriefing and is organized according to the protocol that was used to conduct the session (Attachment A includes the protocol). General comments are provided for each section followed by item specific comments. The item numbers referred to in the specific comments are from the August 25, 1997 SPD Draft Questionnaire. Question wording for specific items follows the item number. When comments were made regarding a series of questions, only the wording for the first item is shown. The content of the other items is given in parentheses.

    A. Question Wording and Sequencing

    1. Employment and Earnings
    General Comments

    The questions in the employment section were reported to be burdensome for elderly retired respondents. FR's suggested incorporating dependent interviewing for retired status (as is done in CPS), or trying to get the elderly retired people out of this series sooner by allowing them to mark "retired" earlier in the employment series.

    The Labor Force Activity Worksheet was not particularly helpful in situations where the person worked for one employer throughout the year. In fact, such persons often left the Worksheet blank. Some FR's found the Worksheet useful in more complicated work situations in which the respondent had several different jobs that started and stopped throughout the year. FR's generally thought that the Worksheet looked too complicated, needed to be simplified, and couldn't be used for phone administration. They also thought the explanation provided to the respondent of how to use the Worksheet was verbose. FR's indicated that some respondents used the Worksheet to answer other questions in the child care and school enrollment sections. In general, they thought a calendar with the weeks marked on the side would be more helpful.

    There should be consistency in the instrument for the manner in which information is entered.

    Item Specific Comments Item 3 "Last week, did you do any work for pay?"

    Should this question be revised to the CPS wording of "for pay or profit?"

    Item 18 "What was the MAIN reason (name/you) did not work in 1996?"

    FR's stated that for many elderly folks that may not be retired, and for younger persons who have not yet entered the labor force, there was no appropriate response option provided (for MAIN reason they did not work in 1996). Although "other" can be used, FR's thought a separate category should be provided, particularly for those who never were in the labor market.

    E-Review List of employers

    FR's mentioned that this item is meaningless as currently worded.

    Items 19, 32 "Please mark on the worksheet the weeks during 1996 that you did any work at all, even for only a few hours."

    For respondents who worked intermittently throughout the year, such as substitute teachers, there were not enough entry spaces. FR's also mentioned that the screens are too cluttered. Some FR's would prefer to enter dates in these items because their respondents know the date they began a job.

    FR's were not sure if teenagers doing casual work (babysitting, mowing lawns, etc.) throughout the year should be reported in item 19. In terms of reporting weeks worked, they thought it would be much too difficult for such workers to remember the exact weeks they worked.

    Item 29 "How many employers did (name/you) work for in 1996?"

    One respondent was a union hall contractor who worked for two-week periods for a variety of different employers throughout the year. Neither the respondent nor the FR considered this person as being self-employed. The respondent answered the maximum four employers for the series and did not provide information for the dozen other jobs the respondent worked in 1996.

    Another respondent was self employed with two different businesses. He reported only one employer in item 29 (self). Subsequent questions on the name of his employer/business, major activities/duties, earnings, etc. were difficult for him to answer since he kept records separately for the two businesses.

    Item 30 "How many hours did (name/you) USUALLY work per week in 1996?"

    One respondent was unclear whether to report the hours usually worked or the hours the person was paid for.

    Item 32 "The following questions refer to your main job, that is , the job at which you worked the most hours in 1996. Please mark on the worksheet next to the line labeled Employer 1 the weeks in 1996 that you worked for this employer."

    Farmers have difficulty with the concept of "main job." Farmers often have one job that is stable income plus their farming income. The CPS definition of main job is the one at which you work the most hours. In many cases, farmers spend more hours farming but consider their other job to be their main job because it provides year-round income.

    Items 41, 43, 43A "What was the name of the (company/non-profit organization/government agency) for which (name/you) worked?" (name of business; address)

    One FR had several sample household members who worked for the same company and requested that there be some way to bring up the employer information for the previous person in the household so they don't have to retype it. FR's also thought it would be helpful to use dependent interviewing in the industry and occupation questions including the employer's name and address.

    Item 43A "What was the address?"

    In all sessions comments were made by FR's questioning the need for the employer's address. The FR's report that most respondents could not provide the street address or zip code. The did not have difficulty in providing city and state. It was suggested that an inquiry be made to the sponsoring division to determine if city (town) and state would be sufficient information to meet the data needs of analysts.

    Item 45 What kind of work were you doing, that is, what was your occupation, as of (fill with the last month worked at this job) 1996?"

    FR's asked if the reference period for this question should be >as of yesterday.=

    Item 51 "During 1996, what were your total earnings from this business/farm AFTER expenses?"

    One respondent rented land to farm and reported negative earnings for the year. The FR was unable to enter a negative dollar amount for farm-related earnings.

    2. Income Sources

    General comments

    FR's requested that we streamline this series of questions and ask only those questions that are appropriate for the household. For example, don't ask questions about receipt of alimony to people who have never been divorced or about receipt of WIC to a single man. FR's also suggested adding income screens to avoid asking wealthier people about receiving public assistance and having enough to eat (food security questions) or asking poor people about mutual funds and royalties.

    There were comments about the redundancy of some questions. For example, in the employment questions, we ask if the person applied for unemployment compensation after leaving a job. Then in the income sources, we ask three additional questions about unemployment compensation (did anybody receive it/who/what type). Later we ask the amount received. One FR said that most respondents gave all the information in the first question, i.e. "yes, I got $250 from the state a week." She felt that having to read the entire question was not necessary.

    FR's suggested having a "range" follow up item for items where respondents are unable to provide specific income amounts.

    Respondents and FR's were unclear whether money earned on IRA's, Keogh's or other retirement accounts was supposed to be reported when questions about interest-earning and dividend-earning accounts were asked. The confusion arose in part because the interest or dividends were being reinvested into the account and the respondents were not drawing off the account. Respondents also had little idea how much these accounts had earned last year.

    The FR's from the Kansas City RO who interviewed many farmers think that the income source questions do not adequately capture farm income. They mentioned that CPS does a better job of this by including a question about receiving farm subsidies. They would also like to see a flashcard with farm-related income listed on it to improve reporting of farm income.

    Item specific comments

    Items 200, 203 "This is a list of different sources of unemployment compensation payments. Did receive any unemployment compensation payments at any time during 1996?" (worker's compensation)

    FR's questioned why these items are asked for persons who reported earlier that they had worked 52 weeks last year.

    Items 206-211 "During 1996 did you receive any Social Security payments?" (who; Social Security for children)

    A problems with the flow of Social Security questions was mentioned. There seemed to be too many questions to get the information about who was receiving Social Security in households with children.

    Items 246, 249a, 251a, b, 254
    "At any time during 1996, did you have money in any kind of savings account, interest-earning checking account or money market fund?" (interest from bonds, treasury notes or CD's; mutual fund shares; rentals property; rental from roomers; royalties)

    Many of the FR's expressed a preference for these income sources to be grouped together on one screen. They thought this would be more efficient.

    Item 249 "At any time during 1996, did you own: Any mutual fund shares? Any shares of stock in corporations?"

    Respondents that have mutual funds in retirement plans sometimes get confused and it is unclear how they are to respond to this item. They suggested an introductory phrase "Excluding retirement funds,..." be added to the question.

    Item 264 "This is a list of other sources of income your household may have received. During 1996, did anyone in this household receive any of the following types of income: National Guard or Reserve pay, casual earnings from a side business or hobby, income from a farm, lump sum payment, income assistance from charitable group, any other sources of income?"

    Some FR's didn't know what "income from a charitable group" was.

    Item 300 " I have recorded that (you/name) received (READ LIST)" " Have I listed anything that should not be there?"

    Change wording to "Is this correct?" rather than "Have I listed anything that shouldn't be there?" One FR reported that the "read list fill" did not appear during one interview.

    3. Income Amounts

    Item specific comments

    Items 445-450 "Did you own any of these (interest-earning accounts) jointly with your (husband/wife)?" (joint interest received; average amount in joint accounts; sole interest-earning accounts; amount of interest received; average amount in sole accounts)

    The section on joint versus sole ownership of interest-earnings accounts caused confusion for the respondent.

    4. Eligibility

    General comments

    FR's said they would like additional training regarding why questions on assets and debts are included in SPD, so they can provide that information to the respondents when they request it. Item specific comments Item 604 "About how much would this (house/apartment) sell for if you were to put it on the market today?"

    Add a probe to ask for the respondent's "best estimate" when the respondent is unable to report how much their house would sell for.

    Item 605 "How much were your total property taxes, including city, county, and school taxes on this (house/apartment) in 1996?"

    Farmers often don't know the property tax paid on their home by itself because their tax bill includes their home and land, with the latter worth considerable value.

    Item 606 "How much did you pay for homeowner's insurance, that is, what was your premium in 1996?"

    Farmers may not know the amount paid for home insurance because they have umbrella policies that include their home, cars, and other liabilities.

    Item 607 "Do you have a mortgage or home equity loan on this property?"

    Farmers don't know how much they pay for mortgage on their homes because the mortgage includes the land they own as well.

    Items 608, 625 "Including any second mortgage or home equity loan, about how much is the remaining principal of this mortgage?" (principal on second home)

    Several FR's indicated that respondents don't know the principal left on their homes. They know their mortgage and how many more years they have left to pay on their house, but they don't know the principal. Not all banks print this information on their statements. One respondent did not understand the term "principal." The FR suggested including a definition in a help screen.

    The phrase "including any second mortgage," caused confusion for some respondents who had refinanced their homes several times. In some cases, it was interpreted as the second one in the series of refinanced mortgages. With the frequency with which homeowners refinance homes due to shifting interest rates, FR's thought it needs to be more clearly communicated that we are asking about their current mortgage(s).

    Item 618C "How much did this household pay for electricity, gas, and other utilities last month?"

    FR's made two suggestions regarding the question on utilities. One is to include a calculator in the computer so that they can add up the individual amounts that respondents report. The second was to ask the items separately and let the computer add the total. FR's also suggested adding "telephone" to the list of utilities and indicating that we just want the cost of basic service included.

    FR's reported that utilities for farmers often include costs for operating some of the farming equipment, which should be counted as a business expense rather than as a utility.

    Item 619 "Do own any real estate (other than your main home,) such as a second home, land, rental real estate, or money owed to you on a land contract?"

    One FR had a question about assets outside the United States. The respondent owned land in another country and wanted to know whether we wanted to include that or not.

    Item 634 "What is the year, make, model of (the newest/the next newest/the third newest) vehicle?"

    FR's need an explanation of why this question is asked and why we collect information for up to three vehicles. FR's need to be able to add a vehicle to the top of the list of newest, next newest, third newest without having to retype all previously input information. Some FR's suggested having different makes or models listed on the screen (like the SIPP precodes) or in a help screen. Others expressed a preference for the SPD method of typing in the make and model of the car.

    Item 637 "Is this vehicle used primarily for either business purposes or for the transportation of a disabled person?"

    This question should include a category "or neither." FR's did not understand why this question is asked. They were not sure if driving to and from work constitutes "used for business."

    Item 645 "Do you have any (other) debts -- such as credit card charges, student loans, medical or legal bills, or loans from relatives?"

    One respondent had an IRS debt and was unsure if this should be counted as debt. Several FR's reported that respondents commented negatively about the debt question. These respondents felt that this question was too personal.

    5. Educational enrollment/Work training

    General comments

    FR's reported that respondents mix up schooling and job training. The FR's said that after asking the Work Training questions, they have to go back and change answers to Educational Enrollment questions, because the respondent realizes he/she answered the for the other type of training. Item specific comments Item 703 "What was the highest level at which you were enrolled?"

    The concept of "highest level at which you were enrolled" was problematic for the respondent. One FR reported that, in LA especially, there needed to be responses for English as a Second Language in the adult education section.

    6. Disability

    General comments

    FR's suggested asking this series at a household level rather than a person level to save time and to reduce repetition of questions that are applicable to a minority of the population. Item specific comments Item 913 "Which type of aid do you use?

    FR's suggested that "hearing aid" be the first item mentioned since it is likely to be the most prevalent and respondents tend to tune out when they hear wheelchair, cane, etc. (Note that this item appears before item 906 in the instrument.)

    7. Health Care Utilization: Medical Expenditures

    General comments

    FR's suggested adding a transitional statement prior to the health care questions. Item specific comments Item 924 "To what kind of place did you usually go?"

    FR's said that respondents seem to be missing or not understanding 'usually' in the question. Respondents say that where they go for health care depends on the problem -- doctor, dentist, specialist, emergency or check-up.

    Item 924a "During (last month) did anyone in this household pay any doctor, dentist, or hospital bills, for prescription medicines for (name)?"

    FR's suggest adding a better transition statement to let respondents know that this item is just about last month. Also, this question is too long. Separate out the concepts. Ask one question about bills for dentists, doctors or hospital visits and a separate question for prescriptions. FR's said some respondents thought question was only about prescriptions. There is a typo in this question. Question should read "or prescription medicines" rather than "for prescription medicines." Also, Some respondents didn't know whether this question was asking if they had out-of-pocket expenses last month or if they had outstanding medical bills last month (that they may not have paid).

    Item 924b "Not counting amounts that will be reimbursed by insurance, how much was paid last months for (your/name's) medical expenses?"

    One FR said that the respondent didn't know if he/she was to report the copay amount or the entire amount. Some respondents had trouble with this question because they didn't know if the amount they had paid would be eventually reimbursed by the insurance company. Others needed clarification about whether we wanted to know how much was "paid" versus how much was "used."

    Some FR's want to record medical expenses at the household level rather than by person.

    8. Health Insurance

    Item specific comments

    Items 965, 970 "What type of health insurance were you covered by in 1996?" (current plan)

    FR's noticed that there are slightly different lists in questions 965 and 970 regarding type of health insurance. The latter includes a distinction for policyholder vs. dependent in the list. FR's would like both lists to be the same, thereby eliminating one of the flashcards.

    FR's from the Kansas City RO debriefing said she had to type "MinnesotaCare" for each person in the household in item 970. They would like MinnesotaCare added to the flashcard and the screen so they won't have to type this.

    Item 967 "Which answer on this card best describes the reason why you weren't covered by health insurance in 1996?"

    One FR suggested adding a category to the response options, "Not eligible because haven't worked long enough yet," for people who have recently started jobs and aren't yet eligible for health coverage.

    Items 968-970 "Is anyone in this household CURRENTLY covered by any type of health insurance including Medicare and Medicaid?" (who is currently covered; type of plan)

    There was some confusion in the transition from 1996 insurance to current year health insurance. Respondents felt that this information was redundant. FR's suggested using the wording from CPS to ask whether the health insurance is the same currently as it was last year.

    Item 969 "Who is currently covered?"

    FR's raised the question as to why they have to enter "N" for "no more," when the line numbers of all household members are already entered.

    9. Food Security

    General comments

    Many FR's got embarrassed asking these questions to families who obviously had enough food. Some FR's felt uncomfortable asking these questions to poor families. One FR suggested adding these to the self-administered portion. Several FR's would prefer to only ask the first screening question on food security and not the second screening series. One respondent got indignant during this section wondering why the FR was questioning her ability to provide for her family. Item specific comments Item 1005 "We couldn't afford to eat balanced meals." Was that often, sometimes, or never true for your household in the last 12 months?"

    A few FR's said that some respondents had trouble with the concepts of "balanced" meals. The FR's said that the balanced meal concept is not understood by some immigrant groups.

    Item 1007 A>We relied on only a few kinds of low-cost food to feed the children because we were running out of money to buy food.' Was that often, sometimes, or never true for (you/your household) in the last 12 months?"

    A few FR's said that some respondents had trouble with the concepts "low-cost food." One respondent questioned whether low-cost food meant store brand vs. name brand or beans vs. chicken.

    10. Marital Relationship and Conflict

    General comments

    Although most FR's reported that respondents liked using the computer, FR's in Miami and LA had problems with self administration because of literacy and respondents' lack of experience with computers. One FR reported helping the respondent by showing what buttons to press when she was answering. Another FR pointed out to him that this defeated the purpose of the self-administration. A few FR's did not turn the computers around. One FR was experienced in HIS and did not see the need for self-administration since she had asked these questions before. She also had households in which Spanish was the primary language. Another FR had strong objection with giving respondents access to his computer when he had confidential data in his computer. It would be helpful to have the questions printed elsewhere in English and Spanish so that FR's can read them when respondents can't read well. The respondent could still enter the numbered response without the FR seeing the answer. 11. Children's School Enrollment

    Item specific comments

    Item 1109 "Between September 1996 and October 1997, what was the highest grade in which (child) was enrolled?"

    The concept of the "highest grade in which ... was enrolled" was problematic. FR's suggested just asking what grade the child is in.

    In Minnesota, students may be enrolled in both high school and college at the same time. They will take classes in the morning at the high school, and classes in the afternoon at the college. In some cases, they can graduate from high school and a 2-year college at the same time. FR's need to know how they are supposed to code the "highest grade" question for these students.

    12. Child Care

    General comments

    In general, the FR's thought that this series was extremely redundant when there are multiple children in the household, given that most households had the same provider for all children. They prefer that the questions be asked about all children in the household at one time, instead of looping back and re-asking the series for each child.

    Some respondents were unclear as to whether they were supposed to report child care arrangements used while they were not working. In one case, the respondent worked during 1996, but didn't work during September 1997 (the month the mother's work schedule questions are asked about). The respondent didn't know whether we wanted her to report child care arrangements used while she was not working and the FR's were also unclear on this point.

    A few FR's would like a worksheet to keep track of child care activities. It would help determine if there were gaps in the weeks recorded for the various child care activities.

    Item specific comments Item 1301 "Please tell me which of these you used for (child's name) on a regular basis between January 1996 And September 1997."

    Item 2 on the flashcard is unclear. Some respondent interpreted "designated parent" as the person "designated" to pick up the kids or be responsible for the kids at that time. Respondents didn't know that the "designated parent" is the person whose work schedule we just asked about. One respondent who was self-employed and took care of her children in her home while she was working was confused by this series and did not choose this category. The FR wasn't sure why the respondent didn't choose this category.

    FR's said that they didn't know the differences between some of the different types of care on the flashcard such as <7> family day care home, <9> nursery school, preschool, and <12> child care or day care center. The distinctions need to be made clear during training and definitions of the terms need to be included in the glossary.

    A substitute teacher who works when she is called said that her kids were not cared for on a regular basis in any arrangement. The FR was unclear whether this was the correct response to question 1301. (May need to include what we mean by "regular" basis in training.)

    Some FR's wanted a "none" category for the households in which the mother always cares for her children. FR's also want to be able to skip all the child care questions when the parent reports that the child takes care of himself/herself.

    Item 1305 "Of those (hours), how many of them were while you were at (work/school/work training/looking at work)?"

    Respondents didn't know whether they should include travel time when reporting hours child is cared for in the arrangement while the designated parent is working.

    13. Child support

    General comment

    There were very few comments about this series because most FR's did not go through it. Item specific comments Items 1401A, 1401B "Why does (name) not have a (father/mother) living outside this house?"

    An FR mentioned that these items were sensitive questions and that some respondents will not want to talk about the outside parent. Several other FR's looked at the question and agreed.

    14. Contact with Absent Parent

    General comments

    No problems were reported with these questions. Few FR's had a chance to ask them. B. Structure and Flow of the Core Questionnaire

    Shifting between person-level and household-level questions

    FR's liked the use of household level questions (as in the income series). They would like more household level questions (such as in the disability series). FR's said that respondents stop listening as the FR's read the same questions for subsequent respondents. FR's said they would like to ask the child care questions for all children at once since children in the same family are often cared for in the same type of arrangement.

    Some FR's mentioned that because of the shift it was not always clear about whom the questions were being asked. One suggestion was to put on the screen "household level" when such questions were being asked.

    Facilitator at the Kansas City RO debriefing asked the FRs' opinion about moving the food security questions to the self-administered section of the questionnaire. FR's thought this would be a good idea and may make it easier for respondents to report truthfully.

    Shifting of topics within each person Some FR's mentioned that they did not like the current structure of asking health care utilization questions about the adults in the household and then later asking about the same questions about the children in the household. The FR's prefer that questions for a specific topic be asked for all persons in the household before moving on to a new topic. For many content areas, the FR's prefer using the household level screener approach which asks "Did anyone in this household...........," identify who, and ask the detailed questions only about such persons. They particularly lobbied for this type of approach for the disability and functional limitations questions. In general, FR's thought that a topic-based instrument would work better than a person-based instrument. If a topic based instrument is not possible, FR's said they would prefer to have the education and work training series grouped together and the disability and health care utilization series grouped together (two smaller person-level loops). (Note that these loops might preclude the possibility of asking the disability questions at a household level.)
    1. Reference Periods

    2. Some FR's suggested having an aid similar to one used in HIS in which the different reference periods are shown in different colors.

      FR's reported that in the employment series the switch from "last week" to "during 1996" caused some confusion. Another content area in which the shift in reference periods posed difficulty was health insurance, in which questions are asked about 1996 coverage and then about "current" coverage. For many persons, whose current coverage was the same as last year's coverage, it was sometimes confusing because the respondents thought they had already answered those items. FR's also reported having to remind respondents of the time period during the educational enrollment questions.

      The shift in reference periods for the child care series was also confusing. The questions ask about January 1996 through September 1997 and then additional questions on hours, cost, and subsidies are asked about September 1997. Also contributing to the confusion in this series are the preliminary questions about the designated parent's work schedule which are specific to September 1997.

      FR's mentioned that a transition is needed before the health care questions. They noted that some respondents spent time counting visits in 1996 prior to the 12-month reference period.

      D. Use of Records

      FR's in Kansas City RO reported that many R's did use records. FR's in Boston reported that respondents used their records about 20 percent of the time. The type of records referred to were W2s, utility bills, and pay stubs. When respondents did use records, FR's felt that tax forms were the most useful.

      FR's from the Atlanta RO said that as a result of not having received the advance letter, many respondents did not have records available when they arrived. FR's suggested adding instructions to the advance letter about what records would be helpful (as is done on CPS).

      Only one FR in LA reported having someone go to use records during the interview. He said that one respondent got his tax return. The FR's said that all the respondents were giving general, and highly rounded, reports of amounts. Some felt that this was because they were CPS respondents rather than SIPP respondents who are used to answering in details.

    3. Screen Layout
    General comments

    In general, the FR's thought the screen layout was good. They liked the split screen roster and did not have a problem with truncated names, nor did they think it would be problem.

    FR's emphasized that precodes for similar actions should be consistent throughout the survey (and preferably consistent across surveys.) For instance when entering that they can proceed ahead because there is no one else to ask about or no more response options to enter, they want one standard precode throughout the instrument. FR's expressed no preference regarding what the precode is, as long as it's consistent throughout. (They mentioned that currently for similar actions, they are sometimes required to enter a "0," "N," "P," or "Enter.")

    FR's preferred screens without response options to be "Hit Enter to Proceed" rather than "P" to proceed. If an entry is needed, a "1" is preferable to a "P" because FR's are used to entering numbers.

    FR's stated a preference for the method of entering a number and then hitting "N" for no more, rather than the other methods of (1) yes; (2) no for every item in the list or typing an "x" next to the appropriate spot. The FR's also always want the "A" for "all" option on multiple response and roster items.

    FR's requested that the instrument be consistent for inputting year entries. Currently there are both 2 and 4 digit entries used for year.

    FR's indicated that they had some problems knowing who should be answering the question. They thought it would be helpful to have the respondent's name on the top of the screen.

    Some screens, including zip code, house value, and income, do not accept "don't know" or "refused" entries. FR's requested that this feature be added.

    A copy of the flashcards should be built into the screens of items where they are used.

    For "mark all that apply" type questions, FR's liked screens in which they typed the response category or line number (for household roster questions) and an 'x' appeared next to the response category or the household roster. They also liked the program screens where they marked '1' or '2' for participation in AFDC, WIC, Food Stamps, etc.

    FR's suggested a larger font for the screens.

    Item specific comments

    Item WD1 The wedge questions (has anyone lived away 30 days or more) are confusing because FR has to enter "0" for no one lived away for 30 days or more or "N" for no one else. This is a yes/no question. Change first screen to yes/no format. Add a second screen with the household roster if someone was reported as living away 30 days or more.

    Item LN/LD The screen asking for "who is the mother/father of..." is confusing. FR's preferred wording from CPS (note that CPS wording may not be appropriate if it doesn't determine whether child is the biological, step or adoptive child of the parent.)

    Item 19 "Please mark on the worksheet the weeks during 1996 that you did any work at all, even for only a few hours."

    This item has too much information for FR's to look at on one screen.

    Item 246, 249a, 251a, b, 254 "At any time during 1996, did anyone in this household have: money in any kind of savings account, interest-earning checking or money market fund?" (mutual fund shares, share of stock in corporations; rental properties; rental income from boarders; royalties)

    These questions should be listed on one screen instead of on separate screens.

    Item E-REVIEW List of employers

    FR's would like the name of the employers listed on this screen instead of Employer 1, Employer 2, etc. as is currently done. (Note that employer name is collected after item 29 in the instrument, so this change isn't possible without restructuring this series.)

    F. Self-administered Adolescent Questionnaire

    General comments

    Not all FR's in the debriefing sessions had interviewed households with adolescents. All FR's in the Kansas City RO had administered the SAQ. In Boston, only 3 FR's had administered the SAQ.

    In general, FR's thought the administration of the adolescent questionnaire went more easily than they expected and there were few problems with the administration of the SAQ or the procedures. Some FR's said that they were probably more uncomfortable asking the questions than the adolescents were answering them. Although they did not face many problems in the pretest, FR's indicated that information on how the survey data is used might help to convert refusals in the future.

    Some FR's mentioned that they thought this portion of the SPD should not be part of this survey, but should be a separate survey. While their respondents did not seem to have any objection to it, the FR's themselves felt a bit uncomfortable with some of the content.

    Parental concerns Two FR's reported parents refusing to allow their child to participate. One FR reported that the respondent was reluctant to participate in SPD, and was tired of being bothered by the Census Bureau. The parent did not want anyone else in the house to spend time on a Census Bureau survey. The FR did not want to upset the respondent and lose the SPD core, so he did not press on the SAQ. In a second case, the parent said his/her child had a learning disability and would not be able to complete the questionnaire. The FR noted that there is no disposition code for this situation and one should be added before the next administration. Some parents were hesitant. In a third case, the adolescent convinced the parent it was okay. In a fourth case, the parent refused to allow an adolescent to answer the SAQ. The adolescent was 13 and the parent was taken back by the term "sexual activities." (Note that this adolescent would not have been asked the questions on sexual activity.)

    A few FR's showed the question booklet to parents regardless of whether the parent asked to see the questions. Other FR's only showed it when asked, which occurred infrequently.

    The FR's said it was obvious that the booklet was designed to hide the sexual questions, since parents flipped through the first two pages and then looked at the last page.

    Concerns Raised by Adolescents One adolescent refused to participate when the FR called back to conduct the interview by phone. The FR thinks the respondent refused because his parents said they would be interested in seeing his answers to the SAQ and he was concerned about confidentiality. Some FR's reported that some respondents were concerned about the confidentiality of their answers and asked the FR about this. A few FR's reported that some adolescents were uncomfortable answering the questions on stealing and fighting and the questions about reasons for not having sex. FR's suggested changing the wording of question 103 to "Why have you chosen not to have sex at this time?" They also suggested changing the list to read "I think I'm too young," "I think sex before marriage is wrong," etc.

    FR's thought that for the most part adolescents did not have trouble concentrating during the interview, but that a couple of younger respondents complained that the survey was too long.

    Problematic Concepts or Terms Concepts that some younger adolescents had difficulty with included: "criticize," "biological," "respect." One FR suggested changing "respect" to "look up to" or "admire." Some adolescents did not understand the terms "biological" or "highly." One respondent didn't know what we meant by "things that are important to me." Some FR's had adolescents with difficulty reading English.

    Add a "not applicable" category in the FR version of the SAQ to the questions on how often limits were broken to accommodate adolescents who say they set their own limits.

    One FR suggested ordering the response categories in the same direction from "negative to positive" or vice versa so that the categories are consistent. She also suggested that we start off with the positive category rather than the negative one. The facilitator explained that on a paper instrument, respondents are more likely to see categories at the top of the list, whereas in telephone administrations respondents are more likely to hear the last response option.

    Item specific comments

    Items 15-38 Questions on parent/child relationships

    Some FR's said that the reference person in these questions is not clear, so they added "your mother/ your father" to the questions.

    One respondent said >in the middle' for "don't know" during the series of questions about the father.

    Items 64-66 "In the past year, how many times did you run away from home for at least one night?" (purposely damaged or destroyed property; stolen something)

    Some FR's were uncomfortable asking these questions on delinquency.

    Item 69 "How old were you when you smoked a while cigarette for the first time?"

    There is not a response for those who smoked less than 1 cigarette. The previous question asks about smoking even a puff or two. In question 69, we ask about the age of smoking an entire cigarette but there is no box for someone who tried a puff but did not smoke an entire cigarette.

    Items 82-94 Questions on perception and knowledge of welfare.

    FR's wondered how the welfare questions will be asked when different states start to use different names for welfare programs. They suggest that the term 'welfare' will not have any meaning to respondents. It will be replaced by 'warfare' or other state based names.

    Item 93 "Where must a teenager who has a baby live in order to receive welfare benefits?"

    There was a suggestion from an FR to change the wording of "in your state" to "in this state." The FR's and the respondents are in the same state. "In your state" sounds like a condition rather than a location.

    Items 104-119 Questions on sexual activity, contraception, child bearing

    FR's were uncomfortable asking the sex questions. Although the situation did not occur, male FR's expressed concern about asking (during a telephone interview) adolescent females how often they have had sex. They are worried about how this could be viewed by the adolescent and their parents.

    Adolescents' Comfort Level and Attention Span

    Cooperation was not really a problem. The adolescents were interested in participating in the survey. One FR said he had to repeat some questions because the respondent's attention drifted. FR's reported that leaving the answer booklet for respondents to use during the telephone interview helped keep the respondent's attention focused. SAQ vs. Telephone Administration The proportion of telephone callbacks was somewhere between 25-33 percent. In some cases the FR had to call the household back more than one time.

    Some FR's reported that they read the response categories every time when administering the questionnaire over the phone and that this got tiresome. Some FR's did not always read the response categories, but they did in most cases.

    FR's thought it was helpful for the adolescent to look at the answer booklet while the FR was reading the questions over the phone. They noted that in a couple of places the answer categories don't match between the adolescent booklet and the FR booklet and that this caused some confusion.

    One FR suggested having an FR of the same sex as the adolescent respondent conduct the telephone callback.

    Procedures and Equipment One FR mentioned that the equipment for the adolescent survey was less than satisfactory. She said the headset broke and also she had difficulty getting the replacement earplugs on and off.

    The FR's mentioned that they did not always transcribe the cover page information on the answer booklet prior to giving it to the adolescent, therefore it was necessary for them to open the sealed envelope after they left the household and enter the appropriate information. There was concern on the part of some FR's that having the answer booklet cover page information filled out prior to giving it to the adolescents may lead the adolescents to wonder about the confidentiality of the answers.

    The majority of FR's thought that two tape recorders were sufficient. FR's did not think the callback form for the adolescent questionnaire was necessary.

    FR's thought their training on the SAQ was well done and that they were well prepared. They suggested that FR's listen to the adolescent tape during training or as part of their self study.

    FR's recommended two separate questionnaires and tapes be used. One for 12-13 year olds and one for 14-17 year olds.

    Comments provided at the FR debriefing will be reviewed along with other information for the SPD questionnaire evaluation. Recommendations for question revision will be made by December 5, once all the data have been reviewed. Please direct any questions or comments on this report to Jennifer Hess. She can be reached at (301) 457-4968 or by e-mail at jennifer.c.hess@ccmail.census.gov.

    Attachment C

    Survey of Program Dynamics

    October 1997 Adolescent SAQ Pre-test Results

    Executive Summary

    1. Completion rate. In total there were 66 adolescent cases received at headquarters from the pretest. Of these, in 3 cases (4.5%) parents refused permission to interview the adolescent; in 1 case (1.5%) the adolescent refused; and there were 2 cases (3.0%) where the adolescent was disabled and unable to complete the questionnaire. Thus there were 60 completed cases, resulting in a completion rate of 91%. This report is based on these 60 cases. The sample was fairly evenly distributed by age, with 38% aged 12-13, 20% aged 14-15 and 42% aged 16-17.

    2. Survey administration issues. The issues covered in this section are the debriefing questions; whether results differed by the method of administration (phone interview or walkman/self administration); whether the longer version or shorter version of the tape was preferred; and whether sensitive questions caused break-offs or are associated with missing data (blanks).

    The debriefing questions concerning the tape (self-) administration revealed few problems, though the respondents stated they would have been concerned about their privacy if the questions had appeared on the answer sheet. Respondents indicated the pace of the tape was neither too fast nor too slow. The results on whether the answer categories should be repeated for every question were mixed, with some preferring all categories read and some preferring that they only be read when the categories change. If the questionnaire were re-ordered so that the section on non-residential parents was placed second-to-last (before the section on sex and contraception), it would result in a shorter tape for younger respondents without non-residential parents. Another option would be to have two tapes, one for older respondents and one for younger respondents, with the section on nonresidential parents placed last on both tapes.

    Most respondents said that it was not difficult to concentrate on the questionnaire, but there were varied opinions on how interesting the questionnaire was. The majority of the adolescent respondents (from 62 to 78%) were "not at all uncomfortable" with the survey questions. The sections where respondents were more likely to report being uncomfortable were those concerning relationships with parents (8% very uncomfortable), problem behaviors such as running away or stealing (2% very uncomfortable), and sex/contraception (9% very uncomfortable). But respondents who expressed discomfort with a section of the questionnaire were not more likely to leave questions in that section blank; there were few blank questions overall.

    An examination of whether responses differed by administration method (whether self administered or by telephone) revealed for the most part no significant differences.

    Recommendation: Administration proceeded smoothly with few problems encountered. The self-administered tape should have the answer categories read whenever question categories change, or every few questions if there is a series of questions where they do not change. This would result in a tape that is about 30 minutes in length. The answer booklet should continue to have answer categories only (omitting the questions) to ensure respondents' privacy. The questionnaire should be re-ordered so that the section on contact with non-residential parents is second-to-last (right before the section on sex and contraception). No changes are indicated in the pace of the tape, method of administration, or in response to debriefing questions on discomfort with sections of the questionnaire.

    3. Problems with the questionnaire. Problems with the questionnaire are analyzed by examining whether there are blank questions and/or sections; whether there are break-offs and incomplete questionnaires; and whether skip patterns were followed correctly. Revisions are also suggested in response to comments made by the Field Representatives in the debriefing sessions.

    The pretest sample had almost no blank responses to questions. One respondent did not answer most of the substance use questions, and one respondent did not answer the behavior problem section.

    Skip patterns are only an issue for those cases who completed a self-administered questionnaire, since they had to listen to all questions read on the tape even if the question did not apply to them. When the interview was conducted by telephone, the interviewer was able to skip over questions that were not applicable. Respondents to the self-administered questionnaire were able to accurately follow the skip patterns in most cases. Simple revisions to the skip pattern or questions are recommended in a few instances, as outlined below.

    The responses to the parental limits question indicated that the question phrasing was ambiguous: if respondents answered that they decided on their own limits, it was unclear how they should answer the question on breaking limits. This can be simplified by adding an option "Does not apply- I set my own limits" in the interviewers' booklet for the questions on who sets limits. Also, there was some confusion about the cigarette smoking questions, as respondents who had only tried a few puffs of a cigarette were then asked how old they were when they smoked a whole cigarette. This can be clarified by adding an option of "I have never smoked a whole cigarette" to this question.

    The Field Representatives had several suggestions to clarify question wording. Since some respondents did not understand the term "biological mother/father", we suggest adding "(that is, the mother/father you were born to)" to these questions. Because respondents were confused about the wording of the response choices in Question 103 on reasons for not having sex, we suggest rewording from "I'm too young" to "I think I'm too young", etc.

    Recommendation: Rewording is suggested for the three questions on parental limits, one of the questions on cigarette smoking, the screening question for parents/parent figures and the question on reasons for not having sex.

    4. Questions on knowledge and attitudes towards welfare. The results show a high percentage of "don't know" responses to these questions, ranging from 37 to 48% "don't know" or "in the middle" responses on the attitude questions and 60 to 88% "don't know" responses on the knowledge questions.

    Recommendation: While young people must know about changes in welfare regulations in order for such changes to affect their behavior, it appears that this age group is too young to be knowledgeable, or even to have opinions, about welfare. We recommend that a set of questions about welfare attitudes and knowledge be posed to adults in the core questionnaire, possibly in the self-administered section. We recommend dropping the welfare attitude questions from the adolescent SAQ and retaining only the two knowledge questions that address regulations directly affecting teens.

    5. Intervening and outcome variables. Scales and indices were constructed to measure the intervening and outcome variables of interest. All of the scales had a moderate to high degree of reliability. The results indicate that the following items could be dropped without reducing reliability:

    Q. 2 (Frequency household chores done by a family member)

    Q. 6 (Frequency straighten your room)

    Q. 20 (Mother- explains reasons for making a decision)

    Q. 21 (Mother- praise you for doing well)

    Q. 22 (Mother- Criticize your ideas-recoded)

    Q. 32 (Father- explains reasons for making a decision)

    Q. 33 (Father- praise you for doing well)

    Q. 34 (Father- Criticize your ideas-recoded)

    Because of the importance of increasing the involvement of non-residential parents, we recommend dropping one item in this section and adding eight parallel items to those in the residential parents' sections. These are:

    Drop: Q. 58 How close do you feel to your outside parent

    Add: I think highly of him/her

    He/she is a person I respect

    I really enjoy spending time with him/her

    I can count on him/her to keep promises

    Helps you with important things to you

    Blame you for his problems

    Spend time just talking to you

    Show that he/she really cares about you

    Summary of recommendations: No changes are recommended in the method of administration. The questions on parental limits, cigarette smoking, parent/parent figure and reasons for never having had sex should be revised. The section on welfare knowledge and attitudes, except for the two questions on policies concerning teens, should be dropped. Three items from the mother's scale and three items from the father's scale should be dropped. One item from the closeness with nonresidential parents section should be dropped and eight items added. In total, 20 items should be dropped and 8 items added for a total reduction from 119 to 107 questions (10%).

    Summary Table of Questions to be Revised or Cut
     
     

    Section

    Item  Revision
    Household routines Q. 2 Frequency household chores done by a family member Cut
    Housework and chores Q. 6 Frequency straighten your room Cut
    Support/ identification with mother Q. 15 Which category best describes the mother you live with? Is it: Your biological mother who lives with you. Q. 15 (revised) Which category best describes the mother you live with? Is it: Your biological mother (that is, the mother you were born to) who lives with you.
    Q. 20 Explains reasons for making a decision

    Q. 21 Praise you for doing well

    Q. 22 Criticize your ideas

    Cut
    Support/ identification with father Q. 27 Which category best describes the father you live with? Is it: Your biological father who lives with you. Q. 27 (revised) Which category best describes the father you live with? Is it: Your biological father (that is, the father you were born to) who lives with you
    Q. 32 Explains reasons for making a decision

    Q. 33 Praise you for doing well

    Q. 34 Criticize your idea

    Cut
    Parental monitoring Q. 47, 49, 51 How often have you broken the limits about staying out late/TV shows & movies/who you can hang out with Add category in interviewer's answer book: 

    ADoes not apply- I set my own limits@

    Contact with non-residential parent Q. 52-57 Move section second-to-last (before sex and contraception)
    Q. 58 How close do you feel to your outside parent Cut
    I think highly of him/her

    He/she is a person I respect

    I really enjoy spending time with him/her

    I can count on him/her to keep promises

    Helps you with important things to you

    Blame you for his problems

    Spend time just talking to you

    Show that he/she really cares about you

    Add
    Substance use Q. 69 How old were you when you smoked a whole cigarette for the first time? Add response category:

    AI have never smoked a whole cigarette@

    Welfare knowledge and attitudes Q. 82-92 Cut
    Sex and contraception Q. 103 What are your reasons for not having sex at this time? You can choose more than one answer.

    1 I=m too young

    2 Sex before marriage is wrong

    3 Don=t want to get pregnant/get someone pregnant

    4 Don=t want to get a sexually transmitted disease

    5 Afraid parent(s) would find out

    6 Don=t have a boyfriend/girlfriend

    7 Waiting for the right person

    8 Not interested

    9 Other reason. Please describe: ________________________

    Q. 103 What are your reasons for not having sex at this time? You can choose more than one answer.

    1 I think I=m too young

    2 I think sex before marriage is wrong

    3 I don=t want to get pregnant/get someone pregnant

    4 I don=t want to get a sexually transmitted disease

    5 I=m afraid my parent(s) would find out

    6 I don=t have a boyfriend/girlfriend

    7 I=m waiting for the right person

    8 I=m not interested 9 Other reason. Please describe: ________________________

end of content rule

SPD Working Papers

SPD Publications


Skip bottom navigation groups

Contact: (dsd.survey.program.dyanmics@census.gov)
URL: http://www.census.gov/spd

 

Introduction to SPD |  Survey Design & Content |  Data Editing |  Finding SPD Info |  Sampling & Weighting | 
Linking Files | Publications |  S&A  |  News&Notes |  Users' Guide 

 


Census 2000  |  Subjects A to Z  |  Search  |  Product Catalog  |  Data Access Tools  |  FOIA  |  Privacy · Policies  |  Contact Us  |  Home
separator rule
U.S. Census Bureau: Helping You Make Informed Decisions