This paper is released to inform interested parties of ongoing research and to encourage discussion of work in progress. All views expressed are the author’s and not necessarily those of the U.S. Census Bureau. This work has benefited greatly from the contributions of many others at the Census Bureau, including especially the members of the Re-Engineered SIPP Research Group. Several SEHSD colleagues provided key technical assistance with data analysis – in particular, I thank Bri Hillmer, Lindsay Monte, Sharon O’Donnell, and Rachael Walsh for their help. I also extend thanks to those who reviewed and commented on an earlier draft of this paper, especially Jamie Choi, Jason Fields, Matthew Marlay, Ann Marie Middleton, Lindsay Monte, and Joanne Pascale. Their comments and suggestions led to many improvements; responsibility for the paper’s flaws, however, rests solely with me.

The 2010 SIPP-EHC Field Test was the Census Bureau’s first large-scale test of event history calendar (EHC) interviewing procedures using an electronic instrument. Current plans call for the eventual use of these procedures, as refined through a series of field and other tests, in a planned new (“re-engineered”) Survey of Income and Program Participation (SIPP) survey scheduled to go into production in 2014. In order to maximize information yield, the 2010 field test included a number of special features and activities, among which was the recording of some of the interviews. This report describes those procedures, their implementation outcomes in the 2010 field test, the methods used to analyze the recordings, and the results of those analyses. Key among the research findings are the following:

• interviewers (“field representatives” (FRs)) performed quite well at introducing the calendar aid and the “landmark events” task, although their follow-up on the former – encouraging respondents to actually use the calendar aid – was almost nonexistent;
• about two-thirds of all respondents produced at least one landmark event report, and FRs recorded those reports very accurately;
• FRs’ performance on an end-of-training “certification test” – in particular, their performance on a sub-set of test items dealing specifically with EHC procedures – was predictive of the quality of their handling of basic EHC interviewing tasks;
• to a lesser extent, FR experience/tenure was also related to basic EHC performance;
• respondents exhibited overt evidence of recall difficulty – the signal for FRs to employ EHC interviewing methods to assist recall – only rarely;
• FRs “caught” those distress signals imperfectly; about 30% of the time when a respondent’s behavior suggested a recall problem, FRs did not appear to notice;
• when FRs did perceive respondent recall difficulties, the actions they took to assist almost never tried to exploit the supposed strengths of EHC methods, and were of generally low quality;
• FRs’ end-of-training certification test scores predicted neither whether or not they noticed respondents’ recall difficulties, nor, when they did notice, the quality of their assistance efforts;
• FR experience/tenure was predictive of both noticing and taking positive action to assist respondent problems, although the nature of the former relationship, in particular, does not lend itself to easy explanation;
• the quality of FRs’ assistance efforts was positively related to the likelihood of an apparently successful resolution of respondents’ recall difficulties.

The results of the investigation point toward a number of recommendations concerning possible SIPP-EHC procedural modifications and aspects of FR training which need to be strengthened.