| Dates: | Tuesday March 14 through Thursday March 16, 2006. |
| Place: | Bureau of Labor Statistics Conference Center, Postal Square Building, 2 Massachusetts Ave., Washington, D.C. 20212 |
| Sponsors: | The Bureau of the Census and the Bureau of Labor Statistics |
Workshops Program
Program Opening Day - Tuesday, March 14, 2006
A. Plenary Sessions (Tuesday 9:00 am to 12:00 noon)
The meetings will begin with a brief welcome and announcements by John Bosley and two 80 minute, consecutive, plenary sessions.
P-1. Opening Keynote Plenary: Computer-Assisted Advances in Sample Design.
Colm O’Muircheartaigh, Professor, University of Chicago, and Senior Fellow, National Opinion Research Center.
P-2. Panel Plenary: Mixed-Mode Survey Design. What Are the Benefits and Costs?
Chair: Bill Mockovak <mockovak.william@bls.gov>. A Panel of Experts to be chosen from both government and private survey agencies will present their views. Time will be saved for audience participation.
B. Concurrent Sessions (Tuesday 1:30-4:30 pm)
B-1. Recent Innovations and Lessons Learned at Participating Organizations
This session has replaced the traditional Round Robin Organizational Reports. Following the model of past year’s approach, the organizational reports will be voluntary. Only organizations that have recent innovations to share with their colleagues are asked to report. Because presentations in this session are generally limited to 10 minutes each, we ask that they be focused on true innovations. Descriptions of new or continuing surveys using familiar CASIC methods may be distributed as handout supplements rather than part of the verbal presentation. The innovations can be in organization, types of surveys undertaken, software, hardware, communications, training, research, or what have you. We are hopeful that at least the larger Federal survey agencies and Federal contractors will have something to contribute, especially on topics that do not fit into one of the half-day technical sessions. If nothing new is happening at your organization, come and see what other organizations are doing.
Schedule of Speakers:
| Time | Topic | Presenter(s) |
|---|---|---|
| 1:30pm | BLS and Census -- Introduction | Nicholls & Bosley |
| 1:35pm | Bureau of Labor Statistcs | Bill Mockovak |
| 1:47pm | Mathematica | Mark Pierzchala |
| 1:59pm | Statistics Canada | Susan Lensen |
| 2:11pm | SRC-Michigan | Patty Maher |
| 2:23pm | Market Strategies | Reg Baker |
| 2:35pm | Blaise- Statistics Netherlands | Lon Hofman |
| 2:50pm | BREAK | |
| 3:05pm | Census Bureau | Cheryl Landman |
| 3:17pm | Westat | Jim Smith |
| 3:29pm | Research Triangle Institute | Karen Davis |
| 3:41pm | NORC | Judy Petty |
| 3:53pm | ORC Macro | Robert Dana |
| 4:05pm | CASES - U.C. Berkeley | Tom Schnetlage |
| 4:20pm | Summary | Bill Nicholls |
B-2. Software and Application Demonstrations
This year we will continue to offer demonstrations of CASIC instruments and software in a mini exhibit hall setting, where attendees can move among exhibitors throughout the demonstration period. Space is anticipated for nine (9) concurrent demonstrations.
Only representatives of Federal agencies or Federal survey contractors may make presentations. Software vendors may participate in demonstrations only when invited by a Federal agency or Federal survey contractor to assist in its presentation.
A list of presenters should be available prior to the workshops and posted here.
Coordinator: Louis Harrell <harrell.louis@bls.gov>.
B-3. Survey Uses of Metadata
This session will concentrate on metadata for questionnaire design and the interview process. Generally, statistical metadata are data that describe some aspect of the survey life-cycle or statistical data. They convey information to someone about a survey and its components and can be used as parameters to drive automated processes. Metadata are data, so they can be managed in a database. Metadata management is used to preserve corporate memory (i.e., an aspect of knowledge management.); provide a record of past designs and processes in order to enhance survey redesign; help harmonize or even standardize concepts (e.g., question wording) across surveys and time to improve survey quality; and provide a reference for people needing to understand statistical data.
Presentation Materials:
- Economic Census' Metadata and Instrument Design (
, 401kb), William Samples <william.r.samples@census.gov> - Survey Data Management Using Metadata (, 611kb)
- Using and Developing Survey Metadata (, 1.3mb), Michael Volynski <mhv3@cdc.gov> - Center for Disease Control
- Web Based Metadata Tracking System Designed for the National Survey on Drug Use and Health (, 1.2mb), Nathini Ganapathi <ganapathi@rti.org> - Research Triangle Institute
Coordinator: Dan Gillman <gillman_d@bls.gov>.
WEDNESDAY MORNING – March 15, 9:00 am -12:00 noon
Concurrent Sessions
WA-1. Implementing, Designing and Evaluating Web-based Surveys.
Web-based surveys have started to mature, with common capabilities, standards and practices starting to take shape. Minimally, a basic understanding that it takes more than a Web server and an intern or graduate student to field a Web-based survey, has sunk in. In this session, we will explore the maturation of Web-based surveys with a variety of topics that will be expected to raise some very good discussion. Presenters will discuss their experiences with the practice of implementing Web-based surveys, the designs being implemented, and how quality is being evaluated. This session will close with a panel discussion with all presenters and the audience. Audience experiences and input will be strongly encouraged.
| Time | Topic | Presenters | Materials |
|---|---|---|---|
| 9:00 - 9:05am | WELCOME / INTRODUCTION | ||
| 9:05 - 9:30am | The Impact of Respondent Computer Environment on Web Questionnaire Design | Mark Pierzchala, Mark Brinkley, Leonard Hart, Scott Reid, and Chris Rankin (Mathematica Policy Research, Inc.) | presentation (, 230kb) |
| 9:30 - 9:55am | Evaluating a Web-based Establishment Survey of U.S. Academic Institutions using a Web-based Response Behavior Survey | Emilda Rivers (National Science Foundation), Scott D. Crawford (Survey Sciences Group, LLC) | presentation (, 184kb) |
| 9:55 - 10:20am | CES Web Data Collection: A Simplified Version | Richard Rosen, Antonio Gomes, Louis Harrell, Jason Chute (Bureau of Labor Statistics) | |
| 10:20 - 10:35am | BREAK | ||
| 10:35 - 11:00am | Web data integration: The Collection Side | Sylvain Hayes (Statistics Canada) | |
| 11:00-Noon | PANEL DISCUSSION / QUESTIONS |
Coordinators: Scott Crawford <scott@surveysciences.com>.
WA-2. Best practices in CAI Authoring.
This session will discuss best practices for the development of CAI instruments. We will discuss a variety of methods, including automation, that can be used to reduce development burden and increase the quality of fielded software. Topics will include specification development, development management metrics, and testing.
| Time | Topic | Presenters |
|---|---|---|
| 9:00 - 9:10am | Introductions | |
| 9:10 - 9:40am | Automated Testing of the Census CFU Instrument | Gilbert Rodriguez and Bharathi Jayanthi (RTI) |
| 9:40 - 10:25am | A Metadata Authoring Tool for Surveys | Debra Reed-Gillette, Shannon Corcoran, Michael Volynski, Ph.D. (DHNES) |
| 10:25 - 10:40am | Coffee break | |
| 10:40 - 11:10am | Automated Testing at Statistics Canada: The Compuware Solution | Michel-Éric Velleman (Statistics Canada) |
| 11:10 - 11:35am | Questionnaire Development within ONS | Colin Setchfield (ONS) |
| 11:35 - Noon | Round Table Discussion: Common Problems During Application Development |
Coordinators: Charlotte Scheper <cscheper@rti.org> and Mike Egan <mike.egan@statcan.ca>.
WA-3. CASIC Methods for Establishment Surveys.
This workshop will look at the application of CASIC methods to establishment surveys and include presentation on current methods of mass data capture.
Rob Dymowki (Westat) will discuss NAEP E-filing, an online application that allows schools, school districts, and state education departments to upload files of student information. These files are then standardized by the user informing us about the contents of the submission. The user maps their variable names to ours and their variable definitions to ours. This is done for about 10 variables. During the process, the application is running QC checks to help ensure solid data.
Jim O'Reilly (Westat) and Lon Hofman (Statistics Netherlands) will discuss BASIL (Blaise Application for Self-Interviewing), including the general Blaise approach and its development using Delphi and Blaise elements (language, routing and checking, data storage). BASIL system is a Windows application with flexible layout and is easy to download and install.
Jamie Isaac (RTI International) will describe Strategies for Contacting Participants and Enhancing Data Quality, lessons learned from a large ongoing web-based collection of postsecondary institutions data.
Coordinators: Daniel Wellwood <daniel.wellwood@census.gov> and Shawn Eaton <shawn.l.eaton@census.gov>.
WA-4. Usability.
We believe it is now generally accepted (at least in theory) that design of CASIC systems and instruments should be user-centered, that is, focused on system and instrument users and how easy it is for them to perform their required tasks. To that end, many agencies and organizations have developed laboratories, tools, and procedures for evaluating the usability of systems and survey instruments and to improve their design.
In this session, we have asked participants from different organizations and agencies to tell us what they have been doing recently to apply usability principles and testing procedures to design and evaluation of surveys and systems. These include evaluations of a Web self-administered survey and a browser-based CAPI survey. Presenters will describe design challenges, procedures and tools used, and usability test findings, and there will be time for open discussion about issues raised.
Presentations:
- Introduction and Overview, Sue Ellen Hansen <sehansen@umich.edu>, University of Michigan, and Kath Straub <kath.straub@gmail.com>, Bureau of Labor Statistics.
- Usability Evaluation of Blaise/WVS, a Browser-Based CAPI Application Ryan Hubbard <ryanhubbard@westat.com> and Ruth Malloy <ruthmalloy@westat.com>, Westat
- Usability Testing of Alternative Design Features for Web-Based Data Collection: Selected Issues, Results and Recommendations, Authors are Elizabeth D. Murphy <elizabeth.d.murphy@census.gov> and Susan M. Ciochetto <susan.m.ciochetto@census.gov>, U. S. Census Bureau
Coordinators: Sue Ellen Hansen <sehansen@umich.edu> and Kath Straub <kath.straub@gmail.com>.
WEDNESDAY AFTERNOON – March 15, 1:30 – 4:30 pm
Concurrent Sessions
WP-1. The Web in Survey Research - Going Beyond the Simple Web Survey.
As the Web-based survey matures as a mode of data collection, we find that the Web in general is also pushing us into new possibilities for the support of and conduct of self-administered data collection. Sample management, respondent support, data reporting, collecting new kinds of data, are all areas where the Web is now beginning to flourish. So no longer can one be so clearly understood when they say they are involved in Web surveys… it now may mean more than the originally conceived scrolling or interactive form interface that presents a questionnaire to a respondent. In this session you will hear about new innovations and applications, new tools and infrastructures, and a variety of Web related topics that all push the Web to the next level in survey research.
| Time | Topic | Presenters | Materials |
|---|---|---|---|
| 1:30 - 1:35pm | WELCOME / INTRODUCTION | ||
| 1:35 - 2:00pm | Distributed CAPI Interviewing with Blaise IS for the Web for the Adolescent Trials Network | Rick Mitchell <rayc1@westat.com> (Westat) | presentation (, 6.6mb) |
| 2:00 - 2:25pm | E-Mail Data Collection: Can it Work? | Richard Rosen, Hong Yu, Antonio Gomes (Bureau of Labor Statistics) | |
| 2:25 - 2:50pm | Integrating Interactive Geographical Maps in Web-based Surveys | Brian Hempton, Scott D. Crawford (Survey Sciences Group, LLC), Robert F. Saltz (Prevention Research Center) | presentation (, 42kb) |
| 2:50 - 3:05pm | BREAK | ||
| 3:05 - 3:30pm | Customization in Internet Surveys: Making Large Surveys Seem Smaller | John M. Kennedy (Indiana University Center for Survey Research) | presentation (, 329kb) |
| 3:30 - 3:55pm | NYC Department of Health and Mental Hygiene: Online Panel Management System | Kevin M. Kelly, Steve Young (ORC Macro) | presentation (, 821kb) |
| 3:55 - 4:30pm | PANEL DISCUSSION / QUESTIONS |
Coordinators: Scott Crawford <scott@surveysciences.com>.
WP-2. Mobile Survey Computing, Including Wireless and Pen Based Environments.
In this session, we will address issues regarding the design and implementation of mobile computing devices in survey data collection. Presenters will discuss lessons learned in implementation as well as results of research efforts.
Coordinator: Jean Fox <fox.jean@bls.gov>.
WP-3. Multi-mode CASIC Surveys.
Multimode CASIC surveys are increasingly supplanting single-mode surveys in an effort to address single-mode limitations including response rate declines, and technological and legal obstacles. Multiple survey modes offer the respondent a choice of mode, and for the survey organization differing channels of communication, and the ability to play off one mode against another. Yet multimode surveys are inherently more complex and challenging to field than their single-mode counterparts.
This session focuses on methodological and technical issues that arise in fielding multimode surveys. Maximizing response rate and reducing (or at least controlling) costs are major themes of this session. Within those themes specific topics may include instrumentation, survey management, survey operations, cost modeling, role of survey personnel, incentives within the context of multiple modes, and data quality and comparability.
| Time | Topic | Presenters | Materials |
|---|---|---|---|
| 1:30 - 1:35pm | INTRODUCTION TO SESSION | ||
| 1:35 - 2:00pm | A Multi-mode CATI-Web Survey Experience with Blaise | Craig Ray <rayc1@westat.com> (Westat) | presentation (, 452kb) |
| 2:00 - 2:25pm | Using a Multi-Mode Design to Maintain Response Rates | Duston Pope (Market Strategies, Inc.), Amy Vincus, Sean Hanley (Pacific Institute for Research and Evaluation), Scott Crawford (Survey Sciences Group, LLC) | presentation (, 680kb) |
| 2:25 - 2:50pm | Telephone Collection as Part of a Multimode Survey | Mark Pierzchala, Debra Wright (Mathematica Policy Research, Inc), Claire Wilson (Insight Policy Research), Paul Guerino (Education Statistics Services Institute) | presentation (, 230kb) |
| 2:50 - 3:00pm | BREAK | ||
| 3:00 - 3:25pm | Implementing the Canadian Multimode CATI and CAPI Labour Force Survey | Eric Joyal <eric.joyal@statcan.can> (Statistics Canada) | presentation (, 8.6mb) |
| 3:25 - 3:50pm | Challenges of Designing and Implementing Multimode Instruments | Melissa Cominole <mcominole@rti.org> (Research Triangle Institute) | presentation (, 2.6mb) |
| 3:50 - 4:15pm | Planning for a Multimode Data Collection System at Statistics Canada | Chad Mackinnon (Statistics Canada) | |
| 4:15 - 4:30pm | DISCUSSION AND QUESTIONS |
Coordinators: Mark Pierzchala <mpierzchala@mathematica-mpr.com>.
WP-4. Capacity Planning (time and staff management and related organizational issues).
For organizations running CATI call centres, capacity planning means ensuring that interviewing capacity will be available when it is needed. Workload fluctation (number of surveys, sample sizes) is only one element involved in managing and scheduling interviewers. Unionization leads to additional challenges. Some surveys require interviewers with specialized skills (eg. ability to speak other languages) or with certain characteristics (gender, age). Use of "best time to call" and placing controls on when calls are made to cases can limit the number of cases available at a given time of day, thereby impacting the number of interviewers scheduled to work. On the other hand, information from call history files can be used to better plan work schedules and to identify patterns of low productivity.
This session will be for participants to discuss the challenges they currently face, and the information, tools, and strategies that could be used to help make decisions.
Coordinator: Lecily Hunter <lecily.hunter@a.statcan.ca>.
THURSDAY MORNING – March 16, 9:00 am -12:00 noon
Concurrent Sessions
TA-1. Security Issues in Federal Data Collection.
This session will feature presenters and discussion centered around current issues of security in the data collection environment including such topics as voice over IP; firewalls; security in non-centralized research environments; mobile computing; authentication methods; spam, viruses and trojans; and other related topics.
The session will begin with a broad overview of security issues followed by more focused presentations on specific areas that are of current concern with time allowed for questions and spirited discussion.
Each attendee will receive a free yellow sticky note with room for multiple passwords that you can stick near your computer when you return home.
Presentation Materials:
- IT Security Challanges for Distributed Science (, 1.8mb), Douglas Gatchell <dgatchell@nsf.gov> - National Science Foundation
Coordinator: Bill Connett <bconnett@isr.umich.edu>.
TA-2. New and Emerging Technologies, Including Those Based on XML.
This session will focus on new technologies which offer solutions to persistent survey problems or extend the capabilities of survey systems. Based on the latest developments and available presenters, possible topics may include: collaborative research tools, including portals and other approaches to knowledge management, peer collaboration, and document search, and interactive data dissemination; automated data cleaning, editing and management tools, including current COTS offerings; integrated systems which combine data collection, metadata, and project management; advances in voice computing, including coding and post processing of voice and video recording; voice to text systems; word and phrase lookup in speech recognition, voice browsing and voice/web synchronization; and various messaging technologies.
Coordinators: David Uglow <duglow@rti.org> and Jim Kennedy <kennedy.jim@bls.gov>.
TA-3. Accessibility.
In this session, we will take a look at how Section 508 affects computer-assisted surveys. The session will include:
- An introduction to Section 508 and when it applies
- A demonstration of a screen reader (JAWS) and a screen magnifier, two adaptive devices commonly used to present web and software information to visually challenged users.
- Lessons learned in designing for accessibility
- Methods for incorporating accessibility into the development process.
Presentation Materials:
- Blaise IS and Accessibility (, 98kb), Jim O'Reilly <oreilly@westat.com> - Westat
- The Basics of Section 508 (, 115kb), Ben Rosset <rosset_b@bls.gov> - Bureau of Labor Statistics
- Accessible Web Survey Design (, 1.4mb), Lawrence A. Malakhoff <lawrence.a.malakhoff@census.gov> - U.S. Census Bureau
Supplemental Materials:
- SSA Guide to Applying Section 508 Standards for Application Developers (, 997kb)
- SSA Guide to Accessible PDF Documents (, 6.8mb)
- SSA Guide to Applying Section 508 Standards (, 1.5mb)
Coordinators: Jean Fox <fox.jean@bls.gov> and Jim O’Reilly <oreilly@westat.com>.