Skip Header

Communicating Data Use and Privacy: In-Person Versus Web-Based Methods for Message Testing

Sat May 14 2016
Written by: Aleia Clark Fobia and Jennifer Hunter Childs, Center for Survey Measurement
Component ID: #ti1551385569

Communicating messages about privacy, data use and access, and confidentiality is critical to earning and keeping the trust of respondents and to ensuring their willingness to participate in surveys. Informing respondents about their rights and how their data will be used is often required by law. However, there is currently not much data available on how respondents react to these messages or how they understand the meanings we try to convey.

Component ID: #ti1033196165

Our research addresses this gap in our knowledge about respondents’ understanding of an intended message. We focused our research on sets of messages that convey different types of information. The sets of messages informed respondents of a range of factors including:

  • Who has access to survey responses.
  • Survey responses are confidential.
  • Data are for statistical use only.
  • Individuals will not be identifiable when statistical data are released.
  • Responses will not be shared with law enforcement or used for eligibility for government benefits.
  • Data are sometimes shared with other federal agencies.

Component ID: #ti764066195

Four additional sets of messages tested statements that the Census Bureau is legally required to provide, such as the mandatory nature of the census, confidentiality protections, burden notifications and other language from the Paperwork Reduction Act.

Component ID: #ti764066194

We used both web-based and in-person methods to test respondent comprehension of messages about privacy and confidentiality. First, we used an open-ended internet instrument to collect qualitative data on respondent comprehension of these messages. Web-based testing was remote and respondents did not interact with an interviewer. We analyzed these data to identify high and low performing messages. We then tested some of those messages in a smaller-scale, in-person cognitive test with 30 respondents.

Component ID: #ti764066193

Combining these two methods allowed for a larger scale data collection than is typical in a qualitative study while retaining the ability to elicit rich description and allow for spontaneous probing. This research not only helps us understand how respondents comprehend our messages, but also facilitates exploration of web-based methods for testing survey questions.

Component ID: #ti764066192

One of the central limitations of in-person interviewing is the difficulty of obtaining respondent diversity. In-person interviewing is also costly in both labor hours and respondent compensation. However, this type of interviewing allows for considerable interviewer flexibility. Through direct comparison with more traditional methods, this research highlights the advantages and limitations of using alternatives to traditional in-person cognitive interviewing. For an update with our findings, come see our presentation at the American Association for Public Opinion Research on Saturday, May 14, 2016, in Austin, Texas.

  Is this page helpful?
Thumbs Up Image Yes    Thumbs Down Image No
Comments or suggestions?
No, thanks
255 characters remaining
Thank you for your feedback.
Comments or suggestions?
Back to Header