Skip Header

Working Paper Number RSM2018-07
Elizabeth M. Nichols, Brian Falcone, Ivonne Figueroa
Component ID: #ti77671113


In 2016, staff in the Center for Survey Measurement of the U.S. Census Bureau usability tested the 2017 National Survey of College Graduates (NSCG) online survey. During the usability sessions, respondents accessed the survey either on Census provided laptop computers or on their own mobile devices, including tablets and smartphones. This report documents the observations concerning the use of a grid response format with small mobile devices. A grid response format is a matrix where there is one question stem; each row heading contains a unique question topic or item, and the column headings contain a response option or value. Several questions on the NSCG paper form used grids and for mode consistency purposes, grids were used for those same questions for the NSCG online reporting option. During usability testing, we did not observe any difficulty with respondents using grids while answering on a PC. However, when respondents accessed the survey via a mobile phone, the grids were more difficult to use because respondents had to scroll back and forth horizontally to see the question items and all the response options. Because of this difficulty, halfway through the usability testing, the matrix design was changed to an item-by-item design for smaller mobile devices. In this design, each question that was originally presented in the grid became a stand-alone item with the response options placed vertically underneath the item as a series of radio buttons. While this design increased the amount of vertical scrolling needed on the page to see and answer each question item, the usability of the question improved with the elimination of the horizontal scrolling. Other researchers have also documented the advantages of an item-by-item solution for grids on mobile (de Bruijne 2015; Borger and Funke 2015; McClain & Crawford, 2013; Lattery & Park Bartolone, 2013). This report provides another source for documenting the item-by-item solution for grids on mobile as it relates to usability and user experience. This report does not make any claims about the impact of the design on the resulting survey response distributions as the research was qualitative in nature. 

  Is this page helpful?
Thumbs Up Image Yes    Thumbs Down Image No
Comments or suggestions?
No, thanks
255 characters remaining
Thank you for your feedback.
Comments or suggestions?
Back to Header