Examining the Meaning of Vague Quantifiers in Higher Education: How Often is “Often”?
Document Type
Article
Publication Date
1-2020
Keywords
Survey research, Response options, Vague quantifiers, Convergent validity
Digital Object Identifier (DOI)
https://doi.org/10.1007/s11162-020-09587-8
Abstract
Researchers, assessment professionals, and faculty in higher education increasingly depend on survey data from students to make pivotal curricular and programmatic decisions. The surveys collecting these data often require students to judge frequency (e.g., how often), quantity (e.g., how much), or intensity (e.g., how strongly). The response options given for these questions are usually vague and include responses such as “never,” “sometimes,” and “often.” However, the meaning that respondents give to these vague responses may vary. This study aims to determine the efficacy of using vague quantifiers in survey research. More specifically, the purpose of this study is to explore the meaning that respondents ascribe to vague response options and whether or not those meanings vary by student characteristics. Results from this study indicate a high degree of correspondence between vague and numeric response and suggest that students seem to adapt the meaning of “sometimes,” “often,” and “very often” based on the appropriate reference for the question. Overall, findings provide evidence of the utility and appropriateness of using vague response options. Some differences by student characteristics and the implications of these differences are discussed.
Higher education researchers, assessment professionals, and faculty increasingly rely on survey data collected from college students to make pivotal curricular and programmatic decisions (Picciano 2012). Typically, the questions on these surveys ask students to judge the frequency (e.g., how often), quantity (e.g., how much), or intensity (e.g., how strongly) of their activities, behaviors, or opinions. The response options given for these questions are usually vague and include responses such as “never,” “sometimes,” and “often.” Additionally, many national college student surveys use these ordered, natural-language response options known as vague quantifiers to understand student behaviors, activities, attitudes, and judgments (e.g., Beginning Postsecondary Studies Longitudinal Study, CIRP Freshman Survey, and National Survey of Student Engagement). However, there has been little research exploring how the use of vague quantifiers may influence results, especially in higher education research. Studying response format is important for ensuring that the data collected from college students is valid for the faculty, staff, and administrators making strategic decisions.
This study aims to determine the efficacy of using vague quantifiers. More specifically, the purpose of this study is to understand the meaning that college students ascribe to vague response options and to examine whether those meanings vary among different groups of students. Understanding if vague quantifiers vary by student demographics is essential because survey results are often used to compare subgroups and making such comparisons assumes that the meaning of the vague response options is consistent across the groups being compared. However, prior research has shown that vague responses may elicit varying meanings (Schaeffer 1991; Schneider and Stone 2016; Wänke 2002). For instance, Wänke (2002) noted that although respondents did not differ in their absolute response, their responses to the vague quantifier were influenced by the information about the target population. In other words, respondents recalibrated their responses based on how their own behaviors compared to the perception of what “often” looks like for the general survey population. Because of this possible variation, as well as other concerns, some researchers have questioned the validity of survey data derived from college students (Porter 2011). Given the frequent use of survey data in higher education to make curricular and programmatic changes (Picciano 2012), understanding the meaning that students give to survey questions using vague quantifiers is critical. This study aims to address some of these issues by examining the use of vague quantifiers in a large national survey of college students.
Was this content written or created while at USF?
Yes
Citation / Publisher Attribution
Research in Higher Education, v. 61, p. 229-247
Scholar Commons Citation
Rocconi, Louis M.; Dumford, Amber D.; and Butler, Breanna, "Examining the Meaning of Vague Quantifiers in Higher Education: How Often is “Often”?" (2020). Leadership, Counseling, Adult, Career and Higher Education Faculty Publications. 302.
https://digitalcommons.usf.edu/ehe_facpub/302
