Big Data, Learning Analytics, and Social Assessment Methods
Document Type
Article
Publication Date
2013
Keywords
big data, writing assessment, social pedagogy, datagogies, transfer, curriculum standardization, peer production, communal agency
Abstract
This article explores the value of using social media and a community rubric to assess writing ability across genres, course sections, and classes. Since Fall 2011 through Spring 2013, approximately 70 instructors each semester in the first-year composition program at the University of South Florida have used one rubric to evaluate over 100,000 student essays. Between Fall 2012 and Spring 2013, students used the same rubric to conduct more than 20,000 peer reviews. The rubric was developed via a datagogical, crowdsourcing process (Moxley, 2008; Vieregge, Stedman, Mitchell, & Moxley, 2012). It was administrated via My Reviewers, a web-based software tool designed to facilitate document review, peer review, and writing program assessment. This report explores what we have learned by comparing rubric scores by project and semester on five measures (Focus, Organization, Evidence, Style, and Format) by project, section, semester, and course and by comparing independent evaluators' scores with classroom teachers' scores on two assignments for two semesters. Findings suggest use of the rubric across genres, sections, and courses facilitates a high level of inter-rater reliability among instructors; illustrates ways a curriculum affects student success; measures the level of difficulty of specific writing projects for student cohorts; and provides a measure of transfer. WPAs and instructors may close the assessment loop by consulting learning analytics that reveal real-time, big-data patterns, which facilitate evidence-based curriculum decisions. While not an absolute measure of student learning or ability, these methods enable tentative mapping of students' reasoning, research, and writing abilities.
Rights Information
Was this content written or created while at USF?
Yes
Citation / Publisher Attribution
The Journal of Writing Assessment, v. 6, issue 1.
Scholar Commons Citation
Moxley, Joseph M., "Big Data, Learning Analytics, and Social Assessment Methods" (2013). English Faculty Publications. 120.
https://digitalcommons.usf.edu/eng_facpub/120
