Presentation (Project) Title

The Influence of Examiner Fidelity on First Graders’ Oral and Written Measure

Mentor Information

Trina Spencer (College of Behavioral & Community Sciences)

Presentation Format

Event

Abstract

The extent to which examiners adhered to assessment protocols can influence the outcomes of a study. Yet, assessment fidelity data are rarely reported in evaluation studies (Reed et al., 2014). Little is known about the extent to which examiner adherence to standardization procedures influence student outcomes on oral and written language measures. Using extant data from a randomized group study investigating the effects of an academic language intervention on 155 first grade students’ oral and written language skills, we examined assessment fidelity. Expository oral retell (ERO) and narrative writing (NGW) language samples were collected from students at three time points: pretest, posttest and follow up. Research assistants, who were blind to student group assignments, independently transcribed and scored samples. About 30% of samples were randomly selected for fidelity examination, which involved another research assistant listening to audio recordings of the data collection session. Fidelity and student outcomes for ERO and NGW measures were analyzed using a two-level hierarchical linear model (HLM) with participant data (Level 1) nested within schools (Level 2) across three time points (pretest, posttest, and follow-up). We hypothesized that low examiner NGW and ERO fidelity to assessment standardization will negatively affect student outcomes.

Streaming Media

This document is currently not available here.

Share

COinS
 

The Influence of Examiner Fidelity on First Graders’ Oral and Written Measure

The extent to which examiners adhered to assessment protocols can influence the outcomes of a study. Yet, assessment fidelity data are rarely reported in evaluation studies (Reed et al., 2014). Little is known about the extent to which examiner adherence to standardization procedures influence student outcomes on oral and written language measures. Using extant data from a randomized group study investigating the effects of an academic language intervention on 155 first grade students’ oral and written language skills, we examined assessment fidelity. Expository oral retell (ERO) and narrative writing (NGW) language samples were collected from students at three time points: pretest, posttest and follow up. Research assistants, who were blind to student group assignments, independently transcribed and scored samples. About 30% of samples were randomly selected for fidelity examination, which involved another research assistant listening to audio recordings of the data collection session. Fidelity and student outcomes for ERO and NGW measures were analyzed using a two-level hierarchical linear model (HLM) with participant data (Level 1) nested within schools (Level 2) across three time points (pretest, posttest, and follow-up). We hypothesized that low examiner NGW and ERO fidelity to assessment standardization will negatively affect student outcomes.