Graduation Year

2012

Document Type

Dissertation

Degree

Ph.D.

Degree Granting Department

Educational Measurement and Research

Major Professor

John M. Ferron

Co-Major Professor

Robert F. Dedrick

Keywords

educational accountability, Gain Score Model, Layered Effects Model, sensitivity analysis

Abstract

The Adequate Yearly Progress (AYP) Model has been used to make many high-stakes decisions concerning schools, though it does not provide a complete assessment of student academic achievement and school effectiveness. To provide a clearer perspective, many states have implemented various Growth and Value Added Models, in addition to AYP. The purpose of this study was to examine two Value Added Model specifications, the Gain Score Model and the Layered Effects Model, to understand similarities and differences in school effect results. Specifically, this study correlated value added school effect estimates, which were derived from two model specifications and two outcome measures (mathematics and reading test scores).

Existing data were obtained from a moderately large and rural school district in Florida. The outcome measures of 7,899 unique students were examined using the Gain Score Model and the Layered Effects Model to estimate school effects. Those school effect estimates were then used to calculate and examine the relationship between school rankings.

Overall, the findings in this study indicated that the school effect estimates and school rankings were more sensitive to outcome measures than they were to model specifications. The mathematics and reading correlations from the Gain Score Model for school effects and school rankings were low (indicating high sensitivity), when advancing from Grades 4 to 5, and were moderate in other grades. The mathematics and reading correlations from the Layered Effects Model were low at Grade 5 for school effects and school rankings, as were the correlations at Grade 7 for the school rankings. In the other grades, correlations were moderate to high (indicating lower sensitivity). Correlations between the Gain Score Model and the Layered Effects Model from mathematics were high in each grade for both school effects and school rankings. Reading correlations were also high for each of the grades.

These results were similar to the findings of previous school effects research and added to the limited body of literature. Depending upon the outcome measure used, school effects and rankings can vary significantly when using Value Added Models. These models have become a popular component in educational accountability systems, yet there is no one perfect model. If used, these models should be used cautiously, in addition to other accountability approaches.

Download

Share

COinS