Graduation Year

2019

Document Type

Ed. Specalist

Degree

*Ed.S.

Degree Name

Education Specialist (Ed.S.)

Degree Granting Department

Curriculum and Instruction

Major Professor

Robert F. Dedrick, Ph.D.

Committee Member

John Ferron, Ph.D.

Committee Member

Yi-Hsin Chen, Ph.D.

Keywords

item response theory 2-PL IRT-likelihood ratio test, Cochran-Mantel-Haenszel, differential item functioning, item bias, Logistic Regression, DIF

Abstract

The main goal of this study was to investigate differential item functioning by gender in the Fundamental Mathematics (FMS) and Mathematics subtests (MS) of the MSPC-2018 Higher Education Institutions Examination. Each test consists of 40 items and for both subtests random samples of 10.000 students were received from the MSPC separately. To compare non-IRT (Classical Test Theory) and Item Response Theory (IRT) approaches, Cochran-Mantel-Haenszel (C-M-H), Logistic Regression (LR), and 2-PL IRT-LR statistics were used.

For the FMS, C-M-H, LR, and 2-PL IRT-LR procedures identified 18, 16, and 10 out of 40 items that had DIF, respectively. Based on the non-IRT approaches, the items, which favor females, divided into three mathematics subtopics, which are number, algebra, and geometry. There were only two items, which were item 5 and item 11 in the number subtopic, in Category C (large DIF) based on ETS delta scale. On the other hand, the items, which favored males, divided into three mathematics subtopics, which were arithmetic, advanced math, and geometry. There were only two items, which were item 18 and item 29 in arithmetic and advanced math subtopics, respectively, in Category C based on the ETS delta scale. Based on 2-PL IRT-LR results, the items, which favored males, divided into same subtopics with non-IRT approach results.

For the FMS, females tend to outperform males in four-operation skills, whereas males have higher performance on higher level mathematics (i.e., problem-solving, analytical thinking) and arithmetic skills than females.

For the MS, C-M-H, LR, and 2-PL IRT-LR procedures identified 22, 18, and 9 out of 40 items that had DIF, respectively. Based on the non-IRT approaches, the items, which favored females, divided into three mathematics subtopics, which were number, arithmetic, and algebra. There were no items, that favored females, identified in Category C. On the other hand, the items that favored males, divided into two mathematics subtopics, which were advanced math and geometry. There were only two items, which were item 22 and item 30 in the advanced math and geometry subtopics, respectively, in Category C based on the ETS delta scale. Based on 2-PL IRT-LR results, for the nine items with DIF, item 1 favored females, whereas the other items favored male students.

To compare groups based on total scores, the two-group approach was used for both tests. After analyzing the items, which were flagged as DIF, item 10 in the FMS was identified as moderately difficult and not discriminating well item and items 16, 30, 31, and 37 in the MS were identified as very difficult and not discriminating well items. Therefore, those items were not categorized items with DIF, and they require revisiting.

Download

Share

COinS