Using the Rasch Model to Investigate Differential Item Functioning in the PET Reading Test

Project Number
Pearson-Using the Mixture Rasch Model

Project Duration
January 2017 - January 2018

Status
In-Progress

Abstract
This research proposal aims to investigate the latent class differential item function (DIF) in the receptive skills section (reading and listening) of the Preliminary English Test (PET) Academic. DIF is the differential cognitive load or functioning of test items for different groups of test takers of the same ability level (Aryadoust, Goh, & Lee, 2011). DIF analysis is conventionally intended to detect potential sources of construct-irrelevant variance (CIV) and explore whether a test constitutes a fair tool to estimate the language ability of test takers of all backgrounds. This stream of research in language and educational assessment has mainly focused on the role of certain manifest variables such as first language or race. Despite its usefulness, two major problems have persisted and led to some kind of stasis in this field: (1) a focus on certain manifest variables such as race and covariates for DIF analysis; and (2) the lack of a well-established theoretical (as opposed to speculative) explanation concerning the choice of covariates in DIF analysis. The present research proposal seeks to address these gaps by applying a mixture Rasch model (MRM; Rost, 1990) to the PET Listening and Reading Tests. MRM, which combines latent class analysis and Rasch measurement, has achieved success in educational assessment, but has rarely been used in language assessment. In my previous research, I have shown that MRM is specifically useful in the assessment of listening and reading comprehension skills, since it factors in the differences of individuals and helps researchers to ground the results within well-established reading and listening theoretical models (see Aryadoust & Zhang, 2016, for reading, and Aryadoust, 2015, for listening). I aim to extend this line of research by developing a cognitive theory of DIF in this proposed study. To do so, I leverage three sophisticated techniques, which are further discussed below: MRM, neural networks, and computational linguistics. In what follows, I will initially review the theoretical background of the proposed study alongside DIF analysis using manifest variables (covariates) and MRM. I will then present research questions and discuss method and data analysis.

Funding Source
Pearson English

Related Projects