Document Type

Conference Proceeding

Publication Date



It is well documented that students perform differentially on assessments if they are more or less sensitive to what the students have experienced through instruction (Airasian & Madaus, 1983; Madaus, Airasian, & Kellaghan, 1980; Ruiz-Primo, Shavelson, Hamilton, & Klein, 2002). This concern has been repeatedly raised by researchers who observed that student scores on large scale tests reflect factors other than the quality of instruction or educational programs implemented (Burstein, Aschbacher, Chen, & Lin, 1990; Commission on Instructionally Supportive Assessment, 2001; Leinhardt, 1983; Popham, 2006, 2007a, 2007b; Wiliam, 2007). Most research done on instructional sensitivity focuses on evaluating assessments already developed and administered as “after the fact,” but is silent on how to construct instructionally sensitive assessments. This paper aims to test the assessment development approach we proposed (Ruiz-Primo & Li, 2008) to systematically manipulate item characteristics to construct items varying in instructional sensitivity. The question that has guided the test of the approach is, what module characteristics can be systematically manipulated to develop items at different distances (close and proximal) that prove to be instructionally sensitive? Both judgmental and empirical evidence is provided to examine the appropriateness of this assessment development approach.