State says flaws in exam for students with disabilities came from attempt to reduce test “burden”
June 3, 2016
By: Shaina Cavazos
Source: chalkbeat.org
It’s not yet clear how Indiana plans to resolve problems with its test for students with severe disabilities, but state education officials say they’ve learned a lesson.
The test, which was given to approximately 7,000 students with severe cognitive disabilities across the state this year, had two major design flaws according to a presentation to the Indiana State Board of Education — having too few questions and an unreliable test scale. Because of those problems, a testing advisory committee recommended Tuesday night that the scores not be used in calculations for 2016 school A-F accountability grades.
Now, testing company Questar has developed a proposal for how to move forward, which will be examined by the advisory committee on Thursday. But it’s already clear to state officials that some problems with the test could have been avoided — especially the test having too few questions to be considered valid.
Michele Walker, testing director for the Indiana Department of Education, said the department didn’t require more questions when the test was first created because it was worried about burdening students with tests that were too long.
“I think that would be one of those areas that would be a lesson learned, both for our vendor and for ourselves,” Walker said. “Because the time was limited to create the items and because we didn’t want to have an overabundance of items for this group of students, we were extremely conservative.”
Other problems with the exam, which was given for the first time this year, had to do with the way it was split up into three shorter tests that were given throughout the year, with the first test being the easiest and the third being the most difficult.
The scale used to assign scores to students wasn’t properly designed to accommodate the variations in the difficulty levels of the three sections. That could make the scores hard to interpret and could prevent educators from comparing student scores from one year to the next.
Walker said too little time was a primary challenge in creating ISTAR — a reason given for many of Indiana’s recent testing debacles as lawmakers have voted to quickly switch from test to test. The state had about six months to create ISTAR, versus the almost three years it took to create and implement the test the state used for students with special needs in 2015, Walker said, a fairly well-regarded national test from the National Center and State Collaborative.
On Tuesday, Katie McClarty, the chief assessment officer for Questar, suggested some options for how Indiana could move forward, such as releasing score reports that show what “levels” students performed at and what skills are typically mastered at those levels.
Cynthia Roach, the state board’s testing director, said Wednesday that the idea will be considered, along with others that might address the problems with the test’s scale, so that schools aren’t penalized for faulty results.
“If Questar’s proposal isn’t approved … we have to look at alternate means for some sort of ‘hold harmless’ for ISTAR results,” Roach said. “We’ve got some ideas we’ve been bouncing around.”
Questar will be presenting to the state’s advisory committee on Thursday. That committee will oversee any changes to the test, which is scheduled to be administered again in 2017. The state paid Questar about $2.5 million per year to develop the test.
Whatever the advisory committee decides will likely need to come back before the board in July, Roach said.
Board members seemed concerned that it took so long for the exam flaws to be found. Ed Roeber, a member of the advisory committee and a former testing director from Michigan, said the advisory committee, which was formed late last year, will work to help avoid such problems moving forward.
“We do get called on to fix things after the fact,” Roeber said. “But it’s a whole lot better for everyone involved if we get called on before.”