News and Tribune

June 11, 2013

Education head hires company to examine ISTEP results; local problems prevalent

By MAUREEN HAYDEN and JEROD CLAPP
newsroom@newsandtribune.com

INDIANAPOLIS —

Indiana Superintendent of Public Instruction Glenda Ritz has hired an outside expert to determine the validity of ISTEP+ test scores of nearly 80,000 students who were knocked offline while taking the high-stakes standardized test.

Thousands of students in Clark and Floyd counties were affected by system kick-offs and other issues while taking the multiple-choice section of the test in April and May, officials say.

Karen Spencer, supervisor for assessment in Greater Clark County Schools, said about 2,000 students were kicked off the system while testing or experienced other issues. About 4,800 students in the district were required to take the test.

She said along with giving serious test-takers additional anxiety in completing the test and possibly affecting their ability to do their best, she said this year’s hiccups give her concern about how the results could affect the district.

“If kids didn’t do their best, it will reflect not only on the school, but on the district and A through F grades [for schools],” Spencer said. “The growth model depends on how they performed this year over last year on that one test. This multiple-choice test weighs in on the applied skills test they took [in March].”

John Reed, assistant superintendent of West Clark Community Schools, said 648 out of 2,173 students in his district were reported to the Indiana Department of Education for problems they experienced during testing. He said some students saw questions repeated after they already answered them along with the other issues other districts had.

“We had so many issues that I don’t want to say that it was limited to a certain amount of kids,” Reed said. “We had issues in every school in our district [that takes ISTEP]. In one way or another, our students experienced disruptions. If it wasn’t one student, it was the ones next to them that had distress, and that put the ones without problems in distress.”

He said even if the state reviews the results of students who experienced issues, he thinks teachers and school districts won’t be happy if those results are admitted for accountability purposes.

 

MONEY PROBLEMS

But Spencer said with federal funding tied to those results, she’s not sure what the solution is to the problem of admitting those results.

“I know federal dollars depend on your accountability status, so I’m not really sure how that’s all going to play out,” Spencer said. “Their hands are kind of tied up there on what they have to do for their end, but I don’t know the solution. I think it’s going to be hard for everyone.”

In Clarksville Community Schools, a number of students experienced the same issues as others across the state, Superintendent Kim Knott said. She said even though the state’s working to investigate the results, she’s doesn’t see how accountability can justifiably fit in with this year’s results.

“I think we’re all just going to have to sit back and wait to see what the [Department of Education] does about this,” Knott said. “I know they sent out a press release about their plans, but we’re all going to have to see how it shakes down. I just can’t imagine that it won’t have an impact on test validity, especially as it relates to A-through-F scores.”

She said she wasn’t sure how many students were affected overall in her district, but she said her schools reported several more than IDOE did. 

In New Albany-Floyd County schools, 3,239 students were reported to IDOE as either being completely kicked off, encountering audio issues or other problems with logging on, Director of Assessment and Student Information Sally Jensen said. 

“When things don’t go quite as we want them to, it takes a lot of flexibility, a lot of patience and a lot of good people that work for us to figure out how we’re going to get the job done,” Jensen said. 

 

THE COMPANY

Ritz announced Monday that the state Department of Education has contracted with New Hampshire-based National Center for the Improvement of Educational Assessment to analyze the results of tests interrupted by computer server problems at CTB-McGraw Hill, the test administrator. 

Ritz said the need for an independent review was critical given that the test results affect teacher pay, school ratings and student placement. 

“Because the stakes of this test are so high, the results must be beyond reproach,” Ritz said.

Nearly one in six Indiana students who took the ISTEP+ test this spring experienced some kind of disruption during the online test, Ritz said. Some students were booted off for a few seconds before they could log on again, while others experienåced longer outages.

Ritz said the “alarmingly high volume of test interruptions” was frustrating for parents, students and teachers alike. 

“These interruptions were simply unacceptable and they call into question the validity of the test scores,” Ritz said. 

The state Department of Education will pay $53,600 to National Center for the analysis, to be done independently of a similar review being conducted by CTB-McGraw Hill. Results from the analysis are expected by mid-July. 

Ritz stopped short of saying whether any of test results will be tossed out, as some school administrators have called for. But the Democrat Ritz made clear her disapproval of the weight the testing now carries, under measures passed by the Republican-controlled legislature, in determining such things as teacher compensation and the grades schools get under the state’s A-F accountability system. 

Reed said if the state decides to admit the test results into accountability standards, he hopes districts will get an opportunity to see how the results were assessed by National Center.

 

EVALUATING THE SITUATION

Ritz said the standardized test is no longer used as intended — for measuring student learning. “I’m hoping that the state of Indiana wants to reduce the high stakes attached to this test,” Ritz said.

She’s already told local districts that they have the option to downgrade the significance of the test scores in regards to teacher evaluations. 

House Education Committee Chairman Bob Behning, an Indianapolis Republican who supported the testing measures that Ritz opposes, said it’s too early to make any decisions about throwing out test scores. 

“Let’s not get too far out on the issue, before we even know where we stand,” said Behning, who will lead a legislative review of the testing problems this summer. 

“Everybody in the General Assembly is very aware of what happened and we’re very concerned,” he added.

After reports of computer problems during the test-taking period in late April, the state stepped in and extended the test period into May. In all, about 482,000 students completed the ISTEP+ test, most without experiencing problems. 

To determine the validity of the tests taken by students who experienced the computer problems, National Center will compare student test answers pre- and post-interruption, and look back at prior-year test scores to statistically determine the validity of this year’s results.

Whether the test administrator, CTB-McGraw Hill, will be asked to repay the state for the analysis is yet to be determined. Ritz said the state has several options, including financially penalizing the company under its four-year, $95 million contract with the state. 

Indiana was one of at least three states that had major problems with CTB-McGraw Hill this spring related to the standardized tests that are federally mandated. The company, which controls about 40 percent of the testing market, issued an apology in May, saying it regretted the impact of “system interruptions” that caused delays for thousands of test-taking students. 

It was the third straight year that Indiana students experienced service interruptions during online testing administered by the company.