News and Tribune

Education/Schools

July 29, 2013

7P.M. UPDATE: Consultant: ISTEP scores not largely affected by computer problems

Education head still plans to let school districts decide on teacher evaluations

INDIANAPOLIS — The testing expert hired by the state to determine the impact of computer problems that plagued the online ISTEP+ assessment said he was surprised by his findings that the computer glitches had no negative impact on student test scores.

Richard Hill, co-founder of the National Center for Improvement of Educational Assessment, told the state’s Education Commission on Monday that student scores actually improved this year, despite repeated computer problems that pushed about 80,000 students offline while they were taking the high-stakes assessment test.

Hill said that the vast majority of students who took the test scored as well as they would have had the interruptions never happened.

It wasn’t news that many commission members expected, nor news that was particularly welcomed by Indiana Superintendent of Public Instruction Glenda Ritz, who hired Hill in June and paid him $53,000 to do the study.

Ritz has been critical of the weight given to ISTEP tests; they play a key role in teacher evaluation and compensation, and in the “A to F” rating system that gives a grade to every school in the state. Schools that consistently score poorly are at risk of being taken over by the state, while schools that score well are in line for more state funding.

“I was expecting to find several points lost at all grade levels,” Hill said, after the commission meeting.

Instead, after analyzing ISTEP tests taken by all 495,000 students in grades three through eight this spring, he found just the opposite: An across-the-board increase in test scores that almost mirror the increases of the past four years.

“I don’t know what the reaction is going to be,” Hill said of his analysis. “I’m sure it’s going to be one of general disbelief.”

It seemed that way among some commission members Monday, who grilled Hill about how he did his analysis and speculated that some students may have actually benefited by having their tests interrupted, giving them more time to figure out test answers.

Hill rejected that notion and said he analyzed the data in many different ways to find some flaw in the test results. He found, for example, that students who were bumped offline for a few minutes didn’t fare much differently than students whose tests were postponed for a day.

“I cut the data a lot of different ways,” Hill said. “When you keep coming back to it, it’s clear: Student scores rose.”

Hill’s conclusions run counter to what Ritz had anticipated when she ordered the ISTEP analysis in May, after computer problems with the testing company, CTB-McGraw Hill, brought the ISTEP tests to halt in late April. The company later apologized for the problems, which it attributed to lack of memory space on their servers.

Ritz and other educators feared student test scores would be negatively impacted by the repeated interruptions and delays in the testing.

Officials in some school districts had called on Ritz to throw out the ISTEP test results. While she had stopped short of that, she did tell local districts in June that they could minimize the weight of the ISTEP results in determining teacher evaluations and pay. On Monday, she said she was sticking by that decision.

“I have spent the last several months talking with Hoosiers about the impact these interruptions had in the classroom,” Ritz said. “Although Dr. Hill’s report found that the statewide average score was not affected by the interruptions, there is no doubt that thousands of Hoosier students were affected.”

That decision by Ritz — a Democrat who’s been critical of the use of ISTEP scores in teacher evaluations — didn’t sit well with Republican House Education Committee Chairman Bob Behning of Indianapolis.

Behning has previously said that Ritz should have waited until the ISTEP analysis was completed.

“Now it appears she mis-shot,” Behning said.

Individual ISTEP results have yet to distributed to students and schools. Ritz said the DOE is now processing ISTEP student results, to be available online to parents and students by late August. In the meantime, the DOE is in negotiations with the testing company, CTB-McGraw Hill, to cover the cost of Hill’s analysis and to pay additional penalties for allegedly violating its contract with the state.

In his analysis, Hill credited students for recovering quickly from whatever stress the interruptions the computer problems may have caused. He repeated that sentiment after Monday’s meeting, saying the ISTEP test may cause more stress for adults than students.

“The are high stakes for the adults,” Hill said. “But the kids don’t have same level of stakes and so don’t perceive it that same way. So when there is a glitch in the system, the adults tend to react more strongly to it than the kids do.”

1
Text Only | Photo Reprints
Education/Schools
LOCAL MAGAZINES
2013 Photos of the year


Take a look at our most memorable photos from 2013.

SPECIAL CONTENT
Twitter Updates
Follow us on twitter
Hyperlocal Search
Premier Guide
Find a business

Walking Fingers
Maps, Menus, Store hours, Coupons, and more...
Premier Guide
AP Video
Raw: More Than 100,000 Gather for Easter Sunday Raw: Greeks Celebrate Easter With "Rocket War" Police Question Captain, Crew on Ferry Disaster Raw: Orthodox Christians Observe Easter Rite Ceremony Marks 19th Anniversary of OKC Bombing Raw: Four French Journalists Freed From Syria Raw: Massive 7.2 Earthquake Rocks Mexico Captain of Sunken SKorean Ferry Arrested Raw: Fire Destroys 3 N.J. Beachfront Homes Raw: Pope Presides Over Good Friday Mass Raw: Space X Launches to Space Station Superheroes Descend on Capitol Mall Man Charged in Kansas City Highway Shootings Obama Awards Navy Football Trophy Anti-semitic Leaflets Posted in Eastern Ukraine Raw: Magnitude-7.2 Earthquake Shakes Mexico City Ceremony at MIT Remembers One of Boston's Finest Raw: Students Hurt in Colo. School Bus Crash Deadly Avalanche Sweeps Slopes of Mount Everest
2013 Photos of the year


Take a look at our most memorable photos from 2013.