Mishaps give wrong score for right answer

Last May, a few weeks before commencement, Jake Plumley was pulled out of the classroom at Harding High School in St. Paul, Minnesota, and told to report to his guidance counselor.

The news was grim: Jake, a senior, had failed a standardized test required for graduation. To try to salvage his diploma, he had to give up a promising job and go to summer school. "It changed my whole life, that test," Jake recalled.

Jake actually had passed the test. But the company that scored it had made an error, giving Jake and 47,000 other Minnesota students lower scores than they deserved.

An error like this -- made by NCS Pearson, the nation's biggest test scorer -- is every testing company's worst nightmare. One executive called it "the equivalent of a plane crash for us."

But it was not an isolated disaster. The testing industry's missteps have affected millions of students who took standardized proficiency tests in at least 20 states.

An examination of recent mistakes and interviews with more than 120 people involved in the testing process suggest that the industry cannot guarantee the kind of error-free, high-speed testing that parents, educators and politicians seem to take for granted.

Now President Bush is proposing a 50 percent increase in the workload of this tiny industry -- a handful of giants with a few small rivals. If Congress approves, every child in grades 3 to 8 will be tested each year in reading and math. Neither the Bush proposal nor the congressional debate has addressed whether the industry can handle the daunting logistics of this additional business.

Already, a growing number of states use these so-called high-stakes exams -- not to be confused with the SAT, the college entrance exam -- to determine whether students in grades 3 to 12 can be promoted or granted a diploma. The test results are also used to evaluate teachers and principals and to decide how much tax money school districts receive. How well schools perform on these tests can even affect property values in surrounding neighborhoods.

The recent flaws all occurred as the testing industry was struggling to meet demands from states to test more students, with custom-tailored tests of greater complexity, designed and scored faster than ever.

In recent years, the four testing companies that dominate the market, NCS Pearson, Harcourt Educational Measurement, CTB/McGraw-Hill and Riverside Publishing, have experienced serious breakdowns in quality control. Problems at NCS, for example, extend beyond Minnesota. In the last three years, the company produced a flawed answer key that incorrectly lowered multiple-choice scores for 12,000 Arizona students, erred in adding up scores of essay tests for students in Michigan and was forced with another company to rescore 204,000 essay tests in Washington because the state found the scores too generous. NCS also missed important deadlines for delivering test results in Florida and California.

"I wanted to just throw them out and hire a new company," said Christine Jax, Minnesota's top education official. "But then my testing director warned me that there isn't a blemish-free testing company out there. That really shocked me."

One error by another big company resulted in nearly 9,000 students in New York City being mistakenly assigned to summer school in 1999. In Kentucky, a mistake in 1997 by a smaller company, Measured Progress of Dover, New Hampshire, denied US$2 million in achievement awards to deserving schools. In California, test booklets have been delivered to schools too late for the scheduled test or arrived with missing pages.