Has school destroyed your creativity and self-confidence? I'm working on a book called Recovering From School, to help you heal the damage caused. Join the Patreon or Newsletter to be notified about updates. Paid Patreon members will get early draft previews, as well as a free digital copy when it's done.

US: SAT Errors Raise New Qualms About Testing

The scoring errors disclosed this week on thousands of the College Board's SAT tests were made by a company that is one of the largest players in the exploding standardized testing business, handling millions of tests each year.

The mistakes, which the company, Pearson Educational Measurement, acknowledged yesterday, raised fresh questions about the reliability of the kinds of high-stakes tests that increasingly dominate education at all levels. Neither Pearson, which handles state testing across the country, nor the College Board detected the scoring problems until two students came forward with complaints.

"The story here is not that they made a mistake in the scanning and scoring but that they seem to have no fail-safe to alert them directly and immediately of a mistake," said Marilee Jones, dean of admissions at the Massachusetts Institute of Technology. "To depend on test-takers who challenge the scores to learn about system failure is not good."

These were not the first major scoring problems that Pearson has experienced. The company agreed in 2002 to settle a large lawsuit over errors in scoring 8,000 tests in Minnesota that prevented several hundred high school seniors from graduating. It also has made significant scoring errors in Washington and Virginia.

After those problems, company officials had assured clients that they had vastly improved their quality control. But the new problems on the October SAT turned out to be the most significant scoring errors that the College Board had experienced.

Pearson said yesterday that the SAT errors, which affected 4,000 students out of 495,000 who took the October test, arose partly because of excessive moisture that caused the answer sheets to expand before they were scanned at the company's large test-processing site in Austin, Tex.

Another factor, the company said, was that its scanners did not pick up some lightly marked answers.

The company said in a statement that it was taking steps to make sure that "this unfortunate situation will not happen again."

Chiara Coletti, the College Board's vice president for public affairs, said yesterday that the College Board has continuing confidence in Pearson.

"Pearson says they now understand the technical issues fully, and we know they can control for those issues now," she said. "We are confident of that because our operations people have been talking to their operations people steadily."

The College Board has said that most of the students affected had higher scores than were reported to colleges. The scores were off by as many as 400 points out of a possible 2,400 on the three-part exam covering mathematics, reading and writing, although most errors were smaller.

Pearson said yesterday that it had examined the scoring of all the subsequent SAT's, which were administered in November, December and January, and found no further problems.

But some critics were not reassured. Shawn Raider, the lawyer who represented the Minnesota families who successfully sued Pearson, questioned whether the company had made good on its promise to improve its procedures.

"They certainly said in the course of our lawsuit that they not only were going to, but already had, implemented new quality control measures," he said.

The Pearson testing unit, a subsidiary of Pearson PLC, the giant publishing company that also owns The Financial Times, was awarded the contract for scanning the SAT answer sheets in 2003, taking over some functions previously performed for the College Board by the Educational Testing Service. They began the work last year.

It was one of many contracts that have helped make Pearson a giant in a field that has grown enormously since President Bush signed the No Child Left Behind Law in 2002, spurring demand for state testing.

For 20 years, Pearson has worked on the Texas testing program that was the template for Mr. Bush's national testing initiative.

Nationally, the company scored more than 300 million pages of answers last year and about 40 million individual tests.

Even as the company explained what went wrong yesterday, new complaints emerged from students and educators who questioned how they could continue to have confidence in the nation's testing apparatus.

Joe Giglio, director of admission at St. Peter's College in Jersey City, said, "It seems that there is a need for some sort of outside auditing of their processes to insure the integrity of the testing from this point forward."

Philip Benoit, a spokesman for Franklin & Marshall College in Lancaster, Pa., said yesterday that at least one applicant whose SAT score was revised upward by more than 100 points, now qualified for the school's merit-based Marshall Scholarship of $12,500.

Beatrice Bradley, a senior at the Williams School in Connecticut, who discovered that her reported score on the writing section of the SAT exam should have been 700 instead of 690, said one of her friends had also had an Advanced Placement score increased last year after raising questions about it.

"You have to wonder how many things go unchecked," she said.

The SAT errors, which the College Board started to investigate only after two students questioned the scores they received in late December, were not unprecedented.

As testing expanded sharply in the last decade, many more errors have occurred and almost all of them have been detected by students, parents or school officials challenging the accuracy of scores. Pearson said yesterday that it did not learn of the SAT problems until early February.

Some testing industry executives acknowledged yesterday that the SAT errors will add to the pressures the industry is already facing.

"There's no question that the testing industry is challenged," said Stuart R. Kahl, president and chief executive of Measured Progress, a nonprofit testing publisher in Dover, N.H., that provides testing services to 24 states. "But with the growth in business, most companies are implementing systems to make this job doable, so I don't get a sense that there is likely to be an exponential growth in errors."

But Mr. Kahl said standardized tests at all educational levels were constantly being revised. "The SAT's have been undergoing a lot of changes," he said yesterday. "And when you're putting out new forms of tests every year, the challenges are tremendous."

Some testing critics like FairTest, a nonprofit organization that opposes most uses of standardized testing, also raised questions about the College Board's selection of Pearson to handle scoring given its history of problems. "Looks like we have a scoring recidivist to deal with," said Robert A. Schaeffer, public education director for FairTest.

For now, college officials, who were caught by surprise by the mistakes at the height of the admission season, said they were working to take the revised scores into account so that students were not disadvantaged by the errors, almost all of which lowered student scores. Although some of the mistakes cost students more than 300 points, the College Board said that 83 percent of the score errors were from 10 to 40 points.

Posted by: lifeischeese
Source Article

Posted in: News on March 13, 2006 @ 12:00 AM

Tags: School

If you like what we're doing here, you can become a Patron and sign up for our newsletter!