Read the entire piece (replete with links) and post a comment on washingtonpost.com.
http://voices.washingtonpost.com/answer-sh...ay-my.html#more
-----
Author: 'My Misadventures in the Standardized Testing Industry'
Today my guest is Todd S. Farley, who worked for years in the standardized testing industry and authored the new book “Making the Grades: My Misadventures in the Standardized Testing Industry.” I asked him to write about the biggest problems he encountered and here is his account. Here is what he wrote.
By Todd S. Farley
For 15 years I was employed by the K-12 testing business, working for many of the biggest players (Pearson Education, Educational Testing Service, American Institutes of Research, etc.) on many of the biggest tests (National Assessment of Educational Progress, California High School Exit Exam, Florida Comprehensive Assessment, Virginia Standards of Learning, etc.).
While I did enjoy the career (good money, nice people, fun trips), it also left me completely convinced of the utter folly of entrusting decisions about American students, teachers, and schools to the for-profit industry that long employed me. I don’t know how anyone who’s seen what I’ve seen could feel any differently. ...
I’m not even talking about the well-publicized disasters that have occurred in recent years, when mis-scored tests kept students from getting into their preferred colleges (SAT, 2006); kept teachers from earning their deserved certifications (Praxis, 2004); and kept teenagers from graduating high school when they should have (Minnesota state assessment, 2000).
Any Google search will result in ... testing tragedies, but I’d say the scandals that make the news are only the tip of the iceberg. In fact, I’d say there aren’t scoring problems on some standardized tests—my experience suggests there are scoring problems on all of them.
Multiple-choice tests can be scored easily by machines, but constructed-response items (short-answer questions, essays questions) that students answer in their own words need to be read and scored by humans, which is where I think it all goes to hell. From my experience, that human scoring of tens of millions of student tests every year goes to hell for five main reasons:
The tests get scored each year by a motley crew of temporary employees earning low hourly wages, and while many of those people are earnest and conscientious employees, many others are not. ...
The only way to get large groups of temporary employees to score the tests in a standardized way is to establish stringent scoring rules and not waver from them.
This regularly results in rules that are rigid to the point of absurd. ...
In the cases when there aren’t rigid scoring rules to establish, the scoring that gets done is largely subjective, with the scores earned resulting as much from which temporary employee reads a response as the quality of student work. When assessing essays/writing, the scorers are supposed to differentiate between essays that are “skillful” and essays that are “sufficient,” for example, which is largely a matter of opinion. Since when are people all supposed to come to the same conclusion about a piece of writing?
The number of tests that need to be read and scored each year is so massive that every conceivable shortcut is taken to get that job done. The testing industry works exceedingly hard to meet deadlines and get scores put on to tests, while I saw much less interest in getting the correct scores put on them. ...
Finally, and perhaps most importantly, the test-scoring industry cheats.
It cheats on qualification tests to make sure there is enough personnel to meet deadlines/get tests scored; it cheats on reliability scores to give off the appearance of standardization even when that doesn’t exist; it cheats on validity scores and calibration scores and anything else that might be needed. I don’t want to just point fingers here, because I am guilty too, and over the years I fudged the numbers like everyone else.
Statistical tomfoolery and corporate chicanery were the hallmark of my test-scoring career, and while I’m not proud of that, it is a fact. Remember, I was never in the testing business for any reason other than to earn a pay check, just like many of the testing companies are in it solely to make a buck.
I’ve spoken to educators in Florida lately about the hard times they face: teaching jobs being cut, extracurricular activities being cut, book shortages in schools and workbooks that have to be recycled from year to year. Meanwhile, the state just signed a $250 million dollar contract for its Florida Comprehensive Assessment Test program.
I don’t claim to have all the answers about the problems in American education, but neither can I imagine how giving a quarter-billion dollars to a for-profit testing company is better for students than hiring teachers and buying books—especially not after I’ve seen what the testing companies do for that money.
Todd Farley is the author of Making the Grades: My Misadventures in the Standardized Testing Industry.