Testing, Testing

by Susan Schorn

This year, our eleven-year-old daughter was scheduled to take the State of Texas Assessments of Academic Readiness, a name which implies that Texas schoolchildren live in a constant state of readiness, like Minutemen, ready to dash out and deploy their multiplication skills in defense of their homeland. As far as I’ve noticed, our kids have never been mobilized, or even called to muster, so I hope whatever threat they’re supposed to be ready for (China?) is a distant one. The STAAR moniker is also misleading in that the tests aren’t administered by the state of Texas but by the state’s proxy, Pearson PLC, which styles itself as “the world’s leading learning company,” and has been paid eleventy billion dollars (approximately)1 in public education funding to determine which of America’s fifth graders are hastening the downfall of our nation by not learning at a government-mandated pace.

We told Lilly she didn’t have to take the STAAR tests this year, even though she has taken them every year since third grade and done well on them — and even though, according to the increasingly dire messaging we received from our local school system as the tests approached, Texas law has decreed that “your child must meet the passing standardon this assessment in order to be promoted to the next grade.” She was happy to skip the tests; they are stressful and stupid, she said, and if you finish them early you aren’t allowed to do anything, even read — you can only check your work or put your head down on your desk.

My husband and I wrote a letter to our daughter’s school, “informing them respectfully” that “in accordance with our deeply held spiritual belief in justice and compassion,” and “with heartfelt appreciation for all you do for our child,” that our child would not take another STAAR test. In reply, the principal — a lovely woman who has a thankless job — somewhat shamefacedly handed us a letter from the local school board (dated two years prior; obviously we weren’t the first parents to raise a fuss about this). It stated that Texas law recognizes no parental right to absent a child from school to avoid testing.

The state of Texas assumes that all state-sponsored tests are good, fair, accurate tests. But we know, for example, that Pearson regularly screws up the answers to its own test questions. To cite just a few examples: In 2000, it incorrectly told nearly eight thousand Minnesota high school seniors that they had failed a state math test, causing some of the students to be denied diplomas. In 2012, New York had to throw out some Pearson test questions that asked fifth graders to find the perimeter of a trapezoid that could not exist mathematically. Last year, Pearson botched the answer key on a fourth grade practice math test in New Jersey, and only corrected the error when a concerned mother brought the problem to its attention. This past spring, New York State education officials discovered flaws in the Common Core English Language Arts exam; they simply scrubbed four questions after the tests had been administered, without saying anything to the public. (States and testing companies regularly tinker with the cut-off scores for passing the tests, even after the tests have been taken.)

Because Pearson demands that teachers and students adhere to confidentiality agreements promising not to talk or write about the tests — and since state educators prefer to obscure any problems — it’s safe to assume that the errors we’ve heard about are only a fraction of the whole. I have further reason not to share my state Board of Education’s faith in the quality of corporate-produced standardized tests. I once “did a little work,” as we used to say, for the testing industry, back when I was a brand-new Master’s candidate in English. At that point, I had never taught anyone anything (unless housebreaking my dogs counts), but I wrote “practice” language arts test question, under the direction of who seemed like2 MBA candidates. Their emails radiated a palpable disdain for literature and, indeed, literacy. These Project Managers regularly “revised” questions by mangling the existing grammar and punctuation; they confused basic parts of speech; they mis-identified simple literary devices, like “foreshadowing” and “personification”; they used expressions, apparently as part of their normal conversation, like “I think this is a go”; they were, in plain terms, people you would gnaw your own arm off to avoid sitting next to on an airplane; and they should not have been in charge of judging any child’s mastery of reading. Yet there they were, producing content to be sold to schools as practice tests to help students and teachers prepare for the real tests, which were made in the same sausage factory from the same rancid meat.

Many of the questions I helped write, if answered “correctly” according to the people revising them, would teach students things that were the opposite of true. I can’t give you any examples, because my contract required me not to share anything. But I can attest that there were errors, and that the people supervising me were either blind or indifferent to those errors. Was my experience anomalous? I’m sure Pearson would insist that it was. Yet I have reason to believe the rot extends throughout the testing industry, because I used the the money I earned writing test questions to complete my Master’s and PhD, and then I took a job in higher education — specifically, in writing instruction.3 One of the perks of my current job is that I sit in meetings with high-ranking administrators and tell them how truly bad and terrible all of the things they get excited about really are.4 Last year, a major U.S. educational publisher — not Pearson, but a big one; you’d know the name, your kids use their textbooks5 — tried to sell our very large public research university a series of online writing “modules” that incoming freshmen could work through to get “up to speed” on grammar.

The salespeople for this product allowed me to peruse the modules. Within ten minutes I had come across this sentence, in a module on noun phrases: “The rain rolled into the bay, so the soaken wet tourists spent most of their time looking for a restaurant to dry off.” Note well: The module didn’t ask students to revise the sentence. The sentence was presented as perfectly correct. All the module wanted was for students to identify a noun phrase in the sentence. What I wanted to do was find the people responsible for that sentence, and bludgeon them with a dictionary while screaming, “How do you dry off a restaurant?”

There were other errors; many more. I was not at all surprised. Things had been bad enough back in my day, when educational testing companies still employed graduate students. Now Pearson hires people off Craigslist who may or may not even have bachelor’s degrees. How likely are they to know that the OED pinpoints the most recent use of “soaken” to 1898?

And that was just one set of grammar modules from one publisher; I am sure equally bad ones are being bought by high schools and colleges around the country, because everyone agrees we need to fix students’ terrible grammar. Why is their grammar so bad? It’s a mystery that, somehow, no set of test results ever helps us solve. And so the snake continues to swallow its own tail.6

Lilly stayed home during the week of STAAR exams (one day of testing, plus several make-up days during which, if your child comes to school, they must take the makeup test). On Day One of her exile, she read Daniel Pinkwater’s novel Borgel. On Day Two, she read some articles about how Pearson had “adapted” an excerpt from Borgel, concerning a rabbit who challenges an eggplant to a race, for a mandatory New York State reading test. Pearson’s crack team of Project Managers stripped the episode of Pinkwater’s absurdist frame, loaded it down with pointless expository digressions, and then asked a series of unanswerable questions about it. For example, “What would have happened if the animals had decided to cheer for the hare?” This is a bit like asking, “What would have happened if the crowd in the marketplace had said kind and supportive things to Hester Prynne instead of taunting her?” It’s an intriguing mental exercise, and could make you ponder Hawthorne’s formative years, whether he was ever bullied at school, and how any such experiences might have informed his narrative choices, but it doesn’t get at anything like testable knowledge.7

Our daughter also wrote her own test about Borgel and then worked with me to revise her questions so that they were a) accurate, and b) fair. After all that, I dragged her to the office of our state representative to talk about what she had learned. I am in many ways a terrible mother, but I am thorough.

“It’s actually illegal for us to be here right now,” I informed our representative after Lilly had recounted her crash course in ethical test development. He had been up until 4:30 in the morning the night before trying to pass a budget in the company of legislators who believe post-rape forensic analysis is the same thing as abortion, but I pressed on ruthlessly. If I were homeschooling my daughter, I pointed out, she wouldn’t have to take the STAAR tests, and people would pat her on the head and tell me what a dedicated parent I was. If my husband and I were sending her to school without a measles vaccine, to potentially infect and kill other children, we’d be within our rights as long as we pretended to have “reasons of conscience.” So why, I asked our groggy legislator, is it illegal to refuse to involve my kid in a test that demonstrably sucks? He nodded blearily, said he understood my frustration, and suggested I contact some other legislators who focus on education issues. To his credit, he didn’t summon the Sergeant at Arms and have us arrested.

All in all, I thought our visit gave Lilly a good glimpse of the legislative process. I was glad because, as I discovered that day, she didn’t have the slightest understanding of how government works: no idea how a bill becomes a law; no sense of the role the government plays in her education; no clue how our representative ended up in his office or why he would meet with us. Our daughter didn’t know any of that because, in her six years of public schooling, she has never been subject to a state-mandated test assessing her knowledge of government or civics, so no one has bothered to teach her any of these things; she is only taught what she will be tested on. As a result, her civic ignorance is vast and pristine, like an unmarked field of freshly fallen snow. In seven more years, she’ll be able to vote.

Having worn out our welcome in the Halls of Government, we chose to become fugitives during the next round of tests, two weeks later. On the first day of the Science and Math STAAR exams, Lilly and I went to Chicago, like a couple of Vietnam-era draft dodgers fleeing to Canada. There we visited the Field Museum, a vast nineteenth-century edifice stuffed with knowledge. Dead animals and dinosaur skeletons and artifacts from vanished civilizations filled its labyrinthine corridors: We saw mummies from Egypt and Vodou spirit figures from Haiti; viewed the museum’s display of ancient ceramic urns from the island of Marajó, where the indigenous culture was wiped out by the Portuguese; and Lilly posed in front of Sue, the famous T-Rex who measuring forty feet long and weighing 6.4 metric tons when she was alive. It isn’t easy to capture a child and a giant lizard in the same frame, so I settled for a shot of Lilly directly in front of Sue’s enormous, bared teeth.

The museum was not without flaws. Its collection, after all, was built on the remnants of the 1893 World’s Columbian Exposition, a loud and triumphant hosanna to colonialism. Walking through it, you had to wonder where all those religious and ceremonial artifacts came from, and whether anyone missed them. Thousands of animals, some of them now extinct, were shot to create the museum’s wildlife dioramas. Still, the museum wasn’t mandatory, and it was transparent. It acknowledged our right to reflect. There were no multiple-choice answers. We asked our own questions.

Our walk back to the hotel took us past the Harold Washington Library and its nearly one million square feet of knowledge. The citizens of Chicago floated a hundred and seventy-five million dollars in bonds to pay for it. “Did you know that rich people used to build libraries?” I asked Lilly as we stared up at the giant aluminum owls on the library’s roof. “And museums?” The men who extracted great personal wealth from America in the nineteenth century — the Fields and Carnegies — saw no profit in museums and libraries. Yet they built them anyway. Now the common citizens have to go into debt to build them, if we decide we want them at all.

Lilly and I flew home from Chicago at the end of the week and shortly thereafter received a letter from our school district. It said that we could be taken to court if our daughter missed more than ten instructional days during the school year. There were a total of twelve STAAR testing days this year.

Pearson and the rest of the “educational” conglomerates have discovered the vast profit to be mined from ignorance: Selling tests untethered from meaning, denying anyone the right to scrutinize them, then selling other shoddy products that promise to undo the damage they’ve inflicted. It’s as if the automobile industry had hit upon the idea of making seat belts out of duct tape, forbid anyone to inspect or talk about accident scenes, and then offered to sell us all cheap Styrofoam crash helmets.

Perhaps this was inevitable. The reservoir of public funding we all pay into, supposedly for the common good, is an obvious resource for corporations to exploit. Taking advantage of it has helped Pearson generate solid profit for its shareholders — thirty-four pence per share for the year ending May 1st.8 Never mind that it has made our educational system more broken and our kids dumber.

In the end, we did not go to court. We were summoned to our daughter’s school for a “grade placement committee meeting,” where the teachers and administrators who had taught our daughter daily for six years unanimously determined that she was ready to be promoted to the sixth grade. It seemed like a perfectly sensible way to make the decision, but the meeting unfortunately didn’t generate any shareholder profits.

Maybe someday, a rare copy of a state-mandated Pearson reading test will lie under glass in a museum. Perhaps scientists will work busily while tourists watch, trying to uncover the logic behind the questions, to reconstruct correct answers from the inscrutable options. I imagine the museum’s visitors will marvel at the old bones of the educational giants, and wonder how they ever grew so big and powerful.9

Photo by Shutterstock.com

1. A 2012 Brookings Report estimated Pearson’s U.S. assessment revenue at $258M annually; Politico reports that 55 percent of Pearson’s $1-billion-plus profit in 2013 came from North America.

2.The only contact I had with them was through several CC’d layers of corporate email addresses. All our conversations were being watched by Someone.

3. It wasn’t anywhere near enough money, and choosing a career in academia only made things worse. Our oldest child is starting college this fall and I’m still paying off my student loans.

4. Some days that’s the only thing that keeps me going, honestly.

5. It was McGraw-Hill.

6. “Its own tail” is a noun phrase.

7. Pearson’s answer options for that question were:
A. The pineapple would have won the race.
B. They would have been mad at the hare for winning.
C. The hare would have just sat there and not moved.
D. They would have been happy to have cheered for a winner.
Pearson has steadfastly declined to state which of these answers it considers correct.

8. With current exchange rates, that amounts to fifty-three cents per share in USD.

9. As I was writing this piece, the Texas Education Agency announced that most of Pearson’s $90 million annual contract with the state will not be renewed. The STAAR testing contract is being handed off to Educational Testing Services, which has, among other acts of sterling civic engagement, attempted to censor MIT researcher Les Perelman, who has thoroughly and brilliantly exposed the poor performance of ETS’s “Automated Scoring Engine,” an automated essay-grading system they are currently marketing to colleges as an “instructional tool.”