In my former life as a ne’er-do-well (during my 20’s when I had a variety of jobs) I worked on a couple of small fishing boats off the Oregon coast. We fished mostly within 50 miles of the beach, far enough out to lose sight of land, and I was grateful for the navigational equipment we had on board whenever it got dark or the weather was snotty.

We had radar and radios to keep us from running into other boats, and we had LORAN (a WWII radio technology, pre-GPS) to tell us exactly where we were. There were also buoys to show us the channel into the harbor. Some of the buoys had bells or horns on them. There were lighthouses, and we had marine charts, too. This stuff was reassuring. And the fact that we had multiple ways of knowing where we were, telling other people where we were, knowing where other people were, knowing where we wanted to be and where we didn’t want to be was important. We used all this stuff to help us find fish, too, which for fishermen is maybe as important as getting back to the docks.

So I wonder, since I have such warm feelings toward this navigational equipment, why do I dislike and distrust standardized testing and “data-driven decision-making” as I do? After all, testing does pretty much the same thing. It tells us where we are relative to fixed landmarks, and it lets other people know where we are. It helps us to see and be seen. Eduwonkette blogged about the significance of research statistics recently, and plans to say a little more about data-driven decisions later this week, which is how I happen to be thinking about this now.

One important thing to remember about all the navigational aids - someone still stands watch. There’s never a time when we’d just set the autopilot and all take a nap. If you see something, or hear something that shouldn’t be there, you might also need to DO something. And there’s a lot of stuff that radios and buoys and blinking lights can’t tell you. Fog, clouds, wind, the height and direction the seas are running, drift logs, the sound of the engine - all the messy details require human awareness, and most importantly, knowledgeable judgment to respond correctly.

It’s no different in the classroom. But in the classroom, standardized testing with high stakes attached to results privileges one kind of data over every other kind. Instead of using multiple ways of seeing, and being seen, as we do on a fishing vessel, we’re being encouraged to pay close attention to only one - the test with teeth - in school. Observational data and other forms of authentic assessment get little respect from pundits and policymakers, but those are the very things that teachers need for making decisions. High stakes tests lead us off course because they are not sensitive to local conditions, the contexts for learning. Sandra Mathison and E. Wayne Ross point out that that there are unintended consequences for high-stakes accountability schemes. And they outline four basic accountability principles:

  1. Improvement. Use of a wider range of strategies to improve the quality of schools and learning, such as professional development.
  2. Equity. Closing the race, ethnicity, and class achievement gaps and overcoming the consequences of poverty and racism, through the provision of health and social welfare care as well as academic care.
  3. Democracy. Control over and responsibility for schools must be grounded in sound principles of participatory democracy, such as informed involvement of local stakeholders.
  4. Informing the public. Providing accurate information about the functioning, successes, and problems of public education, such as information about libraries, health care, availability of enough and current textbooks, clean and equipped bathrooms, and so on.

Authentic accountability is characterized by:

  • local authentic assessments.
  • school quality review model.
  • low-stakes standardized testing in literacy and numeracy.
  • annual local reporting by schools to their communities.
  • consequences at the school level, not the child or teacher level, for failure.

[h/t Where the Blog has No Name]

The art of teaching is in learning how to interpret and use contextualized data.

For a long time I thought that the weaknesses of standardized tests were embedded in their design. But I see now, thanks to a link from Susan Ohanian, that the testing industry gets it wrong on the test-scoring end of the game, as well. Read about the Tests that Fail to see what kind of incompetence is behind how they’re handled after all the bubbles have been fully and carefully marked. David Glovin and David Evans won a reporting award for their story. The testing industry, as Glovin and Evans show, isn’t accountable to anyone.

Test scores mean little to me. They’re like the oil light on the dashboard of my old truck - broken, and not to be trusted.