I was in the doctor’s office a couple of days ago with a broken hand (from a fall while walking our dogs on an icy road). The nurse put a thermometer in my mouth. I told her I wasn’t sick. She said, I know, but if I don’t do this the doctor will want to know why I didn’t. As a general indicator of well-being the thermometer seems to the instrument of choice. I’m thinking about thermometers now since there’s been some discussion about using “thermometers” for monitoring reading fluency in the comments on this blog. I was curious to find out more about what seems to be a fluency movement (a bit of a pun there), so I did this little research project. For readers not involved with US elementary schools, you may read this as an example of our fascination with labels and measurement.

DIBELS, in a minute
I do think that test data should inform instruction - if you have good test data. And with the Dynamic Indicators of Basic Early Literacy Skills (DIBELS), a test that claims to measure the National Reading Panel’s 5 Big Ideas in Beginning Reading, (phonemic awareness, phonics, fluency, vocabulary, and comprehension), we don’t have good data. My fourth graders get screened for “Oral Reading Fluency” with the (DORF), which tells us…what? The number of words a student reads aloud in one minute.

According to Roxanne Hudson, (and my blog commenter) it’s just a quick little assessment, a simple “thermometer” which “tells you if you have a fever, but not what the cause of the fever is or what the treatment should be.” As a measurement tool, DIBELS announces loudly and clearly how the tester defines reading. It tells us, and the kids who get their reading clocked, that the people in charge equate reading fluency with rate, more than any other variable.

Fluency and comprehension
The National Reading Panel noted a correlation between fluency and comprehenson (Put Reading First, p. 23)[download the pdf], but there is no indication of causality. We don’t know if comprehension causes fluency, or whether fluency contributes to comprehension. Word-level fluency is believed to “free up cognitive capacity” and allow the reader to focus on meaning (Pressley, p. 47).

Fluency is only a piece of what an accomplished reader does. Michael Pressley’s Instruction and Development of Reading Fluency in Struggling Readers offers a broad list of what we want good readers to do in addition to accurate and expressive reading. We want readers to:

  • read words fast and accurately and with expression (i.e., read prosodically)
  • to read with high comprehension;
  • overview text and scan it;
  • relate their prior knowledge to ideas in the text;
  • notice when they are confused or need to reread;
  • construct images in their mind’s eye reflecting the content ofthe text;
  • summarize, and interpret;
  • understand, appreciate, and think about the ideas in text;
  • think hard about what they are reading;
  • know when to slow down and employ the comprehension strategies previously described.

Predictive validity
Some screening tools are better than others. So we should use several. Since when do you go to the doctor and only have your temperature taken? Most of the time when I go, they weigh me, and take my blood pressure. The doctor pokes around, listens to my heart and lungs, and asks me questions. All of those things together suggest whether I get to go home right away. If any of those little indicators is far out of order, off I go to the lab for blood work or other invasions.

DIBELS is supposed to have “predictive validity,” which means it can tell us which kids are at risk of not passing the state benchmarks. Let’s take a look at how well it identified reading troubles in my classroom this year. Using the unscientific but personally meaningful sample of the scores of my students,

  • 3 of the 11 students who tested as Advanced on the State Benchmarks for reading were identified by DIBELS as needing “intensive intervention.”

Oops.

  • 6 more whose state scores were Proficient were identified as needing “strategic intervention.”

Oops again. Maybe they’re slow talkers, too.

  • It did accurately identify the 3 neediest in the group.

But anyone who heard them read would know that. So much for predictive utility. Why do I need such a crude test? Why bother with it at all when better alternatives, like running records, or simply listening to a kid read, are just as easily available - for free?

Consequences
In the foreward to The Truth About DIBELS, P. David Pearson described reading skills in terms of their being either constrained or unconstrained, meaning that some skills are mastered to gain control over a limited and relatively simple (constrained) domain. These might include letter naming and letter sounds, for instance. Unconstrained skills, on the other hand, are by definition not subject to mastery, and represent the capacity for life-long growth. These would be things like comprehension strategies, word knowledge, and critical thinking, which Pearson calls “the real stuff of literacy - the important things that we go on to” after we’ve mastered the basics. Pearson pointed out that DIBELS monitors the more basic word-level skills which, when they become the focus of instruction, limit the literacy horizons for students (p. xii).

Robert J. Tierney, in Is DIBELS Leading Us Down the Wrong Path?, pointed to the consequences of embracing narrow, easily measured definitions of literacy. Maybe the thermometer isn’t as benign as some would have us think.

DIBELS may be perpetuating the literacy gap it has promised to eliminate. By closely subscribing to only those five components defined as essential by the NRP and assessed by DIBELS, the definition of “literacy” has been narrowed for the most vulnerable students. For those school/districts who are not high poverty nor low performing, children are less likely to be held to this narrow view of literacy. These children have a more balanced literacy environment that includes viewing, writing, and other critical literacies. Those children in schools receiving funding from Reading First are more likely to be restricted to the five major components of reading as defined by NRP and tested by DIBELS.

I shared this article with my principal and some teachers I work with. They found it worthwhile, but DIBELS continues down the hall. Note that Roxanne Hudson’s well-documented opposing viewpoint, Reading fluency assessment and instruction: What? why? and how? cites only correlational “linkage” to support explicit instruction in what amounts to a rather narrow instructional reading regimen.

Alternatives
In How Will Literacy Be Assessed in the Next Millennium, Robert Tierney shares a more expansive vision of assessment - one that more closely resembles a conversation than a medical procedure.

I would encourage educators to look for approaches to assessment that are both just and empowering and that assess the literacies of learners richly and in all their complexities, without fear of what is not quantifiable or uniform. I look for assessments that consider the quality and usefulness of information that is gleaned for teachers and learners—descriptions and possibilities versus numbers and crude labels. Mostly, I look for a future that views evidence from assessments as conversation starters that engage the learners as decision makers with the support of teachers, parents, and others.

As for me, I prefer to take temperatures informally, by putting a hand to the forehead. With reading, that amounts to saying, Would you mind reading some of this to me? And then I ask a few questions. I take notes. I can tell whether the problems are decoding or vocabulary-related. I can hear whether the kid pays attention to punctuation, and uses an expressive tone of voice. I get a good idea of the types of books that are appropriate for him/her to read.

Of course, nobody makes any money off of that.

Sources:
Armbruster, B.B., Lehr, F., & Osborn, J. (2003) Put Reading First: The Research Building Blocks for Teaching Children to Read. Developed by the Center for the Improvement of Early Reading Achievement (CIERA), retrieved Dec. 2006.

Hudson, R. (2006) Oral Reading Fluency Assessment and Instruction. 2006 International Reading Association Conference

Hudson, R.F., Lane, H.B., & Pullen, P.C. (2005) Reading fluency assessment and instruction: What? why? and how? The Reading Teacher, 58(8), 702-713

Pressley, M., Gaskins, I.W., & Fingeret, L. (2006) Instruction and Development of Reading Fluency in Struggling Readers. In What Research Has to Say About Fluency Instruction, Samuels, S.J. (Ed.) (47-69). Newark, DE: International Reading Association.

Pearson, P. D. (2006). Foreward. in K. Goodman (Ed.), The truth about DIBELS: what it is, what it does (v-xix). Portsmouth, NH: Heinemann

Tierney, R.J (2000) How Will Literacy Be Assessed in the Next Millenium? Reading Research Quarterly; 35(2), 244–245

Tierney, R.J. & Thome, C. (2006). Is DIBELS Leading Us Down the Wrong Path? in K. Goodman (Ed.), The truth about DIBELS: what it is, what it does (50-59). Portsmouth, NH: Heinemann