Tuesdays in the Concord, New Hampshire, public school district are alive with academic inquiry. Groups work together, ask questions, examine assumptions, and tear apart data. This scene, however, is not playing out among students, but among teachers participating in "Data Tuesday Training Days." Using an Eisenhower grant from the U.S. Department of Education to fund release time for professional development, Concord is training all mathematics teachers in the district to use a collaborative inquiry process to improve their programs.

The process modeled at the Concord training sessions is detailed in Using Data /Getting Results: A Practical Guide to School Improvement in Mathematics and Science, by Nancy Love, forthcoming from Christopher-Gordon Publishers. The book grew from Love’s work as a professional development specialist for the Regional Alliance at TERC. Collaborating with a network of schools throughout the Northeast, Puerto Rico, and the Virgin Islands, the Alliance piloted many of the principles and techniques now outlined in the book. Teachers in those schools were using a process of inquiry to improve their curriculum, instruction, and assessment practices. Love and the Alliance realized that the process, which involves asking generative questions, collecting and analyzing data, examining assumptions, and formulating and testing hypotheses, could help other educators achieve their reform goals. Data As a Tool, Not a Club

Using Data presents a rationale, guidance, and reams of practical data collection and analysis tools for school districts that want to use data to improve student learning. The author challenges conventional assumptions about what data are and how to use them in the context of educational reform. She explains, "The standards movement has shined a flashlight on improving learning for all students. However, many school reform efforts have been based on intuition and speculation, rather than on rigorous use of data, and this has led to a flurry of activity without effective ways to measure whether students are benefiting. Over-reliance on standardized and high-stakes tests–which often lack rich information about instruction–compounds the problem.

Educators have learned to fear data because data have been used so often "as a club against schools, administrators, and teachers….For a variety of reasons, including this fear, [educators] often don’t use data for their own purposes–to diagnose problems, spark action, and improve results" (Love, 2001, p. 1.11).

Some schools, responding to pressure to increase scores on state tests, may focus their efforts on raising the scores of those students already doing well. Schools may appear to be improving, when, in fact, the needs of the lowest-performing students remain unmet. A recent study of fourth-grade reading results from the National Assessment of Educational Progress is just one example of this problem. The study found that while the scale scores for higher-performing students have increased since 1992, the gap between the highest- and lowest-performing students is widening (Donahue, Finnegan, Lutkus, Allen &. Campbell, 2001). "Quick fix" methods focused on raising the average test score often leave some students behind. Increasingly, educational leaders are turning to a more rigorous collection and use of data to inform decisions and guide sustained improvements in the system so that all students attain higher levels of achievement.

Creating Data Users

Love asks educators to become "data users, not just data givers." They need to collect and analyze data to develop informed responses to student needs. In addition to standardized tests, schools need to use other measures such as local performance assessments, enrollment figures, and dropout rates. Further, using data about curriculum, instruction, and assessment practice may help educators understand underlying factors related to the student learning results.

Looking Through Many Windows Improves the View

Once a number of potential data sources are identified, Using Data recommends "triangulation," defined as "using multiple (two to three) independent sources of data about the same issue or problem" (Love, 2001, p 2.20). The book references researcher Richard Sagor’s analogy showing how students would use "triangulation" to investigate life in a terrarium. Students would be expected to observe from above and from each side to draw a three-dimensional understanding of the life inside. An animal hidden under a leaf or behind a rock would be overlooked if the observations were made through one window only (Love, 2001, p 2.20). So too, educators gain insight by examining questions from multiple perspectives before embarking on a path of reform.

Data from different sources may highlight issues that raise crucial follow-up questions. For instance, standardized tests showed one school that students were underperforming in geometry. While it was initially assumed that teachers needed to spend more time teaching geometry, subsequent data collection revealed that instruction time was not the problem. That pointed the school to look at differences in instructional practices to improve results. This example illustrates how "Data can get to the root cause of problems, pinpoint areas where change is most needed, and guide resource allocation" (Love, 2001, p.2.10).

Disaggregation: Looking At Each Chapter To Learn The Whole Story

Disaggregating the data is another critical step to gaining increased knowledge from the collected information. Disaggregating data involves delving more deeply into a set of results to highlight issues that pertain to individual subsets of results, such as those for a specific grade level, gender, ethnic, or socio-economic group.

For example, the public schools in Providence, Rhode Island, looked at enrollment data as well as standardized test scores to address poor performance in math. The enrollment data revealed that students of color were underrepresented in high-level mathematics courses. The district also looked at research showing that minority students who do not take algebra or geometry in high school are 40 to 60 percent less likely to complete college. In response, the district untracked mathematics instruction, offering the same algebra instruction to all students. After six years, the district reports failure rates much lower than before, as well as an increased enrollment in third and fourth year mathematics courses (Love, 2001, p. 7.16).

Concord Data Sessions Identify Issues

Chris Demers is the Concord-based educator who is implementing the Tuesday Data Training sessions to introduce all of Concord’s math teachers to the collaborative inquiry process. Early efforts at triangulating and disaggregating data are pointing participants in constructive directions.

Demers tells of how one group used disaggregation to learn more than aggregate scores alone could tell about sixth-grade science performance. Teachers suggested taking the results of a sixth-grade science test and pulling out questions that called for knowledge taught exclusively during sixth grade. Demers explains that by taking this extra step "they came up with some interesting, disturbing findings. Kids were progressively doing worse on the grade 6 content as they looked at tests back all the way to 1996 through to the current test in 2000….The average score [for questions covering sixth-grade content] dropped over the five years, whereas the K—5 content stayed primarily level….They took that as a starting point for further inquiry."

Time Well Spent

As the New Hampshire example illustrates, data analysis leads more often to the need for additional data collection and analysis than to immediate action. The collaborative inquiry process is a means to continuous improvement, not a sprint to short-term solutions. The process does, however, eliminate time and resources wasted when systems ask the wrong questions or take action based on faulty assumptions and incomplete knowledge.

Chris Demers calls Using Data/Getting Results "a natural fit" as his district strives to meet state requirements to measure a district’s actions in terms of its contributions to student learning. Rather than resisting the time commitment required to faithfully follow the process, teachers have reacted positively because using data will meet educational goals and satisfy demands for accountability.

Demers observes, "We see this [the collaborative inquiry process] as a more viable type of professional development, as opposed to offering a workshop on a new way to teach without knowing if it’s right for your school. The process also models the process we want teachers to use with their students, which helps them be better teachers when they leave the session."

Instead of viewing data collection and use as an "add-on," school systems are encouraged to use data to advance the work of established committees and planning groups. Teachers need to have professional learning imbedded in their jobs in the same way that most other professions have meetings and planning sessions included in the weekly workload.

Collaboration Creates Ownership and Informed Action Plans

When Chris Demers began the data use workshops in Concord, he feared teachers might reject the collaborative inquiry process, noting that teachers have been asked to jump on and off so many educational "next-big-thing" bandwagons. He found, however, that many of the teachers who were most skeptical about the utility of the process left the workshop fired up to take the process further. Teachers welcomed the collaborative framework of Using Data’s approach, which diminishes the frustration that comes when change is dictated from external sources alone.

According to Demers, "The teachers like that the process values them as intrinsic to the solution. Because it is grounded in dialogue with no predetermined outcomes, the process mirrors what so many teachers want to achieve in their classrooms. As teachers delve more deeply into a question, they begin to challenge assumptions and reflect on their own personal classroom practices, and open up to different ways of looking at both problems and solutions."

A Cycle for Perpetual Progress

In the collaborative inquiry process (see title graphic), it should be noted that five steps precede taking action and, more importantly, taking action is not the final step. A cycle emerges, which calls for constantly monitoring results to adjust to the ever-changing educational landscape. Reaction to the process, as it is being introduced in Concord, indicates that teachers and administrators appreciate the fact that the approach does not begin with loyalty to any specific curriculum or instructional practice. Because the process uses data to identify problems as much as to solve them, it does not allow for data to be manipulated to justify any predetermined agenda for action.

While data are essential for school improvement, reports alone will not create better schools. Only by working collaboratively to use the numbers and observations can schools attain lasting progress. Using Data/Getting Results points to better use of data as "the compelling evidence that grounds conclusions in actual results, not in speculation." An inquiry-based approach requires time to frame solutions, but results in solutions that hit the mark with greater accuracy and are revised as needed to keep hitting the mark in dynamic educational environments. "In inquiry-based schools, teachers and administrators continually ask questions about how to improve student learning, experiment with new ideas and rigorously use data to uncover problems and monitor results….Researchers in both business and education agree that these qualities are hallmarks of the most successful organizations" (Love, 2001, p.1.11).

References