Rethinking assessment to rebalance education

Testing helped me be successful in school. And it was horrible for my learning. 

I was good at tests. The more standardized, the better. Multiple choice questions were my jam. I specialized in figuring out the correct answer even when I didn’t understand the material. My *bs* abilities were off the charts, which helped for open response questions. I could memorize all sorts of stuff I didn’t care about. For just long enough to ace a test, anyway.

But wow did this eventually catch up with me. I vividly recall an intro literature class where my new college roommate, who hailed from a nontraditional schooling background, waxed poetic about a poem. We were looking at the same words on the page, but he saw things there that I didn’t. Later that night we were hanging out and he made a bunch of connections between the poem and a song we were listening to. He hadn’t just been putting on a show for the professor; he was simply thinking and relating to the world on a different level than me.

Where had I gone wrong? I realize now that it wasn’t just me. Our school systems over-value easily quantified measures of educational achievement. We need to rethink the ways we collect, analyze, and act upon data and evidence so that we can rebalance education and restore the humanity that should be at its core.

Data: Less satellite, more street

It took me a couple of years to get my footing intellectually at that school. I had a lot of catching up to do, however good I looked on paper.

As students internalize these measures, they subscribe to narrow ideas about learning. They too often end up boxed in to superficial extrinsically motivated pursuits of success. Or even worse, they believe the messages the system sends them about how they aren’t good enough.

This is an equity issue. In the book Street Data: A Next Generation Model for Equity, Pedagogy, and Transformation, Shane Safir and Jamila Dugan make the case that our obsession with thousand foot view indicators like standardized test scores and drop-out rates has gotten out of hand. While that kind of quantitative “satellite data” still has its uses, they argue that it has outsized influence and is overly simplistic. 

By attempting to distill the kaleidoscopic process of learning into a metric and promoting a narrow discourse of achievement, satellite data contribute to a long, racist history insinuating students of color have lower intellectual capacity rather than differential access to opportunity.

From Street Data, p. 56.

Instead, they urge that we center qualitative “street data” that represent the full complexity and nuance of human experience. By widening our conception of actionable data to include interviews, observations, and artifacts, we gain a richer understanding of the situation. This is especially important for addressing equity challenges, which are tied deeply to layers of context and identity that are too fuzzy from a bird’s eye view.

To address equity challenges and fully honor the humanity of our students, superficiality will no longer cut it. To get to deep learning and real belonging, we need to operate at street level.

Relational over analytical

When done well, the very act of gathering street data is an act of equity because it prioritizes humanity and connection. For example, educators strengthen relationships with students while they interview them about their aspirations, observe them with an appreciative eye, or examine their work to uncover genius and opportunity. 

In contrast, gathering satellite data is often an impersonal and, for too many students, inherently harmful process. Hours spent taking standardized tests send wrongheaded signals about what is valuable about learning and reinforce for some students their perception that they aren’t smart.

There is emerging evidence that the preoccupation with analyzing satellite data has been a colossal waste of resources. Jill Barshay reported on several recent studies that examined the effectiveness of teacher teams analyzing standardized test data. All of these studies showed no or minimal improvement in student learning. This despite huge investments in standardized and “interim” assessments, data analysis programs and protocols, and the time teachers spent getting trained and performing the analyses. 

Barshay summarizes the downfall of the analytical approach, 

Why doesn’t data analysis work? All three researchers explained that while data is helpful in pinpointing students’ weaknesses, mistakes and gaps, it doesn’t tell teachers what to do about them.

Jennifer Barshay

On the other hand, the street data approach rejects a deficit view. It allows educators to prioritize relationships and affirm the dignity of the humans involved. Street data “are asset based, building on tenets of culturally responsive education by helping educators look for what’s right in our students, schools, and communities instead of seeking what’s wrong” (p. 57).

Taking an appreciative stance is fundamental to Culturally Responsive Practices because it requires us to move beyond our biases. We need to see students not as in need of fixing, but as partners in transformation.

Trust the process

We must partner with students throughout the change process because they are at the heart of educational systems. And if we are focused on equity, we must center students who are being marginalized by inequitable systems. 

Safir and Dugan recommend Equity Transformation Cycles that start with listening in order to collect street data. After digging into the data to understand root causes of inequity, we reimagine new systems together. Unlike other design and inquiry processes, street data involves students and other stakeholders throughout the cycle. Again, equity is seen as a process, not just an outcome. Those who are most directly impacted by inequitable systems are involved as partners and change agents in order to avoid paternalism and saviorism. 

This radical inclusion is one of the reasons why Jennifer Gonzales in a Cult of Pedagogy blog post about Street Data said that she had “never seen an approach that [she] thought was more promising.” She summarized her appreciation this way:

With that data in hand, all stakeholders co-design a path forward, building something that meets the unique needs of their learning community with the unique assets of that community. This is what makes the Street Data approach so special: It’s not a one-size-fits-all ‘program’ that all schools can follow for improvement; it’s a method for study and reflection that will give every school a customized solution, one that will keep evolving over time.

Jennifer Gonzalez about Street Data: A Next Generation Model for Equity, Pedagogy, and Transformation

The Street Data ethos can be applied at all levels of an education system. Let’s end by considering a few examples.

Example 1: Classroom street data via self- and peer-assessment

At the instructional level, teachers and students constantly generate data as part of the assessment process. Proficiency-Based Learning environments reject ranking and sorting in favor of partnering with students to center growth. Grace Gilmore’s grade 7-8 social studies classroom, for example, used student self-and peer- assessment to create a feedback rich environment where reflection on data and evidence was the norm.   

Trevor McKenzie sees student ownership of assessment as an inherently asset-based approach. In a recent voiceEd podcast he asked teachers to reflect on the question of “does your assessment culture tell students that they can, or they can’t.” 

Mackenzie asks us to go beyond self- and peer-assessment to imagine an assessment culture where students have a voice in reporting to families. This is the type of transformation that Street Data opens the possibility for. The classroom is a start. But we must think about change as interconnected and systemic if it is going to be transformative.

Example 2: Team street data via listening sessions

This has been a tough year for many educators. All sorts of data, from satellite to experiential, reflect the fact that student behavior is increasingly challenging. At the middle school level in Vermont, where teachers commonly work in teams with consistent groups of students, we have seen “reset” attempts such as an outdoor retreat at the Flood Brook School and a team-wide integrated unit on social courage at Williston Central School.

One team that I worked with as a school change coach asked me to help them take a Street Data approach. This group of 8th grade teachers were at wits’ end trying to figure out how to repair the classroom culture of a particular cohort of students. 

How could we start restoring community with these young adolescents? Ask them for their ideas.

Listening sessions

We embarked on an Equity Transformation Cycle by studying Safir and Dugan’s concept of listening sessions. Then I and others covered classes so that teachers could have one-on-one listening session conversations with select students. In the past I might have gathered this data and brought it to the team. But these sit downs between teachers and students directly strengthened relationships and built community. 

The data gathering was an act of equity in and of itself. And the uncovering stage was an eye opener for teachers. Each teacher collected their data on a chart and came together to look for themes. They had been braced for negative feedback. But they found that the concerns communicated by students were primarily about their peers. Even the most disruptive students expressed that they wanted to strengthen their learning community.

Coming out of the listening sessions, the team decided to immediately implement time for clubs during the school day. Clubs would strengthen relationships and community in the short term. One teacher noted “it was incredibly powerful to take this time to really listen to students. Though I know this is just the beginning, it feels like something is already shifting.”

School breaks and other things got in the way of us fully building on the momentum. We weren’t able to reimagine alongside students the types of sustained changes that would transform the team culture. But this experience reinforced for me the potential of Street Data. What if listening sessions were the cornerstone of continuous collaborative inquiry to advance equity and inclusion?

Or, to really dream, what if listening and an expanded definition of evidence were the basis for a new model of school quality?

Example 3: Systems street data via better school quality measures

Testing isn’t just bad for individual students. Ask almost any teacher or paraeducator, and most administrators, and they’ll tell you that testing hurts the entire education system.

This is the premise of the book Beyond test scores: A better way to measure school quality, by Jack Schneider. Schneider details the ways that “test-based accountability” doesn’t work: it hasn’t improved schools generally, it has actively harmed many schools by shaming and/or closing them, and it provides very little useful information to families about what they truly care about.

Schneider noted that “We have two decades of evidence that current approaches to educational measurement are insufficient and irresponsible. Each day we fail to act, we ignore the fact that we can do so much better” (p. 13). 

Over the past several years Schneider has acted. He has worked with researchers and educators to create the Massachusetts Consortium of Innovative Education Assessment (MCIEA). The eight MCIEA districts are piloting a School Quality Framework that goes far beyond test scores. It includes five areas: three “inputs” of teachers and leadership, school culture, and resources; and two “outputs” of academic learning and community and wellbeing. The latter are measured using performance assessments and surveys of teachers and students.

At a systems level, what if we were committed to moving beyond satellite data at all levels of our improvement efforts? I wish for a world where we could put humanity and equity at the center, redefine our goals at each level of the system, and then seek the evidence that matches our ambitions.

Take it to the street

In Vermont, our state has a robust plan for using multiple indicators for school accountability. Yet this remains lip service when test scores continue to drive the conversation. While I wait for the state to implement its own plan, or perhaps take notes from models like MCIEA, I will keep working in whatever spheres of influence are available.

One thing I will fight internally, and every time I hear it come up, is the learning loss narrative. In some circles, the disruption of the pandemic has brought satellite data, testing, and deficit views back stronger than ever. In a panel related to lessons learned from pandemic schooling, high school student Celilo Bauman-Swain spoke eloquently about her experience:

I think the biggest part for me, though, was the emotional aspects. Because my favorite part of school is my relationships with my teachers, and I consider teaching an act of love, and it was just really heartbreaking to have to kind of leave that relationship and go into Zoom.

Celilo Bauman-Swain, high school student

This is an important piece of street data. We hear a student seeing teaching as an act of love. Let’s love her and all of our students back by prioritizing their humanity. Not from a thousand foot elevation, but up close. Listen, learn, and legitimize their experience by recognizing it as every bit as crucial as any metric available. I firmly believe this is our route to rebalance, restore, and partner together toward transformation.