Transcript: Restorative Practices: Using data to strengthen your practice

Lan Nguyen:  Welcome everyone. And on behalf of the California Center for School Climate, I’d like to welcome you to our session Restorative Practices: Using data to strengthen your practice. My name’s Lan Nguyen and I’ll be your moderator for this session. And I’m also joined by Rebeca Cerna, Director of the California Center for School Climate, who will be supporting in chat. 

And so, just a quick look at our time together today. We’ll have some brief welcome and introductions. And we’ll get to our main session on restorative practices and data use. And we’ll end our time talking about some upcoming CCSC opportunities. We’ll also be sharing a survey at the end, and we invite you to fill that out. So, just a heads up on the that. 

And so, just a little bit about the California Center for School Climate. We provide free supports to LEAs throughout the state of California, as it relates to school climate improvement. And we do that by delivering relevant, responsive, and engaging technical assistance on a variety of topics. We also support best practices for data use as it relates to school climate improvement. And we also serve as a connector across the state to promote and disseminate best practices. And also, we support LEAs with building partnerships that best serve their communities. You’ll see on the screen, there is a QR code. It will also be dropped in the chat, and we invite you to visit our website to learn more about the center itself, upcoming opportunities, and any other information as we update it. 

And so, for our welcome from the California Department of Education, we have Tom Herman, who’s an Education Administrator. He couldn’t join us today, but we do have a recorded welcome. And one thing that I missed out on for sharing the California Center for School Climate is that it is an initiative of the California Department of Education, and it’s operated by WestEd. And so, we’ll just show a brief video clip of Tom. 

Tom Herman (via video):  Good morning, my name is Dr. Thomas Herman. I’m the Education Administrator of the School Health and Safety Office. I want to welcome you to this California Center for School Climate virtual event. 

The California Center for School Climate was launched in January as the CDE, key legislators, and stakeholders are acknowledging the importance of school climate in addressing the fundamental needs of all students to have a safe and supportive learning environment. 

We have learned a lot about school climate over these last few years. We know, for example, that by improving the various measures of school climate, such as caring relationships between students and staff, and giving more voice to student concerns, that academic achievement for all students improves. And that also, we see a decrease in youth risk behaviors, such as substance use, and chronic absenteeism. 

Today’s session will focus on restorative practices and using data to strengthen that practice. Again, I welcome you, and thank you for the good work you’re doing in schools. 

Lan Nguyen:  All right and so, as I shared earlier before, my name is Lan Nguyen and I’m a Technical Assistance Provider with the Center [CCSC], and Sean Darling-Hammond will be today’s main presenter. However, along the way, I’ll be providing some anecdotes and stories from my time both in the classroom and in my work with school and district leaders as they implemented restorative practices programs. And so, with that, I’d like to pass it to Sean Darling-Hammond. 

Sean Darling-Hammond:  Hello, everyone. Good morning. Thank you so much for joining us for this important session on how to measure restorative practices so that they can realize their intended outcomes. I’m going to my slide deck now so that everyone can follow along. 

Before I jump in to try to convince you all that we should be measuring restorative practice utilization, I want to give a brief introduction to me and my work so that you understand how I come to this work, sort of what my expertise is and isn’t, what kinds of questions I might be able to answer, and how I can help. 

So, educationally, I have a BA from Harvard in sociology, a JD, and a PhD in public policy forthcoming in the next few weeks. Time flies. A nice way of describing myself might be that I’m a lifelong learner. Another way might be somebody who doesn’t know how to leave school. But I’ve kind of combined this background in research and law and policy to foster a research agenda that’s really focused on how we can leverage quantitative methods and quantitative data to improve policy outcomes. So, a lot of my work on restorative practices melds those backgrounds. 

In terms of my actual work experience, I was the director of Berkeley High School’s restorative student court for a couple of years. And so there I got, I guess you could say proselytized, and totally bought into the notion that restorative practices can transform lives. I am the director of Bend It To Justice, LLC, which is a consulting firm that I’ve been leading since 2006. And I’m a soon-to-be professor at UCLA Schools of Public Health and Education in July, where I’ll be doing a lot of work to try to identify ways to improve utilization of restorative practices and ensure that students of all backgrounds thrive in schools. My big life goal is to bridge K-12 research policy and practice silos, and combat racial disparities in schools, and expand belonging, both in schools and in other contexts. And a kind of fun fact is that I am a now seven–time competitor on NBC’s American Ninja Warrior. I have got to say, after having a toddler, as you can see there, I definitely lost a step. So, I’m no longer an elite competitor on the show, but I still show up and do my best. So, that’s me! 

But what are we going to be talking about today? I’m hoping to cover four topics with you, and I’ll welcome questions throughout. I know that you can type questions in the chat so if you have them, please do type them in, and I’ll be happy to pause to answer them as they come up. But our agenda is: one, I want to talk about what restorative practices are, and whether or not they work. There is quite a bit of interesting research energy around the effectiveness of these practices, but there’s also quite a bit of confusion about what they are and are not. So, I want to start grounding our conversation there. 

Next, I want to cover why I believe educational institutions should measure their utilization of restorative practices. Then, talk about how they can do so. And finally, talk a little bit about how we can think about the continuous improvement movement in conjunction with restorative practices and implementation strategies around restorative practices. So, let’s dive on in, and talk about what restorative practices are, and whether they work. 

Restorative practices are often depicted as being either in three tiers or in two buckets. I’m a fan of the two-bucket approach, which is to say that there are kind of two basic kinds of practices. There are practices designed to repair relationships. That is two students have gotten in conflict, maybe one has bullied the other, maybe they’ve both had a fight about something, and these restorative practices are being leveraged to try to repair their relationship. As a means of ensuring that there’s more glue binding them together, they can learn from the experience, et cetera, rather than just saying, “Well, who was more at fault? And we’ll suspend that person.” That’s the repair side of restorative practices. And those include formal conferences with stakeholders, that can be respondents who’ve made a mistake, that can be the victims, that can be community members who are also affected or have the ability to be supportive during that sort of conference. And those are meant to try to help folks resolve conflicts, but that can also be informal repair conversations. So, if a teacher sees two students having a little bit of a tiff, they can step in and say, “Hey, can we chat this out? Who did what? What did you experience? Tell me from your perspective.” These I statements “What did you experience? Tell me from your experience.” These I statements “Can we empathize with one another? What can we do to try to avoid having this happen in the future?” So, sort of guiding that repair that is really such a classic and important human function, but often we forget how important it can be for students at sensitive developmental stages. That’s the repair side. 

The other side is community building, or proactive restorative practices. And those are designed to do two things. One, to create more community cohesion, so folks feel invested in relationships and therefore, want to repair them. And two, to inculcate social and emotional skills and conflict resolution skills so when conflict does occur, students are empowered to overcome the conflict and repair their relationships. And those can be community building circles that happen in your main classroom, your homeroom. On Mondays, you come together, you talk about your experiences with your fellow student peers. You get to know their social roles. You feel more connected to them. 

It can be situational role playing around conflicts that folks are facing not only in school, but maybe in their life or in their home, to help folks inculcate, help students inculcate, these social-emotional skills. And it can be reentry circles. So, some folks often would consider that to be a kind of hybrid tier one/tier two practice. But those are designed to help students who have made mistakes or have been out-of-school for a while to come back into the school environment. So that can be, for example, a student who was suspended for a while and they’re coming back. But it can also be a student who was ill for a while and they’re coming back. You’re trying to make sure that they feel connected to the school community. Again, it’s very proactively avoiding downstream conflict rather than being just responsive to it. So, you can think of it as reactive and proactive, or responsive and proactive practices. And I thought Lan, with her incredible background in working in restorative practices, would be able to give a little bit of color to this idea of what it means to be in a restorative school. So take it away Lan. 

Lan Nguyen:  Thanks, Sean. I’m just going to tell you a brief story about a school I observed, and I’m based in San Diego, so my work in restorative practices has been in San Diego, and so these are the examples I’m bringing to you. There’s a particular school, it’s a comprehensive high school, and it had about 1,000 students with a very diverse student body. In my work, it was shared with me that this school was really doing some amazing work. Really, the story starts with me parking my car in the parking lot, and I remember stepping out of the car. The best way I could describe it is that there was something in the air. And so, as I walked towards the main office, most schools have you sign in, so you can get a visitor’s pass, and you use that visitor’s pass to kind of go around the school. And so, as I walked through the front office, and I signed in, I was kind of listening to the conversations that were going on in this main office. I really noticed this sense of this was like a place that students and adults wanted to be. And I noticed the front office staff actually talking with some students who were kind of hanging out there because they were waiting for their parents to pick them up, and it was very clear that there was a relationship between the front office staff and these students. They were asking about, “How did your math class go last week?” and asking them about these personal details about their life that they seemed to have. There was this rapport and relationship built over time. And so, that was kind of my first inkling into “Wow, this feels like a different kind of space.” 

And then, I found my way. I just kind of walked around, and I found myself actually in an 11th grade math classroom. So, I walked into the classroom, and I sat in the back. And, of course, there’s usually that moment where the students kind of look at you like, “Oh, there’s a stranger here,” but then eventually, after a few minutes, they kind of move on and continue their lesson. What I observed was these students were actually beginning a new unit in math. This particular math teacher actually started a circle, and one of the first questions that this teacher asked wasn’t about math and really struck me. And they started their circle, of course, with a talking piece, and it was really cool. The talking piece in this particular class was actually a decorated calculator. 

And the first question the teacher asked, because this unit was going to be particularly difficult and students typically struggle with it, is they asked students, “What values do you draw on when you are working through something that’s really hard?” And so, he passes the calculator talking piece to the left, and students chime in with, they use persistence, determination, and hard work, all these things. And then, he asks them, “Share an example of a time where you were learning something new that was really hard and how did you overcome that?” And so, he kind of did a popcorn style and just kind of like let whatever students wanted to share, share their stories. And some students shared, “I was learning a particular skill in soccer, and it was really hard, but I had to be patient with myself.” Another student shared about how they were in dance and learning this particular routine. Another student talked about something they were learning in science class. 

And so it just really struck me because this teacher was using restorative practices with the curriculum, and really giving them opportunities to share stories, and build and share values through the content. And this was, to me, an example of that teacher doing both at the same time. And so, I spent a few minutes in that class and eventually, after visiting a few other classrooms, I made my way back to the main office. I decided to check out the counseling office. And this was really all in one day, which I found really remarkable. And there was a couple people that I knew. And so, I was just stopping by to say hi. And as I was stopping by to say hi, I noticed this friend pair of students, they seemed to be friends. They looked a bit distraught actually, and they were coming up to the receptionist desk and they started signing in. And I was like, “What’s that?” They just didn’t look particularly happy. And so, I went up to the receptionist and I said, “Hey, I’m curious, what is this? What are they signing in for?” And the receptionist shared with me that they actually had their peer mediation program. All of their students were trained in having restorative dialogues and conversations, and that they were running these lunchtime supports for students to actually voluntarily opt in without adults to get support for resolving conflict with their peers. And so, what I was observing actually was these two young people on their own accord, on their own time, going to the counseling office, and signing in to get support from their peers to facilitate a restorative dialogue. I was thoroughly impressed. That’s just a little bit into at least one of my examples of a day in the life of, I think, what’s possible in a restorative school. And so, I’ll pass it back to Sean. 

Sean Darling-Hammond:  Thank you, Lan, for that. That is such a wonderful example of one, how intrinsic some of these desires to be in positive relationship are, and two, how atypical it can be for schools to create a paradigm where students can lean into that natural desire to be in positive relationship. I love the idea of the bedazzled calculator and I definitely want one. I think that’s the best talking stick I’ve ever heard of. But yeah, I think what I wanted to emphasize from Lan’s story is that it is not that students don’t want to be in relationship. It is that we don’t typically socialize, really anyone in society, to think that conflicts can be overcome. And so, I’ll be going into that a little bit about that sort of mindset shift that is necessary to create a more restorative paradigm. 

So, it’s not just the school that Lan mentioned that made that shift towards restorative practices. Many schools throughout the state have actually made a huge shift. And it’s partially because in 2014, January, the Departments of Education and Justice issued a joint dear colleague letter, which actually had a bit of a threat attached to it. They essentially noted that based on Government Accountability Office data, there were large and sweeping racial disparities in discipline, and warned schools that if they were implementing policies which are creating these racial disparities, that the departments could investigate and could claw back Title I funds. They suggested restorative practices as a mechanism for reducing these racial disparities in discipline. They provided technical guidance. They provided a grant through the School Climate Transformation Grant program. And they did issue dozens of consent decrees over the course of the following months. 

And so, what happened as a result of all this policy activity was that between the 2013/14 school year when that dear colleague letter came out and the 2014/15 school year, which was the next year, there was this huge increase in utilization of restorative practices. But an increase in utilization does not necessarily mean an increase in student exposure to these practices, which something we’ll be getting into later. But, at baseline, sort of the genesis of this energy around restorative practices is concern about racial disparities in discipline. 

So, I thought it would make sense to spend just a little bit of time talking about discipline in schools, the role of school practices in driving discipline disparities, the impacts of discipline disparities, et cetera. So let’s start with why all the interest in restorative practices, what are the harms of student discipline? And this is a relatively well studied area. Dan Losen’s book in 2014 was one that really sort of opened the floodgates of research interest, and impacts of exclusionary discipline on student outcomes. And Bacher-Hicks and colleagues just two years ago did really sophisticated causal modeling to identify the impacts, the causal impacts, of exposure to more disciplinary academic regimes. And both of these sorts of works, and this body of work, shows that student discipline is related to or even causes negative educational outcomes. So, that’s graduation attainment, achievement, school climate outcomes, mental health outcomes — that’s depression, suicidal ideation, et cetera — and carceral outcomes. Students who are exposed to more discipline in the sensitive middle and high school years are more likely to be juvenilely incarcerated and are more likely to be incarcerated as an adult. 

And you can see that from this chart on the right, which is from Losen’s book, the sort of shaded bars are students who are suspended one time or more. The blank bars are the students who were suspended zero times. You can see big jumps. So, are you more likely to not get a high school diploma if you’re suspended? Absolutely. Are you more likely to be arrested? Absolutely. Are you more likely to be juvenilely confined? Absolutely. Adult confined? Absolutely. And the more recent work by Bacher-Hicks and colleagues shows that’s not just an association, it’s causal.  

So, there are these really potent negative outcomes, but discipline is also not even. So, when we look at the Government Accountability Office’s data regarding who is disciplined in various academic environments, we see that Black students are overrepresented among those who are disciplined relative to White students in every context. So, this chart shows that Black students are 3.9 times more likely to receive an out-of-school suspension compared to White students. Among boys it’s 3.5 times, among girls it’s five times. Students with disabilities 2.8 times more likely. Among students in poorer schools, Black students are 2.3 times more likely than White students, and on, and on, and on, in every academic context you can imagine, including preschool. So, even among preschoolers, Black students are 3.7 times more likely to receive an out–of–school suspension than White preschool students. And this is true in California as well. So, I have access to the CAASPP [California Assessment of Student Performance and Progress] data, the California Administrative Panel data, which tracks students over time. And for that cohort of 3rd through 8th and 11th grade students, we can see that whether you’re looking at elementary, middle, or high school students in California, Black students are much more likely to receive suspensions than their peers. 

And we could simply stop at, well, we think suspensions are harmful, so it’s, therefore, concerning that Black students are overrepresented. But it’s actually not simply that the harm comes from increased exposure to this negative thing, which is exclusionary discipline. There is harm itself in being in an environment where you see peers who look like you being suspended at a higher rate. When you look at the California Healthy Kids Survey data and review whether Black students in schools with a higher discipline rate feel less belonging, you find that is, indeed, the case. So, if you are a Black student in the school with a higher discipline rate, higher Black discipline rate, you feel less belongingness to your school. And belongingness is tied to a whole suite of important academic, and mental health, and school climate outcomes, so this is a concerning finding. And this is true even if you’re focusing on Black students who were not themselves suspended. So Black students who are never suspended, aren’t sort of in that set of students who tend to have classifications of having behavioral issues. Even if you’re looking at those students, if they’re in a school that disciplines Black students more aggressively, those students feel less like they belong. So, another reason to be worried about Black-White discipline disparities. 

But, even if we think Black-White discipline disparities are harmful, there’s not a whole lot schools can do if they’re not partially generated by teacher practices, or school policies, or administrator decisions, or something that goes beyond student behavior. And, of course, there is a rich debate in literature regarding whether disparities in discipline are really just a byproduct of disparities in student behavior. If Black students are misbehaving more than White students, then that might just be the reason we see these disparities in discipline. But a recent sort of flood of research has suggested that’s not the case. Indeed, even when you do what are called vignette experiments, where you try to create a situation and ask teachers to respond to the situation, holding constant everything except for race, we see that when teachers are responding to the same exact behavior, but they think they’re responding to a Black student, that they’re more troubled by the behavior, and they’re more likely to recommend harsh disciplinary action relative to when they are looking at the exact same behavior but it’s a White student. So, holding behavior constant doesn’t seem to eliminate this disparity in teacher response. And we even see this as early as preschool. So, I mentioned before that among preschool students, Black students are over three times more likely to be suspended than White preschool students. Well, when Gilliam and colleagues in 2016 did an eye tracking experiment to see if teachers asked to look for problem behavior would focus more on Black boys. That’s exactly what they found. So, if you’re sort of priming people to think there’s some problem behavior here, then even preschool teachers will focus their attention on Black students and on Black boys in particular. The sort of twist in that experiment was there was actually no problem behavior in the videos that teachers were being asked to review. It was really just a matter of where would they focus their attention when they were told that there was conflict or misbehavior. 

Even if you look at data from the real world, we see that relative to White students, Black students are more likely to be suspended for their first instance of misbehavior, and we see that when Black and White students are in conflict with one another, the Black student is more likely to have a harsh disciplinary sanction at the back end of that. That’s from Barrett and colleagues from last year.  

So, school practices do seem to have a role to play in generating racial disparities. Is there anything we can do to shift school practices to ameliorate racial disparities? So encouraging work on those dimensions comes from some recent psychological RCTs, or randomized controlled trials, that I’ve had the pleasure of being part of. But in two randomized controlled trials, we helped teachers adopt what I think could be called a restorative mindset. Lan mentioned before that, in a restorative context, it’s all about wanting to improve relationships because you think that doing so will improve other things, right? So, one aspect of a restorative mindset from a teacher’s point of view would be that if a student–teacher relationship is improved, that will improve student behavior, and that student–teacher relationships can be improved with effort. So that’s the exact mindset we tried to inculcate in teachers. Via a sort of successful, 45 minute intervention, we were able to inculcate that mindset in teachers. Thereafter, we saw racial disparities in discipline abridge substantially. So, there is something about helping teachers adopt this restorative mindset, this sort of relational mindset, that was a key to helping them treat students of all backgrounds more equitably. So that’s encouraging.  

But what about studies of restorative practices directly? Well, with some of the folks actually in the wings here at WestEd, I had the pleasure of leading a review of all quantitative research on restorative practices. When you look at qualitative research that looks at pre-post study, so what happens in a school after restorative practices are implemented, you see that on most dimensions there’s strong evidence that restorative practices generate positive outcomes. So on discipline, discipline disparities, misbehavior, school climate, the great weight of studies seems to suggest positive results. So positive is on the left, sort of the lighter bars, and the negative results are on the right. But in academics it’s more mixed. So that’s the pre-post studies. 

 

What about the randomized control trials? There have been a few. So, there have been five randomized controlled trials that have evaluated the impacts of restorative practices, and they’ve had really mixed results. Like pre-post studies, they suggested declines in discipline rates, but mixed results on discipline disparities, on misbehavior, on school climate, and on academic outcomes. As a result of this confusing set of results, the authors of these randomized controlled trials have often been asked to ponder whether there were implementation challenges, whether helping a set of schools get access to training and restorative practices really resulted in those schools being more likely to utilize the practices. 

In one such study, in Acosta and colleagues randomized control trial in 2019, the authors had the good sense to not only randomly select certain schools to receive restorative training, but also utilized student and teacher surveys to figure out if those schools used more restorative practices than the schools that were in the control condition. That’s what these two little histograms show. If you’re not familiar with histograms, essentially the shape of the histogram tells you how much of something there is. So, what you would expect to see is two very different shapes for control and treatment schools. What you see instead is in control schools and treatment schools, we see about the same amount of restorative practice utilization. So, the trainings themselves did not shift the level of utilization of the practices. Not surprisingly, then, if the level of utilization is not different between these schools, then the outcomes did not end up being different either. 

So, what we can glean from the pre-post studies and the randomized control trials is that restorative programming and restorative practices are two different things. There’s actually quite a bit of a journey we have to travel in order to get from restorative programs all the way to student outcomes, or even to upstream of that, exposure to the restorative practices. This visual can seem a little daunting, but I think it’s important — and actually not as daunting as it seems — way of understanding why we should be so focused on measuring restorative practices. The first point is that restorative programs do not necessarily result in teachers utilizing restorative practices. That’s for a couple of reasons. One, teachers may or may not be culturally aligned with the practices. So, you can have the best training in world and that could still result in teachers not being able to really take the training and run with it because they’re so focused on using disciplinary tools that it just seems out of step, or out of the realm of what seems appropriate for them, to shift to this relational approach. So, there can be a restorative readiness problem or a cultural misalignment. 

The other reason is the programming itself might not be very effective. You could have a situation where you end up having every teacher in your entire school signing up for and attending a restorative training or program that doesn’t really provide them with the tools they need to make a shift in their practices, to feel confident that they can take on both the proactive and responsive types of practices when opportunities present. 

That’s just at the front end. But let’s say that those two issues don’t exist: you’ve got great programming and really good cultural alignment between teachers in the program. Now you need teachers to actually utilize the practices and to use them across situations. So, you could imagine situations where certain teachers may feel more called to utilize these practices than others because it fits better with the way they normally teach. So maybe the English teachers, the social studies teachers, they’re feeling more like this is aligned with the way that I already teach. It’s already very energetic and didactic and communicative, and so we do see these kinds of relational blossoming happening already and we already focus on trying to overcome conflict. But maybe in math classrooms that’s less the case, and it’s more of a classic Socratic style of lecture, and so there’s not as much attention to relationships. It’s harder for those teachers to dive in and use these practices, so you may end up with sort of variation in teacher utilization.  

Okay, well, let’s say you don’t have that problem, and all your teachers are using restorative practices at least some of the time. You can also end up in a situation where teachers are more comfortable using the practices with certain students and not with other students. There’s quite a bit of research on race matching, to the extent to which when you have a Black teacher and a Black student that students tend to have certain benefits in their performance, and their engagement, and their attainment, et cetera. You may wonder whether the match or the alignment between the race of the student and teacher could be a driver of the extent to which the teacher feels not only empowered but comfortable using such intimate relational practices with the student. Really sort of calling on the student to shift their mindset, shift the way that they relate to their peer. That might be easier to do in some context and harder do in others. 

And then, even if all of that is true, you still want to see if student exposure to restorative practices isn’t conditional on something that isn’t about the teacher but is more about tracking. Maybe students in certain contexts within the school environment are less likely to be exposed to the teachers who are getting the training, or less exposed to the teachers who already got the training and stuck around because they didn’t turn out, right? There are all these things that can happen between you pay for the training and you get an outcome on the back end. 

So, what happens when we measure restorative practice utilization directly, when we measure student exposure to these practices directly? What are the effects of the practices on student outcomes when we’re able to get around all of this complexity and just focus on what’s in the box here? Students got exposed to the practices, what were their outcomes? I won’t spend a ton of time on this aspect of it, but I just want to emphasize the point that there’s good reason to want to get to that point in this long journey, to get to that point where students are actually being exposed. What we see is students who have more exposure to restorative practices see smaller exposure to out-of-school suspensions and smaller racial disparities. So, you can see here for Black students, for Hispanic students, for White students, and for Asian students, those who had more exposure to restorative practices had lower probabilities of experiencing out-of-school suspensions. I can give you all details on the modeling strategies being utilized here. There are a panoply of modeling strategies that were utilized to generate these results. This is actually my dissertation research, but I imagine that is not the focus of this talk. 

Anyway, what this means is that at lower levels of restorative practice exposure, we see a larger Black-White disparity in suspensions, and at higher levels of restorative practice exposure, we see really, really small levels of Black-White disparity. So strong indication that more exposure to these practices could be a strategy for reducing racial disparities. 

We see much the same story when we look at the number of days suspended, right? So again, Black students, more exposure to restorative practices, fewer days suspended. Same for Hispanic students, same for White students, same for Asian students. As a result, Black-White disparities are smaller at the higher end of the restorative practice exposure range. So, the more exposure we see the less there is a disparity in out-of-school suspensions and in days suspended. 

A similar story with academics. More exposure, better performance on the Smarter Balanced English-language arts assessment across racial groups, and marginally smaller disparities, although it’s more of a story of everybody wins, nobody loses. Same thing for the math Smarter Balanced assessment. Everybody wins, nobody loses. 

What about what happens as schools become more restorative? When a school makes a big shift from being less restorative to more restorative, from using these practices less often to more often, what happens to student-body-wide outcomes? Well, we see behavior goes down, gang membership goes down, victimization goes down, students’ expression of depressive symptoms goes down, their sleep deprivation and illness go down. That’s actually quite consistent with a body of research around the impact of exposure to discipline on students’ stress responses, their allostatic load, and other drivers of long-term and short-term health. We see their substance use goes down, so that’s also consistent with research on how students cope with experiences with discipline. So maybe less discipline, less substance abuse. We see absences go down marginally, but it’s not a statistically significant effect. We do see their GPAs go up, and we see their school climate scores go way up. A very, very, very strong effect on school climate. 

Again, if what we’re interested in as a sort of first order question here is “Does exposure to restorative practices make a difference? Does it matter when schools become more restorative?” The answer seems to be yes, but there’s a twist here. It’s not simply the case that when schools become more restorative, they see positive outcomes. It’s also the case that when schools become less restorative, they see negative outcomes. So here we’re seeing on the X axis schools shift in restorative practices. So, if they’re to the right of zero, that means they became more restorative over time. If they’re to the left of zero, they became less restorative over time. On the Y axis, we’re seeing how schools shifted in their school climate. So, if they’re north of this zero, then school climate improved. If they’re south of this zero, then that means school climate declined. What we see is schools that became less restorative saw declines in school climate. Schools that became more restorative saw improvements in school climate. So that relationship between restorative practices and school climate that are reported before, that’s a function of movement in both directions. That’s important because when we think about not only why we should measure and practice utilization, but how we should measure it. That means that we should be sensitive not only to growth, but also to backsliding. 

So, I’m going to summarize, and then I’m going to take the second to answer questions that folks have. But just to summarize so far, when teachers use more restorative practices and when students see more exposure to them, we do see improvements in students’ academic outcomes, disciplinary outcomes, behavioral outcomes, and in school climate outcomes. However, as we see from randomized control trials and pre-post studies, restorative programming does not necessarily lead to teacher use of, or student exposure to, restorative practices. There’s this long journey we have to go to get from “We hired someone to do the programming,” all the way to “and now students are being exposed to the practices and we can expect outcomes.” So then the question becomes, “How can we drive increases in utilization and exposure to these potentially potent practices?”  

I’ll pause there because I would love to answer questions folks have about this lead-up before we go into our next questions, which are why we should measure these practices, how we can measure these practices, and how we can merge measurement of restorative practices with continuous improvement. So now I am pausing. 

Lan Nguyen:  A question came up for me that I think maybe some folks in the audience might be thinking of, so I’ll ask this question, and if you have questions, please feel free to put those in the chat. One of the things you mentioned earlier, Sean, was around mindset shift, right? Restorative mindsets. In your experience or research or conversations you’ve had with people, is there anything that you’ve seen work? How do we help people have restorative mindsets? You talked about that 45 minute intervention. I don’t know if you want to share more about that. 

Sean Darling-Hammond:  Yeah. There’s the basic answer of how do you help people make a shift from one mindset to another, and then there’s a more nuanced answer of what was the psychological manipulation utilized to achieve the mindset shift that we saw in our randomized control trials. On the front end, what is necessary for folks to make a mindset shift? I think a lot of it is it’s a proof-in-the-pudding kind of thing. When people see that their current mindset does not align with the experiences of those around them or their aspirations, that can be a strong incentive for them to shift their mindset. For teachers, there is, I think, often an implicit and not completely inaccurate perception that deterrence is a powerful driver of student behavior, that if we simply make students worried enough that there is a disciplinary outcome around the corner if they misbehave, then students will not misbehave. Sometimes it’s helpful for teachers to realize that that is in line with a school of psychology that is very antiquated and has been largely roundly rejected by decades of psychological research. That sort of an explicit, extrinsic motivation rationale — something outside of you will happen if you do the wrong thing. That doesn’t tend to work across paradigms and across situations. Intrinsic motivation is substantially more effective. So, helping teachers realize that they’re going to get a lot more bang for their buck if they can generate some kind of intrinsic motivation for students to behave properly can then make them wonder, “Well, how can I do that?” 

Then the other piece of it is helping people feel like possibilities, it is possible for student-teacher relationships to improve. I mean, it’s easy to get discouraged, right? If you have had many student–teacher relationships where things just kind of settled into a groove or a rut, and you felt like there was nothing you could do, either you got along or you didn’t, then it’s very hard to say, “Well, now I’m going to put extra effort in these relationships that aren’t as positive.” So, it’s helping teachers make that shift. 

In terms of how we did that, we used a “saying is believing” paradigm. So, we first introduced teachers to statements by their peers about the importance of student–teacher relationships, and the malleability of student–teacher relationships, and the benefits of students being in good relationship with their teachers. Again, sort of getting at this notion that this is a pathway to something that they really want. Then we ask teachers to write letters to their peers about how to generate more positive relationships with students. So that’s the “saying is believing” manipulation. When you’re asked to advocate on behalf of a perspective, it creates this sort of cognitive dissonance in your mind that makes you say, “I wouldn’t have said that if I didn’t really, really believe it.” So, getting teachers to say, “We can really improve student-teacher relationships and here’s how,” makes them then really believe that we can really improve student-teacher relationships, and then lean into the ideas that they created about how to do it. Then we let them go off into the world and see it, see the proof in the pudding. When teachers invest in relationships, they’re often shocked at how much a relationship with a student can change, particularly for middle school students who are really looking to their teachers to be guides about how to be in a relationship with the entire world. So I think that’s a lot of what achieved those outcomes. 

Lan Nguyen:  I wanted to just add, too, to what you were sharing, Sean, about getting teachers to think a little bit more deeply about their belief systems. One of the things I’ve found that’s really powerful is hearing from students, that students actually are really interested in building relationships or knowing their teachers more. That’s something that I found to be really powerful. So, if you’re out there thinking about restorative mindsets and mindset shifts, getting students to share, in their own voice, what it means to them, I’ve seen change a lot of minds very quickly. 

Sean Darling-Hammond:  Yeah, cosigned. That was my experience at Berkeley High. I was shocked with how quickly relationships between teachers and the students that we worked with could improve, and how much that not only helps students feel deeply connected to the school environment, but helped teachers feel more hopeful. We saw teacher satisfaction improve markedly, and that is something we see in many pre-post studies, not all. But most pre-post studies that evaluate teacher climate measures show improvements on those dimensions. 

Yeah, so Julia, that’s interesting that you mentioned. It’s kind of like a communitarian growth mindset. We actually fielded a randomized controlled trial with Stanford’s character lab that was utilizing… we called it a communitarian growth mindset, but the idea that we all grow together when we invest in each other. Yeah, that’s definitely implied within the restorative practice milieu   the idea that being in better relationship with one person isn’t just about what’s good for that person and you, it’s about what’s good for the entire community. 

 And the reason in a restorative paradigm we often see community members participating in restorative conferences, formal conferences between a respondent and a victim or two respondents, is community members have a vested interest in the health of that relationship and those two individuals have a vested interest in being in better relationship with the entire community. So yeah, it broadens the size of one’s human family, which also, I think, is part of what drives the school belongingness and school climate improvements. When you feel like your whole school is part of your family, it’s a lot easier to feel connected to your school, and to want to be there, and want to invest your energy in repairing relationships and improving relationships in that paradigm. 

Yeah, my pleasure. All right. So now we’re into the next section here, which is why should educational institutions measure restorative practices? I’ve already touched on a little bit of that, but it really comes down to this graphic here, right? You can imagine that there’s a lot that can go awry between the restorative program being implemented and getting to the student exposure and outcomes, right? So some questions we might be asking along the way that would help us make sure we’re going from programming to outcomes is, “Is the programming itself effective? Are teachers ready for restorative practice implementation? Is there a cultural fit? Is the school structured so teachers can use restorative practices?” 

I didn’t mention this before, but another thing that can really impinge upon teachers’ ability to use these practices is there even time in the day for them, right? Community building circles require time set aside for teachers to bring students together and help them deepen their sense of connection to one another. If there’s no time in the day for it, it’s really hard to ask teachers to do that when they already have big demands on their time, accountability–oriented testing metrics that they have to hit, et cetera, et cetera. So, making sure that there’s a structural alignment between the goals of restorative practices and the lives of teachers. The next question is, are teachers actually using restorative practices? Do we see variation of which kinds of teachers are using it and are they using with all kinds of students? Do we see variation in which students get exposed? Then finally, I didn’t mention this before either, but this is really important, are teachers sustaining their use of restorative practices? Sustained use can be a function of one of two things at minimum. One, it can be a function of teachers feeling like they can keep trying to use these practices even when they hit setbacks. So not every time that a teacher invested in a student-teacher relationship will result in a big win. Sometimes it can be discouraging. So are teachers sustaining their use of these practices even when they have a discouraging interaction? 

And the other reason is teachers come and go. So, you may have a cohort of teachers that were trained in these practices, but by year two, only 30% of those teachers may still be around. And that actually was exactly the case in the main RCT. We only had, it was actually by the end of year two, only 30% of those trained were still around. So it was really hard to expect the outcomes to continue to flow when those who had been trained had moved on, and those who came in had not been trained.  

These are the reasons that we might want to measure restorative practices. But this issue around student exposure not being uniform is one that deserves additional attention, sort of added weight. When we look at students’ levels of exposure to restorative practices and ascertain whether students of different backgrounds have different levels of exposure, what we find is Black students, Hispanic students, and economically disadvantaged students have markedly lower levels of exposure to these practices, and this is particularly concerning because these are the three groups where we see stark overrepresentation in student discipline. So to the extent that we think student discipline is partially a function of exposure to these practices, and those who are exposed to these practices are sort of inoculated against discipline, it’s quite concerning that these are the groups who are not being exposed to these practices. 

And when I say that these are the groups who are not being exposed, I don’t just mean aggregates statewide or at the level of the district. I mean at the level of the school. If I look within a school and I look to see if Black students within that school have more or less exposure to restorative practices than White students, I will find that averaging across schools, Black students have less exposure. And that really gets at the possibility that race matching matters, or that teacher familiarity or comfort with the ability to communicate with different kinds of students matters. It’s not just a matter of certain schools use these practices. It’s, within the school, teachers are discretionarily using the practices more with certain kinds of students than with others. And I doubt that that’s intentional, but that’s definitely going to impinge on the extent to which the training realizes the desire for student exposure, uniform student exposure to the practices, which is the predicate, the necessary condition for student outcomes. 

So, I just wanted to put a little bit of added emphasis on this part: are teachers using restorative practices with all kinds of students? Evidence suggests that that’s often not the case. So as we measure and try to drive towards exposure and outcomes, that’s one of the things we should be trying to measure. Here again, I thought that it’d be really helpful to pass it to Lan to talk about, what do these implementation challenges look like on the ground, to add more color to the importance of measuring restorative practices. 

Lan Nguyen:  Sure, yes. There’s a particular story I have in mind. And I just want to say, Sean, I appreciate the graphic because I think you’re right there’s just so many points that we tend to focus on in the programming part. And I think that that’s, to me anyways, I’m listening, a really important thing to keep in mind walking away from this webinar. But there’s a particular high school I’m thinking of, and this high school had, it was a smaller high school, it had about 700 students with a new administrative team. And this particular school was actually known for high standardized test scores, high AP test scores and pass rates. And so the new administrative team was actually really eager to bring restorative practices to this particular site, and they drafted a professional development calendar, which lasted over the course of a year at staff and faculty meetings. And as they brought this programming to these faculty and staff meetings, all the staff were actually really enthusiastic and many of them expressed the need. They understood why this might be important. And so after the year ended or near the end of the programming, the administrative team started going around and doing observations. And they actually saw that, in their systematic observations, they weren’t noticing an increase in the utilization of restorative practices. 

And, so, they became curious and what they ended up doing was talking to teachers about, “Do you feel ready to use these?” or “Why, basically, are you not using them in your classroom?” And what they found was that many teachers in this school were really having a hard time contending with the pressures of things like keeping up with the school’s reputation as having high test scores and high AP test pass rates. They identified, or they were able to find, that the implementation challenge was really on the teacher utilization part. And what teachers reported was, “How do I do this and cover content? It seems like it’s taking time. Like, how do I think about that?” It seemed like they needed some supports thinking about how both can actually really happen at this same time. And so that is one particular example of an implementation challenge and I’m sure Sean has other examples. 

Sean Darling-Hammond:  That was perfect. And it really gets at the extent to which often it’s not a matter of teachers not wanting to use these practices, it’s just it’s not easy for them to do so if there’s not a structural fit, or if there’s not alignment and encouragement from all directions. That was perfect. Thank you, Lan. And I see we’ve got some more the chat. Okay, good. 

So, let’s move into the next stage of this, which is “How?”  which is really what you all came to hear about, right? It’s good that we’re getting to the meat of this presentation. How can educational institutions measure practices? And I do want to make sure that folks are asking questions as they come up. Especially in these next two sections, if there’s something that feels particularly relevant or salient to your school context, but you need a little bit of guidance to think about how it could be more directly applicable, please do ask questions throughout. Because the more we can infuse this chat and presentation with grounding on what’s happening in real schools and districts, the better. So please do feel free to ask as you go.  

All right. We’ve covered the why. Now let’s talk about the what we’re measuring, how we’re measuring, and when we’re measuring. On the what, I mentioned that there’s this sort of set of things that we have to ensure happen in order to get from training to outcome. So, what are we measuring when we measure practice? Well, we should be measuring the effectiveness of the training, which is to say the extent to which the training generated learning for those who attended. We should be measuring the extent to which teachers are utilizing these practices. We should be measuring the extent to which students are being exposed to these practices. And, obviously, all of this is predicated on the notion that this will move the needle for students. So, we should be measuring outcomes, not just for students, but also for teachers as well  so school climate outcome measures for teachers  desire to stick around in the school, feelings of strong relationship with students, dealings about student behavior, et cetera, the extent to which teachers feel like they are able to utilize these practices over time with support in a sustained way. We should be measuring these outcomes as well. And as I mentioned before, it’s not merely the fact that movement towards restorative practices is related to improvements. It’s also the fact that movement away from restorative practices is related to declines and backsliding a lot of outcomes. So, we should be measuring movement in either direction  towards or away from implementation, towards or away from exposure, towards or away from utilization, et cetera. That’s the what.  

When are we measuring restorative practices? My recommendation is that there are four moments in a typical school year when it would make sense to measure restorative practices. It doesn’t mean you need to do it at all four points, but these are the four to consider. One is before any professional development related to restorative practices occurs, and this is important because it allows you to capture the baseline, and not just the baseline on student outcomes, which is of course where folks usually focus their attention. We need to know how school climate was before the training so we can know if school climate gets better at the back end but that, of course, is like a very reductionist view of how the training will impact the outcomes. We actually need to be measuring not just the outcomes of students’ but also teachers’ level of utilization of these practices. 

It’s not the case that the only way the teachers can come to use these practices is through restorative practice training. Many teachers actually now get exposure to restorative practices as part of their certification processes. They get exposure to it by collaborating with peers and being part of professional learning communities that either span their schools or go beyond their district. They learn about these practices through media, or it’s just sort of aligned with the way that they normally teach. So, they’re using these practices already. It’s important before the training occurs to consider evaluating the extent to which teachers are already using these practices, and then you can see if after the training the utilization does improve, the utilization does increase, and where it does and where it doesn’t.  

Not surprisingly, the next moment where I recommend institutions evaluate restorative practices is after the training, to capture the extent to which there was learning, or there was growth, and to start to capture some short-term outcomes. Often when teachers are given an opportunity to participate in interventions that shift their thoughts about relationships, they feel this boon of goodwill, of hope, of energy in their teaching, and you can pick that up in a soon-after-training measure. And they may also shift in marked ways how they to relate to students, and that can get picked up in school climate measurement at the student side. 

Then the next point is a few months after training, and that captures some of the outcomes we expect to see movement on in the short term. That’s, of course, school climate. That’s also discipline. The thing that happens most quickly after a practice implementation is a shift in the extent to which teachers use discipline. Sometimes that’s a function of just the school saying, “Don’t do this, do that instead.” But sometimes it’s a function of teachers now feeling like they have two tools in their toolbelt when they used to only have one. You often see a decline in discipline and in discipline disparities after a short time period. 

And then, many months after training, towards the end of the spring, that would be another great time to evaluate, and that’s where you can get that academic engagement. Or is there a drop in absenteeism? Is there an increase in academic engagement? Is there improvement in academic performance that can be in some meaningful way attributed to the training? This is less a talk about how to identify a causal effect, so I won’t get into how one can utilize this measurement to tease out the causal impact of restorative practices on these sorts of downstream outcomes because it gets more complicated. But I will say there are ways to do it. Folks are curious. Please do feel free to reach out to me after the talk. And I’m happy to help you think about how to evaluate these practices to generate a causal estimate. But those are the four points on the when side. 

I see something came in the chat. Let me check. Connie says, “You mentioned the challenge of the time teachers have for hosting restorative conversations. What are some creative examples for how schools have addressed this?” That’s a great question, Connie. It’s not so much a creative one, but it’s an effective one. One of the things that schools have done to try to ensure that teachers have time to do both proactive community-building and responsive repair is to build time into the week that is devoted to restorative time, community time, relational time. One of the best times to do is at the beginning of the week, and one of the other really great times to do this is at the very end of the week. 

At the beginning of the week, it gives students an opportunity to really come into the school environment and provide their teachers and the set of staff that are there to care for them with the most information they can about what’s going on in their lives. You can imagine it’s a lot easier to avoid misbehavior from a student who recently lost a family member if you know that is the case, and you can make sure they feel supported. It’s a lot easier to help a student who is feeling traumatized by seeing violence on the way to school, to feel at peace, if you know that and you can say, “Hey, if you need to go to the nurse and take a second to just talk that out or relax and let your system sort of flush that experience, I’m here for. Or if you need to take a day off. What you saw is really hard. We’re happy to support that.” It’s easier to do that if you’re getting that information. Building it in to the week can be really, really effective.  

And then on the side of relational repair, often it can be helpful to ensure that there is a space for that kind of repair in the school. Lan mentioned that in some schools, they actually have a restorative room. When used well, those can be phenomenal places for students to repair with one another, for teachers and students to repair, as long as it doesn’t feel like folks are going off to a rife therapy session or something. As long as it feels more like this is a place where I can be myself  maybe I go here on my own just to take a breath, maybe I go over with a friend to be supportive to folks who want to have those kinds of conversations, or maybe I go here with a teacher. But that can be helpful too. Creating space and creating time can be really, really effective tools on those dimensions. Yes. I think Lan, that is exactly right. It’s not about time consuming, it’s about making a down payment that will actually save time and energy down the line. I think that’s totally right. 

That’s the when. Now let’s talk about the how. How are we measuring restorative practice? So, I recommend using both quantitative and qualitative means. On the quantitative side, I think that there are some both low cost and already-at-your-fingertips quantitative tools to evaluate the extent to which teachers are feeling like the training is effective, teachers are utilizing these practices, students are getting exposure to these practices, and students and teachers are benefiting. You can use surveys to do this, I think, very effectively and very quickly. And these surveys can be with students, teachers, and restorative coordinators, and with school administrators. Some folks here are at the district side and principals and vice principals have a really good lens, often, into the incentive structures in schools around these practices, or the communication that’s been given to staff about commitment to these practices and sustaining that commitment, or the structures in place to empower the utilization of these practices. Surveys can help capture that and give district folks a sense of, “Well, why might we see more uptake of these practices within this school environment but not within that one?” Administrators can often be an important part of that puzzle. I also think qualitative interviews and focus groups can be incredibly helpful sources of information, but when used appropriately.  

Quantitative surveys are great at capturing variation between folks over time. You can use a survey of a large number of the students within a school to figure out if certain kinds of students are getting more exposure to these practices, and you can use a follow up survey to see if you can move the needle on that dimension  if you can increase the extent to which, say, Black students are getting exposed to restorative practices, or special ed students, or certain grade levels. Quantitative surveys are phenomenal for that over time, and for between-group kind of estimation, that comparative estimation.  

Qualitative data are not phenomenal for that in this context in particular. In my experience, it’s partially because if you put a call out to students and teachers in a school to talk about restorative practices, the folks who come tend to have opinions about restorative practices. You’re not going to find a student who didn’t even know there was a restorative program in the school and get them to come in. And so you’re not going to find out about student-level variation exposure. And you’re not going to hear from the teacher who didn’t show up for the training, because they didn’t think it was useful. You’re not going to get a sense of teacher-level variation in utilization or even in participation, like uptake in the training at the front end. I think qualitative interviews are not great for that kind of over-groups and over-time comparative analysis. 

But they are phenomenal for getting a really deep sense of why something is or is not worth working. What you’re trying to get a sense of is: we have a paradigm right now where we have community building circles on Mondays at this time, and we structure them so students are asked to talk about their lives, and then we do a little bit of role playing of conflict situations, and we want to see if that’s working. A focus group is a phenomenal way to get real student perspectives on the ground and real teacher perspectives on the ground of whether that setup is working or should be tweaked in some meaningful way. Surveys often can’t get to solutions, but they can get to, “Is this working?” Focus groups and interviews can get to, “What should we try differently?” I recommend a mix of quantitative and qualitative approaches to answer different, but also important, questions. 

 

Just a couple more things I wanted to say about how we measure and then I’m going to check into the chat because I love what’s happening there. One, I think, source of trepidation that schools and educational institutions often have about measuring any school practice is fear that it is cumbersome, expensive, and often fruitless  like a fruitless endeavor. What I want to recommend here is the use of free technologies like Google Forms to create teacher surveys that can track implementation with the kind of temporal granularity that one would need to be able to say teachers learned from this training or teachers increased their utilization, and also the spatial granularity to be able to say, “This teacher is using the practice, but this teacher needs some more support.” Or, “This school is using the practice, but this school needs more support.,” Or, “This subset of the school, the grade level,” or some schools have schools within schools “this like arts set of students or this tech set of students, they’re having a really hard time getting exposure, but this set of students is having no problem at all.” So, you can get that with Google Form surveys, fielded with the right temporal granularity with, so frequently enough, with teachers and students. One of the things that’s phenomenal about Google Forms is if the surveys are created in a streamlined enough way, they can be done on a phone, and they can be done in a very small amount of time. If you’re only asking 15 or so questions on one to seven Likert scales that are exactly the question you want to ask at exactly the time you want to ask it, you can get phenomenally useful data in a day, by just putting it out to the school and community and saying, “We’re all going to take 10 minutes right now for everyone to just take this very brief phone-based survey, to get a sense of how you feel about this restorative practicing we’re doing. And this is for all the teachers and all students. Let’s just take 10 minutes really quickly to fill out this form-based survey.” You can do that before the training, after the training, a few months later, and at the end of the semester. You’re talking about an investment of maybe 40 minutes of school time over the course of the year to get incredibly useful data. 

And I know 40 minutes for the whole school is a lot of time because it’s the whole school. But it’s a lot less than the amount of time that’s often used to do sweeping, large surveys that ask a lot of questions you might not need answered to get a sense of restorative practice utilization and identify challenges along that continuum from training to outcomes. Yeah, I strongly recommend using Google Forms. And just to give an example of the kinds of questions that might be on there, if you’re trying to get a sense of if the programming itself effective, you can ask, “Did you attend the restorative practice training?” This would be for a teacher survey. “Did the training provide clarity about what restorative practices are and how to use them? Did it persuade you that implementing these practices can improve outcomes for students?” There you might see disparities in teachers’ level of training being a driver of their sense of persuasion around this. Teachers who have been there longer might be more likely to say, “You can tell me what you want, but I’ve seen a lot of school practices come and go, and I’m not going to change my practices. It’s worked for a very long time.” And many times they might be right, but that might let you know you need to target some of your more veteran teachers and say, “Okay, what can we do to help you try this out?” 

You can ask after their RP training, “Did you feel like you knew what you needed to know to implement these? Did you feel empowered to actually go out there and do it? What more do you feel you would need to know?” And that could be a qualitative, enter-it-as-you-go response, just typing a little bit of content there. 

Sean Darling-Hammond:  “Please indicate the extent to which you believe the RP training empowered you to overcome the following implementation challenges.” You can imagine a handpicked set of four or five important challenges you know teachers will face. “I have a difficult relationship with a particular student and now I feel like I know how to start to repair it.” Or “There’s just not enough time in my day to build community, but now I feel like I can imagine where I would fit it into my day.” Or whatever. And then, “How would you suggest improving the RP training?” So, this is just like the kind of thing you might ask before and after the restorative practice training, to really get a sense of the effectiveness. And then of course you can use qualitative tools to get a lot more depth into how to improve things. 

And then on are teachers using restorative practices and are these practices being used when teachers are interacting with all kinds of students, you can imagine teacher surveys where you simply ask, “To what extent do you use each of the following practices?” And have a list and give teachers an opportunity to use, like a drag bar or something, a thermometer display, or a one to seven Likert, or one to 10 Likert, whatever fits what your teachers are used to, to indicate their level of utilization of these practices. 

But critically, you do the same thing with students. And this is important because often teachers … And this is not just a teacher thing. This is anybody who interacts with a lot of folks at another level of hierarchy. If they use a practice with some folks, they feel like they use it with all folks. We see this with CEOs and employees, we see this with teachers and students. We see this with professors and students and sort of higher ed. If you do a thing with 20% of your students or 20% of your employees, in your head you do it with all of them. The only way to get a sense of whether a given teacher is actually using these practices with all students is to ask the students, because that’s where you get that variation in, or get a sense of the variation of, who’s actually getting exposure to these practices.  

In the same way that you ask the teachers, “Do you use these practices” on a one to seven or thermometer display or whatever, on the students survey you ask, “To what extent do your teachers do the following things?” And here you can see there’s sort of a shift in language. With teachers, you can say, “Use the following practices”, because they think of teaching practices. With students, you probably want to use different language, and you probably want to adjust your language for the grade level. Right? So if it’s a middle school survey, you might use even more simple language. As an elementary school survey, even simpler still, right? But then once you have data from lots of students, about the extent which they feel like their teachers are using a given set of practices, you can disaggregate their responses by race, within classroom, by socioeconomic status, by special ed status, by gender, by whatever, to get a sense of whether there’s meaningful variation in students’ exposure to these practices. And, then again, there are qualitative approaches we can use. You can have interviews or focus groups with teachers about the extent to which they’re using given practices and the challenges they’re facing in using them. And the same with students, about whether they’re seeing their teachers use certain practices and a little bit of color about what their experiences have been.  

All right. So let me see if there are there any questions here? Yes, these are great. And I love that you all are interacting with each other. So let me just see, now this question. “We are facing staff turnover this year, so inconsistent training and therefore implementation and buy-in. Admin does not have full buy-in yet, so they are not modeling expectations. For example, staff meetings do not start with any type of check-in or community building. Is there a source for good questions for measuring RP?” 

Yeah. So, we’ve actually shared on the second question, we’ve shared a resource, which is RP-Assess, which has a sort of set of 12 measures of RP implementation. But I do think that some of the questions I’ve given today are meant to sort of prime the pump and give you all a sense of the kinds of questions you should ask and the buckets that you want these questions to be responsive to. The reason I think there’s no one size fits all approach to asking or answering these questions is every school context has a different set of challenges. You don’t want to be asking questions that aren’t going to be particularly useful to identifying challenges and overcoming them. Again, the goal here is to have 10-minute surveys done maybe three or four times in the year that give you exactly what you need to know and nothing you don’t, rather than the 100-question survey with lots of qualitative answers that you do once and then you kind of puzzle and scratch, like what do we do with all this data? So I think pulling from the things that you’re seeing here, picking and choosing and creating your own surveys, is going to be the way to go. 

Then on the staff turnover and inconsistent training, that’s just so consistent with what a lot of districts face, and there’s no obvious answer to it other than as staff are coming in and as administrators are sort of thinking about the benefits of restorative practices, it’s probably beneficial to try to have administrator–facing communication about the extent to which utilization of these practices is a priority, but also that it’s rooted in science. It’s rooted in good practice. It’s rooted in the kinds of things that can improve relationships and improve student outcomes over time. And communicate the support from the district for administrators to use these practices in whatever ways you can, so folks don’t feel like this is panacea of the day, but it’s something that is going to be a priority in 2030. So it’s worth investing in it now, so you can get the benefits of it now. 

So, one last thing on the “how.” So, many of you all have access to California Healthy Kids Survey data, and within the California Healthy Kids Survey module there are actually eight questions that I’ve used to ascertain the extent to which schools are using restorative practices, which I want to recommend you all use as well. So here are the eight questions in the California Healthy Kids Survey that I think get at three critical elements of restorative practices. One is, “Are there repair practices in the school?” So, that is, the school helps students solve conflicts with one another. And, “If I tell a teacher that someone is bullying me, the teacher will do something.” That gets at the school is proactively repairing. Two is community building practices  the school is inculcating conflict resolution skills of various kinds. Then the third is measures of breadth  to what extent are restorative practices being experienced by students of all backgrounds. 

Sean Darling-Hammond:  So, “Teachers show it is important for students of all different races to get along.” “The adults in the school respect differences in students.” These aren’t necessarily the perfect questions, but this is a reliable scale that I generated with the assistance of Anne Gregory and others to utilize a California Healthy Kids Survey to measure restorative practice utilization at the level of the school. So what one can do is take the California Healthy Kids Survey data that you have and average it across all students within a grade or within a school to get a sense of the school’s level of utilization, or the grades’ level of exposure, at that point in time, and then you can track progress over time using that data. A couple things to note if you’re going to use California Healthy Kids Survey data, data is not collected at a high frequency. Typically, it’s either every other year or every year, so that means, when I was talking about you want to measure this three or four times throughout the year, California Healthy Kids Survey data is in a very different paradigm and so should be used for very different things. It’s more for:  we had a huge investment in restorative practice, training, and coaching, et cetera over the summer, and so we’re going to use our data from the 2018-19 school year before that huge investment and compare it to our data from the 2019-20 school year and see if that investment paid off in terms of students’ levels of exposure to these practices and teachers’ levels of utilization. Because you can use the California Survey of School Staff as well to get a sense of teachers’ levels of utilization, and they have the exact same eight measures in that survey. So, that’s an example. 

Another challenge is Cal-Healthy Kid Survey tends to measure data on odd grade years or odd year grades, so fifth, seventh, ninth, 11th, and not on the even grades. That may or may not matter for your school context, but it may be very important to get a feel for what’s going on for 10th graders or for 12th graders. It probably will matter that you’re not getting fifth grade data if you’re in middle school and that’s because I sort of glossed over this, but when you look at grade as a predictor of escalation of discipline experiences, the grade where we see the biggest jump in student exposure to discipline is sixth grade, without questions. From fifth grade to sixth grade is such a sensitive moment in students’ growth. So, if you don’t have CHKS data from sixth graders, which most district do not, that’s something to think about. That might be a hole that you need to plug to evaluate the effectiveness of the PD. 

Next thing is, sample sizes vary within schools, in many instances within grades, within subgroups, so just be sensitive to that. If you’re using CHKS data, try to get some measure of what percentage of students within a given grade participated in the survey in your school. CHKS aims for 70% for all grades that are evaluated, but it might be lower in your school or higher in your school, same for with any given subgroup. So, and again, the goal is 70% of all, say, Black seventh graders will fill out the survey, if they’re a target of a given year, but you might not have gotten a 70%. You might have gotten a lot lower. And then when you are disaggregating your results, you might be a little bit more skeptical about the validity of your data. So just being sensitive to sample size. 

And then the last thing is, there can be real misalignment between the CHKS data, which does have this grade level specificity of the grade the student is in, and the California Survey of School Staff, which doesn’t have a lot of specificity in which grades teachers teach. The CSSS data, the staff data, it simply asks, “Do you teach fourth grade or below or fifth grade or above?” That’s not that granular. So, if you’re trying to get a sense that, you felt like middle and high schools, of whether you’re getting more utilization of these practice among your high school teachers, you’re going to get a mix of middle and high school teachers that you’re going to be able to look at data from. 

So, again, you might want to plug some holes if you’re trying to get a sense of grade level specific teacher utilization measures, but it can still be a really helpful tool that you’ve already paid money for, and invested in, and you already have access to from prior years. You can get baseline and from future years, so you can get growth. So, I encourage folks to consider using it.  

So, this is great. I will say, Jennifer mentioned Youth Truth. They do have better breakouts, but they don’t have data on restorative practice utilization. So, I think if folks have different opportunities to invest in different tools, or to create their own, my strong recommendation is just to be very thoughtful about what you want to measure  not just when you want to measure or how you want to measure, but what you want to measure, because there is real differentiation in which questions are being fielded in each of the sort of stock surveys that are out there. That’s the only thing we have for now. So I’ll keep moving.  

So, I just want to spend a couple minutes talking about continuous improvement as a paradigm for utilizing this great data that I hope you’ll collect on restorative practice utilization. For those who are not maybe familiar with continuous improvement, it’s essentially a paradigm to try to steadily drive progress towards a goal. And in this paradigm, you identify some challenge, you set a measurable goal, you make some change to try to drive improvement, and then you track progress. That’s it. Just identify a challenge, set a goal, make a change, see what happens, right? So how could this be applied in a restorative context? There are a lot of questions that could be a target. “Teacher uptake of voluntary professional development is too low” or “teacher utilization is uneven.” So, let’s talk about those. 

Let’s say you identify the challenge that teacher uptake of the training is low. You can set a measurable goal: let’s have 80% of incoming teachers participate in fall professional development. Well, now we need to make some change, right? So, you need to do something to try to increase the proportion of teachers who are participating. Let’s say that we’re going to make the change of inviting teachers to a series of paid sessions to discuss and alleviate concerns about implementing RP. And those can happen over the summer before the PD opportunity occurs, right? So maybe your bet is, folks aren’t participating because they just don’t think it’s going to work, right? So, you’re going to make this bet, and you’re going to make a change, and then you’re going to track progress. So, then you can identify the percentage of incoming teachers who participate and see if you hit that 80% target. So that’s an example of a PDSA cycle around trying to increase the proportion of teachers who take up. And then if you do spring training, you can make another change, or add to that change, or sort of double down on that change, and scale it, depending on how things go in that first cycle. Sorry, and PDSA is Plan, Do, Study, Act. It’s just a continuous improvement target.  

So, another example, let’s say the challenge I identify is teacher utilization of restorative practices is super uneven and that your math teachers in particular are having a really low level of utilization. So you set the measurable goal:  by the end of the year, we’re going to have all math teachers at least at we’ll call a medium level of utilization. If you’ve sort of typified your teachers as having low, medium and high levels, you want to have of all your math teachers hitting at least medium, so getting from a lot of low to at least everyone at medium and maybe someone high. So, what change you’re going to make, provide math teachers with supplemental PD regarding how restorative practices can be used in math courses. Maybe they need, sort of more targeted professional development. And so, you provide that, and see what happens, and track progress at the end of the spring. You identify math teachers’ levels of utilization, and you see if you really did hit a minimum of medium levels of utilization.  

So those are just two examples, and that is the end of our journey from what restorative practices are, why we should measure them, how we should measure them, and how we can think about continuous improvement in restorative practices.  

Lan Nguyen:  I just wanted to thank Sean for his insightful work into restorative practices. As a practitioner myself, I think his work is so important for advancing restorative practices in schools — to really think critically about what are we measuring and what aren’t we measuring, and how does that relate to our improvement efforts as it relates to restorative practices? So I want to thank you for sharing all of your knowledge and research.  

So just a little bit it more about the Center. I shared a bit about what we do, the free supports we provide. I just want to kind of share a few more. School climate data use webinar series is a part of our offerings. We also have peer learning exchanges, which is a bit more of an interactive small group session to learn about a school climate topic. We also offer various professional learnings, virtual courses. This annual event is part of the professional learnings. Then we also have briefs as well as technical assistance support. So, these are one-on-one consultations. All of those things can be found on our website. 

And again, if you’re interested in the event alerts, we have a newsletter that’s on our website. We invite you to subscribe, email us at [email protected], and then our website once again. And so with that, we’d like to thank you for joining us and we really appreciate you taking the time to come and learn about all the great things that Sean had to share with us today. 

Sean Darling-Hammond:  Thank you so much. 

Lan Nguyen:  Thank you everyone. And I hope you have a great rest of your afternoon and week.