Lan Nguyen: Good afternoon, everyone. Welcome and thank you so much for joining us for our webinar, “Using School Climate Data to Meet the Moment.” This webinar is being hosted by the California Center for School Climate (CCSC). My name is Lan Nguyen. I’m a Technical Assistance Provider with the Center, and I’m joined by the team, Nisha Bala, Rebeca Cerna, Tran Keys, as well as Lora Markel and Stacy Miles. Today some objectives we have are to, first, learn how to develop questions and to align them with data sources. Our second objective is to identify data sources that are relevant and accessible in order to address questions. And, finally, to increase awareness about the quality and characteristics of the various types of relevant data sources. And to get us there, we’re going to begin with acknowledging the moment. I’m talking about school climate domains, thinking about what it means to ask the right questions, identifying data sources, and then we’ll close our time out with some questions and final thoughts. If you notice on the screen, items two, three, and four are a coral color and they will match a tool that we’ll be sharing very shortly that accompanies this webinar.

Just to get us started, there’s a diverse group of people that registered for today’s webinar, professionally speaking. We want to get a pulse check on what your feelings are around school climate data, and so, in a moment, we’ll launch a poll. And the question we just wanted to ask you was, “Which picture best represents your feelings when it comes to school climate data?” We have four options. The first option, A, I would say, looks something like excited and happy when it comes to school climate data. Option B looks pensive and thinking. Option C is maybe, I don’t know, trying to figure it out. And Option D might be surprise or even in awe. And so we’ll go ahead and launch that poll right now and give you about 15 seconds to answer that question.

All right, great. So it looks like we have a mix, but it looks like by far the category that most folks are feeling is option B. You’re thinking, maybe you’re wondering, “What can I do with school climate data?” Some of us are in the excited and happy. And I’m glad to see there are a few of us that feel like you seem confused, maybe you don’t know. We hope that by the end of this webinar, you feel a little bit less like that, but thank you for sharing your responses. It’s helpful to know.

As we said in the agenda, we want to start by acknowledging this bigger moment that we’re in, and what we mean by that is just how all of these factors have really played into and impacted teaching and learning. Of course, the ongoing COVID-19 pandemic, the ongoing broader conversations around social and racial justice, social and economic instability, mental health impacts, school staffing shortages, and all those things together really create an environment where our schools need to respond to these ever-evolving conditions. And so it’s against that backdrop that this webinar is happening, and we acknowledge that and appreciate you showing up today.

One of the things that we’re hoping to accomplish with this webinar is to make a connection between the moment, or the moments, really, that we were just mentioning, and what school climate data can help us with. The first thing is, during uncertain times, we can really think about what are some problems of practice and how we can turn those into questions. We’re using the tools and the data that we have right now. And the second thing we wanted to offer was that data inquiry is for everyone, regardless of your background. It’s not just for researchers, and, really, it’s for anyone who wants to answer a question. So it’s our hope that by the end of this presentation, anyone that has a question can use data to help inform their work at any level of education.

Rebeca Cerna: Thanks, Lan. Before we move forward, we would like to provide some foundational language on school climate. My name is Rebeca Cerna, as Lan mentioned at the beginning. What do we mean by school climate? Here on this next slide we see school climate refers to the qualities of a school environment, and it’s reflective of the school culture and the experiences of everyone in the school community, including students, staff, and other school partners who may be supporting your schools. School climate is about the relationships, the teaching and learning practices, and also about the organizational structure. Research also shows us that school climate sets the conditions for learning and teaching to happen. It’s very foundational to school and student success.

And, we pulled this statement from the California Safe and Supportive Schools website. We know that learning happens when all members of the school community feel safe physically, emotionally, and psychologically, when they feel valued, when they feel cared for, respected, and engaged. As we know, as you can imagine and as some of you know, there are several school climate domains, and on this next slide, we have several key school climate domains for you. Here you will see those, including engaging families, knowledge and perceived fairness of discipline rules, mental health/ mental wellness, respect for diversity, safety—as we just mentioned, physical, emotional, psychological—school engagement, school supports for students, student agency, the quality of a physical environment, and the quality of relationships. And relationships, often considered to be one of those essential aspects of school climate.

But under each of these domains, there are also subdomains. One of those things that we’re offering as part of this is our school climate data inquiry tool. We will be sharing the link in the chat to that tool in just a moment. Thank you, Lora. The first section in that tool looks a little bit deeper at these domains and we have offered examples of subdomains for each of the domains. That way, for example—in this case that you see, you see school engagement, and knowing those subdomains, it’s helpful to know, “What is part of school engagement?” And these examples that we have for school engagement are school connectedness, academic motivation, high expectations, and attendance. It’s important for us to know what these various components of school climate are so that it can help you consider data sources as you’re moving forward in your data inquiry process. Now I’m going to pass it to Lan so she can guide us forward.

Lan Nguyen: Thank you. We’re really going to talk about this idea of what it means to ask the right questions. But before we get into that, we really want to hear from you in terms of, wherever you’re sitting, what’s a school climate challenge that you’re particularly interested in learning about in this moment? And we would like for you to share your questions or challenges anonymously, using Menti.com. So we’re going to have the access to this pasted in the chat, but there’s three ways to get to this. The first one is the direct link, which you can click on. The second one is to go to Menti.com and enter the code 6 7 3 6 5 1 6 6. And the other option, as maybe some of you have already realized, is to use the QR code on the screen. If we could just take a few moments to do that, and we’ll check out, what are some of the responses or pressing school climate challenges that are top of mind for us right now?

So far it looks like we’ve got some responses around chronic absenteeism. That’s a big one. Re-engaging disengaged students and staff. Student attendance. Teacher burnout. SEL. Chronic absenteeism, again. Morale; feeling valued and respected so staff don’t quit. Yeah, that’s a big one. Mental health and wellness. Increase in maladaptive behaviors this year/post-distance-learning. I’m interested in how the pandemic has affected school climate. Attendance. How do we get more families involved at the secondary level? Concern with building relationships in a virtual environment. Effects of two years of pandemic learning. Engaging families. Adult wellness. Student engagement. Student mental health. Many professionals are not aware of their own biases. Triangulation of data. The impact that school climate has on students with disabilities. Process, procedures, templates, ideas for digging into data. Motivation. Microaggressions. Parent views.

It looks like we have a lot of different challenges that we’re thinking about right now that have really been exacerbated by the pandemic. And many of these are interrelated. We want to thank you so much for sharing the school climate challenges. This will also help us, as a Center, provide more relevant supports for you all. So we really appreciate that.

When thinking about this idea of asking questions, there are some certain goals that we might be hoping to achieve. And among those are seeking new understanding of particular phenomena. And some of you all shared that, whether it was chronic absenteeism or engagement from students, parents, in the community, exploring current school climate practices, identifying root causes of inequities, and measuring progress, among many goals.

When we use this word, “right” questions, really, it’s about questions that are responsive, relevant, and meaningful. And when we create questions like that, we can establish a connection between data and an authentic need. We can help contextualize the findings with what we already know. And importantly, in a world where we often have a lot of data, focusing our approach to the data can be an important outcome of creating these meaningful school climate questions. And in this way, when we think about our larger challenge, those challenges that you all shared are oftentimes really big, right? So how can we turn those into questions that can allow us to understand root cause, which then would hopefully allow us to make decisions based off of that better understanding of our school climate challenge.

How might we go about developing what we call the right questions? Keeping in mind that idea that school climate challenges can be complex, and, in fact, they may spark many questions. Sometimes we actually might have too much data and too many questions. Focusing on one question at a time can actually make it easier—or even multiple related questions can make it easier for us to reasonably act on. Understanding these questions can even reduce overwhelm in terms of understanding that challenge. So, we want to use an example, let’s say you’re a principal at a school that has been noticing some possible decreased levels of school connectedness during the pandemic. And so, as we seek to understand, what are the root causes, potential root causes, for that, we might want to first begin with, “What are our assumptions about why that is?” As a principal in a school, or anyone, really, you have some information based off of your daily experience. So it’s important to address those assumptions, and those assumptions may very well be correct. The second thing is, “How does this particular phenomenon or challenge impact different groups of students differently?” So, for example, we could be talking about ninth grade students, fourth and fifth grade Latinx students, undocumented students. And then, finally, patterns over time. Are these patterns really the result of the pandemic? Do they even—sometimes even time of day or time of year, those things might be different. These are really great places to start when we want to think about, what are some questions that we might want to investigate, and what are some focal points we might want to consider as we continue to explore the possible root causes of our school climate challenges?

Earlier we shared the tool, and perhaps we could share it once more. And these sections match with the tool Section B: Asking the Right Questions. And we created this tool to help organize your thinking in thinking about the challenges and the possible questions that they might elicit. In this chart we have, “What is your school climate question?” “How does the question improve your understanding of your school climate challenge?” and “What are some data that you currently have access to that might help you further answer your question?” In the tool, we actually have further guiding questions to continue getting you thinking about the different aspects of this challenge and maybe particular focal points. With that, I’m going to go ahead and think about identifying data sources, and passing it to Nisha.

Nisha Bala: Hello, everyone. Like Lan said, I’m Nisha Bala and I’m here to talk you through some of your questions through Identifying Data Sources. But first, we do want to hear from you, and we want to really put you into the mind space of thinking about what data you already have access to, to help you answer some school climate questions. So we’re going to be using the same Menti link that we used for the last question. If you already have it open, you just need to go back there. If not, we’ll be sharing that link in the chat again, so you’ll be able to use the direct link or the code, just as we did before. And we’ll be heading to answer, “What types of data do you have access to, to help answer your school climate–related questions?” I’ll give everyone a couple of moments to think about it and go ahead and put that in.

All right. Reports are a good one, especially because a lot of people who are working directly with students get all sorts of reports all the time, not just with report cards. Student surveys are the most typical one that people think about. A lot of the time, student surveys are a big source of where student data comes from in California. Empathy interviews are a really great answer. I think that they’re a really valuable source of talking to students. Panorama surveys, another great source of survey data. I think that district climate surveys are more granular. It has different aspects, more specific aspects, that it might be giving you information about. Listening sessions are a really lovely idea. Behavior, really trying to understand how the students are behaving in your class as a source of data, is a wonderful idea. Educational partner feedback. That’s a really wonderful idea too. So you’re thinking about reviews of your own work, evaluations using that. DataQuest, another wonderful, wonderful source of data, another classic source of data. Lots of mentions of survey data, which makes sense because that’s usually what people think about when we think about school climate data. It’s the most obvious source. And that’s sort of what we’re here to talk about: how we can supplement or add to our understanding. Along with school climate data, how school climate data can be used alongside our other sources of data.

We’ll be going on to really thinking about data sources. One of the most important steps in starting your inquiry is to start inventorying your data sources. Really understanding what data you have access to will help you use it better. Just simple as that. Your first step is inventorying. Start with the data you have and really understand how it was collected, how it can be best used. Really understand how these sources can be explored further, what new types of analyses, what new patterns you might see, how can you merge the ideas. And then you also have different sorts of additional data. These additional data are data that you, in your various roles, especially, can collect and systematically add to your repertoire of data. You can brainstorm what additional types of data that you might need. This data, this process, does not need to be complicated. It just requires you to collect and store data that you already have access to. And, using these two types of data, you can help figure out how you can answer your questions a little bit better.

Moving along, there are also different types of data that can help you answer your questions. The most typical divisions that we see are quantitative data and qualitative data. As it sounds like, quantitative data has to do with numbers, and therefore, the questions that quantitative data can help you answer have to do with numbers. Questions like, “What is happening and how many people are it happening to? Who is this happening to? How often does this happen?” Questions that numbers are the answers to. Qualitative data relies on words, non-numeric data. And you can use this to answer questions about why and how many.

This can be a little vague, so let’s look at some concrete examples right now. And a lot of these examples are examples that you brought up while we were doing this exercise. Classroom observations. School and staff survey data. Administrative data. We talked about reports. Also, attendance data is usually a really great source of understanding where school climate is. We have dosage data, which is what we think of as the length of time that people have been exposed to certain programs or certain services and how experiences changed because of the length of time. But, also, the vast variety of different assignments that teachers can give, or different tracking that teachers can do for themselves, or school districts or schools can capture for teachers. We’re thinking of these focus groups and conversations. We can think of interviews, or even diary accounts, collecting photographs. These are all really meaningful sources of information that can add to what we are getting from our quantitative data, such as school climate surveys. One of the benefits of qualitative data is especially the fact that the people closest to qualitative data really have the best scope at analyzing such data.

But one of the most important things to realize is that quantitative and qualitative are usually used in opposition to each other. Using these trends creates a false dichotomy. And what is really important is the fact that we can marry these methods together. And this is what we call mixed-methods data. We can also call this methodological or data triangulation. And this really is important to answer the kinds of questions that materially impact lots of you in schools today, these multidimensional questions that have to do both with how many people are these affecting and each individual that is being affected.

Let’s take the example of chronic absenteeism. A lot of you have stated that’s a really big concern, and it’s very reasonable. There has been a large surge in chronic absenteeism during the pandemic, and a school district might be interested in understanding how this has impacted their district, how this has impacted this ward. So quantitative data would help you understand who these students are that are absent, how many of them are absent, what days they might be absent on, what could be causing these absences, so which groups of students might be absent. Are there any situations that might affect that? While qualitative data really helps you answer, “Why?” So if you go up to a student and ask them why they haven’t been coming to school, that inherently gives you a piece of data, and you collect all of these various types of data to create a beautiful picture of how, how deep, and how wonderful—how deep the understanding of these data can be. We really do believe that understanding something like absenteeism, like this, can help policy implementation, because mixed-methods data really does a wonderful job of pointing out how the impact of policy implementation works. How will people react to the policies that get implemented at a classroom level, when teachers implement a class policy for their students, to a district level, when district officials implement policy for schools? We do know that research of this sort can be driven by needs, and needs to be responsive to external events. And that’s what we think that mixed-methods research using data triangulation is particularly important for.

Moving along, we believe that identifying the various data sources in a systematic way is a really critical part, and so this has been incorporated in the tool as well. This slide has two sections corresponding to the two sections available for this on the tool. The top one references existing data, so, how you can inventory that existing data and learn how to better use it. And then the bottom section talks about the additional data that you might need. It gives you tools to brainstorm what additional data you could get to collect, how to collect this type of data, how you can sort of think through the various different types, especially, of data that you can collect as a practitioner or a district administrator. We know that also, of course, thinking through all of this logic is really difficult. And so we have another suggestion for how you can organize your thinking through of the process of going through all of these different types of data. This is drilling down, as the graphic that we have on this screen shows us. And the idea is that you drill down from a general level, a broad level, of understanding to a more specific issue. So you start with aggregated data, data consisting of all of your context, and you go to exploring one specific domain of school climate data. You then disaggregate it, you consider how different groups are affected by this. Then you consider subdomains. In a lot of instances, this can look like, as various parts of it, you can look at various questions. You make the part of school climate that you’re looking at smaller and smaller, so you can have a more and more insightful response. And then you collect direct testimony. You take this to your adults, the teachers involved, the educators, the parents, the students, and you have them respond to this data.

Let’s go to an example of this. Let’s say that, based on your experience, you are interested in understanding declining levels of student engagement during the pandemic. You can be whoever. I’m going to actually give a few specific instances of how different people can experience this in different ways, going from district administrators to teachers. Think of this through your own perspective. Student engagement has been declining through the pandemic. One of the first ways you can look at this is just generally considering dashboard data or report data that is sent to you. Most people, especially in California, have access to this type of data, both in the context of their state and of their district. Potentially comparing these both might give you an idea of how you can holistically view this situation in your district. It just gives you a broad overview idea, just really briefly, to understand contextual clues. Then you drill down one level and you start thinking about domain-level data. And in this case, we’re talking about school student engagement. We’re interested in how students are participating in school, what self they’re bringing to school. And, of course, this has lots of parts, but right now we’re thinking about the whole piece. We might be interested in comparing this to different domains. We might be interested in comparing this across times, so we might be interested in looking backwards in the past to understand what student engagement looked like before the pandemic. But that’s not all. We’re going to drill down one step further to disaggregate the data. We might find that data for older students, such as 11th graders, might have been—student engagement might have been dropping for 11th graders, or student engagement might be dropping for people of economic disadvantage. Understanding which groups these changes have happened for really helps us ensure that we understand the construct better. It can point to various issues as to how different people might appreciate it. Why would student engagement be lower in older students? In a lot of cases, because older students have different responsibilities or a different perspective on what’s happening in the world, and so the pandemic would be affecting them differently. Understanding these contextual cases in the context of the larger domain that you’re working with will really help you understand that domain better. Understanding each little individual part will help you put together the whole.

So in that spirit, we drill down one step further. One way we can do this is going through the parts of student engagement, as Rebeca mentioned earlier. We’ve got high expectations, student connectedness, academic motivation, attendance, as some of these parts. We might be interested in academic motivation, which we find, academic motivation really has also dropped during the pandemic. We might be particularly interested in particular survey items, such as “I try hard to make sure I’m good at schoolwork,” or “I’m always trying to do better in my schoolwork.” And what we would do with this is really start to explore a little bit more deeply. The survey questions, the survey constructs, at this point, are just suggestions for how you can build more data, access more data, to help answer these ideas in your world.

If you are a teacher, what you could be doing at this point is, you could collect exit slips after every lesson, asking students, “Did you try your best today?” And that would give you moment-to-moment data that you could really put together, in conjunction with these data that you already have been looking at with your report card data, to understand how this affects your classroom. If you are a district-level official, maybe you’re interested in surveying parents to directly ask them what factors they believe may be affecting the academic motivation of their wards. You might also be interested in collecting information from teachers. That would also be something that would help you understand some of these larger forces at play, the really broad ideas of what is changing in your district. Collecting extra data at this point in time, need not be, again, collecting new data. It can just often be systematizing the way you’re storing data that you’re already collecting, but this can really make sure that you are drilling down on the important aspects that, within the broad general idea of student engagement, are affecting your specific context, your specific district.

And, like I said, the most important step, in many, many instances, is what comes further, which is taking it back to your communities. Community partners really have a lot of important insight as to tell you why. When they’re presented with information, they’re able to respond to that and give you concrete suggestions as to how to make changes within your context. And these changes are invaluable because nobody can speak to the betterment of self as much as the person in the community can. And so it’s important to give community members these opportunities to react to the data as you see it, as they see it. This also has the additional benefit of making them feel included in the process and making sure that there is better transparency and decision-making. And all of these really go together as a whole package, as a whole systemic way of considering data to improve communication and really make sure that there is comfort with what decisions are being made.

Rebeca Cerna: OSo on this next slide, we want to just provide a quick overview of the School Climate Data Inquiry Tool that we just reviewed. Some people might need to explore further domains related to school climate. Others of you, of course, may have a greater understanding and could possibly be here presenting as well on this topic. There are many domains, and understanding school climate isn’t an essential piece to this. This also helps you identify what your existing school climate challenge is, once you have that understanding, and also asking the right questions to focus on your challenge.

Once you’ve identified those, exploring what existing data you have and what additional data you might need are the next key steps, and also to develop a plan for next steps. In this webinar, on this next slide we’re offering—we covered the beginning steps of addressing your school climate challenge, the first steps being exploring various aspects of school climate, asking the right questions, identifying data sources for your data inquiry process, but here we’re also recommending next steps that we can offer as you move forward with your data inquiry process. Which is, if you need to collect additional data, which data is most important to support your school climate data challenge? It might not be feasible to collect everything that you identified as additional. Think about who needs to be involved in the decision to collect these additional data and develop a plan for data collection. Lastly on this slide is to really think about developing a plan for ongoing inquiry cycles and how you continuously use that data to monitor, to iterate, to modify, as you determine what is working and what needs to improve. These steps you will also find in Section D of the tool.

And then we have a few last reminders on this next slide. Anyone can work with data, start with what you have, and data exists all around us. And, really most importantly, invest in engaging with partners, as some of you had mentioned in the chat. As you gather data, both qualitative and quantitative, dig deeper. Work with your students, with families and staff, representing every role in your school or in your district, and with those community partners that really make your schools in your district those school communities that you want them to be.