Measuring Student Mental Wellness with the California Healthy Kids Survey

Arturo Chavez:

So for today’s session, I would like to welcome you to our webinar called Measuring Student Mental Wellness with the California Healthy Kids Survey. This session is part of the California Center for School Climate’s Data Use Series. Next slide please.

So today’s session is being hosted by the California Center for School Climate, a California Department of Education initiative led by WestEd that provides free support and trainings on school climate and data use to local agencies in California. This webinar is in partnership with the CDE, WestEd, Project Cal-Well, and the University of California San Francisco, UCSF.

So our plan for today, our guests will provide an overview of the mental health supports survey module. They will discuss how the mental health support survey module data are used to address student mental health, and you will learn how to access data from school, district, and county CalSCHLS data dashboards about student mental health and mental health supports. We plan to have some time for Q&A so please use the Q&A feature to write any questions that come out throughout this webinar.

So before we begin, just a little bit about our presenters. So today we have Hilva, and Hilva is the education programs consultant with the CDE and oversees school climate and mental health projects. Sarah is an evaluation director with the School Health Evaluation and Research Team at the University of California, San Francisco. Samira is an associate professor with the University of California, San Francisco, where she conducts research and evaluation on a variety of adolescent health and school health projects. And finally, we have Tom, he serves as the director at WestEd of the School Climate Health and Learning Survey Systems.

And now to get us started, I’m going to hand it over to Hilva with our presentation. Thank you.

Hilva Chan:

Good morning, everyone. I’m Hilva Chan with the CDE. Thanks for joining us today. So we are very happy to talk to you about Project Cal-Well, a CDE mental health project, how we began collecting student mental health data, how we use the data, and how we collaborate with WestEd to make the surveys available to all districts statewide. So just to get started, Project Cal-Well is a mental health project led by the CDE and funded by SAMHSA, Substance Abuse and Mental Health Service Administration. It allowed us to work with targeted districts in the past 10 years to help them develop sustainable student mental health infrastructure through leverage partnerships and resources. So we’re able to implement interventions, to increase mental health awareness, address prevention, and implement interventions to increase access to appropriate mental health services. Next, please.

So this map really shows where we have been in the past 10 years. So, you’ll see that we started back in 2014 with five districts, all the way down in Southern California, the green ones. And then we are currently closing out Cohort 2 of our project up north, all the way up in Humboldt, and down north, and also work with some districts in Central Valley. And we just began our Cohort 3, starting last October, working with eight districts in two counties in Santa Cruz, and other counties.

So the next slide, this slide shows our three component model. Component one is about addressing school climate, where districts implement school-wide interventions such as MTSS, SCL, and Youth Mental First Aid. This is the prevention side of student mental health and where we encourage districts to really align the mental health work with their outcome and focus on system alignment to ensure cohesion and sustainability. And component two is about increasing access to school-based mental health services where districts hire additional staff, bring in college interns or contract with local agencies. And Component three is about promoting community collaborations to improve mental health referral pathways. Now, in order to really assess the impact of Project Cal-Well on our districts, we work with UCSF since 2015 to develop student and staff survey questions to collect outcome data for our project. Now, I’m going to turn it over to my colleague from UCSF to talk more about a survey and the data.

Sarah Geierstanger:

Hi everybody. Again, my name is Sarah Geierstanger, and I work at UCSF with the School Health Evaluation and Research Team. And I’m happy to have the chance to talk with you all today about how we’ve used the mental health supports module and provide you some resources so you can see the data in action. Next slide, please. Thanks.

So in 2015/16, our team at UCSF, in collaboration with our project Cal-Well Partners, developed the mental health supports module as part of our comprehensive evaluation of Project Cal-Well, the initiative that Hilva just described. We developed two versions, one for elementary students and one for secondary. It’s a relatively brief module, and we very carefully selected the questions to include. The questions students’ perceived stigma related to mental health, their mental health help seeking attitudes, behaviors, and perceived barriers, and access to and utilization and mental health services and supports.

In 2018, WestEd and CDE began offering the module statewide as an optional add-on module to the core module of the Healthy Kids survey. As awareness of the need to address student mental health grew after the pandemic, we were excited to see that from 2021 to 2024, about 100 school districts statewide administered the module to more than 100,000 students, with some financial support from CD and West Ed to support the implementation of the module. And starting this fall, for the 2024/25 school year, the mental health supports module will be incorporated into the new and broader behavioral health module that Tom will talk about in a little bit. Next slide please.

So I just wanted to briefly mention to you how we at UCSF, and in collaboration with Project Cal-Well, have analyzed and disseminated the mental health supports module findings. We developed detailed Excel workbooks for the grantees as part of Project Cal-Well, and they have shared and discussed those findings with their partners. Hilva and I met with each of the districts annually to review their findings. And they’ve used the data in program planning as part of their LCAP goal setting endeavors, and grant proposals, and presentations to partners. And then each year, we also develop a lot of state level infographics that are housed here on our website, which is available on the link tree that WestEd provided for today’s presentation. Next slide please.

I wanted to highlight three websites that are really important in terms of accessing resources developed through Project Cal-Well and in relation to the Mental health supports module. The first is the California Department of Education’s Project Cal-Well website. And here you can learn all about the many years of evaluation and research through Project Cal-Well and all the implementation of programs. The second is West Ed’s California Safe and Supportive Schools website, which I’m sure you’re all very familiar with. Fabulous, fabulous website, and there’s a lot of important resources on there.

And then the third is, as I mentioned, our website, which is a place for us to house the infographics we’ve developed. As you can see here, if you’re interested, there’s one on Youth Mental Health First Aid, there’s a principal survey we administered, and then the school staff survey data findings from that module are also available on our website. Next slide.

And then finally, bringing us back to the mental health supports module. These two infographics are on our website and they include state-level data from the 2022/23 school year, highlighting key findings from the mental health supports module, both at the elementary and secondary school level. Next slide.

Oh, now I’d like… Samira is going to actually get into a little bit more detail and share with you data on the actual indicators.

Samira Soleimanpour:

Thanks, Sarah. And I’m Samira Soleimanpour, and I co-lead the Project Cal-Well evaluation with Sarah at UCSF. So again, as we heard Sarah and Hilva, project Cal-Well aimed to decrease mental health stigma and increase awareness and access to services for youth. And we use the mental health supports module to track our progress towards these larger goals. In the next few slides, we wanted to share some specific indicators from the mental health supports module that we examined to monitor our progress.

So first there’s a question on the module that complements the mental health related questions on the core, which are related to sadness and suicide ideation. And we asked on the mental health supports module for students to report how lonely they felt in the last month. This was especially important, given the emerging research on the impacts of the pandemic and loneliness in teens. And we saw a decrease in the percentage of youth who felt very lonely in the past month in the Cal-Well schools over time. And while we can’t necessarily attribute this to Cal-Well, it was an important indicator that things were moving in the right direction across our schools. Similarly, we looked at whether students felt they had an adult they could talk to about their problems at school. And while we didn’t see huge shifts here, they were at least moving in the right direction. Next slide please.

And as Sarah described, the module also has indicators related to help seeking behaviors. Specifically, the module asked students to check from a list of behaviors what they would do if they were experiencing signs of depression. And we saw over time, while Cal-Well was being implemented from 2021 to 2022/23, that what we are asking students to check from a list of behaviors, what they would do if they’re experiencing signs of depression. And so, over time, that the percentages who would be afraid to get help decreased over the three years of the initiative. And although slight, the percentage who said they would get help from a counselor or talk to an adult at school increased, which signals to us again that we were hopefully making progress towards decreasing stigma at these schools, which was one of our larger goals. Next slide please.

The module also helped us look at whether the goal of increasing access to care was being achieved. And we found that the percentage of students who wanted to talk to a counselor stayed the same over the three years of the initiative where we were monitoring the mental health supports data, but the percentage who received help when needed, and also the percentage who received help from school, increased over time. Again, we can’t necessarily say this was caused by Cal-Well directly, but it’s encouraging to see things moving in the right direction, given the investments that these schools were making in supporting students’ mental health. Next slide please.

And one final set of indicators that I wanted to share is a question from the module that asks students to indicate which of a list of barriers would stop them from seeking help if they were experiencing mental health concerns. And similar to the other data that we were reviewed, students reports of selected barriers such as not wanting to talk to a counselor or fearing that other students might find out decreased over time. So again, we were able to just keep an eye on these indicators around mental health needs and supports for students over time to make sure that our efforts were making some progress. And then if we needed to course correct, we could also look at the data and use that information to course correct as needed.

What I didn’t show you today, in the interest of time, that we were also able to do with the module was, in addition to looking at these indicators over time within our Cal-Well schools, there’s also statewide data that we were able to compare our progress against. And we saw notable differences there as well.

So next I’m going to turn it over to Tom, who’s going to share with you how you can access similar data for your districts.

Tom Hanson:

Thank you, Samira. Again, I’m Tom Hanson from WestEd. What we’re going to do next is two things. First, I will describe the wide array of student mental health and wellness related data available from the CalSCHLS system. And then second, we’re going to show you how to examine these data on the CalSCHLS data dashboard. So this slide says mental health… you can go back again. I can spend some more time. This slide says Mental Health and Wellness Measures on the CalSCHLS. I just want to level set here and just note that CalSCHLS is an acronym for the California School Climate Health and Learning Survey. I’m not sure how many of the audience knows this, but let’s just say it right out there. So I refer to the CalSCHLS suite of surveys as the California Department of Education’s whole Child and School Climate Measurement System. Next slide please.

Great. So CalSCHLS is comprised of three interrelated surveys. There’s a student survey called the California Healthy Kids Survey or CHKS. There’s a staff survey, the California School Staff Survey or CSSS, sorry about the… Those aren’t acronyms. Those are initials. And then there is a parent survey, the California School Parents Survey or CSPS. I’m just going to talk about the student and staff surveys here because they have the most relevant data related to student mental health and wellness. Now, each survey has a core module required for everyone taking the survey so that data from a standard set of questions is available for all participants across the state. There are also optional topic specific supplementary modules that districts can combine with the core module to assess other areas in more depth. So there are many, many mental health and wellness related measures on the student core module, the new Student Behavioral Health Module, the Student Social and Emotional Health module, the Staff Student Wellness module, and the Staff Trauma Informed Practice module.

So I’m going to describe each of these fairly quickly in turn. Remember, there’s a Linktree document, so you can always go back to this. Okay, next slide please. Okay, so the CHK’s core module includes six self-report measures of wellness, plus a whole host of measures of school engagement, developmental supports, risk behavior, and school climate. So, chronic sadness, hopelessness has been on the survey for 15 years ,at least. And this is based on a single yes-no question, asking students… Just get ready for this. It asked students if during the past 12 months they had felt so sad or hopeless almost every day for two weeks or more that they stopped doing some usual activities. So there’s a lot in that question, and it’s actually the operational definition, I think, of clinical depression. But it’s asked with a yes-no question. What’s amazing about that question, is it seems to really tap into student depression in predictable ways based on how it’s related to other measures.

There’s also a measure of suicide ideation. And then during the pandemic year, we added a social-emotional distress scale, which is a four-item scale developed by Mike Furlong and colleagues from UC Santa Barbara. That scale is designed to identify students who are experiencing anxiety, depression, social stress, other emotional difficulties. On the core now is the three-item optimism scale, which measures the degree to which students have positive expectations about their personal future. In 2021/22, we added a life satisfaction measure. This is an overall subjective, well-being measure and taps subjective, well-being or life satisfaction in five areas, friends, family, self, living environment and school. And then coming next fall, we will be adding the California Student Wellness Index, which incorporates subjective well-being, and that is life satisfaction and psychological distress, which, that is social-emotional distress, into a single index to monitor students’ complete mental health. So the next slide briefly describes the components of the new behavioral health module.

The behavioral health module was developed in collaboration with the California Department of Healthcare Services and CDE. The module includes all the questions in the mental health supports module described by Sarah and Samira earlier. It also includes questions from the student trauma module, which was developed by CDE in collaboration with a multi-agency work group in the 2021/22 year. And then there was a previous version of the behavioral health module. So it includes all the items from that, as well as items from the alcohol and other drug use module. So how could one module include all those items? Well, it’s a multi-form module. So when the module is administered, half the kids are randomly assigned to take the alcohol and other drug use portion of the module, and the rest of the kids are assigned to get the form that assesses other aspects of behavioral health.

There are about 30, 35, no, 42 items on the module, either form A or form B. So, let’s go to the next slide please. So, okay, and I apologize for this slide. It’s really dense, but it describes the content of the behavioral health module. It’s again, all the slides are available on the link in the Linktree. The measures are bucketed into a model with contributing factors, protective factors, consequences, mental health supports and precursors and consequences of AOD use. Noteworthy measures include social isolation and body image, emotion regulation, co-regulation supports. These are the supports that students get from teachers and peers and parents to help them self-regulate. There are trauma PTSD symptoms. This includes… We use a UCLA scale to measure trauma PTSD symptoms, which focuses on issues, trouble concentrating trouble, trouble feeling happiness, being easily upset. There’s a disordered eating measure on the module. And then stress-associated health conditions, these are things like persistent stomach aches, headaches, heart beating too fast. And then of course, there’s the mental health supports that’s included on this module as well.

Let me just, before we go on, I just want to remind… So currently, there is a biennial CHKS, part of the CHKS, and that is the survey that samples a representative sample of schools across the state to assess the state of the state with regards to school current climate and other indicators. Now, as part of that module, there’s the core module and the behavioral health module. So starting in a year from now, or we will have data on all of the indicators on the behavioral health module statewide. In addition, there are incentives for districts to administer the behavioral health module, both as part of the Biennale CHKS… I’m throwing a lot at you, but also, and CDE is providing some support to administer this module as well. And so far this year, there’s about 120 districts have signed up to administer the behavioral health module.

Okay, let’s go to the next slide. The next slide describes the social and emotional health module. This is a strength-based assessment developed by Mike Furlong at UC Santa Barbara. It’s based on Mike’s Co-Vitality model. The Social Emotional Health module assesses the strengths based on a positive psychology module. You can see that these measures are also organized into buckets, belief in self, belief in others, emotional competence, and engaged living. There’s measures of self-efficacy, self-awareness, persistence, emotional regulation, empathy, behavioral self-control, optimism, gratitude, and zest.

Okay, so the next slide shifts to the staff surveys. There is the staff, excuse me, this staff student wellness module, which parallels the student mental health supports module. And it assesses perceptions of students’ social emotional well-being, students’ mental and behavioral health needs, availability of mental health services. Two unique things about staff wellness and capacity are also measured. There’s staff efficacy to provide trauma-informed care and staff compassion, satisfaction and fatigue. Let’s go to the next one. Okay, so this next slide shows the staff trauma-informed practice module. It measures the extent to which the environment is trauma-sensitive. Staff attitudes related to trauma-informed care. These measures come from the ARTIC survey items. They were developed by the Traumatic Stress Institute, and Dr. Courtney Baker from Tulane University. The ARTIC scale scores can be used to assess their staff readiness for trauma-informed care implementation, as well as monitoring changes and attitudes related to trauma-informed care. There’s also some measures of trauma-informed practice implementation and measures of staff self-care practices.

Okay, so next. Those are the modules that are relevant. There’s more, but that’s most of it. And that’s a lot. So how do we access these data? So on the CalSCHLS website, there are two sets of dashboards to access data. There’s the public dashboard, and there’s the district private dashboard. So the public dashboard is available to everyone, while the district private dashboard is available to LEAs with a subscription. So with the public dashboard, results are available for the state as a whole, counties and districts. And with the district private dashboard, results are available for districts and schools. So a big difference between the public dashboard and the district private dashboard is the public dashboard is updated annually with the prior year’s data in the November after the academic year, and the private data dashboard is updated continuously. With both of these dashboards, you can examine trends across time and disparities across demographic groups. The only difference is that with the district private dashboard, you can also look at differences across schools.

So I think what we’re going to do next is we’re going to drop… I’m going to give you a live demonstration, and please be patient because I’m slow. But to access the dashboards, you would go into the CalSCHLS.org website. This is the front page currently. You could access the dashboard by clicking on the menu item on the far right of the screen, or there’s a little button right here in the middle of the screen. Let’s go here. Let’s go in this dashboard here. Okay, so this gives you access to the data dashboards. This first section you see that I’m highlighting, I hope people can see, there’s this public dashboards that’s available to anyone and everyone. They can go there, they can examine their data. If someone has a subscription to the private district dashboard, you simply sign in. And if you do have a subscription to the private data dashboard, you can still access the public dashboards when you sign in.

So I’m going to sign in. And as you see here, here’s a list of dashboards. Ordinarily, what you’d also see in this dashboard, this is for district coordinators and principals and district staff. You’d also see a list of dashboards that shows the current response counts for the districts that are administering. So if a user clicked that, they would see by school how many students had already completed the school. And that’s live data, they can see that right away. But let’s go… We’re going to go to the public dashboard, and we’re going to look at the secondary key indicators. Just for some context, this is a lot. This is not in the Linktree, but there’s item level dashboards that show results for every single item on the survey. And there’s key indicators that are usually constructed, variables or selected items of interest that are aligned with the items on the CalSCHLS reports.

So we’re going to go in the key indicators, and this is going to open up data. It defaults to showing results for academic motivation. This is for a sample district, but what we’re interested in is the mental health related, not that academic motivation is not related to mental health, but we want to look at more measures more related to mental health. So we’re going to click a domain, and we’re going to click social and emotional health. And then under selected measures, you can see what we have here from the core, we have considered suicide, optimism, chronic sadness, hopelessness, life satisfaction, and social emotional distress. Let’s look at chronic sadness, hopelessness.

So what we get here for this sample district, which is a sample district, we didn’t want to identify districts on the webinar. So you can actually get this for a specific district on the public dashboard. But we see the percentage of students by grade that report that they had been chronically sad in the 12 months prior to the survey. If you hover over this item, you can see here’s the actual question. During the past 12 months, did you ever feel so sad or hopeless almost every day for two weeks or more that you stopped doing some usual activities? And the answer choices are yes or no. If you hover over the bar, you’ll see the percentage, the measure, the year, and then the count of students that this is based on. So this is based on 2,916 students. What we see here is a typical pattern where chronic sadness tends to increase as students progress at higher ages. So 38 percent of 11th graders report being chronically sad.

So what if we wanted to look at disparities across this measure? It’s very easy. We just select one of the characteristics, select student characteristics. So let’s choose gender. And here we see results broken down by gender. If we just focus on seventh grade for now, we see that 22 percent of males report being chronically sad in the past 12 months, versus 39 percent of females. This is very typical. There are huge gender disparities on these depression measures. So we see males are pretty advantaged in terms of their reports of chronic sadness, females and non-binary and students who report some other gender identity are also disadvantaged with regards to chronic sadness. Now, these numbers are based on really small… I mean, these estimates are based on very small numbers. You can see the non-binary for 11th grades based on 31 students. But you also see 90 percent of these students indicate that they were chronically sad. Also, note that when there’s an empty bar, that means that 15 or less students actually were in this category and responded to the question. So we suppress results when there’s very few students answering the question.

So you can look at all kinds of disparities across this world, not all kinds, but many. So what if we wanted to look at trends over time? So I’m going to go back to all students to see trends over time, click trends over time. And then, the thing about the chronic sadness is we’ve had this measure for so long, you can look at very long-term trends. If we focus on, say, ninth graders here, and this is true with regards to the state as a whole, we’ve been seeing a gradual increase in chronic sadness since 2016, ’17. And we see a peak in the 2020, 2021 year. That is the year that the pandemic was peaking, and most schools in California were attending school remotely. This is consistent with state data. So this basically shows that at that period, 43 percent of students in this district experienced chronic sadness. I guess the good news here is there seems to be a decline in chronic sadness since the peak of the pandemic.

You see the same pattern for 11th graders. It’s a little… Actually the same pattern for non-traditional students. I mean, we have to be really careful here because there’s usually just small counts for this non-traditional group. So the numbers do fluctuate a lot more within a district. There’s also a county map. It’s about to be updated for 2021/23. But if you want to get a sense of the rates across different counties, you can see this. Is this hard… I bet this is hard to see, but you can see the dark-colored counties are predominantly rural. I think it’s Del Norte, Imperial, Trinity. They seem to have the highest levels.

And also on the right-hand side, you can see it ranks the counties by their rates of chronic sadness. Again, predominantly dominated by rural counties having higher rates of chronic sadness. You can also look at other things as, on the key indicators, we also have the mental health supports outcomes that Sarah and Samir were talking about before. So I just selected that mental health supports module in the domain. And then, let’s look at the first item if you know where to get mental health support. So if we look at the question, it’s like, “I know where to go or who to contact at school to help when I am very sad, stressed, lonely, or depressed.” And this is level of agreement. This is the percent who agree or strongly agree on this item. What’s interesting here is you can see that highest rates among seventh graders, and it kind of goes down by 11th grade.

What if we want to look… I’m just, this is top of the head, this is not… But let’s say you can also disaggregate by chronic sadness. So I wonder if chronically sad kids, kids who have been chronically sad, do they know where to go get help, and how’s that compared to the normal, the kids who were not chronically sad in the past 12 months? So we’re going to disaggregate by that. Oh, my. And so what we see here is the kids who experienced chronic sadness were less likely to know where to get mental health support than the kids who did not experience chronic sadness. And that’s consistent across grade. Although the disparities diminish the higher the grade.

We can also see if there’s gender differences in this measure. And this is the average percent response reporting. Strongly agree or agree. I know where to get help. The gender differences are not that large compared to the differences that we see in depression and the chronic sadness. So that’s how we use the dashboard. There’s other things. I guess, there’s one other thing I’d like to show you since I think we do have some time. Let’s go look at the item level data. And this has a slightly different layout, but we want to look at the mental health supports module results for the sample county. And sorry, so this is mental health supports, and we’re going to choose the barriers to obtaining mental health services or help from a counselor or a therapist that Samir was presenting earlier. And so this is the question: if you were very sad, stressed, lonely, or depressed, would any of these things stop you from talking to a counselor or a therapist?

So this gives all the responses to this category. A big response is, I don’t know, 23 percent or 25 percent of seventh graders say, I don’t know where to get help. 26 percent say… This is a mark all that apply item, by the way. 26 percent say they wouldn’t understand, 27 percent say you don’t want to talk to a counselor or therapist. So this is just a version to talking to a counselor or a therapist. And of course, the highest group is the item does not apply to. So basically, the major action here is they don’t know where to get help. They wouldn’t understand, which is similar to not wanting to talk to a counselor or therapist. And your parents might find out. So these are the barriers, and there are differences across the different domains.

And again, one can look at every single item here on the mental health supports module and disaggregate by demographic characteristics. So to disaggregate, you would disaggregate with this select student characteristic here. So the format of the item level dashboard is a bit different, and it changes over time. You can’t really look at trends across time because the items on the survey switch order. And so, the item level dashboard is year specific. So that’s the demonstration I have for the CalSCHLS data dashboards. I think we want to pass it over to Arturo or someone, and we can start some questions and answers.

Arturo Chavez:

Thank you. Thank you, Tom. Yes, we do have a few questions here in the Q&A, so if you do have questions, feel free to use the Q&A feature. So our first question is, and this could be for Tom or Hilva, do you have an estimate as to when the next Biennial State report will be published for 2021 through 2023?

Tom Hanson:

That’s a great question. The current report is currently in review. It will probably be a couple, two to three months before it’s publicly disseminated.

Arturo Chavez:

We do have some questions related to the surveys. So you mentioned in that the dashboard, what are the advantages of subscribing to the dashboard?

Tom Hanson:

The advantages are there’s instant access to data. So you can get your data immediately, and you can also… Oh, I should have demonstrated this. You can look at school specific differences. So a district coordinator or anyone in the district can examine differences across different schools and even disparities within schools to pinpoint where the greatest needs are. It’s basically, having access to the district dashboard allows district staff to really pinpoint and examine the data more flexibly than is the case on, say, the typical reports, which do have a lot of data, but you have to know where to find the information you’re looking for.

Arturo Chavez:

Thank you. So we have for Samira or Sarah. So thank you again for sharing the data on project Cal-Well. Can you share a bit about what the districts did to help support their mental health efforts, or where can we learn more about what has been done?

Sarah Geierstanger:

I feel like it’s more of a Hilva question.

Hilva Chan:

So we are actually putting together some implementation briefs that we imparted with like UCSF and WestEd. We are putting some implementation brief to really summarize what the districts have done with the impacted data. So we expect those to be available within the next few months. If you come back and check on our website, you’ll see all these beautiful briefs summarizing what they have done. So please check us out in a few months.

Sarah Geierstanger:

But for now, you can go to our website at UCSF or the CDE website that I highlighted when I was speaking. And there’s a lot of briefs that we’ve described explaining the project Cal-Well initiatives across the years.

Arturo Chavez:

Thank you. We have one that I feel this one’s going to be back to Tom. Can you just aggregate the data by the same student groups that appear on the California dashboard?

Tom Hanson:

By this… Wait, can we-

Arturo Chavez:

Yeah. And maybe the person who wrote this, maybe they have clarifying… But this is what they wrote. So can you just aggregate the data by the same student groups that appear on the California dashboard?

Tom Hanson:

I think maybe this question, and if the person wants to ask a follow-up, we provide PDF reports that disaggregate the data by all the LCAP subgroups, including long-term English learner status, foster students, unhoused students. Most of those are also available on the… Actually all of those are available on the dashboard. Plus, there’s more things that can be used on the dashboard that can be used to disaggregate. Like you saw the demonstration, we can disaggregate by chronic sadness to see if there’s disparities across other indicators on that. We can disaggregate by the extent to which the students have caring relationships with adults at school. And there’s high levels of that, versus lower levels. And you can look at differences in mental health outcomes and academic engagement outcomes based on their level of exposure to caring relationships with adults. Those disaggregations can only be, they’re only available on the dashboard. They’re not available in the reports.

Arturo Chavez:

Thank you. Another question is asking about the turnaround time. So what is the turnaround time for LEAs? So from survey administration to being able to access their data.

Tom Hanson:

Okay. So if it’s access the data for the survey on the dashboard, the data are updated two times a week. There is a quality control process to make sure those data, they’re logical. We look for mischievous responders and things like that, and that may affect the data. But so twice a week the data are updated on the dashboard. That’s fairly continuous. The survey counselor update in real time. That’s the dashboard.

For the reports, as soon as we are informed that the survey has been completed, in general, reports are produced and delivered a week after that time period.

Arturo Chavez:

So this next one might-

Tom Hanson:

Can I just-

Arturo Chavez:

Yeah. Go ahead. Go ahead.

Tom Hanson:

The trick is we have to know when surveying is complete. That’s the thing that holds us up. So if we know that we’re good. We can proceed very quickly.

Arturo Chavez:

Thank you. So this next question might be for Hilva or Tom. Is there access to raw data sets for publicly available information?

Tom Hanson:

Yes. There’s a way to access the data. It’s usually, it’s for research purposes and there’s sort of an MOU process that those data can be accessed for research, or, of course, the districts can access those data too, and there’s a process for that. Is there anything to add to that, Hilva?

Hilva Chan:

No. So as long as it’s for research, and then you don’t compromise the… Because it is an anonymous survey. So as long as you do not disclose the school or the district name, everything is from the district, you go through an MOU process. And then as long as we understand the intent of the study, we’ll approve that. So if you’re interested in requesting the raw data set, please contact WestEd. CalSCHLS, right? CalSCHLS at WestEd.

Tom Hanson:

You can send an email to calschls@WestEd. There’s also a phone number on the website. There’s actually an MOU that you can access on the website as well. A lot of times it’s easier just to send us a message or call us. We’re always available. Responsive.

Arturo Chavez:

Thank you. So I know you weren’t sharing information on parent surveys today, but there’s a question about the language that it’s available in. So are parent surveys provided in multiple languages or who should they contact to get an additional language?

Tom Hanson:

Yes. Shoot. And I don’t have the list of languages, but the parent surveys have been translated into 19 or 20 different languages, and that can be worked out with the CalSCHLS technical assistance provider. We have most languages covered, but sometimes we don’t. And then we’ve worked with districts that have developed translations. If there’s a significant group that was not being covered with the survey, I mean, there’s a very good chance that we have the language already translated, and we will work to create a translated version.

Arturo Chavez:

Thank you. So the next question is about how often the surveys are given and when are they given in the year?

Tom Hanson:

Gosh, is there questions for anybody else? No, I’m just kidding. So the questions are currently, the school climate questions are a good source of local control and accountability plan measures. We have all the measures that are suggested for the local control accountability plans on the survey. Plus, we have the ability to disaggregate by all the LCAP subgroups, and is required that districts administer the school climate survey annually.

For the LCAP, we recommend that students are at least served annually. We also have the capacity to provide access to the survey more than one time per year if anyone wants to look at within year changes. Very few districts actually do that, but some do. But anyway, I guess our recommendation is to administer the survey annually to monitor progress in addressing student needs and improving school climate.

Arturo Chavez:

Thank you. So we’re going to give Tom a break, and we’ll see if anybody can answer this question. So what were some of the key lessons learned based on the evaluation for Cal-Well? Are there any takeaways you have for people here in the field?

Sarah Geierstanger:

I can just take a start, and then Samira, if you want to add onto it. Just that the model works, using all three components of the module, the school-wide, tier one interventions, the more tier two group level interventions, and then developing systems to refer students to services when needed outside the school or within the school. Just having that comprehensive view of student mental health was really, really successful. And the collaboration between the different Cal-Well partners was great. We met monthly, and so there’s a lot of collaboration and support in terms of sharing ideas, but it’s really the comprehensive picture that seemed to have made a difference. Samira, do you want to add anything?

Samira Soleimanpour:

I think I would echo what Sarah said. And we did, yeah, we saw improvements. I mean, like we said during the presentation, the focus was on decreasing stigma and increasing awareness for staff and for students. And so, we were able to see changes in awareness of the availability of services and feeling supported. So in that way, we showed some improvements.

And I said this multiple times, I know when I was presenting, we can’t establish causality. We were looking at data once a year, and just monitoring indicators. But like I said, when I was showing the slides, the indicators were going in the right direction, and that was really encouraging. And I’m trying to think if there were ones that we saw going in the wrong direction. We definitely would talk about those as a team if something came up. I think earlier, in the earlier iterations of Cal-Well, school staff were showing some burnout, which we we’re monitoring through the survey, and some compassion fatigue. And so, there was some resources that were dedicated to making sure teachers were feeling supported. So I mean, it was a real good example of reflecting on the data and making continuous improvements over time.

Hilva Chan:

And I do want to add that from CDE perspective. I think it’s really important to not implement any mental health work separately, silently as a separate program. We’ve seen that numerous times. It’s really, really important to connect to your overall district LCAP, and then your vision and mission. So everything is aligned.

If not, when the grant goes away, when funding goes away, all the work may go away. So it’s really important to really try to align it as much as possible to your system, to create a system piece, and align to your LCAP, and for now, tap into all the activities happening in the county, in the state regarding student mental health. So tap into the schedule, trying to sustain it through the various grants, and make sure after a few years it’s definitely sustainable.

Arturo Chavez:

Thank you. We do have another question for Samira or Sarah, possibly. Do you work with nonprofits or for-profits to build this survey tool for youth serving organizations?

Samira Soleimanpour:

We definitely do. We’ve collaborated with a lot of nonprofits throughout the state, and be happy to connect and either help build capacity internally or if we can be helpful to provide some services. We’re happy to, but we definitely have.

Arturo Chavez:

Thank you. And we have one last question before we move on, just because of time. What is the response rate for students and staff on these surveys?

Tom Hanson:

Yeah, so the response rates, they range the average student response rate for this student survey, it ranges from around 80% for seventh graders, and then it declines around 75% and 70% for ninth and 11th graders. The staff response… wait, was the question about staff response rates? Please don’t ask me about parent response rates.

Arturo Chavez:

Student and staff.

Tom Hanson:

Student and staff. So the staff response rates are… It varies tremendously across schools. Sometimes it’s incredibly low, like 50%, sometimes it’s up to 80%. On average, around 60% for staff. Fewer schools participate in the staff survey than in the student survey. I’m going to say, for parents, it’s tough. Some districts are really, really good to getting large responses from parents, but generally, the response rates are down around the thirties, 30%, 25%. It’s just very difficult to get parents to respond to the surveys.

Arturo Chavez:

Thank you.

Hilva Chan:

Can I add that? Add a couple of things. I think, for staff and parent response rates sometimes reflect the district response, what they have done to really promote it and actually use the data. So you’ll get a high response rate if that’s really a collaborative process and the district’s really able to communicate to staff that your voice matters. We want to hear from you. And we’re going to use the just results to really improve the process. That would definitely increase response rates, versus just do it, give it out, and then not go back to it and act on it.

Arturo Chavez:

Thank you. I want to thank all the presenters again. And I want to thank all of you for taking the time today, this morning to join us. So we hope that you have a wonderful day and that you all take care. So thank you.