Integrated Analyses
Out of the 32 school districts, 16 districts and one charter school are participating in MTSS, 16 districts (50%) are accessing the Bridges to Success program, and 13 districts (41%) have completed at the CA Healthy Kids Survey (CHKS) Core Module in 2020-2021. Twenty-five districts (78%) have accessed at least one of the three options to support and measure student SEL and mental health. Seven school districts (22%) have not engaged in any of these options included in the current county scan of social-emotional learning, student mental health and wellbeing programs/tools.
The analyses completed prior to this section have focused primarily on the relationship between social, emotional, and health programs and student level outcomes at the district level. When examined independently, these programs positively correlate with student level outcomes of interest and trend in the direction hypothesized to measure impact. For example, districts engaging in MTSS display a decline in chronic absenteeism rates. This is consistent with what was found through the statewide data analyses. A similar decline in chronic absensteeism rates was also displayed for districts engaging with the Bridges program compared to districts that did not report the same level of engagement.
The interactive graphics below display the integration of data across the CHKS with the MTSS and Bridges cohorts. The interactive effects are also displayed for chronic absenteeism. Further analyses are being conducted to determine if the interactive effects are statistically significant.
Note: 2019-2020 data is not included due to modified data collection methods and restricted data reporting as specified in Assembly Bill 130 (Committee on Budget) Chapter 44 of Statutes of 2021. Additional analyses will be conducted as additional education and public health data become available.
California Healthy Kids Survey Results: When examining the results of CHKS across years by grade (e.g., 2013-2019, grades 7, 9, and 11), Humboldt reports lower percentages for Academic Motivation and Meaningful Participation across all grades. The trend data for elementary schools include 2020-2021 and the displays for secondary will be updated once these data become available and also allow for an exploration of pre-post pandemic data collections.
Academic Motivation: Grade 7 reported higher percentages across all years with a slight decline from 2017 at 72% to 2019 at 70%. The lowest percentage is reported by the continuation schools at 51% in 2019.
Caring Adult Relationships: Reporting of caring adult relationships in the school setting trends in the low 60% across all grades with slightly higher percentages in Grade 11 (67%) and non traditional grades (66%). In contrast, the state displays a steady decline across all grades that range from 56% to 65%.
High Adult Expectations: A decline is reported across all grades (Grade 7 77% to 73%; Grade 9 72% to 69%; Grade 11 75% to 72% and non traditional 71% to 70%). The state percentages have remained consistent across all age groups and range from 63% nontraditional to 75% Grade 7. These county rates are comparable to state percentages and trends for this dimension.
Meaningful Participation: The lowest percentages across all five factors are reported on this dimension. There are also consistent declines across all grades. The highest percentage reported is in Grade 7 0f 30% in 2018-2019 and the lowest percentage is reported in nontraditional of 26% in 2018-2019.
School Connectedness: Higher percentages are reported for the lower grades with a slight decline across years within each grade. For example, Grade 7 reported 63% in 2015 that decreased to 59% in 2019. In contracts, non traditional schools reported 55% in 2015 that declined to 52% in 2019.
The interactive graphic below allows for a deeper exploration of countywide trends on CHKS by filtering and viewing results relative to MTSS participation (n=16 districts) and Bridges engagement (n=13 districts) relative to ALL districts (n=32).
Specifically, the following questions were examined:
Do the MTSS cohort responses on the CHKS survey trends parallel the overall countywide trends?
How are the MTSS cohort percentages adjusted when also considering district engagement with the Bridges program?
Are the interactions between MTSS and Bridges to Success significant on CHKS responses and student level outcomes?
Findings: When the results are filtered by the MTSS cohort, a higher percentage of students reported LOW levels of Academic Motivation and Meaningful Participation across all grades. For example, around 40% across all grade levels report low Academic Motivation, and about 30% to 40% report low levels of Meaningful Participation. When examining these two subscales in relation to MTSS and Bridges engagement, the higher percentages of low engagement are reduced.
The graphic below depicts the median Chronic Absenteeism rate for Humboldt County School districts. These rates can be filtered by MTSS participation and Bridges engagement.
For example, the top half of the graphic can be sorted by the following combinations:
Bridges Engagement by ALL MTSS Participation (both MTSS and Non-MTSS Participation)
Bridges Engagement by Non-MTSS Participation (Bridges and Non-MTSS; Non-Bridges and Non-MTSS)
Bridges Engagement by MTSS Participation (Bridges and MTSS; Non-Bridges and MTSS)
The bottom half of the graphic can be sorted by the following combinations:
MTSS Participation by ALL Bridges engagement (both Bridges and Non-Bridges Engagement)
MTSS Participation by Non-Bridges (MTSS and Non-Bridges; Non-MTSS and Non-Bridges)
MTSS Participation by Bridges (MTSS and Bridges; MTSS and Non-Bridges)
When sorting Chronic Absenteeism by initiative, the strongest effect seems to emerge when examining the MTSS cohort performance and Bridges cohort performance simultaneously. To determine if these patterns were statistically significant and examine the integrated effects on additional student outcomes, district participation in MTSS, Bridges, and CHKS on academic performance, engagement, and conditions and climate was examined using a linear mixed model restricted maximum likelihood analysis. Further, we fit a single model for each parameter with interactions for all interventions in question to understand how the existence of multiple interventions may affect the results of each individual one.
Districts accessing Bridges referral services were associated with an increase of 12.6 percentage points in graduation rates when tested both independently and jointly with the other interventions of interest. Receiving a Bridges referral was also associated with a statistically significant decrease of 10 percentage points in drop-out rates when tested jointly with other interventions. These are especially notable results because of the magnitude of the effects and the strength of the statistical significance. Further, it is interesting that a strong impact of the Bridges program on dropout rates is uncovered when controlling for MTSS Participation in the model. This points to the possibility of a symbiotic relationship between the interventions, wherein certain benefits of the Bridges program are accessible in conjunction with participation in MTSS. Further analysis should investigate these benefits more deeply.
Participation in the 2020-21 CHKS Core Survey was associated with a 3.2 percentage point decrease in rates of suspension when tested independently, and a 3.8 percentage point decrease in rates of suspension when tested jointly with other interventions. Student surveying has been shown to be associated with a decrease in suspensions. Future analyses could investigate other parameters of school climate that we would expect to be positively affected by surveying, such as expulsion and attendance rates.
MTSS Participation was associated with a statistically significant decrease of 0.7 percentage points in dropout rates when tested independently; this effect disappears when tested jointly with Bridges referral. However, MTSS is a targeted intervention, where we would expect to see significant effects on the experiences of specific students but a diluted effect at the school- or district- level. As we are analyzing district-level data, we would expect to find stronger indications in the individual tests. This also means that there may be information to cautiously glean when relaxing the significance level.
Additional details on this analysis are located in the paper below.
