Category: Child / Adolescent - School-Related Issues
School-based anxiety prevention programming is designed to lessen the negative impacts of childhood anxiety in children’s everyday environments (Briesch, Hagermoser, Sanetti, & Briesch, 2010). The FRIENDS for Life (FFL) program is a universal anxiety prevention program designed by Barrett (2004) for classroom administration to students in grades 4 through 6.
Treatment integrity (TI) evaluation assesses the degree to which a program is being implemented as designed, providing program developers with data on what does and does not work with implementation in a real-word setting. Previous TI research of FFL has found rates of session/manual content concordance from 72% - 97% (Essau et al., 2000; Lock & Barrett, 2003; Rodgers & Dunsmuir, 2015). However, few studies have assessed FFL TI in detail (Higgins & O’Sullivan, 2015).
TI of FFL was assessed in ten elementary schools in Newfoundland and Labrador, Canada, as part of a larger FFL program evaluation. The Program Integrity Checklist (Barrett et al., 1999), developed to ensure that FFL sessions are delivered as designed, was used to assess program fidelity. For each session there is a corresponding checklist that outlines topics specific to each session. Observers rate on a scale of 0 (“Extremely well”) to 4 (“Not at all”) how well each session goal was achieved. A random 25% of FFL sessions of participating schools were observed by undergraduate/graduate research assistants who completed corresponding session TI checklists.
Overall, 29 sessions were observed, obtaining a manual/session concordance rate of 71% (calculated by comparing session activities that were covered (rated 1-3) versus not-covered (rated 4)). Although slightly lower than previously observed rates, concordance rate alone is not very informative; therefore, frequency analyses of session-by-session ratings were conducted. Higher scores indicate lower TI. Sessions 1 (M = .62, SD = .87), 3 (M = .51, SD = .33), 4 (M = .38, SD = .45), 5 (M = .88, SD = .96), 8 (M = .35, SD = .10), 9 (M = .25, SD = .35), and 10 (M = 0, SD = 0) showed good TI. Sessions 2 (M = 1.09, SD = .32), 6 (M = 1.77, SD = .00), and 7 (M = 1.77, SD = .00) had the lowest rates of TI. The lower average TI scores in these sessions were due to session content being covered “not at all”. Indeed, when examined across session, sessions 2 (n=3), 6 (n=8), and 7 (n=4) had the greatest frequency of “4” ratings. Although other sessions included ratings of “4” (sessions 1, 3, 4, and 5 also received ratings of “4”), average ratings for these sessions were not as high due to more positive ratings of covered topics (i.e. ratings of 0 and 1 for “extremely well” and “moderately well” covered topics).
In cases where lower TI scores were given, observers noted that facilitators were not adhering to program structure in attempts to 1) adjust programming to the classroom schedule or 2) specialize the session to meet the needs of students. Given the importance of TI to interpreting program evaluation data, future work in this area might focus on more qualitative analysis of those sessions that are not well-adhered to and understanding the reasons for deviating from the manualized content.