Prof.samuel King

Journal of Experiential Education2016, Vol. 39(4) 370 –385 © The Authors 2016 Reprints and permissions:

sagepub.com/journalsPermissions.nav DOI: 10.1177/1053825916668901 jee.sagepub.com Article Preventing Academic Disengagement Through a Middle School–Based Social and Emotional Learning Program Amanda Moore McBride 1, Saras Chung 2, and Anne Robertson 1 Abstract Behaviors that warrant school discipline (e.g., fighting, victimizing p\ eers) is detrimental to school climate and the learning process. This study examines the effectiveness of preventing school disciplinary incidents in middle scho\ ol through an experiential, social and emotional learning (SEL) program. A community youth development organization, two public middle schools in low-income commun\ ities and a local university collaborated to design and deliver the program to all seventh- grade students in social studies curricula. This article describes the design of the intervention and its effect on students’ suspensions, skipping class,\ and failing grades.

The results of this study indicate that a school-based SEL service-learn\ ing program may reduce disciplinary incidents for middle school students. Other atti\ tudes and skills, however, did not change significantly in the anticipated directi\ on. This research demonstrates the effect of SEL curricula and service-learning programs e\ mbedded in school coursework. Implications for practice and research include und\ erstanding mechanisms of change in SEL processes.

Keywords service learning, experiential learning, school discipline, social and e\ motional learning, middle school 1University of Denver, CO, USA2Washington University in St. Louis, MO, USA Corresponding Author:

Saras Chung, Washington University in St. Louis, One Brookings Drive, Ca\ mpus Box 1196, St. Louis, MO 63130, USA.

Email: [email protected] 668901 JEE XX X 10.1177/1053825916668901Journal of Experiential EducationMcBride et al.

research-article 2016 McBride et al. 371 Introduction Many public schools in economically vulnerable communities are constrain\ ed by a lack of resources available to help students learn increasingly difficult academic con- cepts, while also preparing them for a successful launch into society. For resource- depleted schools, the primary mission of education is often complicated \ by the urgent need to address student behaviors, which can be exacerbated by the effects of poverty (Wadsworth et al., 2008). Poverty-related stress can manifest itself in adolescent in\ ter - nalizing and externalizing problems, deviant behavior, and school dropout (Wadsworth et al., 2008), further complicating the already difficult task of educating. These types of stressors can create vicious cycles for vulnerable school\ s. Instead of proactively encouraging positive student behaviors, schools are left wit\ h little choice but to spend their limited time and resources addressing highly disrupti\ ve and often- times dangerous behaviors. On an individual level, paying the consequenc\ es of one’s actions can reduce time that a student remains in the classroom to learn\ . Fragmented attendance in the classroom further diminishes a student’s exposure to instruction, which may increase academic disengagement, causing students to fall fart\ her behind academically (American Academy of Pediatrics, 2013). Lack of academic motivation may further exacerbate their participation in disruptive behaviors, creating a negative reinforcing loop of behaviors for schools to address. The purpose of this study was to examine whether a universal social and \ emotional learning (SEL) curriculum delivered through a service-learning program\ could reduce school disciplinary incidents and increase academic engagement for middl\ e school stu- dents. Specifically, this study included the following objectives: (a) work with an urban\ middle school to deliver an SEL program to an entire grade of seventh-grade students as part of their weekly social studies curricula, and (b) determine wh\ ether the interven- tion could reduce middle school disciplinary incidents and increase acad\ emic engage- ment, as found in previous replications of the program with high school \ students. The intervention design tests a range of academic and social outcomes. F\ irst, this study tests whether the intervention could enhance academic efficacy, civic attitudes, and engagement in schools. To our knowledge, few SEL programs have been coupled with service learning and embedded in school curricula in this way. In addition, this study examines whether delivering the intervention in school curricula could reduce negative school behaviors when embedded in school curricula during the s\ chool day.

SEL as an Intervention to Prevent Negative School Behaviors SEL is defined by the most widely used definition from the Collaborative fo\ r Academic, Social, and Emotional Learning (CASEL) and Zins, Bloodworth, Weissberg, and Walberg (2004) as follows: SEL is a process for helping children develop the fundamental skills for li\ fe effectiveness.

SEL teaches the skills we all need to handle ourselves, our relationships and our work effectively and ethically. These skills include recognizing and managing our emotions, 372 Journal of Experiential Education 39(4) developing caring and concern for others, establishing positive relation\ ships, making responsible decisions, and handling challenging situations constructivel\ y and ethically.

They are the skills that allow children to calm themselves when angry, make friends, resolve conflicts respectfully, and make ethical and safe choices. (Collaborative for Academic, Social, and Emotional Learning [CASEL], 2014, http://www.casel.org/ social-and-emotional-learning/) Similarly, Zins et al. (2004) define SEL as the process through which children and adolescents enhance their ability to incorporate thinking, feeling, and \ behaving to achieve important life tasks. These skills include recognizing and managing emotions, developing caring and concern for others, establishing positive relation\ ships, making responsible decisions, and handling challenging situations constructivel\ y and ethically (Zins et al., 2004). The American Academy of Pediatrics (2013) has recommended that early interven- tion programs identifying students at risk of suspensions and expulsions\ should also teach age-appropriate behaviors. SEL programs seem to align with this need. Previous research suggests that SEL programs can influence the development of appropriate emotion regulation techniques and positive peer interactions, encouragin\ g young peo- ple to follow positive and productive trajectories (Anderson-Butcher, Stetler, & Midle, 2006; Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Zins & Elias, 2006). In schools, SEL interventions have been used to reduce a variety of negative behav- iors and increase positive development for youth. In a meta-analysis of \ SEL programs, successful programs included components that (a) use a connected or co\ ordinated set of activities to achieve objectives related to skill development, (b) use active forms of learning to help youth learn new skills, (c) have at least one compone\ nt devoted to developing personal or social skills, and (d) targets specific SEL skills rather than targeting skills in general terms (Durlak et al., 2011). Social and emotional skills taught and demonstrated during childhood have been tied to numerous positive behaviors and qualities. A 3-year longitudinal study of a universal SEL program for elementary school students indicated that students who engaged in the program experienced preventive effects on population-level rates of aggression, social competence, and school engagement (Conduct Problems \ Prevention Research Group, 2010). In adolescence, there is evidence of indirect ef\ fects on the reduction of anxiety and depression or emotional distress (Durlak et al., 2011; Neil & Christensen, 2009); decreased incidence of conduct problems, such as dr\ ug use (Diekstra, 2008; Durlak et al., 2011); and improved performance in school (Diekstra, 2008; Durlak et al., 2011; Wilson, Lipsey, Jo, & Mark, 2006). Although there are many different types of SEL programs, experiential activities embedded in service learning within a positive youth development framewo\ rk may be one way to build social and emotional skills in early adolescence (Chun\ g & McBride, 2015). Service learning is a pedagogy that is used to describe structur\ ed learning opportunities that engage students in service from research to reflection. These types of experiences have an explicit student learning focus integrated with a\ n equally fun- damental community focus, providing hands-on opportunities for students \ to research, McBride et al. 373 practice, and reflect on their contributions to a community (Bringle, Clayton, & Hatcher, 2013; Eyler & Giles, 1999; Felten & Clayton, 2011; McBride, Pritzker, Daftary, & Tang, 2006). Multiple studies have found that youth who participated in service learn\ ing were less likely to be referred for disciplinary measures and more likely to \ have increased standardized tests scores when compared with those who did not participate in such programming (Billig, 2000). In addition, studies have shown that schoo\ l-based service learning increases grades, attendance (e.g., Bryant, Shdaimah, Sander, & Cornelius, 2013), academic interest, school engagement, and encourages stronger cl\ assroom task-engagement and skills (Schmidt, Shumow, & Kackar, 2006). Classroom-based programs using a positive youth development framework to incorporate ser\ vice learn- ing can also reduce school disciplinary incidents, especially for studen\ ts who are most at risk (Allen & Philliber, 2001). Despite evidence to support the development of SEL in children and adolescents (Durlak et al., 2011), many teachers and school administrators indicate that there are not enough resources or expertise to address the unique needs of individ\ uals who exhibit disruptive school behaviors (Maras, Splett, Reinke, Stormont, &\ Herman, 2014). Changing the school environment may initially require reinforcements that are not available within the school. Mobilizing community resources that proactively sup- port students’ social and emotional development can help address some of these chal- lenges (Epstein, 1995, 2001). According to D’Agostino (2013), partnerships or collaborations can identify and prioritize student needs and strategize \ evidence-based ways to meet them.

The Intervention Wyman’s Teen Outreach Program® (TOP) is a positive youth development program teaching social and emotional skills through curriculum and service lear\ ning. Research on the effectiveness of TOP has been published as early as the 1990s on topics of school suspensions and teen pregnancy (e.g., Allen, Philliber, & Hoggson, 1990; Allen & Philliber, 1991, 2001). Few studies, however, have examined how the program works when embedded within school curricula. In addition, few evaluation\ s have been conducted in the past decade focusing on middle school youth in particul\ ar. This is important, as the intervention has been listed on a number of evidence-b\ ased practice lists from the Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-Based Practices (“Wyman’s Teen Outreach Program,” 2015) to the Promising Practices Network: Programs That Work (2013).

Most of these lists, however, use data from studies conducted over 30 years ago.

Overview of the Study This article reports on outcomes of a 9-month intervention conducted ove\ r the course of 1 academic year. To our knowledge, this is one of the first studies using a quasi- experimental design to determine whether TOP, when closely aligned with middle 374 Journal of Experiential Education 39(4) school curricula, can improve academic engagement for adolescents. This study answers the following research questions: Research Question 1: What is the effect of Wyman’s TOP at improving academic engagement for middle school students when embedded in the school curric\ ula?

Research question 2: What is the effect of Wyman’s TOP on student’s behavioral, academic, and civic attitudes?

It is hypothesized that skills learned during the intervention will help\ students to refrain from engaging in negative behaviors that lead to disciplinary in\ cidents (e.g., fighting, disrupting class, skipping class) for students in the interve\ ntion school. In addition, we hypothesize that students in this experiential curriculum w\ ill have increas- ingly positive beliefs and attitudes toward school engagement and civic \ duties.

Method The Intervention Wyman’s TOP is an SEL curriculum embedded in a service-learning program (Chung & McBride, 2015). In this study, there were four collaborative entities necessary to implement the intervention: a community-based youth development organization, two middle schools, and a local university. The youth development organization provided access to the intervention, trained and supervised program facilitators,\ and provided program materials and replication expertise. The intervention school delivered the program to seventh-grade students through the required social studies cu\ rriculum. The comparison school was asked to participate based on its similarity in de\ mographic composition and size and received “business as usual” followed by \ a delayed interven- tion. The university designed the school and community partnership, provided a\ ccess to master’s-level social work students to serve as program facilitators in exchang\ e for practicum credits, and developed instruments for program evaluation and \ intervention effect testing. Researchers, the youth development organization, and school administrators from both the comparison and intervention school were included in meetings to\ discuss the implementation. Given previous research outcomes on the program and exis\ ting rela- tionships with the youth development organization, school leadership were open to implementing the program in their schools. Agreements from the school were contin- gent upon the university to provide master’s-level social work students to deliver the programming and to orchestrate the community service with training, over\ sight, and program support from the youth development agency. The intervention was implemented during social studies class periods onc\ e a week (45-60 min sessions) across the entire school year (32 weeks). \ Program ses- sions (also called “TOP Clubs”) were led by program facilitators. During this time, classroom teachers would turn over their class to the program facilitato\ r and use this time for lesson planning or to watch as a passive observer. Lessons were led in an McBride et al. 375 interactive, facilitator-led group discussion format on social and emotional skill development. Sessions included topics that were planned alongside existi\ ng social studies lessons, such as decision-making, building healthy relationships\ , communi- cation skills, and issues of social and physical development. The program also required students to participate in 20-hr of school-based service learni\ ng designed to engage students in the planning, execution, and evaluation of their serv\ ice-learning activities. Students designed activities that coordinated with their exi\ sting social studies curriculum. Program facilitators for the intervention were chosen from a pool of qua\ lified mas- ter’s of social work (MSW) degree candidates at the collaborating universi\ ty. Prior to the academic year, facilitators attended a 40-hr, weeklong training led by trainers from the youth development organization. A training manual and curriculum guide, includ- ing lesson plans for all sessions, was given to all facilitators for ref\ erence. Ongoing training, supervision, and professional development opportunities were p\ rovided throughout the year. According to the fidelity requirements of the intervention, program clubs should meet weekly over the course of 9 months, each club provides a minimum of\ 25 curric- ulum-focused peer group meetings, students should complete a minimum of 20 hr of service learning, and meetings should be facilitated by a trained facili\ tator. Students in the intervention school participated in approximately 21.28 hr (SD = 2.89) of com- munity service during the academic year and received approximately 31 pr\ ogram ses- sions (µ = 31.04, SD = 4.22), satisfying the fidelity requirements of the intervention.

Setting and Sample The study took place in two public, Midwestern middle schools during 201\ 2-2013.

The protocol and design of this study was institutional review board (I\ RB)–approved by Washington University in St. Louis (No. 201207120). The intervention school included 661 sixth- to eighth-grade students in the 2012-2013 school year. A majority of the students (69.4 %) were eligible for free or reduced-price lunch\ . Racially, 87.4% of students in the intervention school were African American and 7.6% of students were White. The comparison school spanned Grades 7 to 8, with 1,028 students enrolled in 2012-2013 school year. A majority of the students (92.7%) were eligible for free or reduced-price lunch. In regard to race, 97.6% of students from the compari- son school were African American. All seventh-grade students were eligible to participate in the study. Parental consent for research was received for 148 students in the intervention school (\ 71.8%) and 151 seventh-grade students in the comparison school (46.7%). Both samples \ from each school were mostly female (intervention school: 56.3%, comparison schoo\ l: 60.4%) and African American (intervention school: 85%, comparison school: 95%). Chi- square analyses indicated no significant differences in gender, race, or living situation for these samples (Table 1). School faculty (e.g., teachers) from the intervention school had an average of 14 years of experience and 62% of staff had advanced degrees. The comparison school 376 Journal of Experiential Education 39(4) had an average of 17.9 years of experience and 70.3% of staff had advanced degrees.

The intervention facilitators (n = 4) were all new to the intervention school setting but had prior experience with children and youth in other settings.

Method At the beginning of the school year, the research team worked with facilitators, teach- ers, and school administrators to inform and recruit seventh-grade parti\ cipants for the study. Pre- and posttest surveys were distributed to participating students i\ n the fall and spring by the university research staff. Research staff explained the survey components to students and remained present during the survey administration period to answer questions. Of those who consented, 112 students completed both the pre- and posttest from the intervention school (75.6%). In the comparison school, 106 students com- pleted both the pre- and posttests (70.2%).

Table 1. Sample Description.

Demographic Characteristics Intervention school Comparison school n % n% Total Participants in Each Sample 112100106 100 Gender Males 4943.8 4239.6 Females 6356.3 6460.4 Race African American 8278.9 9591.4 White 98.7 11.0 Hispanic/Latino 11.0 — — Asian or Pacific Islander ——— — Multiethnic 54.8 43.9 Native American/Alaskan Native 11.0 11.0 Other 65.8 21.9 Mother’s education High school diploma or less 1618.4* 2636.6* Some college or more 7181.6* 4563.4* Father’s education High school diploma or less 2131.8* 3258.2* Some college or more 4568.2* 2341.8* Household family composition Two-parent 6159.2 5552.9 Single-parent 3332.0 3937.5 Guardian/other 98.7 10 9.6 *p < .05. McBride et al. 377 Measures and Analytic Strategy Surveys were used to understand changes in student attitudes regarding t\ heir belong- ing and engagement in school, self-worth, perceptions of community safet\ y, and civic duty. Other constructs on prosocial bonding and student autonomy were also mea- sured; however, these scales had poor Cronbach’s alpha scores (α < .70) and therefore were omitted from the analysis.

School belonging. Questions regarding school belonging were modified from items on the Psychological Sense of School Membership Scale (Goodenow, 1993; α = .79).

Students were prompted with five statements: I feel like a real part of \ my school; Sometimes I feel as if I don’t belong here; I wish I were in a different school; I feel proud to belong to this school; and I am happy to be at this school. Likert-type response ratings ranged from 1 (not at all) to 5 (likely true). Negatively worded items were reversed while coding. The mean was taken across the sum of scale items.

Emotional engagement. Emotional engagement was assessed through a subscale (α = .78) of the Engagement versus Disaffection With Learning (EvsD) Measure (Skinner & Belmont, 1993): My classes are fun; I enjoy learning new things in my\ classes; When we work on something in class, I feel interested; When I am in class, I feel good; and In my classes, I work as hard as I can. Students rated items on a sc\ ale from 1 (never) to 5 (a lot), and the scale was summed and then averaged, with higher values indicating greater agreement.

Behavioral engagement. The Behavioral Engagement subscale of the EvsD (Skinner & Belmont, 1993) was used to measure students’ active participation in learning (α = .70). Survey items asked students to rate the following statements from\ 1 (never) to 5 (a lot): I pay attention in my classes; When I’m in class, I participate in class discus- sions; When I’m in class, I listen very carefully; I try hard to do well in \ school; and When we work on something in class, I get involved. Scale scores were th\ en averaged, with higher scores indicating greater agreement.

Academic efficacy. Academic efficacy was measured through the Patterns of Adaptive Learning Scale (PALS; Midgley et al., 2000). PALS ask students to rate themselves on questions that gauge their perception of their ability to complete and l\ earn difficult school- work, master the skills taught in school, and discern and solve difficult concepts (α = .79).

Students were asked to respond with 1 (none) to 5 (a lot) to the following prompts: I’m certain I can master the skills taught in class this year; I’m certai\ n I can figure out how to do the most difficult class work; I can do almost all the work in class if I don’t give up; Even if the work is hard, I can learn it; and I can do even the hardest \ work in this class if I try. Scores were averaged, with scores near 5 indicating higher self-efficacy.

Civic duty. Civic duty was measured using a 12-item scale on civic duty by Zaff, Boyd, Li, Lerner, and Lerner (2010) with an Cronbach’s alpha level of α = .83. The 378 Journal of Experiential Education 39(4) scale measured student attitudes on statements on a 5-point scale from 1\ (not impor - tant) to 5 (very important). Scale items were summed then averaged for interpreta- tion, with higher scores reflecting student attitudes suggesting that civic duty was very important. Statements included phrases such as the following: I bel\ ieve I can make a difference in my community, helping to make sure all people are treated fairly, speaking up for equality, and it is important for me to contribute to my com- munity and society.

School disciplinary events. Students were asked to report academic behaviors (failing grades/courses) and social behaviors (skipping class without permissio\ n and getting suspended) in both the pre- and posttest surveys. Students were asked, \ “Have you [obtained any failing grades; skipped school without permission; been su\ spended] in the past year?” Responses to these questions were dichotomously coded “yes/no.” If a student responded yes, he or she was asked to report how many times. Logistic regression analysis in SAS 9.4 was used to determine whether th\ ere were any significant differences in the intervention and comparison group, while control- ling for pretest responses, parent’s education, household structure, and gender.

Results Social, Emotional, and Civic Attitudes Overall, the differences between pre- and posttest surveys were not significant in the hypothesized direction. In fact, attitudes for certain scales were marke\ d less favorably between pretest and posttest for the intervention students. The two areas where stu- dents from the intervention school were significantly different from students in the comparison group were areas of emotional engagement and academic efficacy.

Emotional engagement. At pretest, the mean score for students in the intervention school was 18.43 and dropped about 2 points at posttest to 16.09. Studen\ ts from the comparison school only dropped about 1 point from 17.30 in the pretest to 16.48 in the posttest (t = 2.61, p < .05; Table 2). These scores suggest that students from the inter - vention school agreed less with the emotional engagement scale in the po\ sttest than they did in the pretest when compared with their peers in the comparison\ school.

Academic efficacy. Academic efficacy scales referred to students’ self-ratings on ques- tions that gauge student’s beliefs on their ability to complete and learn difficult school- work, master the skills taught in school, and discern and solve difficult concepts. The highest score an individual could get on this scale is 25 points, with h\ igher values indicating more agreement with the questions being asked. Overall, the s\ cores on both the pre- and posttests were high for both schools (Table 2). Students in the intervention school decreased their ratings of efficacy by 1.43 points compared with the 0.27 increase in the comparison group (t = 2.25, p < .05). McBride et al. 379 There were no significant changes between the intervention and compariso\ n group for attitudes on school belonging, behavioral engagement, or civic duty.

Student Behaviors Control variables were chosen for regression analysis by examining exist\ ing literature on risk and protective factors for adolescents (Zolkoski & Bullock, 2012). Environmental factors, such as parental education and household structure, as well as \ demographic fac- tors such as gender, were examined. Previous history of the adolescent’s targeted behav- ior (baseline pretest scores) was also included to adjust for the syst\ emic bias and error variance (Barry, 2015). At baseline, there were no significant differences in the number of failing grades, suspensions, and classes skipped reported by students\ in the interven- tion and comparison school. By posttest, however, logistic regression results indicated that students who participated in the intervention were 62% less likely than the compari- son group to skip class without permission (p < .05, χ 2 = 24.39, β = −0.96, odds ratio [OR] = .38, 95% confidence interval [CI] = [0.16, 0.92]) and 78% less l\ ikely to report failing grades (p < .05, χ 2 = 38.20, β = −1.50, OR = .22, 95% CI = [0.09, 0.54]) even while controlling for previous history of the outcome behavior, parent’s education, household structure, and gender (Table 3). The results suggest that students who went through the intervention were less likely to engage in behaviors that ar\ e correlated with negative academic outcomes and behaviors. Discussion Wyman’s TOP, an SEL curriculum with experiential service-learning components, has been delivered and evaluated in a variety of contexts, though primarily \ in after-school, extracurricular high school settings (Chung & McBride, 2015). Few stud\ ies, however, Table 2. School Comparisons of Pre- and Postscales (Full Sample).

Variable α Pretest Posttest Difference t Intervention Comparison Intervention Comparison Intervention Comparison M SD MSD MSD MSD M M School belonging .8 3.79 12.78 0.93.27 1.032.53 0.9 −0.54 −0.271.8 Emotional engagement in school .8 3.69 0.83.46 0.83.22 0.933.3 0.9 −0.47 −0.152.61* Behavioral engagement in school .7 4.28 0.54.2 0.63.92 0.84.02 0.8 −0.34 −0.161.9 Academic self- efficacy .8 4.31 0.64.2 0.74.03 0.844.16 0.7 −0.29 −0.052.25* Civic duty .84.04 0.63.7 0.73.76 0.713.59 0.7 −0.27 −0.131.5 *p < .05. 380 Journal of Experiential Education 39(4) have examined the effectiveness of this SEL service-learning program when delivered in middle school. This lack of nuanced research is important as the program continues to be implemented in a variety of school settings. This study’s focus was to understand whether delivering Wyman’s TOP within middle school curricula during the school day could prevent negative disciplinary outcomes for youth. Results from this study suggests that adolescents who participated in th\ e yearlong intervention experienced statistically significant reductions in failing\ grades and skip- ping classes compared with the comparison group, even while controlling \ for a variety of demographic factors. Although adolescents from the comparison group did not dif- fer significantly in initial rates of failing grades and skipping classes without permis- sion, there were significant reductions in these negative school behavio\ rs for adolescents from the intervention group by posttest. These outcomes support the hypothesis that the intervention may have positive effects on behaviors related to aca- demic success.

Limitations Limitations of this study encompass issues with sampling and the occurre\ nce of major unanticipated events. First, the sample included students whose parents \ actively con- sented for them to participate in the study. Therefore, students who did not participate in the research sample may represent those who are already disengaged an\ d possibly most at risk. Future assessments of the program should be made to understand this sample and how they compare with students who elected to participate. In\ relation to the limitation of sampling, there may also be issues with social desirab\ ility present in this study. For instance, when students are asked questions about negative school behaviors, they may have answered in a way that they can be perceived mo\ re favorably, Table 3. Logistic Regression for Behavioral Outcomes—Students Receiving Wyman’\ s Teen Outreach Program.

Outcome Model statistics Wyman’s Teen Outreach Program Model χ 2 Max rescaled R 2 β SEOR 95% CI Failing courses 41.14.31−0.41.42.670.29 1.53 Failing grades 38.20*.32−1.50*.46.220.09 0.54 Skipping class 24.39*.21−0.96*.45.380.16 0.92 Suspensions 47.02.39−0.88.47.410.16 1.05 Note. Control items include the following variables: Gender, mother and fathe\ r’s education, family composition, and previous history of the listed behavior. Coding was con\ ducted as follows: Gender (1 = female), mother/father’s education (0 = less than high school\ ; 1 = high school graduation or more), Intervention School (0 = Comparison School; 1 = Treatment School), hou\ sehold composition (0 = two-parent family; 1 = single parent; 2 = guardian/other), and pr\ evious history of X behavior (0 = no; 1 = yes). OR = odds ratio; CI = confidence interval.

*p < .05. McBride et al. 381 as they understand the intervention was focused on reducing these types \ of outcomes.

We tried to minimize these effects by making responses anonymous in the survey administration process. Furthermore, the lack of randomization poses ano\ ther limita- tion in regard to the generalizability of the results. Another unforeseeable incident that may influence our findings is that t\ he compari- son school lost its accreditation shortly after the first intervention y\ ear. Therefore, underlying issues within the school may have affected students in ways that are unac- counted for in the design of this study. Although the comparison school was undergo- ing various accreditation issues, there are also aspects of the interven\ tion school that may have altered our findings in unintended ways. Like many schools, the\ re were a number of positive behavioral interventions being implemented. For insta\ nce, at the time of this study and unbeknownst to the researchers, the intervention school was also implementing strategies to reduce tardiness among students. It is unclear what effect these programs had on the intervention students, but it is important to \ point out that many school-based studies experience similar issues with conflation. Although the data are informative for the case study, they should not be considered generalizable to a larger context.

Examining the Findings Why were there positive effects on academic engagement in the form of reduced fail- ing grades? The intervention is highly experiential. Instead of lecture-based classr\ oom sessions, lessons are tied to existing school curriculum through opportunities for rich group discussion and hands-on learning. Even the additional service-lear\ ning projects required by TOP were closely aligned with school coursework. Students were asked to participate in the planning, development, and execution of service-learning activities.

These types of activities are considered to be critical to the success o\ f service learning.

In addition, when service learning is tied to opportunities for reflecti\ on and class time, as it was in the TOP intervention, it enhances the academic climate (Levesque-Bristol, Knapp, & Fisher, 2010). We conjecture that the hands-on activities connected with cognitive learning, as conducted in the intervention, were critical to engage students in the educational process. These outcomes have also been seen in other studies where service learning has been suggested to be an effective method of increasing academic performance (Celio, Durlak, & Dymnicki, 2011). Contrary to our second hypothesis, however, behavioral, emotional, and civic atti- tudes did not differ significantly from pre- to posttest. The results of the student atti- tudes in our study are perplexing. The attitudes compared with the self-reported behaviors measured in this study seem to contradict each other. Although some behav- iors, such as getting failing grades in school and skipping classes with\ out permission, improved in the intervention school, attitudes on behavioral and emotion\ al engage- ment did not. This paradox might be explained by a variety of possibilities related to\ school-based interventions and adolescent development. For instance, per\ haps partici- pation in this intervention made adolescents realize that their initial \ self-ratings in the pretest were unrealistic. It may be possible that middle school students\ increased in 382 Journal of Experiential Education 39(4) understanding of what it meant to be behaviorally engaged after receivin\ g targeted instruction on how to engage in school. This type of measurement bias has been described as response-shift bias (Howard & Dailey, 1979). Response-shift bias is unlike social desirability bias, when survey part\ icipants respond in a manner that would be viewed favorably by others. According to Howard & Dailey (1979), response-shift bias is a consequence of many interven\ tions that aim to change a subject’s awareness or understanding of the variable being mea- sured. It refers to a type of measurement bias where survey respondents \ rate them- selves lower in the posttest because as a result of participating in an \ intervention, they learn their initial understanding of the question was flawed. Evide\ nce of this bias has been noted in evaluations of education programs teaching assert\ iveness and learning basic helping skills (Howard & Dailey, 1979). Given the educational setting and age group, the use of retrospective pretests may remove this respons\ e-shift bias.

These types of pretests demonstrate significantly greater validity than \ the measure of change used in traditional self-report pretests. Moreover, it would be helpful to use objective measures of change, such as grades or triangulation with teachers, peers, and parents. Inconsistencies in attitudes on social and emotional outcomes are not ne\ w in SEL research. Whitehurst (2016) suggests that this has less to do with the attitudes\ themselves than it does with the nature of traits we measure to gauge these SEL constructs. In his Brookings Institution report, Whitehurst argues that the concep- tualization and measurement of these soft skills are conducted in a way \ that psy- chologists approach human personality and therefore schools have difficulty influencing differences among students (Whitehurst, 2016). He gives examples of charter schools that have effectively raised student achievement by focusing on character development but have had no or reverse impact on students’ self-reported soft skills. Our findings are consistent with many other attitudes which\ indicate that studying the measurement of SEL is just as, if not more, important than implement- ing the interventions (Cohen, 2006; Humphrey, 2013). Although this study only compared students from two schools, the lack of change and unexpected im\ pact on social and emotional attitudes support the theory that there may be a wi\ der issue regarding SEL measurement in adolescence. Although there were inconsistencies between behaviors and attitudes, the results of this study did point to possible benefits for student behaviors. Stud\ ents in this program reported that they were less likely to skip class without permis\ sion and attain failing grades after a year of engaging in the intervention curri\ culum. And yet, though these behaviors improved, attitudes toward school engagement actu\ ally decreased. As previously discussed, these changes could likely be the result of response-shift bias or social desirability. These findings may also be indicative of processes that are inherent for early adolescent development. Given thes\ e paradoxi- cal outcomes, future research is warranted. Consequent studies should ex\ amine how skills change during adolescence, refining the definition of constructs;\ whether out- comes from these types of interventions sustain beyond their grade level\ ; and how to control for learning biases. McBride et al. 383 Conclusion This study assessed the effects of delivering an SEL program utilizing service learning within middle school curricula. It updated existing studies on a popular\ youth inter - vention, the TOP, and provided a new perspective gauging whether the intervention was effective at reducing and preventing negative school behaviors, such as ge\ tting failing grades and skipping class. Although the intervention seemed to have positive effects on some behaviors, attitudes regarding behavioral and emotional engagement, academic self-efficacy, and civic responsibility did not seem to change in the desired direction. Further research is needed to understand the impact of SEL on this develop- mental phase, especially through service-learning curricula. Particular attention should be paid to sensitivity of design and measurement.

Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect\ to the research, authorship, and/or publication of this article.

Funding The author(s) received no financial support for the research, authorsh\ ip, and/or publication of this article.

References Allen, J. P., & Philliber, S. (1991, January). Evaluating why and how the Teen Outreach Program works: Years 3-5 of the Teen Outreach National Replication (198\ 6/87-1988/89).

New York, NY: Association of Junior Leagues.

Allen, J. P., & Philliber, S. (2001). Who benefits most from a broadly\ targeted prevention pro- gram? Differential efficacy across populations in the Teen Outreach Prog\ ram. Journal of Community Psychology, 29, 637-655.

Allen, J. P., Philliber, S., & Hoggson, N. (1990). School-based preven\ tion of teen-age pregnancy and school dropout: Process evaluation of the national replication of th\ e Teen Outreach Program. American Journal of Community Psychology, 18, 505-524.

American Academy of Pediatrics. (2013). Out-of-school suspension and e\ xpulsion. Pediatrics, 112, 1206-1209. doi:10.1542/peds.112.5.1206 Anderson-Butcher, D., Stetler, E. G., & Midle, T. (2006). A case for e\ xpanded school- community partnerships in support of positive youth development. Children & Schools, 28, 155-163. doi:10.1093/cs/28.3.155 Barry, J. (2015, September 16). Data analysis of pre-post study designs. Cornell Statistical Consulting Unit. Retrieved from https://www.cscu.cornell.edu/news/statne\ ws/stnews79.pdf Billig, S. H. (2000). Research on K-12 school-based service-learning. \ Phi Delta Kappan, 81, 658-665.

Bringle, R. G., Clayton, P. H., & Hatcher, J. A. (2013). Research on s\ ervice learning: An intro- duction. In P. H. Clayton, R. G. Bringle, & J. A. Hatcher (Eds.), Research on service learn- ing: Conceptual frameworks and assessment (pp. 3-25). Sterling, VA: Stylus Publishing.

Bryant, V. C., Shdaimah, C., Sander, R. L., & Cornelius, L. J. (2013).\ School as haven: Transforming school environments into welcoming learning communities. Children and Youth Services Review, 35, 848-855. doi:10.1016/j.childyouth.2013.02.001 384 Journal of Experiential Education 39(4) Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of th\ e impact of service-learn- ing on students. Journal of Experiential Education, 34, 164-181.

Chung, S., & McBride, A. M. (2015). Social and emotional learning in middle school curri- cula: A service learning model based on positive youth development. Children and Youth Services Review, 53, 192-200. doi:10.1016/j.childyouth.2015.04.008 Cohen, J. (2006). Social, emotional, ethical, and academic education: Creating a climate for learn- ing, participation in democracy, and well-being. Harvard Educational Review, 76, 201-237.

Collaborative for Academic, Social, and Emotional Learning, (2014). What is social and emo- tional learning? Retrieved from http://www.casel.org/social-and-emotional-learning/ Conduct Problems Prevention Research Group. (2010). The effects of a m\ ultiyear universal social-emotional learning program: The role of student and school characteristics. Journal of Consulting and Clinical Psychology, 78, 156-168. doi:10.1016/j.biotechadv.2011.08.021 D’Agostino, C. (2013). Collaboration as an essential school social \ work skill. Children & Schools, 35, 248-251.

Diekstra, R. F. W. (2008). Evaluation of effectiveness of school-based\ social and emotional educa- tion programmes worldwide. In Fundacion Marcelino Botin (Ed.), Social and emotional edu- cation: An international analysis (pp. 255-284). Madrid, Spain: Fundacion Marcelino Botin.

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schel\ linger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A me\ ta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi:10.1111/j.1467- 8624.2010.01564.x Epstein, J. L. (1995). School/family/community partnerships: Caring fo\ r the children we share. Phi Delta Kappan, 76, 701-716.

Epstein, J. L. (2001). School, family, and community partnerships: Preparing educators and improving schools (1st ed.). Boulder, CO: Westview Press.

Eyler, J., & Giles, D. E. (1999). Where’s the learning in service-learning? San Francisco, CA: Jossey-Bass.

Felten, P., & Clayton, P. H. (2011). Service-learning. New Directions for Teaching & Learning, 128, 75-85. doi:10.1002/tl Goodenow, C. (1993). The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychology in the Schools, 30, 79-90.

Howard, G. S., & Dailey, P. R. (1979). Response-shift bias: A source o\ f contamination of self-report measures. Journal of Applied Psychology, 64, 144-150. doi:10.1037/0021-9010.64.2.144 Humphrey, N. (2013). Social and emotional learning: A critical appraisal (1st ed.). London, England: Sage.

Levesque-Bristol, C., Knapp, T. D., & Fisher, B. J. (2010). The effect\ iveness of service- learning: It’s not always what you think. Journal of Experiential Education, 33, 208-224.

doi:10.5193/JEE33.3.208 Maras, M. A., Splett, J. W., Reinke, W. M., Stormont, M., & Herman, K. C\ . (2014). School prac- titioners’ perspectives on planning, implementing, and evaluating evi\ dence-based practices.

Children and Youth Services Review, 47, 314-322. doi:10.1016/j.childyouth.2014.10.005 McBride, A. M., Pritzker, S., Daftary, D., & Tang, F. (2006). Youth se\ rvice: A comprehensive perspective. Journal of Community Practice, 14, 71-90. doi:10.1300/J125v14n04 Midgley, C., Maehr, M. L., Hicks, L., Roeser, R., Urdan, T., Anderman, E\ . M., . . . Middleton, M. J. (2000). Manual for the Patterns of Adaptive Learning Scales. Ann Arbor: University of Michigan. Retrieved from http://www.umich.edu/~pals/PALS2000_V12Word9\ 7.pdf Neil, A. L., & Christensen, H. (2009). Efficacy and effectiveness of s\ chool-based prevention and early intervention programs for anxiety. Clinical Psychology Review, 29, 208-215.

doi:10.1016/j.cpr.2009.01.002 McBride et al. 385 Promising Practices Network: Programs That Work. (2013). Wyman’s Teen Outreach Program.

Retrieved from http://www.promisingpractices.net/program.asp?programid=1\ 4 Schmidt, J. A., Shumow, L., & Kackar, H. (2006). Adolescents’ parti\ cipation in service activi- ties and its impact on academic, behavioral, and civic outcomes. Journal of Youth and Adolescence, 36, 127-140. doi:10.1007/s10964-006-9119-5 Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85, 571-581. doi:10.1037/0022-0663.85.4.571 Wadsworth, M. E., Raviv, T., Reinhard, C., Wolff, B., Santiago, C. D., &\ Einhorn, L. (2008). An indirect effects model of the association between poverty and child functioning:

The role of children’s poverty-related stress. Journal of Loss & Trauma, 13, 156-185.

doi:10.1080/15325020701742185 Whitehurst, G. J. “Russ.” (2016). Hard thinking on soft-skills. Brookings Institution. Retrieved from http://www.brookings.edu/research/reports/2016/03/24-hard-thinking-soft-\ skills- [email protected]&;utm_campaign=Center+on+Children+an\ d +Families&utm_source=hs_email&utm_medium=email&utm_content=27848577&_ hsenc=p2ANqtz–YZuJXArwpHMQHSvx_jn1 Wilson, S. J., Lipsey, M. W., Jo, S., & Mark, W. (2006). The effects o\ f school-based social information processing interventions on aggressive behavior: Part I: Uni\ versal programs: A systematic review. The Campbell Collaboration, 6, 1-37. doi:10.4073/csr.2006.5 Wyman’s Teen Outreach Program. (2015). Substance Abuse and Mental H\ ealth Services Administration’s National Registry of Evidence-based Programs and Pra\ ctices. Retrieved from http://legacy.nreppadmin.net/ViewIntervention.aspx?id=203 Zaff, J., Boyd, M., Li, Y., Lerner, J. V., & Lerner, R. M. (2010). Act\ ive and engaged citizenship: Multi-group and longitudinal factorial analysis of an integrated constru\ ct of civic engage- ment. Journal of Youth and Adolescence, 39, 736-750.

Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (200\ 4). The scientific base linking social and emotional learning to school success. In J. E. Zins, \ R. P. Weissberg, M. C. Wang, & H. J. Walberg (Eds.), Building academic success on social and emotional learning: What does the research say? (pp. 3-22). New York, NY: Teachers College Press.

Zins, J. E., & Elias, M. J. (2006). Social and emotional learning: Pro\ moting the development of all students. Journal of Educational and Psychological Consultation, 17, 233-255.

Zolkoski, S. M., & Bullock, L. M. (2012). Resilience in children and y\ outh: A review. Children and Youth Services Review, 34, 2295-2303. doi:10.1016/j.childyouth.2012.08.009 Author Biographies Amanda Moore McBride is the Morris endowed dean and professor at the Graduate School of Social Work at the University of Denver. An international expert in civic and community engagement, she has authored or coauthored more than 70 books, special j\ ournal issues, and articles. Recent research focuses on higher education.

Saras Chung is a computational social scientist with the Brown School of Work at Was\ hington University in St. Louis. Her research examines school-based initiatives to foster positive youth development and prevent negative mental health outcomes using systems sc\ ience.

Anne Robertson is an adjunct professor at the University of Denver Graduate School of Social Work. Her research interests include social work and educational interve\ ntions and other inter- disciplinary efforts focused on child and family well-being. Copyright ofJournal ofExperiential Educationisthe property ofSage Publications Inc.and its content maynotbecopied oremailed tomultiple sitesorposted toalistserv without the copyright holder'sexpresswrittenpermission. However,usersmayprint, download, oremail articles forindividual use.