Students Help Design Measures of Social-Emotional Skills

April 14, 2016

By: Evie Blad

Source: edweek.org

Schools in this city, known for its aging casinos, are using a comprehensive social-emotional learning strategy to tackle student engagement and academic success.

The 64,000-student Washoe County district wants to raise its graduation rate, which reached a record 75 percent in 2015, to 90 percent by 2020, an ambitious goal in a state where young adults can make a middle-class salary valet-parking cars without a diploma, Superintendent Traci Davis said.

A growing body of research connects skills like responsible decisionmaking and recognizing and responding to emotions with greater engagement in the classroom and improved academic outcomes such as higher graduation rates. But, until recently, the district had very few ways of measuring the effectiveness of its social-emotional learning efforts, which it launched as a result of a 2010 strategic plan.

It asked questions many educators and researchers are facing: What’s the most accurate way to determine if students are learning so-called “soft skills,” like how to empathize with their peers? And what’s the best way to respond to the resulting data?

Better data would also help answer a core question for the district: Is social-emotional learning contributing to that rising graduation rate? Educators who’ve embraced the strategy believe it’s necessary, but they’ve lacked data in the past that proves how much they are moving the needle for students.

“If we could come up with good measures, then maybe we could measure the mediating effect of social-emotional learning competencies on this risk [of not completing high school],” said Ben Hayes, the district’s chief accountability officer. “That became the kind of learning goal, and from there we found out very quickly that we need to have better measures.”

While Washoe County is way ahead of other school districts on developing more refined and careful measures of students’ skills, educators there say there is much more work to be done before the results should be used for accountability.

With the passage of the Every Student Succeeds Act, the main federal K-12 law, states are now required to include at least one measure other than test scores in their accountability systems, an issue that has raised the profile of social-emotional measurement questions.

Better Measures

With the help of a federal grant and assistance from the Collaborative for Academic, Social, and Emotional Learning, or CASEL, Washoe County administrators developed new survey measures, working with students to understand how they respond to questions about social-emotional skills and with teachers to develop data that could actually be used to change what happens in the classroom.

Alongside the district’s existing early-warning system, which tracks risk factors for dropping out throughout a student’s school career, the data being gathered around social-emotional learning is helping educators ensure that students’ needs are met and their skills are developed as they progress through school.

A majority of the district’s enrollment is students of color. Next to white students, who make up about 46 percent of Washoe County’s enrollment, Hispanic students are the largest group, at about 40 percent. Nearly half of students qualify for free and reduced-price meals, a common measure of poverty, and about 16 percent of them are English-learners.

Social-emotional learning is a key to helping otherwise successful students thrive after graduation and to helping buffer risk factors for students considered more likely to drop out, the district found.

In an analysis of 2014 data, students who were classified as “high risk” in the district’s early-warning system but who also demonstrated strong social-emotional competencies performed just as well as their low-risk peers on state assessments.

The district’s work has been noticed and replicated. Questions developed through its social-emotional measurement project have been adopted by the district in Austin, Texas, and incorporated into statewide student surveys in Alaska and Nevada.

Washoe County is one of eight urban districts that have committed toimplementing comprehensive social-emotional learning practices for students in elementary, middle, and high schools and allowing researchers from the American Institutes for Research to measure their results. Districts in the group, coordinated by CASEL, include Anchorage, Austin, Chicago, Cleveland, Nashville, Tenn., and Oakland and Sacramento in California. All have taken their own approach, reworking discipline policies, boosting student-staff relationships, and gradually adopting direct instruction of social and emotional skills in classrooms.

In Washoe County’s elementary and middle schools, for example, students learn the concepts through role-playing exercises, games, and class discussions.

Older students, like those in Damonte Ranch High School, learn about forming healthy relationships and meeting long-term goals in their traditional, subject-oriented classes and in special advisory periods.

And adults in the school have learned new skills, too.

Administrators now require teachers to solve non-violent problems in the classroom or by calling for a quick consultation in the hallway, rather than sending a student directly to the office.

Principal Denise Hausauer, an exuberant woman who is passionate about the school’s strategies, wears a jingle bell on a lanyard around her neck so students can hear her coming down the hall. She wants to catch them doing something good so she can reinforce positive behaviors through praise.

“When I say, ‘I’ll be there with bells on,’ I mean it literally,” she said.

And the school’s values and expectations for students are painted colorfully in the school’s hallways.

“If you’re not strategic about this stuff, you’re not doing enough for your kids,” Hausauer said.

‘Ceiling Effect’

But how do Washoe County educators know if the strategy is working?

Good data both informs efforts and helps motivate teachers by showing them how they are helping students, said Laura Davidson, director of research and evaluation for the district.

The district first incorporated questions about social and emotional skills into a 2013 version of a school climate survey it administers to students annually, but administrators quickly realized the responses to those questions weren’t helpful.

The surveys asked students to rate on a one-to-five scale whether it was easy or difficult for them to do various tasks, like empathizing with a peer.

But Davidson found a “massive ceiling effect,” she said, because large amounts of students ranked everything as easy. The team working on the measures analyzed the data and found that students were “topping out” for three reasons: they really did have high skills in those areas, they got bored taking the surveys, or they didn’t understand the questions.

“Lots of kids were saying that they had perfect skills or really good skills,” Davidson said. “The problem with that is that we can’t tell, do the students really have these skills or are they really not engaging in the survey?”

So the team turned to the students themselves, holding focus groups to create questions they could understand and relate to and about skills they would find more challenging, skills linked to the district’s social-emotional learning goals.

“By having them talk about these relationship skills as they happen in their classroom, we were able to come up with all sorts of ideas for more challenging items,” Davidson said.

And researchers also teamed up with teachers to generate items they could actually respond to and address, said Jeremy Taylor, CASEL’s director of assessment and continuous improvement.

“If a teacher can’t really do anything about it, it’s not really useful to measure it,” he said.

Through the process, the district came up with 150 items that measure students’ competency in the five areas it seeks to nurture: self-awareness, self-management, social awareness, relationship skills, and responsible decisionmaking.

Within that list, the district identified 17 “anchor items,” questions it plans to include on future student surveys. High student scores on those items correlate with stronger academic and behavioral outcomes, the team found, suggesting they are valid measures.

Washoe County isn’t using its social-emotional data for high-stakes school accountability, and both Davidson and Hayes said they don’t recommend using their survey items for that purpose.

The Future of Measurement

Concerns about how measurements of non-academic skills should be used have gotten a heap of attention since Congress passed the Every Student Succeeds Act late last year. Some social-emotional learning advocates have suggested that using measures of students’ skills in state accountability systems would encourage more schools to adopt such programs.

But some high-profile researchers have said the measures shouldn’t be used for any kind of accountability because they are still prone to biases, and because they could lead to unforeseen consequences, like shallow classroom exercises that focus on how to correctly answer surveys rather than really changing student behavior.

The question of how to responsibly and accurately measure students’ social-emotional strengths and skills is one that CASEL and other groups are working to address, Taylor said.

A 30-member working group of researchers and practitioners will review what measures schools are currently using and how to use the resulting data.

The group will also work to explore the future of social-emotional learning assessments, examining the feasibility of tools like classroom-based performance items and computer programs to measure student strengths, Taylor said.

“We want to determine the characteristics that make any measure as useful and practical for educators as possible,” he said.

As researchers work to refine measures, schools are already putting them in place.

Washoe County plans to make its measures available for other schools to use, along with some resources about how to learn from the resulting data.

And the district wants to continue exploring how to analyze students’ strengths, particularly among younger students, Davidson said. The answer may be a more creative vehicle, like video games that put students in hypothetical situations, she said.

Once Washoe County finalizes results from its most recent survey, it will share them with students in a “student data summit” where they can openly ask questions and learn about how their schools measure up in academics, social-emotional learning, and other comparative data.

When the social-emotional measurement team sought students’ input, they realized just how interested the students were in their schools’ results, Hayes said.

“They know what we’re talking about, and they can really inform the conversation.”