This is the third in an occasional series on new teacher evaluations in Colorado. Previously, we looked at broad changes coming as the state’s new teacher evaluation law takes effect in the fall and early scores released under LEAP, Denver’s teacher evaluation system.
LEAP is not only Denver’s new teacher evaluation system, it’s a work in progress.
Begun as a 16-school pilot in 2011, LEAP — Leading Effective Academic Practice — becomes Denver’s one and only teacher evaluation system beginning in the fall. It coincides with the first official year of Senate Bill 10-191, the so-called teacher effectiveness law.
SB-191 dramatically changes how a teacher gets tenure by no longer linking tenure status to longevity alone. Under the law, non-probationary status will be earned after three consecutive years of demonstrated effectiveness. Teachers will lose non-probationary status — more commonly known as tenure — after two consecutive years of ineffective ratings. However, ineffective ratings will not be counted against a teacher in the first year.
While LEAP is further along than many teacher evaluation systems in Colorado, there are still holes that need to be plugged in anticipation of the fall. For instance, it hasn’t yet been decided how much weight each piece — test scores, professionalism, student feedback, other measures — will be given or whether the weighting will be different for each teacher. And Denver has yet to incorporate standardized test scores and other student outcomes into LEAP, though the law requires student outcomes to comprise half a teacher’s evaluations.
“This year we had hoped to pilot [student outcomes] with a few teachers but we did not get to that point,” said Pam Shamburg, the Denver Classroom Teachers Association representative to LEAP. “It turns out to be more complex than we were aware…When you get into it, do you use one year’s data — because there’s a lot of swing year to year. Or do you use three years of data?”
Work this summer will be focused on incorporating student outcomes into LEAP, along with deciding what numerical ratings translate into which of four categories — not meeting expectations, approaching expectations, effective and distinguished. Recently released partial LEAP ratings revealed that just over 50 percent of teachers landed between 4.5 and 5.5 and straddled “approaching” and “effective.”
While negative ratings won’t count in the first year, Jennifer Stern, executive director of Teacher Performance Management for Denver Public Schools, said the stakes are still high for Denver teachers. The data is already being used to decide whether probationary teachers are renewed — a process that has caused some controversy — and to recognize teacher leaders. Next year, LEAP will be used to help district officials make dismissal and remediation decisions.
“One piece — the scariest piece – is not happening yet,” Stern said. “But next year it’s for real.”
Tweaks made this year based on teacher feedback
Significant adjustments were made to LEAP this year based on feedback from those on the receiving end of the classroom observations and evaluations.
For instance, the district was more focused in terms of how it collected student perception data that is included in LEAP. Students in grades 3 through 12 are asked questions about everything from whether the teacher does a good job explaining difficult concepts, whether the classroom is under control or if the teacher knows whether something is bothering the student.
“Kids know if they’re learning,” Stern said. “They know if the teacher respects them. In some ways, this is the most accurate and reliable [data].”
The rubric was also shortened from 21 sometimes redundant indicators of teacher quality to 12. And the approach to observations was changed. In 2011-2012, there were three full observations by principals and two by peer observers.
“We got a lot of feedback…that the principals couldn’t get all that done,” Stern said.
This year, principals made more frequent — but shorter — visits to classrooms. They filled out the parts of the rubric that they could. They were required to observe at least one complete lesson, along with some “walk-throughs,” or half a lesson.
“We just said, ‘Get in there,'” Stern said. “‘You’re in the classrooms. Capture what you’re seeing.'”
The strategy shift has resulted in more data collected and more observations, which district officials hope will help improve the system’s reliability.
But teachers still have concerns about principals’ ability to perform quality observations and feedback sessions under LEAP.
“What we’re hearing from our teachers pretty overwhelmingly is that they’re beginning to like this theory behind the system…but they’re still really worried about implementation,” Shamburg said. “They’re really concerned about how well our principals and assistant principals know how to implement this system. They’re not as confident about that.”
Stern acknowledged there were gaps in the system this school year.
“Was it where it needs to be with every school?” Stern said. “No. We have work to do to make sure teachers get the amount of feedback they need and deserve.”
One other issue that emerged as more schools joined LEAP this year is that 45 peer observers is not enough. So this year, the district had to figure out who needed a peer observer most. Probationary teachers, who make up a little over a third of those observed, make up one such category. Only half the district’s non-probationary (that is, tenured) teachers had observers popping up in their classrooms this year. The other half will get them next year.
“We didn’t have capacity to give everybody a peer observer,” Stern said.
LEAP staff and the DCTA are developing a system so that teachers below a certain benchmark in terms of effective teaching traits will definitely be assigned a peer observer next year. Teachers above that to-be-determined performance bar will be able to opt into LEAP if they want to.
LEAP staff are also tweaking the professional development options available to teachers by differentiating them, so a teacher gets professional development that is as relevant as it can be — either through videos or face-to-face courses. For instance, a teacher who gets a 2 in one category needs far different supports than a teacher who earned a 6 in that same category. LEAP staff also want to tap home grown talent, including standout teachers in each building who can offer coaching and support to colleagues.
Teacher finds peer observations helpful
One of the big shifts this year has been the overall attitude by teachers about the district’s 45 peer observers.
“At the beginning of this year, teachers were really leery of peer observers,” Shamburg said. “We’ve seen a 180-degree change. It doesn’t mean every teacher was happy with every peer observer, but we’re hearing very strongly, ‘I trust my peer observer. They give me feedback I can use.'”
North High School algebra and geometry teacher Zachary Rowe, 26, said undergoing LEAP evaluations has been helpful, especially as he dove into his “focus indicators.” Teachers hone in on one indicator under the school’s focus area — North’s is masterful content delivery — and another of their choosing.
Rowe, who teaches freshmen, is focusing on creating rigorous tasks for students and boosting communication and collaboration among his youthful charges.
“I like this a lot more,” Rowe said of LEAP. “Under the old system, someone would walk in, write down their feelings about your class, leave you a slip on a desk and walk out.”
More frustrating than that were conversations with principals or school leaders who shared their thoughts and feelings but no tangible suggestions to improve.
“They were not really telling you specifically what to do to get better or even where you could look to get better,” Rowe said. “It was really frustrating as a new teacher. One person thought you were great, and wrote you really nice notes, then the evaluator the following year thought I had a lot to work on, and wrote not-so-nice notes.”
Under LEAP, Rowe has meaningful conversations “grounded in a framework.” A peer observer helped him trouble shoot ideas to get more students to volunteer to solve problems on a smart board in front of the class. The observer suggested sentence starters so students knew how to start talking about the problem-solving process.
Rowe is curious how he will be rated under LEAP next year.
“I have received ‘exceeding expectations’ bonuses in the past,” he said. “Analyzing my data in the past has been positive. I assume LEAP will continue to reveal that.”
Rowe is particularly curious to see how student outcomes scores equate to those ratings given by observers and principals. And as much as he likes LEAP, he knows not every observation will yield a glowing review.
“There are days when I am a distinguished teacher, and an equal number of days I am not,” Rowe said. “I am ‘approaching’ at best and ‘not meeting’ at worst, and I think that’s OK. As teachers, we can’t be expected to walk on water every single day – and no expects us to walk on water every single day. The trick is how teachers build an environment in a classroom – create a system to ensure that even on the worst days they’re still effective.”
But spikes in performance or consistently low ratings can indicate a problem, he said.
“For me and most of my colleagues we’re pretty much within a one point difference consistently throughout the year,” he said. “We get excited about gunning for 7s.”
Principals also face scrutiny
In 2012-2013, Denver also put its principals in the LEAP spotlight as it piloted LEAP evaluations for school leaders.
There is ongoing discussion about how principal ratings will work, as the numerical system is not the same as the one used for teachers. Instructional superintendents are talking to principals now about their observations, and school leaders are busy setting their goals for next year. Goals will be adjusted once TCAP results are released in the fall.
With more time spent on LEAP, teachers — and principals — are becoming more comfortable with the system, Stern said. But that doesn’t mean it’s perfect.
“People are always going to be anxious when there is something new out there that feels high stakes,” Stern said. “I think we’re always working on how do we make this focused on the right things — make it focused on kids, growth and development. It does have stakes attached.”
Stern said DPS wants to make sure educators are comfortable being held accountable for their performance.
“We’re trying to build that comfort,” she said. “We’re better than we used to be.”