A roomful of teachers sat in a training room in September, surrounded by flip charts emblazoned with the titles of popular movies: Jaws; Tangled; Sleepless in Seattle; The Wizard of Oz.
Trainer Courtney Cabrera asked the group to stand next to the flip chart that most closely matched their feelings about the pending rollout of Senate Bill 10-191, Colorado’s groundbreaking teacher effectiveness law.
By the end of the training — one of 50 sessions designed to get teachers more comfortable with the tenets of the new evaluation system — teachers talked about whether they would pick a different movie title now, maybe one that wasn’t about unsuspecting swimmers having limbs ripped off by a great white shark. Most did modify their movie picks to ones with a gentler tone.
“The more practice [educators] get using our rubric, the more questions that are asked and answered, the more comfortable they feel with the system,” said Katy Anthes, the lead executive on teacher effectiveness for the Colorado Department of Education (CDE). “Honestly, we’re at the cusp of a huge change process.”
Next school year, the teacher effectiveness law will become part of every school district’s approach to teaching and learning in Colorado.
And by July 1, every district in Colorado must tell the state how it plans to evaluate teachers under the new law.
This is the first in a series EdNews Colorado plans to publish examining the impact of a policy shift that is proving pivotal in the national debate over how to improve teacher quality.
Observers may have to hold their collective breaths, however, to learn whether the new system is actually improving the quality of teaching. The first statewide teacher evaluation data won’t be available until December 2014.
SB-191 backstory
The goal of the law, signed by Gov. Bill Ritter in May 2010, is to improve teacher and principal quality by providing meaningful evaluations and conversations throughout the academic year that help both grow as professionals by providing them with resources linked to whatever weaknesses emerge.
SB 10-191 timeline
- 2011-12 – Elements of system piloted for principals
- 2012-13 – Pilot testing for teachers and principals
- 2013-14 – All districts to use the state system or an approved local system; evaluation results won’t affect tenure status
- 2014-15 – Ratings of partially effective or ineffective will begin to affect tenure status
- 2016-17 – First year a teacher could lose tenure (“non-probationary status”)
Key provisions of the law
- Annual evaluations of principals and teachers
- 50 percent of evaluations based on student academic growth
- Teachers lose tenure if rated less than effective for 2 consecutive years
- Loss of tenure does not mean automatic loss of job
- Assignment to a school requires mutual teacher-principal consent
Teacher ratings
- Highly effective
- Effective
- Partially effective
- Ineffective
In the past, the quality and intensity of teacher evaluations varied by district, school and classroom. Concerns also arose about a lack of objectivity by principals who may have held personal grudges against certain employees and favored others based on subjective information.
The law also dramatically changes how a teacher gets tenure by no longer linking tenure status to longevity alone. Under the law, non-probationary status will be earned after three consecutive years of demonstrated effectiveness.
Meanwhile, teachers will lose non-probationary status — more commonly known as tenure and job protections — after two consecutive years of ineffective ratings. That doesn’t necessarily mean a teacher loses his or her job. The law states that a teacher can be assigned to a new post at another school – but only with the consent of the hiring principal and with input from at least two teachers employed at the school in a process known as “mutual consent.” A teacher who doesn’t get a job after two hiring cycles, however, will be placed on unpaid leave without benefits until rehired.
But, in an effort to calm the nerves of teachers who still think of Jaws when they think of the new system, the law states that negative ratings won’t count in the first year.
“If educators don’t have trust in the system it won’t work,” Anthes said. “We want to have that exploratory year.”
The years of work, collaboration and even tears that went into forging 191 represent a historical shift, said Linda Barker, Colorado Education Association (CEA) director of teaching and learning.
“People are seeing school board members sitting with teachers talking about practice,” Barker said. “This is all new for all of us. The doors are open. Everyone is being really collaborative.”
To be considered “effective” under the law, teachers must meet a set of quality standards, including how well a teacher knows the content; establishes classroom environment; facilitates learning; reflects on practice; and demonstrates leadership. This accounts for half the evaluation and is measured through a lengthy rubric, which is still being tweaked. That rubric is filled out by evaluators – normally principals.
The other half of the teacher’s evaluation focuses on student growth. For a teacher who teaches a subject tested by TCAP, those scores must be part of that teacher’s rating, but it’s up to districts to decide how much weight to give TCAP — or other standardized test scores. Barker estimated that TCAP scores will make up 5 to 15 percent of the student outcomes piece, with the rest drawn from other assessments and growth scores.
However, including student growth using TCAP or any standardized test in evaluations is controversial. Many teachers argue that test scores only capture one slice of a student’s academic and social growth. Even though the state makes clear that growth scores should be based on “multiple measures of student growth or student learning over time, not a single assessment,” teachers remain worried.
For teachers who don’t teach subjects tested in TCAP, such as social studies or art teachers, the state has collected — and is still collecting — an assortment of valid measures districts can use. This resource bank comes out of the so-called “content collaboratives,” a group of Colorado educators who are identifying and creating high-quality assessments that are aligned to the new Colorado Academic Standards.
The state is also in the process of working out what criteria to use for evaluating other licensed personnel, such as school audiologists, nurses, occupational therapists, physical therapists, psychologists, social workers, speech-language pathologists and counselors. Rubrics will be tested in 2013-2014.
The sheer number of variables involved in understanding a teachers’ influence on student outcomes has sparked anxiety in many teachers. Barker questioned how the state can accurately measure the impact a music teacher has on a student, for example.
“There is so much complexity around our jobs,” Barker said. “How do you ever have a system that takes all that data, puts it into two buckets and puts you into four ratings. We’re still concerned about what that looks like.”
Lessons learned from pilot programs
Twenty-seven districts are piloting the state’s model evaluation system, which includes rubrics that are being continually updated based on feedback, user guides and training materials. Of the 27, the state picked 15 as the “official pilots” after they submitted applications. These are the districts whose data will be dissected and shared with the state.
School districts can use the state’s pre-packaged evaluation system, or create their own — as long as it meets criteria established by the state. Most districts are expected to use the state’s system with minor modifications. That means there will still be variance by district, but evaluations will be more standard across the state than there has been previously.
“I’m honestly optimistic,” Anthes said. “Next year is a practice year. I am expecting some trepidation in using the system and complete understanding of the system. There is going to be a lot of work that continues.”
The Colorado Legacy Foundation is working with five so-called “integration districts” and one BOCES to implement both the evaluation system and the new Colorado Academic Standards, as a kind of double whammy pilot.
Gretchen Wilson is a veteran fifth grade teacher in Durango, which is part of the San Juan BOCES integration pilot, and spends a day week working on the pilot on behalf of the teachers union.
She described her job in part as “trying to change an entire way of doing business and trying not to have people freak out along the way.”
In Durango schools, in addition to lengthier official observations, principals and assistant principals also do 10-minute walkthroughs in classrooms four or six times a year with iPads in hand, taking notes. The feedback is sent to the teacher, who then responds to it. The sessions are considered more as coaching tools than evaluative measures.
“A lot of this is about getting used to people being in your classroom all the time,” Wilson said, noting that use of the actual rubric can be daunting as well. “Twenty-seven pages on a checklist is very intimidating for very many people.”
The tiny, 4,000-student district hired three new assistant principals this year to help do the work, in addition to three new “integration liaisons,” or classroom teachers who will have a two-year special assignment working and partnering with teachers on a daily basis, Wilson said.
“It’s been challenging, I admit, trying to make it meaningful and effective,” she said.
Wilson said it’s key that she is also on the receiving end of the observations.
“It’s been a bit uncomfortable because it’s new — and change creates a little anxiety,” Wilson said. “I think it’s made me a better teacher. The standards and goals are on the board and I reference them all the time. I never used to do that. Things are very specific. I work with my teammates to make sure we’re teaching math standards. Not just the textbook, but teaching to the standards and what you are doing when kids don’t get it.”
Wilson said teachers in her district have been told they’ll be lucky if they are ranked effective under the system. For teachers who have always gotten excellent evaluations that could be a blow, she said.
“It’s a whole different mindset,” she said. “It’s almost like you’ve failed something… But this is a different yardstick.”
In Denver, as part of LEAP (Leading Effective Academic Practice), the district’s own teacher evaluation pilot begun in spring 2011, 40 professional peer observers make regular rotations through Denver classrooms, using laptops and, of course, a rubric. They take high speed notes while watching a teacher in action, asking questions of students and providing that teacher a detailed consult before doing it all again a few months later.
The observation is always followed by deep conversations and a plan for improvement — ideally, one with adequate supports for the teacher.
While this all sounds good in theory, issues are bubbling up with LEAP in Denver, with some probationary teachers whose contracts were recently not renewed saying scores by LEAP observers in some cases differed from those given by principals, that the discrepancy was never explained to them and that they lost their jobs anyway.
But district staff say the non-renewal decisions were based on a “body of evidence,” not just LEAP scores.
CDE, meanwhile, is developing tools to promote common interpretations of teacher quality and help evaluators provide useful and actionable feedback to educators.
One such tool is an online training system that is being developed in partnership with My Learning Plan meant to ensure that two or more evaluators using the same observation tool give the same rating to an identical observable situation.
The state’s training system will allow evaluators to log onto a website, view a number of short videos of practicing teachers, rate those videos according to the Colorado State Model Evaluation System rubric and receive a score that shows how close they are to rating the videos in accordance with “master scorer” ratings.
Educators who receive scores within an approved range will know they are evaluating professionals within an acceptable, comparable and fair manner. The first set of videos will be available this August.
Principal evals also underway
Meanwhile, under the pilots, principals are evaluated on seven quality standards.
Half of the principal’s evaluation is based on six professional practice quality standards: strategic leadership, instructional leadership, school cultural and equity leadership, human resource leadership, managerial leadership and external development leadership.
The other half of a principal’s evaluation is based on the seventh quality standard — academic growth of students.
Since principals are at-will employees, they don’t lose or gain probationary status under the law. But districts can use the evaluations to promote principals or provide them needed supports.
“This is the first time we’ve had a road map,” Barker said. “Teachers and principals actually know what standards to talk about because they’re the same.”
Rural Moffat County School District in northwestern Colorado was among the 15 pilots. Superintendent Joe Patrone said the new evaluation system is changing the way educators think about their jobs.
“We’re hearing this has been a wonderful opportunity for our principals to have engaging conversations with teachers about their growth and development in areas that have been outlined in the rubric,” Patrone said.
Patrone said the evaluation process has sparked enlightening conversations with principals and teachers about what really matters: student learning.
“It has elevated their appreciation for those opportunities that we believe are available but should be more available,” Patrone said. “We don’t have as much time as we d’ like to have to do what we should be doing — spending more time with teachers about professional growth and development.”
In fact, time remains one of the biggest hurdles to effective teacher evaluations. Many educators fear that there will never enough time to complete comprehensive evaluations and to allow teachers to take advantage of high-quality, targeted professional development so they can shore up areas of weakness.
“The challenge is to do [professional development] well and find delivery systems that allow teachers to stay in classrooms,” Patrone said.
Supporting teachers to improve
To address that issue, Moffat is meeting with other regional districts to discuss the creation of a hybrid online and in-person teacher training module that could be shared.
Through LEAP, Denver Public Schools created an online portal with training videos and other materials to help teachers get the support they need.
Ultimately, the vision for the state’s content collaboratives is that they too become networks for creating and disseminating innovative teaching practices. CDE is also working to build a video library with examples of exemplary teaching practice tagged to each standard in the teacher rubric.
Barker envisions the equivalent of an “individualized learning plan” for each teacher, in which they get exactly the training and support they need while not wasting precious hours getting trained in an area they already have mastered.
Additionally, Barker said that teachers’ colleagues at their own schools will be an essential part of the new culture so all teachers feel supported. Not only can teachers watch videos of teachers demonstrating mastery of certain skills, but they could watch their own colleagues live, in person.
Finally, there are options for teachers who come out with a negative rating that they don’t believe is fair.
Beginning in 2014-2015, all school districts are required to have an appeals process for teachers who lose non-probationary status. The appeals process can be determined through collective bargaining in districts with union contracts.
So, does this mean that all the i’s are dotted and the t’s crossed and all will be well as the new teacher evaluation system becomes reality? Definitely not. But, for now, some teachers are cautiously optimistic.
“This is a roadmap for teacher development,” said Adele Brado, a National Board Certified first grade teacher at Boulder Valley’s Kohl Elementary School in a CEA video on 191. “What we’ve wanted for so long is authentic professional development…The bar is high. The expectations are really high. But that’s what we really need. We need to improve our practice.”