ProComp findings positive, tough to prove

Student achievement is up and teacher turnover is down since Denver Public Schools implemented its merit pay plan for teachers in 2006, but it’s tough to prove a direct link between the two.

Researchers from the University of Colorado Denver and the University of Washington on Tuesday released their report on the plan known as ProComp, or the Professional Compensation system for teachers.

The external evaluation was required as part of the $25 million tax increase approved by Denver voters in November 2005 to fund the innovative pay plan, which rewards teachers based on ten different components and now covers nearly 80 percent of DPS teachers.

“Six years into ProComp, as we release the study … we show the Denver Public Schools in a dramatically different place than when the voters of Denver chose to make that investment in 2005,” DPS Superintendent Tom Boasberg said at a morning news conference at Skinner Middle School.

He ticked off gains in student achievement, in teacher retention and in student enrollment, which surpassed the 80,000-mark this year for the first time since 1974.

“Many factors go into that … but clearly ProComp is a very important part,” Boasberg said, noting, “One example, for every open teaching position, we see five times more applicants than we saw when Denver voters made the investment in ProComp six years ago.”

Henry Roman, president of the Denver Classroom Teachers Association, was an early advocate of ProComp who fought changes to the plan in 2008 that shifted some permanent increases to annual bonuses.

“I know we don’t have definitive answers for many things,” he said at the press conference, “but it’s certainly a trend, it’s trending up, and it’s heading in the right direction and that’s highly encouraging for all of us.”

Difficult to isolate effects of ProComp, among other reforms

ProComp was implemented in DPS along with a number of other changes, including the district strategic reform plan called the Denver Plan. Between 2006 and today, the district also joined the statewide Public Employees Retirement Association, which may have made DPS more attractive to other Colorado teachers.

Robert Reichardt, the research team director with CU-Denver, said it’s difficult to isolate the effects of ProComp from other DPS reform measures.

“I would never recommend that districts stop all reforms so you can study one reform,” he said. “I think DPS has followed the model of, ‘Do whatever it takes’ and implemented a lot of things that may be making a difference.”

Still, “ProComp appears to make a difference,” Reichardt said. “Clearly, ProComp helped build system capacity around data systems and professional development and HR that we think led to other reforms within DPS. Clearly, during ProComp implementation, recruitment and retention improved and there’s some evidence ProComp was a factor in improving retention.”

For example, between 2004-05 and 2008-09, teacher attrition in DPS declined from 17 percent to 13 percent. Attrition also dipped elsewhere in Colorado but not quite as much – it declined from 14 percent to 11 percent for the remaining Denver metro region and from 14 percent to 12 percent for the rest of the state.

“We estimate, at most, DPS was able to retain 160 teachers more per year due to ProComp,” Reichardt said.

In achievement, the numbers of students scoring proficient or advanced on state exams has grown in both reading and math over the past six years. In 2005, 40 percent of DPS students were reading at grade level compared to 49 percent in 2011. Math proficiency during that time rose from 29 percent to 41 percent.

“How much of that was due to ProComp, our design doesn’t allow us to say,” Reichardt said. “We think there is some evidence that some of it is due to ProComp.”

Two ProComp components – rewards for achieving student growth objectives set at the beginning of the school year and for exceeding expectations on CSAP exams in the spring – strongly correlated with teacher effectiveness, he said. Researchers defined teacher effectiveness by using “value-added models” to estimate teacher contributions to student achievement on state tests.

Other components, including the attainment of advanced degrees, did not appear to have any correlation with teacher effectiveness, Reichardt said.

ProComp agreement up for negotiation, expires August 2013

Results of the external evaluation, which mesh with earlier results of internal studies by CU-Boulder researchers, will be used as the teachers’ union and the district begin to negotiate the ProComp agreement, which expires in August 2013. The external study costs $360,000.

Skinner staff on ProComp“The thing I like about ProComp is that it rewards me for my hard work when we see results with the kiddos.”
Joe Waldon, social worker

“In a time when we’re asked more as teachers, we are rewarding teachers equitably for the work they put in.”
Mathew Dennis, teacher

Roman, the DCTA president, said negotiators may look at reducing the number of ProComp components. He agreed with the finding that many teachers and principals still do not understand the complex system, which he described as having “too many moving parts.”

He also expressed concerns about the finding that there’s “limited evidence” of a link between advanced degrees and improved student achievement.

“Just like when you’re trying to jump start the economy, you don’t see results in three months or six months,” Roman said. “Sometimes you see the results a year later or two years later.”

At Tuesday’s press conference, Boasberg presented a symbolic paper check for $233,711.58 to a Skinner teacher and social worker. The amount represents ProComp incentive payments earned by 21 Skinner teachers in 2010-11 – an average of $11,000 per teacher.

Joe Waldon, the school social worker, said he joined ProComp when it launched six years ago.

“ProComp wasn’t going to make me work harder – my work ethic demands that – but the thing I like about ProComp is that it rewards me for my hard work when we see results with the kiddos,” Waldon said. “So it’s really nice to know when I’m here late in the evening or on a Saturday or on a Sunday … that there will be some financial compensation for that.”

At Skinner, in northwest Denver, the numbers of students reading proficiently has increased from 33 percent in 2005 to 50 percent in 2011.

Matthew Dennis, a seventh-grade language arts teacher at the school, said Skinner teachers “push one another” and that ProComp “motivates teachers to work collaboratively to raise the bar for all of our students.”

“In a time when we’re asked more as teachers, we are rewarding teachers equitably for the work they put in,” Dennis said, adding, “It doesn’t necessarily translate to me working harder. The people here at Skinner are hard workers already. But what it does is it shines a light on the good work that we’re doing and it incentivizes others to follow suit with what we’re doing here.”