Test questions in Denver, Aurora

Colorado is not facing a standardized test cheating scandal, the state’s chief testing official says, after a recent newspaper investigation highlighted score anomalies in Denver and Aurora public schools.

Jo O’Brien, assistant commissioner of standards and assessment with the Colorado Department of Education, said her office was already aware of irregularities pointed out in a nationwide investigation into possible cheating published in the Atlanta Journal-Constitution.

“The percentage needs to be improved upon, but it’s not a sign there was something larger,” O’Brien said, noting that the AJC figures on Denver and Aurora are “not signs of a widespread scandal.”

“When the (AJC investigation) came out, we were curious to see what that third party would come up with. We were relieved to see what they had discovered was not much different than what we had seen and/or identified.”

O’Brien said the data did not indicate systemic problems in test administration in Denver or Aurora. The districts were included on the newspaper’s list of “districts with suspicious patterns” in test results.

She said the state would likely delve further into the numbers after this year’s testing window closes later this month. She also said the figures do highlight the need to redouble efforts to train educators about proper testing protocols and test security.

Atlanta is home to the nation’s most notorious high-stakes test cheating scandal. In 2011, a federal investigation found that 178 teachers and principals cheated to boost scores on standardized tests.

AJC conducts national investigation into test scores

The Journal-Constitution published a comprehensive report March 25 highlighting districts from coast to coast where cheating may be happening. It found suspicious test scores in roughly 200 school districts that resembled patterns found in Atlanta.

In Denver Public Schools, 8.8 percent of classes in 2011 were flagged as having irregular test scores. In Aurora, 9.4 percent of classes were flagged as having test scores warranting further examination.

Percent of classes flagged in Denver for irregular test scores

  • 2008 – 8.6%
  • 2009 – 9.1%
  • 2010 – 7.8%
  • 2011 – 8.8%

Percent of classes flagged in Aurora for irregular test scores

  • 2008 – 12.5%
  • 2009 – 13.6%
  • 2010 – 7.1%
  • 2011 – 9.4%
  • See the methodology behind the Atlanta Journal-Constitution‘s cheating investigation

Those figures compare to 25.6 percent classes flagged in 2010 in Atlanta. That figure dropped to 4.2 percent in 2011 – after the cheating scandal broke.

Aurora Public Schools Superintendent John Barry and Chief Accountability Officer Lisa Escarcega said it’s difficult to answer questions about the series without knowing which schools the newspaper examined.

The newspaper has yet to identify specific schools examined in either Aurora or Denver, though it plans to release additional data in coming weeks through Investigative Reporters and Editors. The nonprofit journalists’ group showcases investigative stories and how they were done.

“There’s no evidence we’ve found” of cheating, except in isolated incidents, such as the teacher let go last year, Barry said, adding, “We look real hard at those CSAP scores.”

Barry said district leaders respond quickly to allegations of potential problems.

“We had one teacher last year we had to let go because she influenced the scores and the kids,” he said. “Those things get dealt with right away.”

In that case, students told their parents that the teacher provided help and parents contacted the district. Students were interviewed and it became clear “something was done wrong,” Barry said.

Escarcega said test security is taken very seriously and test books are monitored by an assistant assessment coordinator.

“We keep a very close eye on every single test book,” she said. “Books are not left with teachers, they are not left unaccounted for at any time – there is no opportunity for anyone to change answers.”

Escarcega said the district has done “erasure analysis” on an individual basis when alerted to a potential issue. But she said the district is prohibited by the state from poring over the test books without their permission.

In other cases, district officials have uncovered “honest mistakes” such as providing students note paper during the test, which is a test violation. The district then invalidated those exam results.

Colorado counts about 21 testing mishaps per year

O’Brien said there are about 21 so-called “misadministrations” each year in Colorado, which usually involve a classroom of students.

Misadministration can point to outright cheating – such as erasing wrong answers and filling in the right ones – or innocent violations of testing protocols, such as letting kids leave the room to grab lunch when a  bell rings, then returning to complete a test.

Whether it’s caught or not, cheating cheats students of educational supports or tutoring they need, the AJR reported.

DPS spokesman Mike Vaughn did not indicate whether DPS officials will be further delving into the test score anomalies in Denver schools. But he made it clear the district won’t tolerate shoddy testing practices.

“The integrity of assessment data is of the utmost importance at Denver Public Schools,” Vaughn said in a statement. “Each year, we work closely with our schools and the Colorado Department of Education to ensure proper testing procedures and practices are taking place at all of our schools.”

O’Brien said it’s critical that Colorado – and other states – ensure the validity of standardized testing results since those scores in 2014 will account for half of Colorado teachers’ professional evaluations, thus giving them the moniker “high-stakes tests.”

Additionally, more standardized testing will be happening online in 2014, which opens the door to new security challenges, she said.

The state conducts regression analysis every year in an attempt to spot irregularities. And it investigates testing practices when complaints come in. But O’Brien said more needs to be done.

State may increase security as tests go online, link to teacher evaluations

So far, the state has not invested in full-scale erasure analysis, which can pinpoint when answers have been erased and corrected. But she said costs for that sort of review will be provided to state legislators as they fine-tune plans for the standardized tests that will replace the CSAPs and reflect revised content standards.

“We’re looking at the usefulness of  having erasure analyses done annually,” she said. “We’ve only done it when our own research shows there’s something fishy. It’s usually a grade and/or a school. I don’t believe it’s ever been districtwide.”

The most high-profile cheating investigation in Colorado happened in Pueblo three years ago, and it resulted in the state paying for erasure analysis.

State officials hired an outside firm to look into allegations of cheating at Cesar Chavez K-8 Academy in Pueblo, after the former Pueblo schools superintendent publicly urged then-state education Commissioner Dwight Jones to investigate.

A $25,000 audit by Caveon Security of Utah found “extremely high” rates of special accommodations – more than half the students one year received extra time on the exams – but no evidence of answer sheet tampering or test coaching.

“We’re talking about what the online security needs to be,” O’Brien said. “There are necessary firewalls to prevent a large-scale test breach. The threat is very real.”