Summer of testing debate launches with release of study

Updated May 14, 4 p.m. – School leaders, teachers and parents are worried about the time demands that new state tests will impose and the impact on classroom learning, school schedules and staff time, according to research done this spring for the Colorado Department of Education.

The study, done by the California-based education-consulting group WestEd, included surveys of district assessment coordinators, focus groups in selected districts and additional focus groups with parents and teachers.

“The number one thing we heard over and over from focus groups was the impact on instruction,” said Sheila Arredondo, WestEd lead researcher, told the State Board of Education on Wednesday.

That work was done from February through April, before new online science and social tests were given to more than 250,000 students in four grades and before “field tests” of the coming CMAS assessments were administered to more than 12,000 students in 85 districts. (Details of how that went were presented to the State Board of Education on Wednesday; read the story here.)

WestEd researchers are beginning new surveys and focus groups this month to gauge attitudes now that those tests actually have been given.

Arredondo said she’s already talked to some districts about their experience with spring testing. “The kids seemed to handle it beautifully … it went pretty well.” But she said administrators remain worried about being able to handle testing next year when CMAS exams have to be given in every grade starting with the third.

“I think the study was more validating than surprising,” said Jill Hawley, associate education commissioner, meaning that it raised familiar issues the department has heard before. “I also think it will be very interesting to see how people’s answers change” after having administered the spring tests. Hawley spoke with Chalkbeat Colorado on Tuesday.

The first-phase results were presented to the State Board during its Wednesday meeting in Grand Junction. The presentation marks the start of what’s expected to be several months of intense discussion about statewide testing.

Not only is the second WestEd report coming, but a 15-member appointed Standards and Assessments Task Force is supposed to start work by July 15. Its assignment is to study the impact of testing on teaching time, the interaction of testing with the state accountability and educator evaluation systems and the feasibility of waiving some assessment requirements, among several other issues. (Get more information on the task force in this legislative staff memo.)

The legislation that created the task force also allocated $142,750 to CDE to coordinate the group’s work, commission a testing cost study, pay for a separate review analyzing how different testing schemes would affect the accountability system and obtain legal advice on the implications of letting parents and districts opt out of some testing requirements.

The task force and CDE are to report findings and recommendations to the legislature by next Jan. 31, giving the 2015 session plenty of time of consider the issue. District and parent anxiety about testing was an undercurrent during the 2014 session. But lawmakers rejected bills that would have delayed rollout of new tests next year and cut back on the new social studies tests. The task force bill was the only testing legislation passed. (That measure originally would have allowed districts to opt out of some testing requirements.)

CDE commissioned the WestEd study before the task force bill was passed.

Here are some other key conclusions from the WestEd survey and focus groups:

  • People are worried about the impact of testing on instructional time
  • Districts have only moderate levels of readiness for the new tests
  • Quantity, frequency and duration of assessments are a concern
  • Administrators, teachers and parents want timely, relevant and useful results
  • There’s concern about the burden and usefulness of high school tests
  • Respondents believe there needs to be greater recognition of local assessment systems and practices

Hawley said the survey found respondents want tests that take less time and return results more promptly, but that they also value the academic growth data produced by statewide tests. “That says to me they want shorter, timelier assessments that still measure student growth.”

The surveys also found that respondents generally value local interim tests more than state tests, dislike the high stakes attached to state tests and delayed results and perceive a lack of value for instruction in statewide exams.

School readiness tests and the social studies and science exams were seen as the most burdensome by large majorities of respondents.

The report notes that the results are weighted toward the concerns of smaller districts. Some 87 of 178 districts responded to the survey — 73 percent rural, 16 percent suburban and 8 percent urban. Just over half of respondents were from districts with fewer than 1,000 students.

Focus groups were conducted in the Archuleta, Buena Vista, Cherry Creek, Delta, La Veta, Platte Valley, Strasburg and Woodland Park districts, along with separate focus groups for teachers and parents.

But Hawley said because rural districts make up 80 percent of all Colorado districts, “We feel it’s representative.” She noted that the second part of WestEd’s work will seek to gather more opinions from metro-area and large districts. Those results will be available in July, just as the task force is starting up.

“Districts generally view current English language arts and mathematics assessments (i.e., TCAP) as low value with suburban districts valuing these assessments more than urban and rural districts. Two-thirds of rural and suburban districts view all TCAP assessments as high in burden compared to 29 percent of urban districts. … And although the burden of school readiness assessments is high, urban districts (100 percent) consider it highly valuable in informing student progress compared to 38 percent of rural and 13 percent of suburban districts,” according to the report’s executive summary.

The report also contained some interesting responses about districts’ readiness for new online tests.

“Regarding overall readiness to administer state assessments, 27 percent of districts appear fully prepared, 53 percent reported moderate readiness levels, and 20 percent are not yet ready. The two primary factors influencing district readiness are management (62 percent) and devices (60 percent). … Suburban districts appear to be the least prepared with 79 percent citing management and IT staff and personnel issues,” the report said.

Jill Hawley, Colorado Department of Education

Hawley agreed the main concerns about readiness involve staff time and management of testing and showed fewer worries about technical issues like availability of hardware and connectivity.

(The switch to online testing means that districts will have to spread assessments over a longer period of time so that classes can be rotated through computer labs to take the exams.)

The study found strong sentiment for slowing the move to new tests.

“The solution noted most frequently by focus group participants was that of holding schools and districts harmless until all components of the system are validated and functioning effectively. … Finally, because participants feel overwhelmed and under-resourced, they desire a more gradual pace and seek to slow the roll-out of the new assessment system.”

Arredondo reiterated that point Wednesday, saying, “The number one idea mentioned from focus groups was this idea of holding harmless.”

The study suggested four possible paths for state policymakers to consider as they plan for new tests:

  • Implement the transition plan as scheduled
  • Stay the course with added supports and policy adjustments
  • Purposefully delay parts of the system
  • Strategically eliminate specific assessments

The WestEd report also suggested some short-term options for consideration by the state, including:

  • A phase-in of online assessments and offering paper options
  • Providing emergency funds for districs to purchase devices
  • Reducing the number and length of test sessions
  • Use a sampling approach for social studies exams
  • Make the school readiness assessment optional
  • Make 9th and 10th grade language arts and math tests optional
  • Revert to federal minimum testing requirements and made all other tests optional
  • Hawley noted that while the surveys found wide agreement on testing issues, “There are not shared beliefs on the solutions.”
  • All those suggestions are likely to be key points of discussion as the task force begins its work.