Measuring success

How to measure school quality beyond test scores? State officials count the ways

PHOTO: Shaina Cavazos
Students at Eagle Elementary School in Zionsville line up to for an assembly.

As Indiana — and the rest of the country — moves away from measuring schools based solely on student test scores, there are a lot of options on the table.

Thirty-four options, to be exact.

That’s how many ideas Indiana officials came up with when they surveyed other states to find out how they’re planning to go beyond test scores to assess schools, which the new federal education law requires.

The list ranges from including attendance to class size to student surveys. Some of the ideas could go into practice fairly quickly, while others would require new data to be collected. Some ideas are already in play in other states, but others might not meet federal rules.

Ultimately, the state will have to choose one to use in its new A-F letter grade system, but for now, all are under consideration, state education officials say.

“Nothing is really off the table,” said state Superintendent Glenda Ritz on Tuesday. “We just have to figure out if [the measure] can comply with regulations … and what it is we want to incentivize with our schools to actually improve achievement.”

The list is extensive, and includes factors such as student attendance, chronic absenteeism, suspension and expulsion rates, teacher retention, access to technology, school climate, dropout rate, class size, and school safety, to name a few. The Indiana Department of Education currently collects data on 22 of the measures.

Indiana already has the high school part of the equation down — measuring how many kids take and pass Advanced Placement and International Baccalaureate exams and earn dual credit for college or industry credentials. A box still left to check is how to include a similarly non-test score based factor for elementary and middle schools.

State education officials already know that some “student success” measures on the list presented Tuesday to the Indiana State Board of Education will almost immediately be ruled out because they likely won’t pass muster with the federal government — either because they won’t differ significantly among schools or because there isn’t yet a statistically sound way to measure them.

Surveys on school climate are one option already being used in an effort from the city, Indianapolis Public Schools, and The Mind Trust, but some board members expressed concerns about relying on self-reported information. Other tools would either need to be created from scratch or found in existing research.

At a seminar earlier this month on the new federal education law, groups of educators and administrators gathered to rank what non-test score measures they’d like to see most. Topping the list was a desire for each district or school to establish its own measure that best reflects its needs — but that almost certainly isn’t possible because federal requirements say the chosen measure must apply to all schools in the state.

“I’m all for local decisions,” said board member and Warren Township Assistant Superintendent Lee Ann Kwiatkowski. “But then I’m wondering how we would meet that [federal] requirement.”

Other measures ranked highly were curriculum offerings, climate surveys, and whether a district meets its school improvement plan goals. Board member Steve Yager, a former superintendent from Fort Wayne, laid out a number of things the board should keep in mind as conversations continue.

“The practitioner side of me says be cautious,” Yager said. “Whatever we decide … can it be easily measured? Can it be easily reported and understood? Does research support the effort? Can we afford the effort? And can trends be tied to student success? That’s probably the most important thing.”

Ritz said next steps for the process include further vetting of possible success measures and gathering research on which are already highly correlated with student achievement. Whichever factor is chosen will ultimately count less in letter grade calculations than test-score based measures, however.

Ritz also announced that she has consulted with Senate President David Long, House Speaker Brian Bosma, and Gov. Mike Pence about reconvening the state’s accountability panel, which first met years ago to create the current A-F model. She wants the panel to dive deeper into what changes will need to be made under the new federal education law and report back to the board with a set of recommendations.

“We’ll come back with some research and information,” Ritz said. “We just have to hone in on where we think we want to go. We will be wrestling with this for a little while.”

See the full list of student success measures below.

multiplemeasures

The Colorado Way

Feds approve Colorado’s education plan after multiple revisions, but critics see more work to do

PHOTO: Denver Post file
Students prepare for statewide testing in Michelle Mugatha’s eighth-grade language arts class at Columbia Middle School in Aurora.

Colorado finally received approval for its federally mandated education plan Monday, one year and two revisions after the state first submitted it.

Colorado’s plan was held up longer than any other state’s by a series of disagreements over the best way to measure student achievement, including how to count students who opt out of state assessments. In most of those disagreements, the federal view prevailed, leaving Colorado with two divergent accountability systems, one state and one federal.

“We wanted to stick to our Colorado principles,” said Pat Chapman, executive director of the federal programs unit in the Colorado Department of Education.

Colorado wanted to use its state accountability system developed in 2009 to meet federal requirements, but ultimately the two were not entirely compatible. The state accountability system is more likely to identify schools that are not serving a large share of their students, while the federal system flags schools that aren’t serving certain subgroups, like students who qualify for free- and reduced-price lunch, a proxy for poverty, or English language learners, even if their overall numbers look good.

“What we use the federal system for is to identify schools that need additional support and to get additional resources to those schools,” Chapman said.

Educational and civil rights advocates who have been involved in the development of the plan say that it’s improved in some ways, but they’re concerned that the existence of two accountability systems – or three in the case of districts like Denver that have their own school ratings – will lead to more confusion unless there’s a clear way of sharing information with parents.

Schools identified as “turnaround” or “priority improvement” status under state law won’t necessarily be flagged for improvement under ESSA, and vice versa.

“Our concern with having two different systems is that there may be confusion among parents about which system actually tells them how their school and district is performing,” said Leslie Colwell of the Colorado Children’s Campaign, a member of the Equity in Colorado Coalition.

And bilingual educators say the approved plan fails to address two key problems, the lack of assessments in students’ native language and inconsistent criteria for when students learning English keep receiving services or transition out.

The Every Student Succeeds Act, or ESSA, which replaced No Child Left Behind in 2015, requires each state to submit a plan that lays out how it will measure student achievement and what it will do to improve performance among groups of students who aren’t meeting academic goals.

Without an approved plan, Colorado schools risked losing access to nearly $200 million in federal funds for children from low-income families, and other aid. ESSA also provides $10 million a year to Colorado schools that have been flagged as needing improvement.

Colorado has been a center of the “opt-out” movement of parents refusing to allow their children to be tested. In response, the State Board of Education forbid lowering a school’s quality rating if fewer than 95 percent of its students were tested. The U.S. Department of Education, meanwhile, insisted that Colorado treat students who don’t take the test as if they were not proficient, the lowest ranking. This became a key sticking point.

Under the approved plan, Colorado schools with high opt-out rates will need to come up with plans to test more of their students. This comes even as state lawmakers this year banned the use of rewards like pizza parties for students who take the tests. Instead, schools will have to make the case to parents and students that the tests are meaningful and important.

Two other key differences:

  • The federal government will rate schools based on four-year graduation rates, while Colorado lets schools use the best result from its four-, five-, six- or seven-year graduation rates.
  • The federal government will rate alternative high schools based on their graduation rate, while Colorado looks at completion rate, a broader measure that includes students who get a GED.

ESSA also lets states choose non-academic measures of student success against which school quality can be judged. In Colorado, one of those will be chronic absenteeism, and some schools are trying innovative programs to work with parents to help them get their kids to school.

Colorado’s student data privacy regulations means that the state won’t be reporting detailed information about small subgroups to federal regulators, an issue that advocacy groups say limits the public’s ability to understand how schools are doing.

State education officials have embraced the “flexibility” offered by ESSA in comparison to No Child Left Behind, but during a panel discussion earlier this year, Alexandra Alonso of the Colorado Latino Leadership, Advocacy, and Research Organization, stressed that the new law needs to be understood as a piece of civil rights legislation.

“It’s not intended to create more autonomy for states,” Alonso said. “It’s intended to have more equitable outcomes for our students.”

In that regard, Colorado’s plan still needs work, said Jorge García, executive director of the Colorado Association for Bilingual Education. There still isn’t a consistent enough standard for promoting students out of English acquisition programs, García said. Students languishing in those classes too long suffer, he said.

“It denies them access to electives,” he said. “It denies them access to the entire schedule of classes. It denies them access to the full content of the classes. They don’t take the classes that prepare them for college and a career. This particular high stakes decision is hurting a lot of our students.”

Colorado’s ESSA plan also doesn’t indicate any intention to develop Spanish-language math assessments.

Colorado received wide praise for its work to reach out to community groups, advocates, and school districts as it developed its plan, and Colwell said community groups will continue to work with the Colorado Department of Education to shape implementation of the plan.

“I am incredibly thankful for the amount of time and effort that so many people put in to develop our state plan,” Colorado Education Commissioner Katy Anthes said in a press release. “Colorado has had ambitious education strategies in place. This plan maintains our strong education laws while working together with the federal law to support all students.”

In a press release announcing approval for Colorado’s plan, U.S. Department of Education officials highlighted several features that other states don’t have. Those include a “one-stop process” for schools to apply for services and grants tailored for their specific challenges, a coordinated grant management system, and training for teachers in all subject areas.

Every Student Succeeds Act

The Indiana State Board of Education is hitting the brakes on a plan to overhaul A-F school grades

PHOTO: Shaina Cavazos
Students in IPS School 91's multi-age first-, second- and third-grade classroom work on math activities.

The Indiana State Board of Education is pressing pause on a proposed overhaul of how schools are graded that drew criticism from educators and some education advocates.

Board members said they wanted more time to consider how the A-F proposal — initially created to address new federal accountability law — would work alongside new graduation requirements and to incorporate feedback from educators about how the school grades are calculated, especially for high schools.

That means for this year, the 2018-19 school year, and possibly longer, Indiana schools will be measured according to two different yardsticks — a state model introduced in 2016 and a federal system that complies with the new Every Student Succeeds Act.

Read: Indiana has a curious plan to sidestep federal rules — give schools two A-F grades next year

The board met Wednesday to continue hammering out the new process for calculating state grades, a draft of which was approved in January. But just as the meeting started, board member Byron Ernest suggested pausing process, aiming instead for a new A-F grading model for the 2019-20 school year at the earliest.

“I would like for us to take a step back and do some research,” Ernest said. Four of the state board members were absent, including state Superintendent Jennifer McCormick. The seven present board members quickly reached a consensus that they should postpone a decision on the A-F rules, though no official vote happened.

As it stands now, the state and federal grading methods for calculating school ratings have important differences. The federal grade calculation, for example, would include school attendance rates and language proficiency of English-learners, whereas the state calculation would mainly rely on state test scores and test score growth. Because Indiana’s calculation also excludes certain students that the federal plan includes, such as those receiving credit recovery services, the final ratings could differ significantly for the same school. Although state and federal accountability metrics have differed in the past, the differences going forward would be more significant.

The differences ultimately add a lot of confusion to a state accountability system designed to be simpler to understand for teachers, parents, and the community.

Cari Whicker, a board member and principal, said the changes Indiana has made to testing and accountability have been exhausting and frustrating for schools.

“Either A-F accountability or testing has changed every year since 2011,” Whicker said. “That’s a lot for schools. What you consider tweaking is truly moving the target for people in the field.”

The pause is also an about-face from a meeting just a couple months ago, where board members shot down a similar proposal from Gordon Hendry to slow down. On Wednesday, Hendry said he was glad to hear Ernest’s proposal.

“That’s what I advocated for in January — wouldn’t it behoove us to take our time,” Hendry said.

In January, educators and education advocates came forward with concerns over the process for creating the new school grades, which they said was far too fast and not transparent. They also took issue with the substance of the state plan, which would have made test scores more important and limited how much test score improvement could have factored into high school grades.

It’s not yet clear exactly what changes the board wants to make in the state A-F grading model that haven’t already been discussed or considered. The Indiana Department of Education released its federal ESSA plan over the summer, and the board has had multiple opportunities to examine that plan and give feedback.

Further discussion is expected at the state board’s April meeting.