Measuring success

How to measure school quality beyond test scores? State officials count the ways

PHOTO: Shaina Cavazos
Students at Eagle Elementary School in Zionsville line up to for an assembly.

As Indiana — and the rest of the country — moves away from measuring schools based solely on student test scores, there are a lot of options on the table.

Thirty-four options, to be exact.

That’s how many ideas Indiana officials came up with when they surveyed other states to find out how they’re planning to go beyond test scores to assess schools, which the new federal education law requires.

The list ranges from including attendance to class size to student surveys. Some of the ideas could go into practice fairly quickly, while others would require new data to be collected. Some ideas are already in play in other states, but others might not meet federal rules.

Ultimately, the state will have to choose one to use in its new A-F letter grade system, but for now, all are under consideration, state education officials say.

“Nothing is really off the table,” said state Superintendent Glenda Ritz on Tuesday. “We just have to figure out if [the measure] can comply with regulations … and what it is we want to incentivize with our schools to actually improve achievement.”

The list is extensive, and includes factors such as student attendance, chronic absenteeism, suspension and expulsion rates, teacher retention, access to technology, school climate, dropout rate, class size, and school safety, to name a few. The Indiana Department of Education currently collects data on 22 of the measures.

Indiana already has the high school part of the equation down — measuring how many kids take and pass Advanced Placement and International Baccalaureate exams and earn dual credit for college or industry credentials. A box still left to check is how to include a similarly non-test score based factor for elementary and middle schools.

State education officials already know that some “student success” measures on the list presented Tuesday to the Indiana State Board of Education will almost immediately be ruled out because they likely won’t pass muster with the federal government — either because they won’t differ significantly among schools or because there isn’t yet a statistically sound way to measure them.

Surveys on school climate are one option already being used in an effort from the city, Indianapolis Public Schools, and The Mind Trust, but some board members expressed concerns about relying on self-reported information. Other tools would either need to be created from scratch or found in existing research.

At a seminar earlier this month on the new federal education law, groups of educators and administrators gathered to rank what non-test score measures they’d like to see most. Topping the list was a desire for each district or school to establish its own measure that best reflects its needs — but that almost certainly isn’t possible because federal requirements say the chosen measure must apply to all schools in the state.

“I’m all for local decisions,” said board member and Warren Township Assistant Superintendent Lee Ann Kwiatkowski. “But then I’m wondering how we would meet that [federal] requirement.”

Other measures ranked highly were curriculum offerings, climate surveys, and whether a district meets its school improvement plan goals. Board member Steve Yager, a former superintendent from Fort Wayne, laid out a number of things the board should keep in mind as conversations continue.

“The practitioner side of me says be cautious,” Yager said. “Whatever we decide … can it be easily measured? Can it be easily reported and understood? Does research support the effort? Can we afford the effort? And can trends be tied to student success? That’s probably the most important thing.”

Ritz said next steps for the process include further vetting of possible success measures and gathering research on which are already highly correlated with student achievement. Whichever factor is chosen will ultimately count less in letter grade calculations than test-score based measures, however.

Ritz also announced that she has consulted with Senate President David Long, House Speaker Brian Bosma, and Gov. Mike Pence about reconvening the state’s accountability panel, which first met years ago to create the current A-F model. She wants the panel to dive deeper into what changes will need to be made under the new federal education law and report back to the board with a set of recommendations.

“We’ll come back with some research and information,” Ritz said. “We just have to hone in on where we think we want to go. We will be wrestling with this for a little while.”

See the full list of student success measures below.

multiplemeasures

hurdle cleared

Indiana’s federally required education plan wins approval

PHOTO: Courtesy of the Indiana Department of Education
State Superintendent Jennifer McCormick greets elementary school students in Decatur Township.

U.S. Education Secretary Betsy DeVos has signed off on Indiana’s federally required education plan, ushering in another era of changes — although not exactly major ones — to the state’s public school system.

The U.S Department of Education announced the plan’s approval on Friday. Like other states, Indiana went through an extensive process to craft a blueprint to comply with the Every Student Succeeds Act, or ESSA, which was signed into law in 2015.

“Today is a great day for Indiana,” state Superintendent Jennifer McCormick said in a statement. “Our ESSA plan reflects the input and perspective of many stakeholders in communities across our state. From the beginning, we set out to build a plan that responded to the needs of Hoosier students. From our clear accountability system to our innovative, locally-driven approach to school improvement, our ESSA plan was designed to support student success.”

The federal government highlighted two aspects of Indiana’s plan. One is a pledge to close achievement gaps separating certain groups of students, such as racial and ethnic groups, from their peers by 50 percent by 2023.

Another is a staple of other states’ plans, as well: adding new ways for measuring how ready students are for attending college or starting their careers. Indiana education officials and lawmakers have made this a priority over the past several years, culminating in a new set of graduation requirements the Indiana State Board of Education approved late last year.

Under Indiana’s plan, high schoolers’ readiness will be measured not just by tests but also by performance in advanced courses and earning dual credits or industry certifications. Elementary school students will be measured in part by student attendance and growth in student attendance over time. Test scores and test score improvement still play a major role in how all schools are rated using state A-F letter grades.

In all, 35 states’ ESSA plans have won federal approval.

Advocates hope the law will bring more attention to the country’s neediest children and those most likely to be overlooked — including English-learners and students with disabilities.

Indiana officials struggled to bring some state measures in line with federal laws, such as graduation requirements and diplomas.

Under the state’s ESSA plan, A-F grades would include these measures (see weights here):

  • Academic achievement in the form of state test scores.
  • Test score improvement.
  • Graduation rate and a measure of “college and career readiness” for high schools.
  • Academic progress of English-language learners, measured by the WIDA test.
  • At least one aspect of school quality. For now, that will be chronic absenteeism, but the state hopes to pursue student and teacher surveys.

The last two are new to Indiana, but represent ESSA’s goal of being more inclusive and, in the case of chronic absenteeism, attempting to value other measures that aren’t test scores.

Because the Indiana State Board of Education passed its own draft A-F rules earlier this month — rules that deviate from the state ESSA plan — it’s possible Hoosier schools could get two sets of letter grades going forward, muddying the initial intent of the simple A-F grade concept parents and community members are familiar with.

The state board’s A-F changes include other measures, such as a “well-rounded” measure for elementary schools that is calculated based on science and social studies tests and an “on-track” measure for high schools that is calculated based on credits and freshman-year grades. Neither component is part of  the state’s federal plan. The state board plan also gets rid of the test score improvement measure for high-schoolers.

While that A-F proposal is preliminary, if approved it would go into effect for schools in 2018-19.

The state can still make changes to its ESSA plan, and the state board’s A-F draft is also expected to see revisions after public comment. But the fact that they conflict now could create difficulties moving forward, and it has led to tension during state board meetings. Already, the state expected schools would see two years of A-F grades in 2018. If both plans move forward as is, that could continue beyond next year.

Read: Will Indiana go through with a ‘confusing’ plan that could mean every school winds up with two A-F grades?

Find more of our coverage of the Every Student Succeeds Act here.

double take

Will Indiana go through with a ‘confusing’ plan that could mean every school winds up with two A-F grades?

Students work on assignments at Indianapolis Public Schools Center For Inquiry at School 27.

Imagine a scenario where Indiana schools get not just one A-F grade each year, but two.

One grade would determine whether a school can be taken over by the state. The other would comply with federal law asking states to track student test progress and how federal aid is spent. Both would count, but each would reflect different measures of achievement and bring different consequences.

This could be Indiana’s future if a state board-approved plan moves ahead at the same time the state is working on a conflicting plan to comply with a new federal law.

If it sounds complicated, that’s because it probably would be, said state Superintendent Jennifer McCormick. Originally, A-F grades were intended to be an easy way for parents and community members to understand how their school is doing.

“It’s extremely confusing to have multiple accountability systems with multiple consequences,” McCormick told board members last week. “All along our message has been to get as much alignment as we can.”

Indiana would not be the first state to consider dual accountability systems — Colorado operated separate systems for years under No Child Left Behind and is now doing so again. Virginia, New Hampshire, and South Carolina have also had two models in years past. But this move would be a big departure from Indiana’s efforts over the past several years to simplify accountability, and education officials warn it could create more problems than it would solve.

Dale Chu, an education consultant who previously worked in Indiana under state Superintendent Tony Bennett, said it’s actually not common for states to have multiple systems, and doing so for political reasons, rather than what helps students and families, is concerning.

“We all know how confusing accountability systems can be when you just have one,” Chu said. “To create a bifurcated system, I don’t see how you gain additional clarity … I would certainly hope that if that’s the direction the state is going to move in, they are very thoughtful and intentional about it.”

The changes come as Indiana works to create a plan to comply with a new federal education law, known as the Every Student Succeeds Act. McCormick’s education department has been working to align the federal system with Indiana’s grading system, and is struggling to bring some state measures in line with federal laws, most notably in the area of graduation requirements and diplomas.

At the same time the Indiana State Board of Education is negotiating this alignment, it is also revamping the A-F grade system.

A new grading proposal approved by the state board last week would put more emphasis on student test scores than the A-F system that now unifies state and federal requirements. Those new rules would include extra categories for grading schools, such as a “well-rounded” measure for elementary schools that is calculated based on science and social studies tests and an “on-track” measure for high schools that is calculated based on credits and freshman-year grades. Neither component is part of  the state’s federal plan.

While that proposal is preliminary, if approved it would go into effect for schools in 2018-19.

Officials were already expecting to issue two sets of A-F grades to schools in 2018 — one state grade, and one federal — as the state continued to work all of Indiana’s unresolved education issues into the new federal plan. Figuring out how to ensure state graduation rates don’t plummet because of other federal rule changes dictating  which diplomas count and incorporating the new high school graduation requirements, for example, will take time — and legislation — to fix.

Read: Indiana has a curious plan to sidestep federal rules — give schools two A-F grades next year.

But ultimately, officials said, if some of the state board-approved changes make it into final policy, and Indiana’s federal plan doesn’t change to accommodate it, the state and federal accountability systems could remain at odds with each other — meaning schools would continue to get two grades after 2018.

The original intent was to have all Indiana’s state grading system line up with federal requirements before the plan was sent to federal officials in September. Then, once the federal government gave feedback, the state A-F revamp could continue.

But just this past fall, after the federal plan had been submitted, some members of the state board began adding in additional measures, some of which reflect their personal interests in how schools should be rated.

Those measures were added after board members had multiple chances to discuss the federal plan with the education department, conversations that were held in an attempt to ward off such changes this late in the game. Yet even last week at the state board’s monthly meeting, where the new grading changes were approved, some board members didn’t seem to realize until after the vote that the A-F systems would not match up.

David Freitas, a state board member, said he didn’t see the conflicting A-F grade rules as a problem. The board can make Indiana’s state A-F system whatever it wants, he said, and there will be plenty of time to iron out specifics as the rulemaking process unfolds over the next several months.

“We’re not banned from having two different systems,” Freitas said. “But we need to consider the implications and consequences of that.”

Read more of our coverage of the Every Student Succeeds Act here.