The Virginia Department of Education issued a report in May on the current status of its efforts to “restore excellence and close K-12 achievement gaps.” The report presents a series of facts about declines in student achievement levels on state and national assessment tests in recent years, concluding that “Virginia’s trend lines on key measurements are headed in the wrong direction” due to lowered expectations and a lack of transparency and accountability from school officials.
Though Virginia students have historically scored well on state assessment tests on average, the report says the results “mask widening student achievement gaps in the Commonwealth’s schools and a recent slip in comparison with other states on a range of academic achievement measures.” The report focuses particularly on what it calls the “honesty gap”—the spread between state and national measures of proficiency—which for Virginia students is among the widest in the nation. Here is a breakdown of the report’s claims.
Testing Pass Rates
The report focuses on two sets of standardized test results and their trends from 2017-19: Virginia’s statewide Standards of Learning tests (SOL’s), given each year in grades 3-12, and the National Assessment of Educational Progress (NAEP), given nationwide every other year to students in grades 4, 8, and 12.
As an example of diminished achievement, the Department of Education (DOE) report notes that between 2017-19, Grade 4 passing scores on the SOL reading assessment declined from 79% to 75% statewide. During the same period, the report said that “Virginia students in grades four and eight posted statistically significant declines in reading performance on the 2019 NAEP.”
In math, both fourth and eighth grade SOL pass rates saw a downturn from 2017 to 2018 followed by a sharp increase in 2019—a jump of four to six percentage points. However, the DOE report points out that in early 2019 the Virginia State Board of Education reduced the “cut scores” on math SOL’s, meaning that fewer correct answers were required to pass the test.
Indeed, the agenda from a March 2019 Virginia Board of Education meeting described a review of the mathematics SOL cut scores due to changes in the tests’ content. For the Grade 4 math assessment, the previous cutoff to pass the test was 31 out of 50 questions answered correctly (62%). Though the cut score for the new test “to maintain the previous level of rigor” was determined to be 29 out of 50, the state superintendent recommended 27 out of 50 (54%) as the cut score. The superintendent’s (lowered) recommendations were adopted for every grade level.
A similar cut score change took place in 2020 for SOL reading standards, resulting in a decrease in the score required to pass the Grade 4 test from 25 to 23 out of 40 questions (57%), even though the analysis showed the cut score should actually be raised to 26 to maintain the same rigor. “The state Board of Education’s decision to lower proficiency standards masked several years of declining achievement,” said the DOE’s report.
Neither SOL tests nor the NAEP were administered during 2020, and the 2021 SOL scores (where pass rates declined steeply from 2019) will be disregarded by state officials due to incomplete student participation. Though the 2021 pass rates showed achievement gaps between races and socioeconomic groups widening among every demographic, Albemarle officials said they cannot use SOL results to assess whether it is meeting its mission to “end the predictive power of race, gender, or special capacity on student success.” Assistant Superintendent Patrick McLaughlin said the division will start over this year in measuring the achievement gap.
“This year is going to kind of reset the bar for us,” said McLaughlin. “We’ve got a five-year strategic plan, and we say that within the course of this five years [beginning this year], our measure of success will be whether we ended that predictive value by the end of those five years. So, when our official SOL results come out in August, we’ll look at what the gaps look like in our schools and we’ll set targets for each of our schools and work to sustainably close those gaps.”
School Accreditation
As part of its description of overall lowered expectations for Virginia’s schools, the report also highlights the Board of Education’s decision to “water down the importance of grade level proficiency in school accreditation standards.” A school is accredited if it meets a certain level of educational standards as determined by state policy, and achievement on assessments is a large part of what is evaluated to achieve a high rating. In 2019-20, 92% of Virginia public schools were accredited. (All accreditation ratings have been waived for 2021 and 2022 due to the pandemic.)
Back in 2016, only 81% of Virginia schools were accredited, but in 2017 the Board of Education significantly adjusted the standards. The board opted to add students who were unable to meet the state’s proficiency standards, but who met a “minimal objective for growth,” to be included as proficient for calculating accreditation ratings for as many as three consecutive years. For instance, in 2019, almost 11,000 students had failed to meet Virginia’s reading benchmark in third, fourth, or fifth grade, but more than 3,000 of those students were weighed equally with passing students in schools’ accreditation ratings for 2019-20 because they had showed some amount of growth over time.
The division’s Coordinator of Research and Program Evaluation Chris Gilman has observed that some of the state board’s decisions in this realm seem to revolve around maintaining the number of schools that are fully accredited, which depends, in turn, on how many students pass the SOL tests. “What would be the knock-on effect [of changing the standards]?” said Gilman. “If we talk about shifting the [SOL] cut scores to get a truer pass rate, would we then have to shift accreditation standards [as well], so that we don’t get so many schools running afoul of accreditation that we can’t have schools operating in the state of Virginia. Because that would be the attendant outcome.”
Honesty Gap
The DOE report addressed the “honesty gap” between state SOL and national NAEP test pass rates, calling the NAEP test “the gold standard of student assessment and a critical tool for measuring if students are on track to be college and career ready.” Virginia’s negative gap was among the three largest in the nation in 2017, and it widened further in 2019. While 75% of Virginia fourth graders passed the reading SOL test in 2019, only 38% were proficient on the NAEP.
The DOE cites this gap as evidence that the state is claiming many more students are “proficient” than actually are. “[Since 2013], most states immediately addressed the gap by better aligning their state assessments to be at least as rigorous as NAEP,” reads the report. “Yet Virginia declined to narrow or close its own honesty gap.” The data broken down by demographic groups is even more stark—while 62% of Black fourth graders passed the SOL, only 19% were proficient on the NAEP.
“The NAEP is tricky,” said Gilman. “They use a nationally stratified sample for testing, and each year we get a letter telling us which two or three schools in our division they’ve chosen to take the assessments. So the decreasing scores may be an artifact of who they’re electing to sample from the state of Virginia. It’s possible that, even though it’s random, the types of places that have been selected from Virginia are not necessarily representative of Virginia as a whole. We get no [county-level] information back from them about student performance—only the individual families receive that data.”
Besides the sampling approach, a key distinction between the two tests is the definition of “proficiency.” The SOL tests “establish minimum expectations for what students should know and be able to do at the end of each grade or course [of study],” according to the DOE, and they measure grade-level performance as Virginia defines it.
The NAEP, developed in 1969 by an act of the U.S. Congress to measure student achievement nationally, is referred to as “the Nation’s Report Card.” The test’s “proficient” standard is that students “have demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter.” The NAEP says its standard “represents the goal for what all students should know.”
The NAEP is often (though not always) more challenging than state-wide assessments, and students nationwide reach the proficiency mark on average in the range of 35-40%. Though Virginia’s pass rates on the NAEP in 2019 are generally at or above national averages, some scores have been slipping in recent years. Grades 4 and 8 posted statistically significant declines from 2017 to 2019, with grade 4 reading scores at their lowest level since 2003, grade 8 math the lowest since 2009, and grade 8 reading the lowest since 1998. These declines, the DOE report points out, have occurred even as SOL pass rates have increased.
Until now, Virginia simply has not put much stock into the NAEP results. “Trying to compare what’s happening between those two [assessments] is like the difference between weighing yourself and checking your BMI,” said Gilman. “[The assessments are] similar, because they’re both about learning, but they’re different enough that you wouldn’t compare them. I think comparisons could also be attributable to, you know, the rest of the world catching up—sometimes it’s not a person slowing down, it’s somebody else speeding up—and so looking at our performance relative to others looks like a decline.”
Closing the gap between NAEP and SOL test passing rates will be a point of emphasis for Virginia’s DOE going forward, which will put pressure on state officials to adjust the SOL standards to be more like the NAEP’s—either in content or in proficiency cutoff levels or both. That effort may run into resistance from parents, said Gilman.
“Which parents would be willing to say, no, my kid really didn’t pass?” he said, “Because that’s ultimately what you’d have to do. These kids are as proficient as they are. To adjust the honesty gap would be turning some passes into fails, but doing it somewhat artificially, because we’re not going to make kids less smart. There is a certain sort of ethos around it—as in, do we have this many kids passing because this is how many parents think their kids are passing?
“Philosophically, we can put that cut score anywhere we want it to be, because for grades three through eight, passing and failing mean essentially nothing,” Gilman continued, “The high school level is the only place that passing the SOL functionally matters [because high school students must pass to earn a “verified credit” for having taken specific courses].” High school students may retake the SOL’s as many times as they need to pass.
Low Standards
One striking set of graphs in the DOE report compares state proficiency standards to NAEP standards in 2019 for every state. The charts use the percentage of students who met the state standard and maps that rate onto their performance on the NAEP, to arrive at an estimate of what the SOL’s “equivalent NAEP score” would be. For both grades 4 and 8 literacy, Virginia’s state standard maps to a level below the NAEP’s “basic” standard, which is itself a level below its “proficient” bar. In both of the reading tests plus grade 4 math assessments, Virginia’s standard as compared to the NAEP is the lowest in the U.S.
A key factor driving this low result is that Virginia’s state-determined pass rates are among the highest of any state—in 2017 for example, its grade 4 math pass rate was 82%. The DOE report cites lowered expectations of students as one of the department’s chief concerns. “We must close the honesty gap by raising proficiency cut scores and improving SOL rigor to align with the demands of the knowledge economy,” the report states.
Rigor, however, is only one of many considerations in Albemarle. “It comes back to what is our philosophy?” said Gilman. “Back in the 2000s we gave a math assessment in fourth grade that really challenged our students and we loved it, because it was so hard that even our brightest students weren’t absolutely knocking the socks off of it. It really tested their conceptual understanding of mathematics, not just the calculations, and we pushed everyone to those rigorous standards.
“As times change,” he continued, “philosophies shift, and then it becomes, well maybe pushing kids [is too much], kind of like if you run a car at 100 mph all the time, you’re going to hit a wall. So how do we balance that, how do we say rigor is important but now we want to do these other things? And so you start to create contours.”
Read the Virginia DOE’s report, titled “Our Commitment to Virginians,” at the state website: www.doe.virginia.gov.
