Bold new evidence: Kentucky does not lead the nation for education improvement

Claim especially misleading for state’s black students

Truth supports need for charter schools in Kentucky

As arguments swirled the past few months over charter schools, Kentuckians have been hearing claims that their state already leads the nation for the most educational improvement since passage of the Kentucky Education Reform Act of 1990 (KERA). As a consequence, that argument goes, this means Kentucky doesn’t need charters.

The latest example of this “leads the nation” claim is found in a March 10, 2017 Herald-Leader Op-Ed by David Hornbeck, one of the major architects of KERA. Hornbeck asserts:

“Kentucky children have made more progress than any other state in the nation.”

It’s a bold statement, but is it true?

And, is it true for all Kentucky’s children?

To explore these questions, we fired up the National Assessment of Educational Progress’ Main NAEP Data Explorer web tool. We used data from the NAEP Data Explorer to assemble the two tables below, which show how Kentucky’s eighth-grade blacks stack up against other states that also had scores for these children of color reported for both the earliest and latest years of NAEP state testing.

Table 1 shows the NAEP Grade 8 math results black students in the listed states received back in 1990, the year KERA was enacted, and 2015 scores – the latest available. The table is sorted by the change in the NAEP Scale Score for math in each state across the 1990 to 2015 period.

Table 1

Grade 8 Math Improvement for Blacks for 1990 and 2015 Ranked

As you can see, Hornbeck’s assertion isn’t just wrong, it’s very wrong when we talk about improvements for Kentucky’s largest racial minority group compared to other states with usable NAEP data for black students.

Kentucky lands nearly at the bottom of the stack when we rank each state’s increase in NAEP Grade 8 Math Scale Scores for black students over time. Only four of the 28 states with data available progressed even less than Kentucky.

If we only consider southern states listed in Table 1, we find that North Carolina, Florida, Louisiana, Georgia and Arkansas all matched or exceeded the national average increase in black students’ math scores between 1990 and 2015. Kentucky never came close to any of them.

By the way, all of those five Southern states have charter schools. At present, aside from Kentucky, only Montana, Nebraska, North Dakota, South Dakota, Vermont, and West Virginia don’t have charters. Thus, except for West Virginia, all the states listed above Kentucky in Table 1 have charter school laws. That is something to think about.

NAEP didn’t start state testing of Grade 8 reading until 1998. Table 2 shows how Kentucky progressed against other states with available data for black students since that inaugural Grade 8 reading assessment.

Table 2

Grade 8 Reading Improvement for Blacks for 1998 and 2015 Ranked

Once again, Kentucky ranks near the bottom, only outperforming score gains for four of the 29 states that reported black students’ scores for both years of NAEP Grade 8 Reading.

In fact, Kentucky’s meager, one-point score improvement for black Grade 8 reading in the 17-year period between 1998 and 2015 isn’t even a statistically significant change.

Did you get that? Kentucky’s blacks have made NO measurable progress in NAEP Grade 8 Reading in more than one-and-one-half decades of testing. That sure isn’t nation-leading improvement.

Look at the performance of those “pesky” Volunteers from Tennessee in Table 2. They rank way up near the top – in fourth place – for their increase in black reading scores on NAEP. Back in 1998, Tennessee scored 11 points behind Kentucky’s black students. Flash forward to 2015 and those pesky Volunteers now tie us. Yes, Tennessee has charter schools.

Florida is an even bigger embarrassment. Their blacks scored 10 points behind Kentucky’s in 1998 but now place four points ahead of the Bluegrass State.

Once again, every state listed above Kentucky in Table 2 – except for West Virginia – currently has charter school legislation on the books. Again, something to think about.

Here’s another point: we also ran the data for Kentucky’s white eighth-grade students in math and reading. Our whites didn’t come close to matching Hornbeck’s assertions, either, though they certainly did better than our blacks. Of course, that isn’t a surprise to our readers, who know that white minus black achievement gaps have been growing in Kentucky ever since KERA began.

Given that Hornbeck’s KERA reforms have shaped Kentucky education for the past quarter century, the fact that our black students remain seriously left behind when it comes to making progress really gives us something else to think about. Quite opposite to Hornbeck’s claims, KERA – at least for our black students – has failed. That makes you wonder if there were things wrong with his advice regarding KERA in the first place. In fact, there was research out there in 1990 from a huge effort called Project Follow Through that shows the types of instructional techniques pushed by KERA are especially unsuitable for disadvantaged and minority students.

So, maybe Hornbeck’s KERA wasn’t the right way to go from the get-go, at least if you really care about ALL Kentuckians.

Comments

  1. Only to a sample of students from each state take part in the NAEP assessment. Thus, all parametric statistics (e.g., average, standard error, etc.) that NAEP reports are only estimates. The NAEP average scale scores for black eighth grade students in 1998 and 2015 for both reading and mathematics from the states referenced in this study all contain a measure of uncertainty.

    These states cannot be rank ordered using their 1998 average scores because of the uncertainty in NAEP results, expressed as standard errors. These states cannot be rank ordered using their 2015 average scores either because of the uncertainty in NAEP results.

    How is it possible to get an accurate “score change” by subtracting a 1998 average score with its standard error from the 2015 average score with a different standard error? This process does not reduce or eliminate the uncertainty in the NAEP averages; it compounds the uncertainty.

    Stoneberg, B.D. (2005). Please don’t use NAEP scores to rank order the 50 states. “Practical Assessment, Research & Evaluation,” 10(9). Available online: http://pareonline.net/getvn.asp?v=10&n=9

    • Richard Innes says:

      Actually, some of the differences are so great that they swamp the standard errors.

      For example, in 1990 Texas scored 6 points lower than Kentucky on NAEP Grade 8 Math. Due to the sampling errors, that wasn’t statistically significant.

      Flash forward to 2015 and Texas outscored Kentucky by 10 points, and that was statistically significant.

      So, Texas definitely moved ahead.

      In 1990 North Carolina scored 10 points behind Kentucky, which was a statistically significant difference (using NAEP Data Explorer, which carries scores out to a number of decimal places so the number doesn’t exactly agree with table in blog).

      In 2015 North Carolina scored 5 points ahead (using NAEP Data Explorer), which was a statistical tie.

      So, North Carolina caught up to Kentucky after being well behind.

      Kentucky does not fare well on NAEP over time.