Bold new evidence: Kentucky does not lead the nation for education improvement

Claim especially misleading for state’s black students

Truth supports need for charter schools in Kentucky

As arguments swirled the past few months over charter schools, Kentuckians have been hearing claims that their state already leads the nation for the most educational improvement since passage of the Kentucky Education Reform Act of 1990 (KERA). As a consequence, that argument goes, this means Kentucky doesn’t need charters.

The latest example of this “leads the nation” claim is found in a March 10, 2017 Herald-Leader Op-Ed by David Hornbeck, one of the major architects of KERA. Hornbeck asserts:

“Kentucky children have made more progress than any other state in the nation.”

It’s a bold statement, but is it true?

And, is it true for all Kentucky’s children?

To explore these questions, we fired up the National Assessment of Educational Progress’ Main NAEP Data Explorer web tool. We used data from the NAEP Data Explorer to assemble the two tables below, which show how Kentucky’s eighth-grade blacks stack up against other states that also had scores for these children of color reported for both the earliest and latest years of NAEP state testing.

Table 1 shows the NAEP Grade 8 math results black students in the listed states received back in 1990, the year KERA was enacted, and 2015 scores – the latest available. The table is sorted by the change in the NAEP Scale Score for math in each state across the 1990 to 2015 period.

Table 1

Grade 8 Math Improvement for Blacks for 1990 and 2015 Ranked

As you can see, Hornbeck’s assertion isn’t just wrong, it’s very wrong when we talk about improvements for Kentucky’s largest racial minority group compared to other states with usable NAEP data for black students.

Kentucky lands nearly at the bottom of the stack when we rank each state’s increase in NAEP Grade 8 Math Scale Scores for black students over time. Only four of the 28 states with data available progressed even less than Kentucky.

If we only consider southern states listed in Table 1, we find that North Carolina, Florida, Louisiana, Georgia and Arkansas all matched or exceeded the national average increase in black students’ math scores between 1990 and 2015. Kentucky never came close to any of them.

By the way, all of those five Southern states have charter schools. At present, aside from Kentucky, only Montana, Nebraska, North Dakota, South Dakota, Vermont, and West Virginia don’t have charters. Thus, except for West Virginia, all the states listed above Kentucky in Table 1 have charter school laws. That is something to think about.

[Read more…]

Kentucky’s Real Progress on the National Assessment of Educational Progress (NAEP)

On Friday, March 3, 2016 the Kentucky House made history when it voted for the first time in favor of a charter school bill and sent it on for Kentucky Senate approval.

The vote was contentious.

Debates in the morning meeting of the House Education Committee and during the eventual deliberation and adoption of the bill by the full Kentucky House sometimes were bitter – even tear filled. And, there were lots of inaccurate statements along the way.

One entirely too prevalent assertion mentioned by many legislators was that Kentucky has made great education progress since passage of the Kentucky Education Reform Act of 1990 (KERA). Sadly, while the state’s public education system has made some progress in the past quarter of a century, it’s a real stretch to say “great” progress has been made. Let’s examine why inflated claims of great progress are out of order.

Figure 1 shows the NAEP Grades 4 and 8 reading and math proficiency rates for all Kentucky students from the earliest available year of testing and the most recent, 2015 results. There obviously has been progress, more in Grade 4 than Grade 8, but calling this a “great” accomplishment just isn’t right.

For example, only 40 percent of Kentucky’s fourth graders tested at or above NAEP’s Proficient level in 2015 in both fourth grade math and reading. That means that after a quarter of a century of KERA, 60 percent of our fourth graders – well over half – still don’t meet muster in either subject. After a quarter of a century, with so far yet to go, does it seem right to talk about “great progress?”

In the eighth grade NAEP, results were even worse. Only 36 percent of the state’s eighth graders scored at or above the NAEP Proficient level in reading. Far more disturbing, only a truly disappointing 28 percent of Kentucky’s eighth graders met muster in NAEP math. That means 72 percent of the state’s eighth grade students – as of 2015, a full quarter century after the launch of KERA – still don’t perform adequately in math.

Figure 1

Kentucky's NAEP Proficiency Rates on 4th and 8th Grade Reading and Math Assessments, Earliest Year Tested and 2015, All Students

Based on the known rates of progress that can be calculated using the data shown in Figure 1, the Bluegrass Institute projected the number of years following 2015 that remain before Kentucky can anticipate that at least 80 percent of its students will score proficient or above on the NAEP. You can see those projections in the table inserted in the upper right side of Figure 1. Those time estimates to reach 80 percent proficiency rates on the NAEP range from at least 34 more years required in Grade 4 math to an astonishing 126 more years for Grade 8 Reading.

With so much left to do, it is obviously inappropriate to crow about already making “great” progress. A large amount of progress simply hasn’t happened.

By the way, the situation looks MUCH worse when we examine the NAEP performance of Kentucky’s black students. Claiming “great progress” once this actual data is examined is simply unacceptable.

As Figure 2 shows, even as 2015, the NAEP reports only depressingly low percentages of Kentucky’s black students scored proficient or above in both Grade 4 and Grade 8 reading and mathematics.

Figure 2

Kentucky's NAEP Proficiency Rates on 4th and 8th Grade Reading and Math Assessments, Earliest Year Tested and 2015, Black Students Only

In two cases shown in the table insert in Figure 2, the trends on NAEP tell us Kentucky is nearly a century away from seeing a desirable math proficiency rate for its black students. In eighth grade math, the goal is the better part of two centuries away. In the case of Grade 8 Reading, the 80 percent proficiency rate goal is more than 2-1/2 centuries away!

This is simply unacceptable.

Clearly, Kentucky’s actual NAEP performance renders claims of great progress to be greatly exaggerated.

[Read more…]

Kentucky’s real “progress” under Common Core

With the Kentucky legislature coming back into session, a number of education issues are heating up. One of the more hotly debated topics will be whether the state continues to use the Common Core State Standards for its English language arts and math standards.

Because Kentucky was the first state to adopt the Common Core and has more experience with it than any other state, the coming battle is likely to get considerable attention outside the borders of the Bluegrass State.

Already, lines are forming, and shots were fired last week in an Op-Ed that appeared under the title “Column: Common Core: If it isn’t broken, don’t fix it” in the Community Recorder newspaper in Northern Kentucky and under a different title in the Courier-Journal. Unfortunately, as typically happens with an advocacy piece, there is a very large amount of what the late Paul Harvey used to call “The rest of the story” that didn’t make it into the Op-Eds.

But, our readers deserve to know the rest of the story, so let’s take a more informed look at Common Core.

We will start with the most important issue, one related to assertions in the Op-Ed that Common Core is working. Well, you certainly can’t see that in reading and mathematics scores for Kentucky from the National Assessment of Educational Progress. Let’s take a look.

Figure 1 shows the overall average score for “all students” in Kentucky from the 2011, 2013 and 2015 administrations of the NAEP eighth grade math assessment. This covers a period from one school term before Kentucky started KPREP Common Core aligned testing in math and reading through the latest available NAEP data.

As you can see, Kentucky’s scores on this 500-point scale NAEP assessment don’t show Common Core working at all. In fact, as indicated by the asterisks by the scores for 2011 and 2013, the scores in both of those years were statistically significantly higher than the latest available score for 2015. To make this crystal clear, NAEP shows Kentucky experienced a definite decline in eighth grade math performance after Common Core came on line.

Figure 1

G8 NAEP Math for KY 2011 to 2015

This isn’t looking at results for Advanced Placement tests (which cover material far more advanced than anything in Common Core) or a Harvard study that looks at changes over a two-decade long period as the Op-Eds did. Those can’t provide very precise information about what happened as a result of Common Core.

This is looking directly at students in grades that are fully under the influence of Common Core in Kentucky, and the obvious result isn’t encouraging.

The bad news for progress under Common Core doesn’t end with eighth grade math, either. To see still more of “The rest of the story,” just click the “Read more” link.

[Read more…]

NAEP tells more about why Kentucky needs charter schools

Kentucky has a major problem with white minus black achievement gaps and it is especially apparent in the state’s results over time from the National Assessment of Educational Progress (NAEP) Grade 8 Reading Assessment.

The NAEP didn’t start to collect state level eighth grade reading data until 1998, but by now we still have a rather long trend line to examine.

And, the gap trend isn’t good.

The figure below shows the percentages of Kentucky’s white and black students that scored at or above the performance level NAEP defines as Proficient for eighth grade reading.

KY White Vs Black Gap for G8 NAEP Reading 1998 to 2015

Below the white and black score lines is a table that shows the gap in the proficiency rates for each year. The asterisk next to the 2007 figure in the table shows that this particular gap is statistically significantly different from the 2015 gap.

For example, in 2007 Kentucky’s white eighth grade students scored only 30 percent proficient for reading following a decay in their performance that began after 2003.

In 2007 only 14 percent of Kentucky’s blacks scored proficient or more for eighth grade reading.

However, when we consider the gaps, due to the rather pronounced white reading drop in proficiency in 2007, the gap that year was the lowest ever posted. As a consequence the 2007 gap is statistically significantly lower than the gap of 24 points for 2015. Actually, due to statistical sampling errors in the NAEP, none of the gaps for any other year shown in the graphic’s table are notably different from the 2015 gap.

So, as far as the NAEP can tell us, essentially the white minus black eighth grade reading gap in Kentucky is the same as it was in 1998. And, that gap is far too large.

We can see something else that is very disturbing in this graphic. Essentially, black reading proficiency in Kentucky has not changed notably since 2002. In fact, the 2015 black proficiency rate isn’t statistically significantly different from any previous score all the way back to 1998.

To be very clear: Kentucky’s eighth grade blacks are not making reading progress according to the NAEP – NONE!

This isn’t news to us at the Bluegrass Institute. Since we became the first Bluegrass State organization to start pushing charter schools and other school choice options 14 years ago, blacks in our state have not fared well, and that is particularly true for the massively important area of reading.

Clearly, it is past time for Kentucky to move beyond a blind faith that somehow – despite 25 years of failing to do so – the state’s traditional school system will fix this problem on its own. An education lesson now lasting a quarter of a century, one confirmed by the NAEP, says this problem isn’t going to be fixed without some new and different approaches for Kentucky. Since charter schools are showing particular positive impacts for black students, charters are clearly an education tool the Bluegrass State should offer its students.

[Read more…]

Will the nation’s “gold standard” test turn to dross?

As the Huffington Post just reported, the National Assessment of Educational Progress (NAEP), which often is billed as the “gold standard” for education testing, is making a major transition from its former paper-and-pencil testing format to a new, digital administration. However, the NAEP’s transition to digital testing has many thoughtful observers holding their breath. Click the “Read more” link to see why.

[Read more…]

Misleading us again

The Prichard Committee is at it again with a new edition of their Top 20 by 2020 report series.

Once more Prichard is improperly ranking National Assessment of Educational Progress (NAEP) white students in Kentucky against lower performing racial minorities in other states, trying to make the Bluegrass State’s educational performance look better than it really is.

Prichard probably won’t ever stop their apples to oranges NAEP rankings, but it does give me a chance to point to better pictures of what is really happening.

In fact, I posted a pretty good blog about this nonsense only about a week and a half ago, so I am not going to waste a lot of time on the new, really same-old, same-old, nonsense that came out today.

But, one area I haven’t really discussed is the eighth grade science picture. I just fired up the NAEP Data Explorer to see how Kentucky’s whites stacked up against their white counterparts in other states in 2015 NAEP science testing.

The answer doesn’t look so good.

g8-science-for-whites-ranked

There we are, way down in the 41st listing in the table. Only two states with scores reported had science performance for their white students that was statistically significantly worse. Even Mississippi’s whites tied with us once you consider the statistical sampling errors in the NAEP scores. Those pesky folks from Tennessee outclassed us, too.

By the way, even Prichard had to admit Kentucky’s eighth grade NAEP math performance looks “disturbing.” But, after you look at the table above and my blog from a week-and-a-half ago, you will know that’s only part of the “disturbing” picture.

‘Nuff said.

KY Ed. Secretary gets it right about Maryland charter schools

KET’s Kentucky Tonight discussion on charter schools yesterday was nothing if not lively. With Kentucky’s Secretary of Education Hal Heiner and “Old Guard” Kentucky Senator Gerald Neal on the panel along with our own Jim Waters and executive director of the Kentucky Association of School Superintendents Tom Shelton, things were bound to be interesting.

However, perhaps most interesting of all for me was a Heiner/Neal back-and-forth about the value of Maryland’s charter schools as a model to build a charter program for Kentucky.

Neal kept pushing Maryland throughout the broadcast until Heiner finally shut him down, saying that Maryland’s charters perform near the bottom.

Wow, I thought. I have not seen anything on that. How do Maryland’s charter schools perform?

Sadly, as Dr. Joseph Waddington, Assistant Professor, Educational Policy Studies & Evaluation, University of Kentucky pointed out to the Kentucky Board of Education last week,

“Not every study meets important thresholds of methodological rigor.”

In fact, a lot of charter research is of poor quality. The iffy quality of the data on charters even extends to the National Assessment of Educational Progress (NAEP). Thanks in part to inadequate student sampling of charter schools, this federal assessment isn’t a very useful tool for charter research. Too often, NAEP scores are not even reported for some states’ charter schools. For example, Indiana and Massachusetts scores for charter schools are not available from the 2015 NAEP.

Still, I decided to take a look at what the NAEP does report for eighth grade reading and math in charters in Maryland and other states. I got a mild surprise when I did that.

It turns out that while many charter states such as Indiana and Massachusetts were missing score information, Maryland and a number of other states did have scores both for their total charter school population and for their black students. When I ranked those scores using the NAEP Data Explorer’s Statistical Significance testing tools, Maryland did indeed wind up consistently around the bottom end of the NAEP stack for both math and reading in Grade 8 testing for all students.

g8-reading-all-students-naep-rank-for-maryland

g8-math-all-students-naep-rank-for-maryland

[Read more…]

Overstating Kentucky’s education progress — again

During Monday’s Kentucky Board of Education meeting about charter schools, one presenter again pushed the idea that Kentucky has made great progress on the National Assessment of Educational Progress (NAEP) since KERA began in 1990. But, the picture that got painted, which only referenced overall average NAEP scores, was incomplete and misleading.

When you only compare overall student NAEP scores for Kentucky to the “all student” scores in other states, it looks like we have made progress. But, the overall numbers do not provide a fair comparison.

As Figure 1, which specifically examines data from the 2015 NAEP Grade 8 Math Assessment, shows, Kentucky’s school enrollment was about 80 percent white while across the nation only about half the students were whites. In some places like California, white students were a notable minority part of the public school enrollment.

Figure 1

comparing-kentuckys-2015-grade-8-naep-racial-makeup-to-national-public-and-california-makeups-percentage-of-students-in-each-racial-category

So, only looking at overall average scores from NAEP just matches a lot of white student scores in Kentucky against a lot of minority scores found in other states. Thanks to the very large achievement gaps in this country, that doesn’t provide a realistic comparison.

For example, comparing Kentucky’s overall average scores against California’s results means that 82 percent minus 25 percent, or 57 percent of all Kentucky’s students who are whites are being matched to students of other races in California. Many of those California students will be lower-scoring Hispanics and an appreciable proportion of those California Hispanics will be English language learners, as well.

If we compare Kentucky to the nationwide public school results, we are still matching a group amounting to 31 percent of all Kentucky’s students who are whites to students of other races in other parts of the country. That won’t provide an accurate picture of our state’s true performance.

When we do break the 2015 NAEP Grade 8 Math results out by race, Figure 2 shows Kentucky’s performance for its dominant racial group does not look very good.

Figure 2

g8-naep-math-map-for-whites-2015

Our white students scored almost at the very bottom on this national assessment.

So, if you want to fairly compare education performance across the states, you have to break the data out by student subgroups. You are just fooling people, otherwise.

[Read more…]

Bellwether Education Partners raising concerns about high school graduation rate validity

I am not the only one who is raising concerns about the education quality behind the increases in high school graduation rates being reported across the nation.

Bellwether Education Partners clearly share my concerns, as this article, “What Good Are Higher Graduation Rates If Students Aren’t Learning More?” clearly states.

Check my multi-part blog series, “More on the quality control problems with Kentucky’s high school diplomas,” for more.

Part 1

Part 2

Kentucky’s new NAEP Science scores look flat

New results from the National Assessment of Educational Progress (NAEP) science assessment for 2015 have finally been released.

Even the Kentucky Department of Education (KDE) admits the results overall look “flat.”

[Read more…]