What education accountability can (and cannot) do, Part II

Once again, Prof. Gary Houchens, a Bluegrass Institute Scholar and Kentucky Board of Education member as well as a professor at Western Kentucky University, has posted a great blog about the goals and limitations of education accountability.

I highly recommend reading Dr. Houchens’ blog.

By the way, at the risk of oversimplification, an education accountability program is somewhat like the speedometer in your car. Without question, a speedometer provides valuable information for the safe operation of the car, but your speedometer won’t make your car go faster or slower. It is only a performance instrument.

Still, it is dangerous to ignore a speedometer and consequences for doing so can be very high.

Thus, the idea that we would want to rip an accountability program out of the education machine is just about as dangerous as the idea of ripping out the speedometer in our vehicles. The key is that we want a speedometer with accurate calibration and easy to read indications. I think our education speedometer needs more work in the accuracy and readability area, but the notion that we could get along without such a performance gauge is not good for our kids.

Mississippi fires testing contractor who made serious mistakes

Kentucky uses same contractor

AP reports that Mississippi has fired NCS Pearson after that testing company made serious grading errors on high school tests that have impacted graduation for as many as 1,000 students.

Some students who actually performed poorly on a high school history test erroneously got high scores while other students who actually did well got inaccurately low scores that might have prevented high school graduation.

The AP article points out that this isn’t the first time Pearson made mistakes on high school exam grading that adversely impacted students. The company also had to pay for a failure of its online testing system in 2015, as well. Pearson reimbursed Mississippi $250,000 for that computer glitch.

Pearson is the prime contractor for Kentucky’s KPREP tests in Grades 3 to 8. So far, problems such as those in Mississippi have not been reported during Kentucky testing.

KY Board of Ed gets bad news about projected math proficiency to 2030

The Kentucky Board of Education (KBE) was heavily engaged in discussions about the state’s new school accountability system today, and one slide, shown in Figure 1, which didn’t make anyone happy, is shown below.

Figure 1

Kentucky Actual and Projected Elementary - Middle School Math Achievement to 2030

This slide shows actual average math scores from Kentucky KPREP testing in 2014 (blue bars) and 2016 (red bars) along with projected scores for 2018 (green bars) and a full school generation of kids out in 2030. Each section of the graph covers a different group of students:

ALL = All Students
W = White Students Only
AA = African-American Students Only
FR = Free or Reduced Cost School Lunch Eligible Students Only
SWD-IEP = Students with Learning Disabilities Who Have an Individual Education Plan

As you can see, even for the highest performing group, the white students, even 13 years from now the Kentucky Department of Education projects only 73 percent will be proficient. That would be an increase of 24 points from the 49 percent that KDE says were actually proficient in 2014. That works out to a proficiency rate growth rate of 1.8 points per year.

For African-American students, fewer that one in two, just 46 percent, will be proficient in 2030. In 2014, 30 percent were proficient, for a growth rate of only 1.2 points per year

Of course, this is based on Kentucky’s somewhat unproved KPREP test results. For an even more disturbing look at the state’s slow rate of progress, click the “Read more” link.

[Read more…]

What happens when your school accountability program misses important problems

This week the Kentucky Board of Education considers ending state assistance to the Robertson County Public School District. The situation provides a good example of how important decisions about education can be seriously hampered when a state school accountability system hides problems.

[Read more…]

Valley High School exits Priority Status????

Last week the media in Louisville trumpeted the announcement that Kentucky Commissioner of Education Stephen Pruitt had declared the Valley High School in the Jefferson County Public School District was no longer in Priority School Status (see WDRB’s coverage here).

That sounded interesting, so I decided to take a quick look at the latest performance in this school for math and reading testing. I looked at math and reading because performing in the lowest five percent of all schools for these two subjects was supposed to be the primary cause to enter Priority Status back in 2010 when these low performing schools, which originally were called “Persistently Low-Achieving Schools,” first started to be identified.

Well, my quick look turned up a puzzle.

This first table shows the lowest 20 performing standard (Class A1) high schools on KPREP End-of-Course testing in Algebra II and English II. These two KPREP tests are used to gauge reading and math for federal reporting purposes. The table shows the combined percentage of students who were rated either Proficient or Distinguished in Algebra II in the first data column and then lists the combined percentage of Proficient and Distinguished students in English II in the middle data column. The next column, on which the table is ranked, shows the average of these two percentages.

Table 1

Valley High KPREP Math-Reading Combined Ranking 2016

As you can see, Valley High School ranked in the bottom five percent of all high schools in Kentucky that had data reported, ranking at 218 out of 227 reporting high schools.

But, the original testing that got Valley High in trouble (it was named a Persistently Low-Achieving School in the spring of 2010) was the now defunct CATS Kentucky Core Content Tests. Since those tests don’t even exist in 2016, I decided to give Valley another chance by looking at its performance on math and reading in the 2016 ACT testing of Kentucky’s 11th grade students. Table 2 shows how that turned out.

The first data column in Table 2 shows the percentage of students in each school that reached or exceeded the Benchmark Score set by the Kentucky Council on Postsecondary Education (CPE) for ACT testing that indicates students will not have to take remedial courses in math. The next column shows the percentage of students that met the CPE’s ACT Benchmark for reading, which also avoids a requirement for college remediation in that area. The two Benchmark percentages are then averaged together in the next column and the table is ranked on this combined average column.

Table 2

Valley High ACT Math-Reading Combined Ranking 2016

Incredibly, if we look at the average of the percentages of students meeting the CPE’s College Readiness Benchmark Scores for the ACT, Valley High ranks even lower than on KPREP!

So, this is a real puzzle. I know the actual method used to determine Priority Status uses a more complex approach than just looking at a single year of data, but when we see Valley High’s latest performance in both Tables 1 and 2, something just doesn’t feel right.

Should Valley High be off the hook?

In any event, based on its latest year’s performances on both KPREP and ACT math and reading, Valley High remains a very low-performing school. I think the public deserves to know that even if our educators are letting Valley off the hook.

[Read more…]

Will Kentucky’s education system be standards driven or test driven?

When the Common Core State Standards for English Language Arts and Mathematics (CCSS) and the Next Generation Science Standards (NGSS) came to the Bluegrass State, Kentuckians were told their state’s education system would be built around those standards.

Well, perhaps not.

There was a presentation about the pending revision to the state’s science assessments during today’s meeting the Kentucky School Curriculum and Assessment Committee (SCAAC). The presenter was asked if all science areas would be covered in the assessments for elementary, middle and high schools. The answer, to my considerable surprise, was “Yes.”

Not certain I heard this correctly, I questioned the presenter during a break and confirmed that all science areas would be fair game in the new science assessments at all school levels. That included chemistry and physics for high schools.

The reason this surprised me, and the reason this is a problem, is because the generally vague NGSS essentially cut off completely after high school biology. Topics from high school chemistry and physics are basically absent even though some at the Kentucky Department of Education don’t seem to understand that.

Furthermore, a well-established legal principal known as “Notice or “Fair Notice” says you can’t give tests that have consequences if you don’t provide advance notice of what is fair game on the tests. The way you provide notice is with the state’s education standards. NGSS can’t give adequate notice for things it doesn’t include.

So, is Kentucky sailing into really troubled waters with its new science assessments? Unless some changes are made, I think so.

[Read more…]

KY State of Education shows serious grading discrepancies by race

Kentucky Commissioner of Education Stephen Pruitt delivered his second annual “The State of P-12 Public Education in the Commonwealth of Kentucky” report today, making extensive and very candid comments about the serious achievement gap situation in the state.

I’ll be spending some time in this report, but I think many at the press conference were particularly struck by results of a new analysis of course grade awards versus performance on Kentucky’s various mathematics assessments. So, I am going to delve into that new research now.

To put it mildly, this new research was a major eye-opener. Aside from showing some very disturbing trends regarding differential course grading by race, the data undermines a long-held notion that course grades are likely to be the best predictor of college performance.

Let’s look at two of the eye-watering graphs in the new report.

Figure 1

Grade 8 Course Grades Vs. KPREP by Race

The graph in Figure 1 is based on a study of Grade 8 math course letter grades and KPREP math scores from 2012 to 2016, and is found on Page 6 in the report. It shows some pretty disappointing things are happening in Kentucky’s public school system.

Looking vertically up from the “A” grade point on the right side of the horizontal axis, we see an example of why the report says:

“For African American students whose average letter grade in their middle school math courses was an A, the chance of scoring proficient on state math tests was 25 percentage points lower than that of white students who also earned an A average.”

Clearly, less is being demanded of Kentucky’s blacks to earn an “A” grade in math class. Across Kentucky, teachers are setting a lower standard for these children of color to earn an “A.” Examination of the graph for other letter grades shows blacks are held to lower standards for every other grade from “B” even down to a “D” score, though the amount of performance difference for whites versus blacks does decline a bit as we move down the grading scale.

[Read more…]

Do Kentucky’s KPREP school assessments do what they are supposed to do?

If so, why is the evidence not available after five years of KPREP testing?

The Bluegrass Institute has discovered a rather extraordinary January 6, 2017 letter from the US Department of Education to Kentucky Commissioner of Education Stephen Pruitt.

This letter says evidence provided by the Kentucky Department of Education only shows that the state’s public school assessments just partially meet requirements of federal education legislation.

The letter lists the following general comments:

  • Reading/ language arts (R/LA) and mathematics general assessments in grades 3-8 (Kentucky Performance Rating for Educational Progress (K-PREP)): Partially meets requirements
  • R/LA and mathematics general assessments in high school (ACT QualityCore EOC for R/LA and math): Partially meets requirements
  • R/LA and mathematics alternate assessments for students with the most significant cognitive disabilities in grades 3-8 and high school (Alternate K-PREP for R/LA and math): Partially meets requirements

The letter continues:

“The partially meets requirements designation for a component means that it does not meet a number of the requirements of the statute and regulations, and Kentucky will need to provide substantial additional information to demonstrate it meets the requirements. The Department expects that Kentucky may not be able to submit all of the required information within one year (underlined emphasis added).”

Keeping in mind that the Kentucky KPREP and End-of-Course tests have been in place since the 2011-12 school term, the letter’s expanded details about the missing evidence are very disturbing.

For example, Under Critical Element 1.2, the US Department of Education says Kentucky needs to provide:

“A description of State stakeholders involved in the development and/or adoption process for the R/LA, mathematics, and science content standards that includes detail on subject-matter expertise, individuals representing English learners (ELs) and students with disabilities.”

This might be really hard to do. Kentucky basically just adopted the Common Core State Standards for English Language Arts and Mathematics (CCSS) at a high level. State stakeholders really had no say in the final decisions about what went into the CCSS. The adoption was made by the Kentucky Board of Education, the Kentucky Educational Professional Standards Board and the Kentucky Council on Postsecondary Education. Subject matter experts were not involved in this widely televised, media event joint meeting of these three boards.

In fact, the adoption of Common Core took place about 3-1/2 months before the final version of the Common Core was even published. It is hard for experts to have looked at something that didn’t even exist at the time of adoption. In fact, the public comment draft of the Common Core didn’t even come out until March 2010, weeks after the three Kentucky boards had already adopted the Common Core, sight unseen.

Under Critical Element 1.5, Kentucky still needs to provide:

“Evidence that the State has procedures in place for ensuring that each student is tested and counted in the calculation of participation rates on each required assessment.”

How’s that? Kentucky can’t provide evidence it really is testing all students with KPREP? Not even after the test has been in used for five testing cycles? That is a real problem.

And, the letter doesn’t stop there. To learn still more, click on the “Read more” link.

[Read more…]

Kentucky’s real “progress” under Common Core

With the Kentucky legislature coming back into session, a number of education issues are heating up. One of the more hotly debated topics will be whether the state continues to use the Common Core State Standards for its English language arts and math standards.

Because Kentucky was the first state to adopt the Common Core and has more experience with it than any other state, the coming battle is likely to get considerable attention outside the borders of the Bluegrass State.

Already, lines are forming, and shots were fired last week in an Op-Ed that appeared under the title “Column: Common Core: If it isn’t broken, don’t fix it” in the Community Recorder newspaper in Northern Kentucky and under a different title in the Courier-Journal. Unfortunately, as typically happens with an advocacy piece, there is a very large amount of what the late Paul Harvey used to call “The rest of the story” that didn’t make it into the Op-Eds.

But, our readers deserve to know the rest of the story, so let’s take a more informed look at Common Core.

We will start with the most important issue, one related to assertions in the Op-Ed that Common Core is working. Well, you certainly can’t see that in reading and mathematics scores for Kentucky from the National Assessment of Educational Progress. Let’s take a look.

Figure 1 shows the overall average score for “all students” in Kentucky from the 2011, 2013 and 2015 administrations of the NAEP eighth grade math assessment. This covers a period from one school term before Kentucky started KPREP Common Core aligned testing in math and reading through the latest available NAEP data.

As you can see, Kentucky’s scores on this 500-point scale NAEP assessment don’t show Common Core working at all. In fact, as indicated by the asterisks by the scores for 2011 and 2013, the scores in both of those years were statistically significantly higher than the latest available score for 2015. To make this crystal clear, NAEP shows Kentucky experienced a definite decline in eighth grade math performance after Common Core came on line.

Figure 1

G8 NAEP Math for KY 2011 to 2015

This isn’t looking at results for Advanced Placement tests (which cover material far more advanced than anything in Common Core) or a Harvard study that looks at changes over a two-decade long period as the Op-Eds did. Those can’t provide very precise information about what happened as a result of Common Core.

This is looking directly at students in grades that are fully under the influence of Common Core in Kentucky, and the obvious result isn’t encouraging.

The bad news for progress under Common Core doesn’t end with eighth grade math, either. To see still more of “The rest of the story,” just click the “Read more” link.

[Read more…]

Author says: ‘I Can’t Answer These Texas Standardized Test Questions About My Own Poems’

Here’s a Huffington Post article worth your read.

Poet Sara Holbrook discovered some of her poems were being used as seventh grade test items in Texas STAAR test. To her dismay, she couldn’t figure out what the answers to the questions really were! And, Ms. Holbrook wrote the poems!

By the way, Kentucky, don’t act smug just because this issue surfaced in Texas. Holbrook says the Texas tests are created by Pearson, the same company that creates the Grade 3 through Grade 8 KPREP tests for the Bluegrass State. For all we know, the same test questions might even be on the KPREP.