BLOG

Let’s do a little graph work with the Long Term Trend version of the National Assessment of Educational Progress (LTT NAEP). We’ll use this graph of the LTT NAEP reading performance, which is cut and pasted from the “NAEP 2008 Trends in Academic Progress.”

On this graph, when scores for earlier years are statistically significantly different from the latest scores for 2008, those earlier year scores are tagged with an asterisk. If an earlier score lacks an asterisk, the score is not statistically significantly different from the latest, 2008 score.

Consider the Age 17 reading results; the scores from 1984 to 1992 are higher than the 2008 score, and each has an asterisk. That indicates these scores are higher than the 2008 score by a statistically significant amount (NAEP uses a 95 percent confidence test for statistical significance). Scores for 1980 all the way back to 1971 lack an asterisk, indicating these are basically tied with the 2008 score. Scores from 1994 to 2004 based on the original assessment format lack an asterisk, so they are just statistically tied with the 2008 score.

Note: Although the NAEP changed the assessment format somewhat in 2004, the people who administer the assessment claim the differences are not very significant and scores from the revised format can be compared to the original format scores.

So, in the early days of LTT NAEP reading, Age 17 reading did improve very slightly. However, it is clear that progress went flat around 1984 and stayed flat until recently, when a statistically significant DECLINE in Age 17 reading started. The most recent Age 17 reading scores are statistically significantly lower than scores for 1984 to 1992.

The bottom line: there is nothing to crow about in the Age 17 results, as a number of researchers have noted.

Now, check the Age 13 results. They have basically been flat since 1992, and the entire score gain over the 37 years shown is a measly five point rise. There is nothing to crow about here, either.

So, the modest improvement in the Age 9 results has never translated to upper age levels, a fact that concerns plenty of education researchers.

If you look at the math LTT NAEP results for Age 17 students, you would find a similar situation. The 2008 Age 17 LTT NAEP math score is not statistically different from the score way back in 1973, and the 2008 score is not statistically different from all scores posted since 1990. That’s nothing to crow about either.

Still, that didn’t stop some people from trying to fabricate a mountain out of the Mississippi Delta with this data. Several weeks ago, Education Week published a Commentary article titled, “Public Schools: Glass Half Full or Half Empty?”

The article and the paper upon which the article is based, “Restoring Faith in Public Education,” make incredible claims such as the educational progress on the LTT NAEP has been “Commendable.” Even our education commissioner got fooled.

However, the truth is the paper engages in some very poor analysis and cherry-picking of data.

You can read part of my reaction to this nonsense in my recently published letter to Education Week.

I also posted two rather extensive comments to Ed Week’s Commentary article. If you don’t want to check that out, just click the “Read more” link for a brief synopsis.