About the only thing being stimulated by the “stimulus funding” is taxpayers’ anger.
Click here to listen to the 90-second audio commentary.
Here are some links for this Thursday! Enjoy!
I raised some questions yesterday about Kentucky’s fourth grade reading performance in the newly released 2009 National Assessment of Educational Progress (NAEP) Reading Report Card.
Today, I’m going to point out some more issues, this time focused on the state’s eighth grade results.
This graph shows results from the two most recent NAEP eighth grade reading assessments and the corresponding test results from Kentucky’s 100 percent testing of all eighth grade students with the EXPLORE Reading Assessment from the ACT, Incorporated.
In sharp contrast, Kentucky’s reading performance on the NAEP experienced a very notable five point rise in the same time interval.
Very simply, these notably different results raise questions about the 2009 NAEP scoring.
It should be pointed out that the EXPLORE test frameworks and scoring did not change, to my knowledge, during the time interval shown in the graph.
In sharp contrast, the NAEP reading assessment was changed in 2009 as explained by this quote from the 2009 NAEP Reading Report Card:
“The Reading Framework for the 2009 National Assessment of Educational Progress replaces the framework first used for the 1992 reading assessment and then for subsequent reading assessments through 2007. Compared to the previous framework, the 2009 reading framework includes more emphasis on literary and informational texts, a redefinition of reading cognitive processes, a new systematic assessment of vocabulary knowledge, and the addition of poetry to grade 4” (Page 4).
While the 2009 NAEP Reading Report Card goes on to claim that the 2009 results can be compared to prior year’s results, the Kentucky EXPLORE and NAEP comparison shown in the graph above raises questions about that assertion.
Clearly, the disparity between Kentucky’s NAEP and EXPLORE reading performance invites further investigation of the 2009 NAEP scoring. The situation also raises concerns about the validity of the 2009 rise in Kentucky’s NAEP scores in grade eight.
Furthermore, if the NAEP frameworks were changed in ways that impacted the trend lines for eighth grade, a similar problem could exist in grade four, as well. Unfortunately, ACT does not offer a companion EPAS test for that level.
The data shown in the graph for each test year is for the same cohort of students. Kentucky conducts EXPLORE testing in the fall of each academic year and the NAEP conducts its testing a few months later in late winter. Thus, it should be understood that the data in the graph above for 2006-07 and for 2008-09 are both for the same cohort of eighth grade students. NAEP didn’t conduct testing in the intervening school term.
It should also be emphasized that both the NAEP and the EXPLORE tests are controlled and scored by organizations outside of Kentucky. However, Kentucky does have some influence on the results from both tests because it has control on the use of testing accommodations. This results in some skewing of the results for both assessments.
For example, while few Kentucky students are excluded from EXPLORE, an appreciable proportion of the state’s students with learning disabilities receive test accommodations on EXPLORE that the ACT Incorporated does not allow on any ACT tests taken for college application purposes.
Kentucky’s accommodations program also impacts NAEP results, but in a somewhat different way. An appreciable proportion of Kentucky’s learning disabled students (48 percent in 2009 in grade four and 55 percent in grade 8) are completely excluded from the NAEP due to state requirements for incompatible testing accommodations that the NAEP will not allow. In fact, Kentucky’s learning disabled exclusion rate for NAEP reading in both fourth and eighth grade is among the very highest in the nation. Only two states had higher exclusion proportions in the 2009 eighth grade NAEP reading assessment.
This exclusion issue is largely due to a very controversial accommodation allowed in Kentucky – many of the state’s learning disabled students actually have all “reading” tests read to them. This totally changes the construct of the test from printed text decoding and comprehension to one of spoken word comprehension. Neither the ACT nor the NAEP will allow this clearly inappropriate accommodation on tests taken for record although EXPLORE does report scores for students tested this way for Kentucky’s internal state uses only.
Find more information:
While many people are somewhat familiar with the NAEP, fewer probably know about the EXPLORE. Briefly, EXPLORE was developed by the ACT as a part of a coordinated college preparation testing program know as the EPAS system.
Kentucky tests all its students with EXPLORE in eight grade, the PLAN in tenth grade and then conducts 100 percent testing of all public school eleventh grade students with the ACT college entrance test.
Each of the EPAS tests has a set of “Benchmark” scores for the subjects of English, math, reading and science which are linked to an empirical study of how well ACT college entrance test scores compare to actual success in the first related college course. Students who score at or above the Benchmarks have a 75 percent chance of earning a “C” and a 50 percent chance they will get a “B” in those freshman college courses.
Learn more about the Benchmark Scores here.
ACT discusses the EPAS system here.
Data sources used to assemble the graph:
2009 NAEP Reading Report Card, Table A-19
EXPLORE Excel Spreadsheet, “EXPLORE_Benchmark_06070910.xls” from Kentucky Department of Education.
Maybe too good
The new National Assessment of Educational Progress results for the 2009 reading assessment were released today, and Kentucky’s overall performance really looks good.
We were the only state in the nation to post a statistically significant score increase in both fourth and eighth grade (NAEP does not report on state high school reading performance).
But, the results almost look TOO good.
For example, this map, found on page 13 in the new NAEP Reading Report Card, shows only one other state, Rhode Island, also made statistically significant progress in fourth grade reading since 2007.
But, hold on. All states have been pushing reading hard since No Child Left Behind was enacted in 2001. Can it really be that hardly any of them are making progress?
Furthermore, this table, which is extracted from a larger table found on page 14 in the Report Card, shows something that seems to conflict with what the map shows for Kentucky.
How is that possible?
Notice that in both Washington DC and Rhode Island, the only other places where reading progress supposedly was made, at least one racial group made progress. Not so in Kentucky. Strange, that.
The answer here may be due to some statistical sampling issues. Those statistical rules require larger score changes before statistical significance can be established when sampled groups are smaller.
However, with whites comprising 84 percent of the 2009 sample tested in Kentucky (Report Card, Page 52), it seems like if the overall scores made a statistically significant rise, then the whites should have a statistically detectable increase as well.
The fact that this didn’t happen makes me wonder if something unusual is happening in this NAEP scores report.
There is another curious thing. Overall, Kentucky’s average fourth grade reading proficiency rate on the NAEP was reported as 36 percent, while the national average proficiency rate was only 32 percent. However, when the disaggregated proficiency rates for whites and blacks are examined, things look quite different. Both Kentucky’s whites and blacks scored LOWER than their peers across the nation. Go figure!
For example, our exclusion rate for students with learning disabilities, once again, was well above the national average. That tends to inflate our scores because these particular students can be expected to score very low if they were to take the NAEP. Their exclusion abnormally raises our scores compared to other states.
In Kentucky, fully seven percent of all the students the NAEP wanted to test were ultimately excluded because they were determined to be too disabled to sit for a reading test. Across the nation, the exclusion rate for learning disabled students was only four percent, a decrease of one point from 2007. Kentucky’s exclusion rate was seven percent in 2007, as well.
Anyway, NAEP analysis has gotten quite involved, and simplistic examination of overall scores which some others like to engage in can be highly misleading. I’ll be looking at this some more, so stay tuned.
Just where does the fed appetite to control and take over our private sector end? Not at your computer, apparently.
A long-awaited plan on broadband Internet connections announced March 15 by the Federal Communications Commission signals government interference in the private sector is about to grow.
President Obama now appears poised to apply his spread-the-wealth philosophy in ways that will create yet another bureaucracy to “redistribute” our Internet capability.
Want to pay $56,000 per home to ensure that folks without broadband can get broadband that’s as fast as the FCC’s broadband planners have decided they deserve?
The government announcement sounds innocent.
But not everyone agrees.
According to the plan’s executive summary, big government control — instead of freedom and entrepreneurship — will be stimulated.
The Internet Freedom Coalition has credible perspective to counter the innocent government spin.
Jerry Ellig, a senior research fellow at the Mercatus Center at George Mason University, challenges that providing Internet access to Americans not currently online can be accomplished without a broken bank or government takeover.
This is nothing more than another federal initiative being downloaded to strike at the heart of more Kentucky freedoms and businesses.
The Bluegrass Institute for Public Policy Solutions works with Kentuckians, pro-liberty coalitions, grassroots organizations and business owners to advance freedom and prosperity by promoting free-market capitalism, individual liberty and transparent government. Join Us