November 9, 2007

One way to measure educational performance

Although demographics obviously are the driving force in measures of student achievement, it is possible for one state to do a better job than another relative to what it has to work with in terms of student potential. One interesting way to analyze the value added performance of a state's public schools is to compare 8th grade scores versus 4th grade scores on the National Assessment of Educational Progress. If a state improves from 4th to 8th grade relative to the rest of the country, this could be evidence that it is doing a good job of schooling (at least in the middle years).

The NAEP is also given to 12th graders, but those score are distorted by the large number of dropouts.

There are lots of data on the NAEP site, so if anybody analyzes it, let me know.

My published articles are archived at iSteve.com -- Steve Sailer

15 comments:

Anonymous said...

This may be the most realistic approach since true double-blind studies would be impossibly unethical. One caveat though.

Can’t academic performance and even IQ testing be temporarily elevated in early life due to positive external efforts like being adopted into high-IQ, high achieving families, moving into high-IQ, high achieving communities or even participation in head start programs (although HS effects only persist a year or two with negative behavior side effects). Doesn’t an individual’s IQ, which correlates with academic performance and many other positive life outcomes, settle down to a largely genetically determined figure around 18 years old?

If so, wouldn’t relatively small black populations in white states like Minnesota show higher academic performance and IQ testing in the early years due to positive external environment which will fall with age until 18 years old? In contrast, large black populations in states like Louisiana lacking a positive external environment would trend toward the bottom of genetically determined performance where the negative effects of the black community become self-reinforcing?

These effects would skew your time-shifted self- and cross-comparisons. Perhaps it's a relatively small amount, but it's worth quantifying and taking into account in your proposed study. You suggestion is better than nothing other than the PC hot air which sets the policy today.

simon newman said...

Oregon shows improvement from 'red' to 'green' - unless it's just that Oregon has suffered a big demographic change recently such that their 8th grade cohort is much better than their 4th grade cohort.

Another point from the maps is that Texas appears to be doing something right educationally - a big southern state with lots of minorities, yet it's in the Green category for both 4th & 8th grade.

By contrast, my wife's home state of Tennessee does poorly - one of the whiter Southern states, yet well in the red on both 4th & 8th. While the recent Hispanic influx might be having an inpact, I suspect it's more to do with the American Scots-Irish's traditionally relaxed attitude to book learning.

Anonymous said...

The astounding acceptance of information provided by newspapers and journals instead of information provided by reputable researchers/scholars in the relevant fields has an uncanny parallel to the acceptance in high school and college debate of "evidence" that is not required to be based on science or research.
Debate evidence merely has to be published, and is usually drawn from newspapers because these sources are the easiest to search. I believe that this was not always the case, although I am not sure.

This was the case in federal courts as well until the Daubert Supreme Court decision in 1993 requiring judges to act as gatekeepers with respect to expert witness testimony. This decision has only recently trickled down into the consciousness of state courts.

Hans Gruber said...

Is the Southwest becoming the educational twin to the Deep South?

http://preview.tinyurl.com/yqu3w5

An outlier seems to be Texas, which scores in the upper third of states overall for math in 8th grade. If you sort by Hispanic you will also see Texas Hispanics score the best in the country. Any theories?

Parentalcation said...

I love the NAEP website.

The most interesting thing I found was that children of blacks and hispanic families with "some education after High School" consistently score higher than children of blacks and hispanics who "graduated college".

I blogged about it and have a screenshot of the results over at Parentalcation

I googled for any other studies on the phenomena, but didn't find any.

Parentalcation said...

Text Savvy followed up on the data over here

Audacious Epigone said...

I went ahead and gave it a shot using math scores.

# 2 pencil said...

"Another point from the maps is that Texas appears to be doing something right educationally - a big southern state with lots of minorities, yet it's in the Green category for both 4th & 8th grade."

U.T., A&M and private universities like Baylor all have excellent teacher education programs. Also, Texas public schools take standardized tests seriously. Teachers are required to give practice sessions for a certain amount of time before the actual test. Some even complain that much of their class time is consumed with teaching to the tests. And it can be monotonous. How many times in one day can a teacher endure reminding students that they must fill in the bubble or whatever completely in order for their answer to be scored correctly.

The good thing about national standards, however, is that they prevent the left-wingnuts from completely destroying the minds of the children in their care. I thought the test would resemble the Texas test that I'm somewhat familiar with but after looking at the sample history questions, the NAEP seems to be very respectable.

I also like Sailer's idea of comparing the performance of a cohort from 4th to 8th grade. Since nationwide standardized scores are available, you might as well use them. There may also be an opportunity to avoid redundancy in testing. Here's hoping they save some money and scrap the state tests like the mindless Texas TASP.

Steve Sailer said...

Sorry, I accidentally lost somebody's question about why Massachusetts does so well.

Well, it's been the center of higher education for 370 years, so that probably has something to do with it.

Mike said...

Gents, I went to the NAEP report card and looked at the various categories.

Putting it bluntly, anyone who thinks that Texas is better than Wisconsin, North Dakota, Minnesota, Vermont and New Hampshire in teaching white 8th graders mathematics, and is only JUST behind Massachusetts in 2nd place is a fool.

I don't know how they are rigging the game, I don't know what their score inflation tools are, but they are cooking them somehow.

And I say this as a proud descendant of Texas revolutionaries going back to 1811 and before.

Didn't Rod Paige start the score diddling in Houston? Has it spread statewide?

TabooTruth said...

But what about the different levels of heritability of intelligence between 4th grade and 8th grade? A fourth grader's test score may be artificially inflateted by an enriching environment at the age of 9, but by the age of 13, couldn't the genes have kicked in more and brought down the potential IQ, thereby casting unfair blame on the school for not allowing the student to fulfill his (inaccurate) 4th grade IQ?

mercer said...

"Oregon shows improvement from 'red' to 'green' - unless it's just that Oregon has suffered a big demographic change recently such that their 8th grade cohort is much better than their 4th grade cohort."

This shift is more likely than not at least partialy responsible based on conversations I have had with elementary school teachers. Many parts of the state are currently undergoing large scale enrichment from points south with uncomfortable consequences for the teaching profession. It is hard not to hear the complaints of longtime advocates of mass immigration and multiculturalism without laughing. Even more amusing is listening to them try to express reality in a coherent fashion and remain fathful to PC taboo. It's a hard fall back to reality for the poor dears.

Anonymous said...

It would be better if, for example, the 8th grade in 2007 were compared to the 4th grade in 2003 rather than the 4th grade in 2007. Even so, the 4th to 8th grade comparison shouldn't be made at all. Educational Testing Service (ETS) is the private-sector statistical corporation with which the U.S. Department of Education contracts to conduct the data analysis and reporting of NAEP results. Several years ago ETS issued a report based on 4th to 8th grade comparisons, and it took some hits for it. Today ETS strongly advises against making the 4th to 8th grade comparsons.

Mark said...

Many parts of the state are currently undergoing large scale enrichment from points south with uncomfortable consequences for the teaching profession.

How rapidly this transformation can take hold is breathtaking. The Derb recently linked to the demographic data of schools in Storm Lake, Iowa. The breakdown: high school 43% hispanic, middle school 53% hispanic, and elementary schools about 63% hispanic. Roughly every 4 years, in other words, the percent of the town's population that was hispanic was increasing by 10%.

Now granted this was a town, and states don't change that fast. But it's still amazing (in a horrible way) how rapidly immigration is affecting demographics.

Jack said...

An anecdote about Oregon's "demographic shift": We here in the Beaver State are currently being favored by Mexican colonization (which of course I celebrate daily). Given the relative youth of the Mexican colonists and their exceptionally high birth rates, this colonization is most strongly felt in the K-12 school system. My aunt, who's worked in Oregon's schools since the 1980s, recently retired early because she couldn't take the disciplinary problems, er, "cultural diversity" associated with this demographic change. One can only take being called various sundry profanities and having to break up fights on a daily basis before one tires of the game. And being a monolingual Anglophone doesn't help. We are all expected to adopt the colonists' language. Much easier than getting the colonists to learn our soon-to-be-obsolete language.

Whether this has anything to do with the education data cited above, I have no idea. Having grown up in southern Arizona myself, I do know that education is not a high priority for a large proportion of the Mexican-origin population.

I would also echo the other comments about IQ and variability by age. Steve knows the literature better than I, but I do recall that environmental controls on IQ are less important as children age. This could possibly confound any attempt to measure the performance of schools. Gains made early in life due to good schooling may be erased as certain populations' genetic potential (or lack thereof) comes to the forefront.