Education Reads to Start Your Week, March 23rd

Hi! This is Scholastic Alchemy, a twice-weekly blog where I write about education and related topics. Mondays usually see me posting a selection of education links and some commentary about each and Wednesday posts are typically a deep dive into an education topic of my choosing. If Scholastic Alchemy had a thesis, I suppose it would go a little like this: We keep trying to induce educational gold from lead and it keeps not working but we keep on trying. My goal here is to talk about curriculum, instruction, policy, public opinion, and other topics in order to explain why I think we keep failing to produce this magical educational gold. If you find that at all interesting, please consider a paid subscription here, or at the parallel publishing spot on Beehiiv. (Some folks hate the ‘stack, I get it.) That said, all posts are going to remain free for the foreseeable future. Thanks for reading!

Norms, Standards, and School Policy

The 74, an outlet funded largely by mid-aughts school reform types, has an article reporting on a paper about the growing gap between the 90th percentile of students and the 10th percentile of students. The general gist of the problem is that kids ranked in the bottom 10th percentile of students (90% of students do better than these kids) are losing ground on the NAEP as the years go by while the top 10% of students (90% of students do worse than these kids) are improving. One of the interesting things about this data is that they break out charter schools, Catholic schools (that choose to participate in NAEP), and the Department of Defense Schools. The 74 brings us this graph.

You can read the whole paper here. One thing that stands out to me is that the authors of the paper devote a section in the discussion to call out teachers’ unions and public health officials for school closures during the COVID pandemic. This is sort of core to their overall point, because they find that that score declines for the bottom 10% worsened post-pandemic, especially in public schools.

The thing is, I’m not sure their analysis is sensitive to differences in Covid responses across different school jurisdictions. Despite talking about unions, there is no comparison of NAEP scores between states with strong unions, states with unions that can’t collectively bargain, and states without teachers’ unions. They simply lump all traditional public schools together as unionized. Likewise, they assume that schools everywhere were closed in similar ways, when the reality had reopenings all over the place. I’m not being unfair here, it’s how they describe their calculations.

The pandemic for them becomes a simple pre- and post- line they draw and form a comparison of scores across either side. If you’re familiar with the NAEP, you will know that the score reports don’t include granular enough data to drill down by district or form cross-jurisdictional comparisons. You might say, James, this is silly, they can’t tell you what you want them to tell you because their data set doesn’t include that kind of information. You’re right! But that statement cuts both ways. There’s not enough data here to make any claims about public health measures or school unionization’s impact on NAEP scores. It is inappropriate for them to argue that their data shows negative impacts of unions and public health measures. I’m not sure why this jab is here except to offer up some grist for the anti-union mill. Covid clearly made things worse but that’s kind of all you can say. Kids in Texas were back in school by Sept 2020. Kids in New Jersey were back literally two years later than that. They don’t get to be in the same comparative bucket.

Similarly, the authors recommend a bunch of policies that they don’t actually examine in the paper.

So, to be clear, public health and unions: bad. Science of Reading, high-dose tutoring, and extended learning time: good. We know this not because of work the authors did in this paper, but because they cited other peoples’ work who make these claims. What, then, is the purpose of this paper. How does this analysis of NAEP scores add anything new to our understanding of learning outcomes? How does this analysis of NAEP scores imply, for example, that the Science of Reading is going to boost NAEP scores? I’m not objecting to the veracity of these claims so much as saying that the authors didn’t do any work justify or support these claims. I’m left not really understating what the point of the paper is.

But, we have to recognize that there is a game being played here, too. Peter Greene recently republished a decade-old essay that touches on the challenges of policy discussions that confuse standards-referenced tests like the NAEP with norm-referenced tests, like the SAT or, appropriate for the paper and article above, percentile distributions.

Greene is largely critical of using normed tests to compare across years of test takers or between meaningfully different groups of students. Moreover, because the process of norming requires fitting kids to a bell-curve, some kids have to fail in order to create the normal distribution.

Now, the NAEP has none of these problems because it’s a standardized test. Passing and failure are possible for all kids, that how we can measure changes in achievement across various groups. Kids’ performance on the NAEP doesn’t depend on the performance of all other kids taking it. Unless, of course, fancy statisticians decide they need to norm the NAEP data in order to make some kind of argument about the relative performance of groups taking the test. For Greene, this presents some problems. He uses the example of what it might mean to read “at grade level”.

So, I’m not exactly sure what to make of the paper. I think the headline data is right and kids performance on the NAEP has been somewhat worse over the last two decades with a big post-covid decline. None of that, though, is new information. That we can see smaller declines in Carters and Catholic schools is interesting, too. After all, they can benefit from selection bias to construct a group of testers more apt to perform well on the NAEP. I they also saw declines, then it shows that something bigger than just public-school policy is going on.

Still, drawing any meaningful conclusions from this study is hard. Certainly some of their implications don’t seem to stem from the data but from prior commitments of the authors. Unfortunately, their proposed solutions seem to get picked up in the education media and reported out uncritically. Unfortunately, this also drives school policy. And we wonder why reforms don’t often work out the way we intend!

Anti-ready

The Curriculum Insight Project continues its fight against EdTech with a post about the problems with iReady.

Now, I don’t have a ton to add here. I’m generally an edTech skeptic and have seen schools and districts waste time and money endlessly pursuing technological fixes when they could be building out better supports for teachers and students. Rather, I think this is an important data point in the ongoing backlash to edtech we’re seeing. Parents are frustrated, kids are unhappy, and teachers are reporting lots of negatives. We should take this stuff seriously! Rather than rehash old points or try to jump on with the linked post, I want to pivot to one of the less talked-about problems with edtech, and that’s a lack of ongoing support.

Three weeks ago, Mark Watkins attended OpenAI’s 2026 education summit. He has a lot to say so definitely read the whole post but one thing he pointed out is that it’s hopelessly naive to assume that students will limit themselves to the educational versions of AI products. OpenAI, however, didn’t seem to discuss that very much.

Still, Mark came away generally enthusiastic about some of the other tools they showed off, especially some of the things they are building to support teachers. It’s too bad, then, that OpenAI is probably not going to put much time and energy into education anymore. The Wall St. Journal reports that OpenAI held an emergency all-hands meeting last month where they previewed a major reorganization of the company toward a focus on business and productivity product lines.

I’m sure some of those productivity and business “quests” will provide some value for education, but it’s clear that schools won’t be a major focus for the company anymore. And that, friends, is the problem with education technology. If you’re a school district that was persuaded by slick presentations at the 2026 education summit, how do you feel right now? If you ponied up millions of dollars for district-wide access to products that might be abandonware, is there some audit or accountability for this decision to spend taxpayer money? Sorry guys, kids’ learning is a “side quest” and we’re not doing those anymore.

So, when I read about iReady or any other edTech, one thing I keep in mind is that at any point the company might just decide to stop supporting the product and leave you hanging. Better to slowly and purposefully develop the capacity to do these things yourselves, at least at the state level.

And that’s more than 2700 words with just 2 things to go read, so maybe let’s stop there. No. Wait. There’s more.

Math Standards Battles

Steven Yoder at The Hechinger Report, uh, reports on conservatives’ efforts to simplify mathematics standards in South Dakota and, potentially, nationwide. Much of this is downstream from the anti-Common Core fervor of a decade ago, now reinvigorated by today’s culture wars. At the center of this effort is a right-wing organization, The National Association of Scholars.

Yoder provides a helpful example of what these changes look like in practice.

They certainly are simple. Oddly, this puts a lot more emphasis on the teachers’ professional judgement as to how to meet these standards. That said, if South Dakota is doing any kind of standardized testing, it’s going to be difficult to know what will be on the tests from the language of the standards. This will mean teachers may not adequately teach skills and concepts that will be tested. Maybe that’s the point?

Okay, now we’re done! Thanks for reading and have a good week.