Education Reads to Start Your Week, June 8th

Higher Ed Publishing Culture, AI Use and Perceptions, AI Writing Sucks

Author

James Shanahan
June 08, 2026

Hi! This is Scholastic Alchemy, a twice-weekly blog where I write about education and related topics. Wednesday posts are typically a deep dive into an education topic of my choosing and Mondays usually see me posting a selection of education links and some commentary about each. If Scholastic Alchemy had a thesis, I suppose it would go a little like this: We keep trying to induce educational gold from lead and it keeps not working but we keep on trying. My goal here is to talk about curriculum, instruction, policy, public opinion, and other topics in order to explain why I think we keep failing to produce this magical educational gold. If you find that at all interesting, please consider a paid subscription here, or at the parallel publishing spot on Beehiiv. (Some folks hate the ‘stack, I get it.) That said, all posts are going to remain free for the foreseeable future. Thanks for reading!

Higher Ed Publishing Culture

Today’s first link is to a TikTok from someone who actively publishes education research. Dr. Chung says that she thinks one of the reviewers for an article she submitted to a journal is AI and is asking her followers what they think she should do.

@drsamchung

Thoughts? #highered #edutok #academic

Now, there are several discussions that could be had here. Maybe AI does a good job reviewing articles? I have no idea. But it’s also not disclosed by the journal or the reviewer and Dr. Chung says that she expects a human reviewer, not an AI one. She wants to reach out to the journal editor but is a little worried about this. There are only a few comments but I want to highlight one of them.

I think a lot of people don’t want to believe that this is the case in higher ed. They want to believe that reviewers and writers are anonymous to each other and that journal editors would be above this kind of thing. They’re not. Dr. Chung and I sat next to each other as a panel of journal editors in our field explained to us that the research community is small, that reviewers usually do know the writer even when it’s supposedly “triple blinded,” and that journal editors need to be finessed and flattered. They flat out told us that poor behavior on the part of authors will get us desk rejected forever. Because early career scholars need to publish a lot, they are in no position to alienate reviewers and editors of major journals because those are the journals where publishing will make or break promotions and tenure.

Does that mean it’s the case at every journal or with every editor and reviewer? No, I’m sure it doesn’t. But how would you know, at the outset, whether you had someone receptive to pushback and questioning reviewers? Do you take the risk that you might get blacklisted from a major journal? It’s a huge dilemma and, frankly, I think it compromises the integrity of the research and publishing process. As I noted in a prior post, I have a lot of problems with higher education research, and this is definitely one of them. Doing research is hard enough. Getting published shouldn’t be easy or automatic, but it should be a process open to discussion and feedback instead of fear. The fact that Dr. Chung had to ask, tells you just how real that fear is.

AI Use and Perceptions

One of the interesting things about polling and surveying is that you end up with findings that show you gaps or problems with people’s perceptions. One famous example that’s lasted for literal decades comes from asking parents their perceptions of education in the US overall and their perceptions of their own children’s education. Invariably, a large majority of parents think that America’s education is bad but a similarly large majority think that their own children’s education is good. This cannot be a factual statement about the quality of education, but it tells interesting things about how parents perceive schools. They generally like where their kids go to school and their direct experiences with schools are usually positive but they live in an information environment that frequently tells them schools are bad so they assume that other schools must be the bad ones. Interestingly, gaps like these can also be a productive because researchers can build techniques to use the gap to better understand the underlying reality.

Speaking of surveys (and AI), some researchers at the University of Chicago surveyed some students about AI use. They were interested in understanding social desirability bias, a challenge for certain types of polling and survey research wherein participants lie about things in order to give an answer that they feel will be perceived positively by others.

Misreporting on sensitive topics due to social desirability bias is common in surveys, often driven by a desire to avoid embarrassment in the presence of an interviewer or to prevent potential repercussions from third parties [98]. This bias can manifest as an overreporting of socially favorable attitudes and behaviors, such as compliance with medical advice or adherence to safety measures, and an underreporting of less desirable ones [72]. Prior work has shown that social desirability bias may be particularly relevant in educational settings, where students are hesitant to state opinions that may be viewed as undesirable by others [15, 75]. Such bias can be measured through survey strategies such as indirect questioning, where individuals are asked about others rather than themselves regarding certain behaviors to elicit more truthful responses on sensitive topics [35]. Indirect questioning, as commonly used in social psychology research, reduces bias in reporting because individuals feel more comfortable attributing behaviors perceived as undesirable within their social group to others than themselves. The gap between self-reports under direct questioning and peer-reports under indirect questioning should reflect misreporting induced by social desirability bias.

The authors note that AI-use surveys don’t usually use mitigation techniques like indirect questioning, follow-up surveys, etc., and they have designed a series of surveys that leverage those techniques to better understand AI use among college students. They find a 40% gap between self-reported AI use and perceptions of others’ use of AI. The gap appears strongly related to negative stigma attached to using AI, indicating the possibility of pervasive underreporting. It also calls into question simple self-report findings that are typical of how we learn about AI use.

I have a few thoughts.

  • There’s a kind of collective action problem here and we’re not solving for the right equilibrium. We’re more or less incentivizing individuals to maximize their outcomes at the cost of the whole system’s efficacy at achieving stated goals. e.g. We’re getting kids degrees but the underlying learning and skills may not be there. The degrees are supposed to mean something, yeah? Easy to say, much harder to develop a policy and enforce it. I never said it was an easy problem!

  • If you look at what the students say about the beliefs and perceptions of AI use, it’s pretty damning. The “kids” hate AI even though they use it. They say that using AI signals weakness, laziness, inability to handle the work, and feels fundamentally dishonest. And then somewhere between 60% and 90% of them use it. I think that should tell us something important about the nature of AI and learning (see above).

  • If people think everyone else is doing something, it forms the basis of justification for their use. We’ve seen this in other areas like drug and alcohol use on college campuses. In those cases, changing perceptions around how many kids do drugs or drink excessively lowered use. Of course, one key difference here is that universities weren’t themselves tapping beer kegs in the dorms or passing out blow in the quad, but they are definitely incorporating AI products into their learning management systems. They have a lot more culpability here.

AI Writing Sucks

Jennifer Trainor shows us the feedback her writing students gave to the AI that gave them feedback and edits on their writing. As she says in the subtitle, “We have this supposedly world-changing technology, and everybody just kinda ... hates it?” That resonates with what we see above. What’s interesting, though is that Trainor tried to build out a process or a system to do what students say they want AI to do.

Students, too, are increasingly skeptical. In focus groups and surveys we’ve conducted at SF State this semester, a vibe shift seems to have occurred: students increasingly want the ability to opt out of AI assignments; they do not trust or like AI; they say it erases their voice, and they worry they are becoming overly reliant on it.

Rather than a synthetic writing generator, students tell us that they want AI tools that provide limited guidance without easy answers and editing support without voice-erasure.

She devised the “AI Glow-Up” whereby the students submit their authentic writing and let AI, a customized version of ChatGPT in this case, take that writing and produce an edited version under a prompt’s guidance to edit for clarity and flow. And, us, they didn’t like it.

when students experimented with an AI “glow up” for their own drafts, their evaluation of the output was almost uniformly negative. The majority reported that the AI revisions did not sound like them and erased their voice. Most rejected the revisions. A smaller group accepted the revisions but didn’t feel positive about the experience, noting that seeing the revisions made them feel less confident as writers and that ChatGPT added too much to their drafts. A few felt that the glow-up showed them new ways of phrasing their ideas and would be helpful in the future but didn’t say anything concrete about how the glow-up helped them learn. Almost all students expressed an uneasiness with the distance that seemed to exist between their drafts and the glow-ups.

Trainor has a really cool point here which is that the way this output just kind of arrives in a complete version as an output of the black box may be to blame.

In the glow-up activity, my students’ ideas were still there -- they wrote the original draft -- but they felt their voices were overtaken and the changes seemed fake. More to the point, the glow-up produced revisions students couldn’t read as revisions of their own work. The design of ChatGPT, its chatbot interface, rendered the glow-up as a fait accompli, with no affordances for writerly agency beyond continued prompting.

Agency is something I’ve been thinking about a lot lately. Part of learning may be that we need the ability to act on the world in some way in order to drive that learning home. With writing, students need to see how changes to syntax and tone shift meaning and make writing more impactful but simply depositing those changes via LLM may not work from a learning perspective because the students aren’t exercising writerly agency. They’re not using their voice, their ability as writers to make their own writing better and what we see is a lack of recognition on their part of the writing that the LLM produces. Moreover, it produces bad, bland, standardized writing. Trainor concludes:

AI seems to work best in my class as a foil —what students don’t, actually, want writing to sound like.

Thanks for Reading!