The Ghostwriter in the Machine, LLM Author Psychosis, and Other Saturday Sundries

Some more AI thoughts and other odds and ends

This is going to be a more scattered post than usual as I’ve spent recent weeks in the novel mines finishing up my forthcoming haunted houses (yes plural) novel Haunted Hills. I plan to write more about that soon. For now, some other things:

Two Anthologies To Look Out For

Despite being deep in the novel mines for the last few months, I did manage to emerge and finish two short stories that are coming out in two very different yet equally cool anthologies.

The first anthology is Thirty Cabins edited by Josh Riedel, which is organized around an old 1918 cabin catalog called Sunset’s Cabin Plan Book. Each author was asked to write a flash fiction piece based on one of the cabin plans and other chapters (my story is based on the “cabin miscellany” decoration section). I love a playful Oulipian constraint like this, and the anthology has a great line-up of writers, many of whom are here on Substack such as Ed Park, a. natasha joukovsky, and Rachel Khong. The zine should be out later this summer. There is also a Thirty Cabins Substack newsletter for the curious.

The second anthology is the aptly named The Speculative Detective Agency edited by Matthew David Goodwin and Richie Narvaez. As the name implies, the anthology collects detective stories with science-fiction, fantasy, or horror elements. The anthology has a shared universe with characters recurring between stories. I had a lot of fun writing my entry. (If you’ve read my first novel, The Body Scout, then you know I enjoy both Raymond Chandleresque hardboiled narrators and weird speculative concepts.) My entry deals with LLMs and black magic. I suppose I shouldn’t say anything more.

This anthology also has a great line-up (Monique Laban, Amber Sparks, Karlo Yeager Rodriguez, Alex Grecian, Megan Chee, and more) and will be published by Diversion Books in October. Pre-orders are open if you’re interested.

A New LLM Author Psychosis?

It’s hard to keep up with the multiplying publishing AI scandals, but I’ve become a little fascinated with one. Not the literally nonsensical “The Serpent in the Grove” story that won a Commonwealth Short Story prize, nor the Shy Girl self-pub-to-trade-pub horror novel, nor the freelance book reviewer who admitted to AI plagiarism, nor the temper tantrums some (mostly STEM) professors have thrown over arXiv imposing temporary bans on academics who submit papers with AI errors. No, what I’ve been following is the flailing of Steven Rosenbaum after his (ironically titled) The Future of Truth: How AI Reshapes Reality was found to contain fabricated or misattributed quotations.

Rosenbaum admits he composed the book with extensive collaboration with LLMs, which he blames for the errors. While it would seem pretty easy for an author to simply say “my bad,” Rosenbaum has chosen a different route: a media complaint tour. Talking to WIRED, New York, The Atlantic, and more, Rosenbaum has begun insisting that it’s unfair he is criticized for making basic errors and not checking his work. Indeed, he claims, his book has fewer errors than most: “it’s five, and of the five, one is a citation error. So it’s four, which my guess in nonfiction is below the average.” He also insists that everyone else, including presumably anyone critiquing him, is secretly using AI to write their work anyway. He told WIRED “82 percent of journalists are using AI every day” and New York that “[a]nyone who is a working writer today […] you’re using AI.” Indeed, even asking him about his LLM use is “like saying, do you beat your wife? It’s one of those accusations that there’s no response to.”

Rosenbaum’s defenses are as sloppy as his quote checking. His claim that 82% of journalists use AI “every day” is hallucinated from a study that found 82% of journalists surveyed have used some form of AI at least once “in the past 12 months.” That study included any use from spellcheck to interview transcription (which responsible journalists check against the audio files) and in no way implies most journalists are using LLMs in the way Rosenbaum does. I also doubt it is true most non-fiction books have more than four hallucinated quotations. While it might be true that there are four errors of some kind in many books, we certainly have no reason to believe the fake quotes are the only errors in Rosenbaum’s book. They were just the easiest to catch.

The strangest part of Rosenbaum’s various interviews was when he told Kate Knibbs at WIRED that he would literally never write again if he had to write without LLMs:

He doubled down on his personal commitment to AI, noting that he still uses it every day. “If the only way for me to not end up with a mistake ever again is to literally stop using AI, that’s just not realistic. If the answer is to stop writing, that’s not out of the realm of possibility.”

I asked him whether he would rather stop writing than stop using AI in his writing process. “Yeah,” he answered.

This is an odd new form of AI psychosis. Rosenbaum is in his mid-60s, and was presumably capable of writing without LLMs for many decades. Using them has damaged the basic credibility of his work. Yet he is already so dependent on LLMs that he cannot even conceive of writing without them…

This is an example of why it’s hard to classify LLMs as merely another “tool” for artists. They can be, certainly, but imagine a professional athlete saying “if I can’t wear the new Nike sneakers, I’d rather never play again,” an established painter saying they’d give up art if they couldn’t use the latest paintbrush, or for that matter an author saying “if I can’t use Google Docs to write, there is no point in writing.”

On the other hand, I could imagine an author who cannot write their own work admitting they would never write again if they couldn’t employ ghostwriters…

The Ghostwriter in the Machine

There’s another aspect of Rosenbaum’s scandal that fascinates me. He seems incapable of understanding that by letting LLMs ghostwrite parts of his work—he’s extremely cagey about the degree—he has in fact not written the work entirely himself. This is something I’ve noticed in many instances of AI use. Authors want to believe that the work is entirely theirs even when they did not write the entire work.

(Or at least the work is theirs when things go well. When things go bad, they have no relationship to the work published under their name. See, for example, Mia Ballard blaming the Shy Girl AI scandal on a friend she hired to edit who “changed a lot of the wordings” and now “my name is ruined for something I didn’t even personally do.”)

Rosenbaum claims that LLMs “hallucinated” his reporting, changing the words and meaning. Yet he also insists “the ideas, reporting, arguments, and final authorship are mine.” In Ars Technica, he says something similar: the “actual reporting, narrative structure, interviews, arguments, and conclusions… [are] entirely mine… There was never a time when AI was writing the book.” These statements are obviously false. If AI generated quotes that were published in the book, then LLMs did write parts of the book. If LLMs changed your reporting, then your book is (at least in part) not your reporting. 2 + 2 still equals 4, even if Claude hallucinates 5.

It reminds me of the first time I caught an undergraduate student using LLMs. This was back when LLM cheating was rare instead of routine. The student had submitted an essay comparing themes in two different poems, but the poems were entirely hallucinated. The poets named were real poets, but the titles and every single line had been invented by an LLM. (Ironically, I caught the paper because TurnItIn—which traditionally caught plagiarism by flagging when text has been copied from elsewhere—had not flagged anything in the paper, including the quotes.)

How do you grade the uncanny valley? I was reading imaginary “analysis” of poems that did not exist. Hallucinated claims about non-existent poems.

The student, like Rosenbaum, admitted to using ChatGPT yet thought they should get credit for the arguments and the work. They had only let an LLM rewrite the text to fix grammar and spelling errors. The paper was still theirs. Except the paper, in a very literal sense, no longer represented their thoughts on the poems because they had not read these poems. The poems didn’t exist before the LLM hallucinated the lines. Plus, the LLM had changed the meaning of their sentences just as it changed the lines of the poem.

There are legitimate uses for LLMs in art and journalism. Good work can be made with the help of programs. But writers who use LLMs to generate some or all of their work should be honest about what they are doing. They are at a minimum collaborating with another “author” and in many cases handing their work over to a digital ghostwriter. The ghostwriter isn’t exorcised merely because it’s a machine instead of a man. Writers who want to collaborate with LLMs should make peace with this fact. There’s nothing wrong with collaborative writing. In fact, I’d say the most interesting LLM-assisted works published so far have all been open collaborations and leaned into the merging of program and human. The problem comes in hiding your use. In refusing to be honest—even to yourself—about what it is that you are doing.

When does an LLM cross the line from tool to ghostwriter? That’s a fair question. It may take some time to sort out. But I think there’s an easy rule of thumb:

A Simple Rule for Using AI in Art

Saturday Reading Links

I’m going to end with a little link round-up here, including several interesting essays on AI.

In the New Yorker, Jay Caspian Kang wrote about a test he developed to have readers guess if paragraphs were human-authored or LLM-authored. Despite people missing about half the time, Kang talks about why human authorship still matters. He also noted a stylistic tic that I’ve also seen in LLM works: the characters don’t do anything:

A.I. also had a weird habit of making its characters fidget constantly, always running a finger along the edge of a table or adjusting a collar. The most reliable marker, though, was something more abstract, and, I suppose, upon reflection, even a little spooky. The scenes generated by A.I. had characters, but, apart from fidgeting, they mostly did nothing.

In The Atlantic, Ted Chiang wrote about Anthropic’s Claude constitution and why “No, Artificial Intelligence Is Not Conscious”:

What would it take to convince me that a computer program is actually conscious and using language the way that people use language? Let me offer an analogy. If tomorrow someone showed me a video of an astronaut in a spaceship orbiting Alpha Centauri, a star that’s 4.3 light-years from Earth, what would I have to see in that video to convince me that it was real? My answer to that is, there is nothing in the video itself that would convince me. No matter how high the video resolution is or how realistic the scenery is, I would feel confident in saying that the video is fake. I won’t pay attention to any video of an astronaut orbiting Alpha Centauri unless I have previously seen good evidence that astronauts have landed on Mars, that astronauts have reached the moons of Jupiter, that astronauts have reached the moons of Saturn, and that astronauts have crossed the orbit of Pluto. Before anyone can credibly claim that they’ve solved an extraordinarily difficult engineering problem, I need to be confident that they have previously solved the many much simpler problems that precede the difficult problem.

Sam Kriss had a great piece / rant on AI here on Substack that pointed out that a world of LLM text is bad regardless of the quality of LLM text:

AI is a bad writer, but that’s not even close to being the whole problem. Let’s say it wasn’t. Let’s say they finally fixed the machine so it was really good, so its default setting was to write exactly like VS Naipaul. The result would be a world in which you’re constantly confronted by cold emails from VS Naipaul, bubbly magazine articles by VS Naipaul, signs in shop windows in which VS Naipaul tells you about the new opening hours, strangely flaccid sexts VS Naipaul ghostwrote for someone on Feeld, and websites in which VS Naipaul fails to say anything in particular about grilled meats. This would not be an improvement; it might even be worse. Any world in which there is only one literary voice, blanketing everything in the exact same tone, is a nightmare.

In the past couple of weeks, two authors I interviewed last year for my “Processing” author interview series won awards. Helen Phillips won the Climate Fiction Prize for Hum and Julia Elliott won the Carol Shields Prize for Fiction for her story collection Hellions. Many congrats to Phillips and Elliott! While these awards have nothing to do with my interviews, it seems a fine excuse to repost them:

Processing: How Helen Phillips Wrote Hum

Processing: How Julia Elliott Wrote Hellions

Lastly, I reviewed Deb Olin Unferth’s lovely and wild science-fiction novel Earth 7 for the New York Times.

---

My new novel Metallic Realms is out in stores! Reviews have called the book “brilliant” (Esquire), “riveting” (Publishers Weekly), “hilariously clever” (Elle), “a total blast” (Chicago Tribune), “unrelentingly smart and inventive” (Locus), and “just plain wonderful” (Booklist). My previous books are the science fiction noir novel The Body Scout and the genre-bending story collection Upright Beasts. If you enjoy this newsletter, perhaps you’ll enjoy one or more of those books too.