Will artificial intelligence ever write a novel? I believe the answer is “Yes.”
Using current technology we could use a text generator to spout out 50,000 words of sequential text and call it a “novel.” But only fans of stream of consciousness narrative would consider that a book. What’s generally meant by the question of whether or not AI can write a novel is if it can write a story with a coherent plot. I think that’s a certainty. Writing good novels might take more time, but I see that as likely as well.
I think what will happen first is that novelists will use AI tools to help do things like generate character names (as I’m already doing), proofreading and adding details. I actually have an upcoming novel with a passage that was written using OpenAI’s API. I’ve also built an app that turns conversations from my interactive fiction stories into narrative fiction.
The reason that people are skeptical about AI writing novels is because it’s a very hard problem to solve and requires AI to understand story structure. Even the most advanced AI models have trouble grasping concepts like plot. When you read a 500-word sample from a state of the art neural network it becomes clear that it’s trying to figure out what sentence comes next and not the story as a whole. It’s easy to walk away with the assumption that an AI just can’t “get it.” But the problem is much more basic than that and solvable.
When you train a text transformer you give it what’s called a “corpus” – a body of data to train on. This could be Wikipedia articles, Reddit discussions, news articles and public domain text. The model takes in samples that might be ten words or a few hundred. Think of a sample as one photograph. If you want a model to recognize cat photos you give it thousands of photos labeled “Cat” and thousands of photos of dogs and other things that aren’t cats labeled “Not a cat.” The model looks at each photo and learns to discern cats from non cats. Text generators work a little differently. It’s not looking at whole images of cats. It’s looking at smaller details like sentence structure, grammar, possessive objects, keywords. This would be the same as your cat detector only training on claws, whiskers and fur, but never a whole image of a cat. It could tell you what was cat-like but have trouble understanding what a cat actually looks like.
This limitation has to do with the processing power of computers. Each sample, whether it’s text or an image, is turned into an array of numbers called a “tensor.” Many of the image recognition models we use today were trained on tensors that were only 256×256 pixels which is about an 8k image. For comparison, one of my novels is about 500k in size. While that doesn’t sound like a lot, to create a neural network you have to hold thousands or even millions of samples in memory at a time.
For a text transformer to understand plot structure it would have to be trained on entire short stories or novels, which tend to be larger than the sample-size any contemporary model is capable of handling. If the model doesn’t know the killer is usually caught on page 390, it’ll never know to set up a red herring in chapter two.
Using a 400-page novel as a sample is a technical limitation for now, but one that will be soon achievable. The next step is to train a model on short stories, which is something that’s achievable right now. For fun, I built an app for my AI|Channels project that generates two-sentence horror stories. The results can be crazy random, but it’s come up with some really creep ones (although it played a little liberally with the “two-sentence” rule.)
“The sound of chainsaws fascinates me. I just can’t seem to look away. It’s like an itch I can’t scratch.”
“They say there’s someone for everyone. I hope that’s true. I’m tired of having all these imaginary people in my life.”
“I once wished that I could have kids. A week later I was arrested for kidnapping.”
While these can be hit or miss, the fact that the API is able to understand that the last sentence should be a pithy response shows that it gets the basic idea. Training it on longer examples would seem likely to show similar results.
I’ve seen OpenAI’s API pick up on patterns and create dialogue where characters have secrets and act out on things planted earlier on. I’ve also been able to get it to do longer narratives than the sample size by using a few tricks. This experience has shown me that as much as I want to believe my ability to write novels is a magical gift that will never be touched by technology, that’s naive.
People can argue that writing requires life experience, but if contemporary authors can write books about Ancient Greece or life on other planets, we’re probably overrating the value of life experience a little. Writing is about patterns. One of the ways you become a better writer is reading other books and discovering new patterns. Neural networks are amazing at recognizing patterns and it seems logical that at some point they’ll be writing novels.
Will they be good novels? Probably not at first. Humans write millions of books every year and most of them are of questionable quality. My first unpublished stories weren’t very good. Eventually they’ll get better and then they’ll get faster at getting better. In time I’m sure an AI will write a better Andrew Mayne novel than I can.
I don’t think this has to be a creative apocalypse. An Andrew Mayne novel a decade from now might mean a book where I chose the topic and then supervised an AI that created the book. In fact, our concept of a book may change as well. When I work with the AI as it creates the book it might also generate an audio drama and a possibly even computer generated movie with digital actors. A book could be a completely immersive experience that lets you explore off the page. I think AI is going to upend everything in entertainment and not just my little corner of publishing. But no matter how advanced the technology becomes I think we’re still going to want some of the things we enjoy and make us human to have also been made by humans.
@AndrewMayne 