Will AI Replace Copywriters? The Evolution and Future of AI Content Writing

As text generation and machine learning advances, there's a lot of talk about whether content writers will be replaced by the myriad of AI copywriting tools that offer autocomplete functionality.

GPT-3, Rytr, Jarvis, Shortly, CopyAI, Frase, etc. - the list of new products with autocomplete popping up on the market is endless. All these products feature a "Write for me" button. You click that button, and a paragraph of text comes out.

It almost feels like magic.

As an artificial intelligence researcher, I've been following breakthroughs in this area for the last 5 years - and it's fascinating how something that started as a way to autocomplete your text messages turned into a technology that can now almost produce entire novels.

But does that mean AI can replace copywriters - is human writing obsolete? ‍The answer is complicated.

To answer that question, we first have to peel back the curtain to see how autocomplete for content writing really works. Armed with that information, we can dig into the trade-offs this artificial intelligence technology brings, and figure out if copywriters have anything to worry about with all these AI writing tools on the market.

How Autocomplete Evolved for Content Writing

With all the hype around new artificial intelligence breakthroughs like GPT-3, it is easy to forget how predictive text technology (autocomplete and autocorrect) has evolved throughout the decades.

Yes, this is the feature on your old iPhone 5 that autocorrects you from "iz" to "is", and it's also how Google suggests you (sometimes hilarious) completions for your search.

Counting on N-Grams to Write For You

You may be surprised, but autocomplete technology can be traced way back to 1948. Since then, it has helped content writers spellcheck and correct their writing.

Let's take a look at autocomplete's humble origins.

Many early autocomplete systems were based on the concept of a language model. This was basically a model that can predict the next word based on a history of words.

The earliest language model, first referenced by Claude Shannon, was called the n-grams model. One application of n-grams was to predict how likely it is for a set of words to appear in the text you are typing.

So for example, if you type "Can you please come" (the history words), the n-grams model will predict that the next word "here" as highly likely (for example, 80% chance). Your phone will then offer to autocomplete your phrase with the word "here".

How do n-grams know which words are likely?

You can create an n-grams model by simply counting the number of times the phrase "Can you please come here" appeared in a corpus of high-quality text (also called the training data). If this phrase appears a lot, it means "here" is likely to be a correct completion - otherwise, it is an unlikely phrasing.

This occurrence count is applied to all possible phrases in the corpus, and this results in a "table of counts".

In the example above, the first row has the highest count because it occurs in the human written language most often. The last row is written in esoteric English and doesn't occur much in modern language often, so it has a lower count.

With this table, whenever someone types, the program will look at this table to find the phrase that matches with the highest count. This best match is essentially a prediction of what the next word should be. This match also allows the program to give you an estimation of likelihood (for example, 80% chance of correct completion based on the corpus).

And there we have it - the magic behind many autocomplete tools boils down to counting words!

In an overly simplified nutshell, that's how you build a model that could predict the most likely next word given a certain set of history words. This is the foundation of how autocorrect and autocomplete systems work.

So are copywriters going to be replaced by an n-grams model?

Definitely not.

There is a fundamental problem with n-grams - it represents language in a sparse way. To put it simply, this means if you have a lot of data, your table becomes too large. That's where neural networks like GPT comes into play.

Going beyond a table with GPT

What we use today has come a long way since the n-grams model.

Modern artificial intelligence for autocomplete relies on neural networks, which are much more powerful than n-grams models. Although more sophisticated, neural networks rely on the same fundamental principle of statistical counting.

The all-mighty GPT-3 (Generative Pre-trained Transformer v3) is a large neural network language model developed by OpenAI and is now the backbone of all the recently released autocomplete writing tools on the market. GPT-3 is part of a trend in natural language processing (NLP) to move towards large pre-trained neural networks.

With GPT-3, we no longer naively use a table to store all possible phrases, rather we store a compressed summary of it in the weights of its neural network. This allows us to train models on billions of phrases and sentences that would've been infeasible to fit in a single table.

‍So how does GPT-3 write for you?

Just like n-grams, when you type a word or phrase, GPT-3 will try to find the most likely word to complete your sentence based on the corpus of text it was trained from.

But it doesn't stop there. Once it predicts the next word you're going to type, it will do this in a loop and keep predicting the next word until it has written a paragraph. This is how it "generates" a paragraph for you.

But wait - if all GPT-3 does is look up probabilities derived from previously written content, does this mean GPT-3 simply repeats what it has read, or is it actually creative?

That's a tricky philosophical question that is prevalent in today's discussions about AI-generated content.

Misconceptions about AI Content Writing

Can AI Content Writing Be Creative?

Many critics have noted that GPT-3, like all AI models, can only generate text that it has seen before. They claim that AI writing lacks creativity and that these tools are only good for spamming regurgitated content.

While this view used to be valid, it is no longer entirely true.

It is easy to claim that an n-grams model from 1948 would simply repeat existing content because it literally stores all its training data in a table and "generates" text by looking through phrases it has seen.

But because GPT-3 is a highly efficient compressor of its training text, it is forced to develop rules and patterns of written content - it doesn't always remember the exact sentence from its training data stored in its memory.

While a few sentences could be generated verbatim, many phrases produced are novel. A quick Google search of generated text will show you that most generations are original.

Whether or not you believe that GPT-3 (or any AI model and AI Tools) can produce original writing is up to debate, and depends on how you define originality. After all, humans have learned from previous great works and created spin-offs from Shakespeare, so are humans really all that original?

While modern AI produces text that may be similar to what is out there, it also can produce text that may surprise you.

It is up to human copywriters and content editors to make the most of that surprise.

Better use of autocomplete should involve writers filtering and choosing the best AI-generated text, or using it as a source of inspiration to break writer's block.

Can AI Content Writing Have Emotion?

One of the concerns around AI content writing is that it will produce soulless, unemotional text.

This is another broad statement that lacks nuance - and is perhaps derived from our science fiction notion of AI being tin-can robots with no feelings.

Once again, simple AI models like the n-gram would be unlikely to produce emotional text because it lacks representational power - it has a practical limit on how much it can learn.

But because GPT-3 learns from a large corpus of text with more context, it can often parrot the sentiment and tone in writing. This means that if you type in a phrase like "I'm feeling sad today," the AI model will try to find the most appropriate words to reflect that sentiment in the generated text.

(The paragraph you just read above was entirely autocompleted by Jenni AI without edits. It has learned to match my tone and writing style from previous paragraphs.)

A study published in ICLR 2020 shows that neural text generators can even parrot too often and start becoming repetitive. However, parroting is not the same as having emotion.

As a writer, you will still need to be in charge of the overall tone and emotion of your writing. While AI can produce text that mirrors human sentiment, it does not have empirical experience of what it is to be human - it is not an embodied intelligence.

Remember, like n-gram models, GPT-3 is trained on a corpus of text (mostly from the internet and produced by a human copywriter).

It hasn't seen or experienced anything else a typical human experiences - it'll never know what a cheeseburger tastes like, nor can it fully empathize. According to OpenAI, it cannot answer questions accurately relating to the physical world, such as "If I put cheese into the fridge, will it melt?".

This is an inherent limitation of modern language models to come in the next few years - at least until AI gets a physical body.

For content writing, understanding this limitation is critical.

This means that to truly leverage the power of AI for content writing, we need to provide guidance and feedback to the AI model to steer it in the right direction.

Why AI + Human Is the Future of Content Writing

These drawbacks may lead many to be skeptical about the advances in AI content writing or fear that our future will be full of spam content.

On the contrary, I foresee a much brighter future.

In 1996, when IBM's AI system  in a game of chess, it was thought that the game of chess was solved and there would be no chess players left.

However, what happened was a resurgence of people learning new chess strategies by studying the AI's moves. A similar phenomenon happened after AlphaGo from DeepMind defeated Lee Sedol, the world's best player at Go in 2016.

Success in AI means humans have to adapt and change - and this change may be uncomfortable but it is usually for the better. While AI can beat humans at certain tasks, humans are better generalists, and we can learn to incorporate AI to augment our overall productivity.

This rings true for content writing, where copywriters need to integrate a high-level content strategy, a company's vision and brand, and an understanding of the audience into their content.

That is why I predict a future where we can have the best of both worlds - humans and AI working together to produce even higher quality content.

Will Copywriting Be Obsolete Tomorrow?

With the exponential development of technology, it's hard not to wonder - is the content writer's job at risk in the future?

If we look at the trend of language model improvement over the years, it is clear that AI is getting better and better at autocompleting text. The perplexity (a measurement of error) of AI on a common benchmark like WikiText-103 has decreased from 40 to 10 in the last 3 years - that's a 4x improvement!

Extrapolating this exponential growth, in the next 5 years, we further expect a further 10x improvement in the quality of autocomplete technology.

That means if all you do for your SEO content writing is produce low-value work - rewriting existing content, filling in templates, copy/pasting listicles, or spinning other people's content - then the answer is yes - you are doomed.

So, what does this mean for the serious and passionate copywriters out there?

Don't "Write for Me", "Write with Me"

There's a reason why we don't use typewriters anymore. It's because content writing isn't about putting ink on a piece of paper.

There's a reason why we don't manually check grammar anymore. It's because grammar is a technicality and not the true heart of your content.

Satisfying user search intent and being seen as subject matter experts in your specific niche will have your readers coming back for more. They will organically share your articles on a larger scale and help your article shoot up in search engine rankings.

Despite all these evolutions in how we write with technology, the writer is still in charge of the vision of the content. Augmentation rather than replacement is the key.

If AI is here to remove the low-level work, as a copywriter, you must shift your methods to perform higher-value work. It's time to think deeper about what content you're producing.

There are 7.5 million blogs published every single day and your content needs to stand out.

Your job is to connect the dots between your marketing strategy, audience, and content - bringing unique information, research, and ideas - and to present it as a story that others haven't told. A story that grabs attention and keeps your readers engaged right to the end of the piece.

That means writing will be less so about the mechanics of putting words on paper and more about the ideas you want to convey and the art of storytelling.

We need to stop relying on AI to write for us, but rather, write with us.

If your job involves empathizing with your reader to create high-quality, engaging content that resonates with your audience and provides real value - your role is safe.

How Jenni can Help

At Jenni, we work hard to make this integration between humans and AI as seamless as possible - and that is why we carefully designed our GPT-3 based autocomplete system to not get in your way, but rather keep you - the content creator - in the driver's seat. Always!

As of March 2022, we have decided to phase out the Write for Me functionality - you know, the button you press, and it magically writes a paragraph for you. Shocking!

That's because we've found through numerous user case studies - over half of new users who were given access to the "Write For Me" buttons would click it to produce ~80% of their content - most of which was of low quality.

The incentive of this button is too easy for a user to create spam, and it keeps you from being the author of your story.

Instead, Jenni will now assist you by actively providing you with suggestions while you're writing and seamlessly integrating with your content creation process.

This will greatly assist to break any writer's block, and will also bring the fun and passion back into your craft.