Experts who confidently assert that generative artificial intelligence (AI) like ChatGPT can never replace human writers must now rethink that position. New search at MIT shows that human readers preferred AI-generated content over that created by professional human writers.
New study finds people preferred persuasive AI-written content over human-written content
The researchers tasked both human experts and ChatGPT 4 with creating two types of content: descriptions of advertising products and persuasive content for advertising campaigns. They evaluated four variants:
- Content reserved for humans
- AI-only content
- Human content edited by AI (“Augmented AI”)
- AI content edited by humans (“Augmented Human”)
Surprisingly, perhaps to some, AI content – either written entirely by AI or with final editing by AI – was preferred by readers.
A second key finding is that revealing that content was written by AI does not change preference for AI content.
AI: the great persuader
Marketers and leaders in general must change the behavior of customers, employees and others. We view persuasion as a uniquely human skill, but this research shows that AI can do better than humans within a set of strict constraints.
The researchers used this initial instruction to human experts for the persuasive content task:
- “Your task is to write compelling content for a campaign in less than 100 words. Your goal is to persuade people to change their behavior after seeing your content.»
Sounds like the kind of task marketers face every day, right? The second instruction offered more details:
- “Please create persuasive content for a client (e.g. an NGO running a campaign) to convince people to [perform the action advocated by the campaign] in less than 100 words.»
The prompts in ChatGPT were “nearly identical.”
Human subjects who read the content were asked: “How convinced are you by the above content of [perform the action advocated by the campaign] on a scale of 1 to 7”? (1 meant “not at all convinced” and 7 meant “very convinced.”)
The product description task included a question about “willingness to pay.”
Across all tasks, the quality of the content created by the AI was judged to be better.
Would people care if AI wrote the content?
In this study, the researchers also tested the perception of content with the disclosure of its AI or human origin. They found that the subjects did not reduce their evaluation of AI content when told it was generated by AI.
Content written by humans received a small boost when topics were informed of its origin.
In this limited task, at least, the AI content performed well, whether or not subjects knew it was generated by the AI. No aversion to AI or bias against AI was found.
The key to remember
The important takeaway from this study is not that content created by AI will be better than content created by humans. Human subjects created their content in a vacuum – no discussion, no editorial input, no rumination or review, etc.
The most important finding for marketers is that AI can produce an output at least as good as that of an average professional human writer. when both are given a simple, well-defined persuasive writing task like those in this study.
As a digital marketing expert Notes by Chris Penn, these results mean that for content creators, “using AI will not hurt your marketing.” Penn also thinks fears that AI will create a “sea of trash” are overblown. (Maybe it won’t be rubbish, but I think we’re already seeing the start of a tsunami of mediocre content aimed at boosting search traffic.)
Should Marketers Outsource Content Creation to ChatGPT?
Writing 100 words is a fairly small task. Create and evaluate campaign concepts and flesh out slogans, graphics, etc. is a much bigger challenge. And rather than just one person writing the copy for a major advertising campaign, many people will likely be involved in crafting and refining the message.
It is unlikely that current AI can create new and highly compelling campaigns like those designed, for example, by advertising legend David Ogilvy. (Of course, many human agencies might also find this difficult.)
Companies shouldn’t lay off their marketing writers, at least not yet.
Where can marketers apply AI today?
Most marketing writing is pretty mundane. Creating product descriptions for websites and catalogs. Writing a weekly newsletter. Create news article content that benefits customers and improves search engine traffic. Unlike a major new advertising campaign, most of this content is written and published with minimal discussion.
It is in the production of this mundane content that AI can be a force multiplier for human writers, saving them time and money.
Human involvement is still necessary. AI must be triggered correctly if optimal results are desired. Of course, prompts can be reused once they prove effective. Automated workflows can be created. Over time, the initial labor of humans can be reduced.
Equally important, AI-created content still needs to be verified. Hallucinations, i.e. made-up facts and non-existent quotes, remain a problem for all models. Models also have difficulty understanding concepts: ask ChatGPT a question about an article or study, and it may come up with an answer that seems plausible but misinterprets the original.
Human writers and editors can save time by having the AI they use check its work, or by handing off content to another AI to check. Future AI models will undoubtedly be better at providing results free of hallucinations and conceptual errors.
Today, AI marketers need fewer core content writers to create the same result. The oft-used maxim about job loss due to AI applies here: writers will not lose their jobs because of AI. Rather, they will lose them to writers who will effectively leverage AI to write better content faster.
(This article was written entirely by humans. By me. Obviously I’m not following my own advice, at least not yet.)