Ever since OpenAI launched ChatGPT last year, users have relied on this new technology to create texts, music, and other outputs that can be made by a talented human. However, not everyone may know about ChatGPT and AI in general. Some might not have an idea of what they are reading online was produced by an AI or human.
The Methodology Used
A survey was conducted by Tooltester, a company that reviews various digital platform tools, between February 20 to 26, 2023. They conducted 1,920 American adults of all age ranges to determine if the text was created by a human, AI, or AI and edited by a human. They used 25 questions for the survey, sourced from the following:
- ChatGPT, which was given prompts to write as an expert on the relevant topic and explain answers “simply;”
- Human-written content from expert sites that provide in-depth content about the topic
These questions were asked with three answers per question: written by AI, written by a human journalist, or created by AI but edited by a human copywriter.
The Findings
Tooltester in its survey reported that people cannot tell the difference between human-written and AI-generated text.
The survey states that only 46.9% of people can correctly identify AI-generated content, which varied on the topic of the content they were reading. However, Tooltester does not conclude to mean that AI content is undetectable by more than half of the people who participated in the survey.
36.3% of the participants thought the AI-generated content was written by a human, and 16.7% thought that it was AI content edited by a human professional copywriter.
Another finding of Tooltester is that people were deceived the most by AI-generated health content. 56.1% thought that the health content was written by a human or edited by a human.
Finally, more than half, or 53% of the participants read AI-generated content and assumed that a human was involved in writing said content. This means that AI bots like ChatGPT can be convincing when it writes content, even without a writer adding their unique style and character.
Similar Reports
There are other surveys conducted on whether people can detect AI-written content. For one, researchers at Cornell University found that untrained humans were able to spot AI-generated text correctly, but only by random chance.
However, a new study claims that AI-generated content is detectable, but only with the use of an AI detection tool. SEO Agency Search Logistics reported that a public AI detection tool was able to identify nearly 90% of the content generated by AI on CNET. CNET Money had utilized an “AI assist” to produce explainers in response to commonly asked questions, resulting in approximately 75 published articles by mid-January.
They also reported that AI-generated content can be identified, but the fact that the AI detection tool used was able to detect AI-generated content on CNET does not necessarily indicate that AI cannot replace human content creators. It should be noted that while the tool can detect AI-generated content, it does not necessarily mean that a human reader would be able to distinguish between content created by a human or AI.
Conclusion
The key findings by Tooltester and other researchers regarding this topic are that those who are more familiar with AI tools can identify AI content better than those who aren’t, but they are only correct 48% of the time. And only 40.8% of people who are not familiar with ChatGPT or other AI tools can identify AI content.
Humans cannot easily identify AI content without the use of AI detection tools, and training the human eye on how to spot AI-generated text may be a stretch for now. But experts and researchers are continuously creating ways to make humans gradually better at detecting AI content when it is presented to them right off the bat.
Sources:
Image Source from Pixabay and Pexels
Tooltester.com
Cornell University
Search Logistics
