Many publishers are forming agreements with OpenAI, the company behind ChatGPT, which has raised concerns based on a study by the Tow Center for Digital Journalism. The study focused on how ChatGPT generates citations for publishers’ content, revealing worrying findings.
The research, conducted at Columbia Journalism School, analyzed citations produced by ChatGPT to identify the source of sample quotations from various publishers. The study included content from publishers like The New York Times, The Washington Post, and The Financial Times, among others, to assess the accuracy of the citations.
The results indicated that ChatGPT’s citations were often inaccurate, with varying levels of reliability in the responses provided. While some citations were entirely correct, many were wrong, and some fell in between. This inconsistency poses a risk to publishers in terms of reputation and commercial implications.
Moreover, the study highlighted that ChatGPT could inadvertently reward plagiarism by citing incorrect sources. This raises questions about OpenAI’s ability to filter and validate data sources effectively, especially when dealing with unlicensed or plagiarized content.
The researchers also noted that ChatGPT’s technology treats journalism as decontextualized content, potentially overlooking the circumstances of the content’s original production. Additionally, the study found that ChatGPT’s responses varied when asked the same query multiple times, impacting the accuracy of the citations provided.
Despite the high-level deals between major publishers and OpenAI, the study suggests that there is no guarantee of accurate sourcing, even for publishers with licensing agreements. This lack of consistency in citation accuracy poses challenges for publishers seeking visibility and credibility in ChatGPT’s search engine.
Overall, the study concludes that publishers have limited control over their content once it is processed by ChatGPT. OpenAI responded to the research findings, claiming to support publishers and creators by enhancing citation accuracy and respecting publisher preferences. However, the study’s results raise important questions about the reliability and transparency of AI-generated citations in the journalism industry.
