The New York Times’ copyright lawsuit against Microsoft and OpenAI sent shockwaves across the business and media sector. Although a threat of a lawsuit had been revealed months ago, the Complaint filed by the NYT was cogent and chock-full of examples of alleged infringement. If OpenAI was trying to negotiate a licensing deal with the New York Times as reported, the price tag may have just gone up — by a lot.
Even more impressive than the NYT Complaint was Exhibit J, an attachment to the Complaint. Exhibit J allegedly provides “One Hundred Examples of GPT-4 Memorizing Content from The New York Times,” spanning 127 pages. Great title–did the New York Times headline writers come up with it?
The side-by-side comparison of New York Times content and GPT-4 outputs apparently copying the language from the NYT article verbatim is very effective. Here’s the side-by-side comparison between the NYT article and GPT-4 output for the first example:
Example 1: How the U.S. Lost Out on iPhone Work (May 31, 2012 NYT article)
The entire Exhibit J is filled with 100 examples of such apparent verbatim copying by GPT-4.
What’s interesting, though, is how the New York Times even found these examples. From their description on p. 2, it appears the Times simply fed lines from its own articles as a prompt on ChatGPT: “The prompt that was given to GPT-4. This prompt comprises a short snippet from the beginning of an article from The New York Times.” And then ChatGPT provided other actual lines from the article.
The NYT used the following “prompt,” consisting of actual lines from the NYT article published on May 31, 2012 titled “How the U.S. Lost Out on iPhone Work” to generate the example above:
This strikes me as a glitch in ChatGPT.
Prompts typically are instructions to a chatbot. For example, “write an essay about X, Y, and Z,” or “create a table using this information A, B, and C.” Maybe I’m not clever enough. But I wouldn’t have thought of just taking actual lines from an article, feeding it into ChatGPT, and expecting ChatGPT would fill in the rest. Maybe this strategy was known by others, who share knowledge about prompting. Or maybe the NYT figured it out on its own.
But it doesn’t seem to be what ChatGPT was set up for. Based on my testing today, the glitch, if it was a glitch, has apparently been removed from ChatGPT. Just feeding lines from articles won’t “prompt” ChatGPT to fill in the rest of the article.
So, let’s assume this glitch is gone. And let’s also assume, in the best case scenario for the New York Times, that all 100 examples in Exhibit J were willful infringement. Putting aside the other claims of infringement, such as in Bing Chat’s synthetic search results, the maximum value in statutory damages the New York Times can receive just for these 100 examples is $15 million (= 100 works x $150,000 per work).
Of course, the New York Times’s Complaint alleges other infringement based on the training of ChatGPT on “millions of Times Works ” (para. 92). If we take the most conservative figure of 1 million works, and assume willful infringement for those works, the maximum value in statutory damages is $150 billion (= 1 million works x $150,000 per work). $150 billion is 50% more than OpenAI is currently valued at ($100B)!
UPDATE: The Complaint has the NYT over 3 million registered works. So the maximum value of statutory damages is $450 billion, assuming willful infringement.