, , ,

The History of Anthropic is the History of Scaling Laws

I thought I was done writing about the scaling laws in AI research after launching an entire webpage devoted to it. Well, I was wrong. On the day of Anthropic’s historic IPO yesterday, the 7 co-founders of Anthropic launched a video that opens with the origin of the company. What did Anthropic’s video focus on? You guessed it, how the scaling laws propelled the company’s research and later success.

SOME EARLY INVESTORS RECOGNIZED THE POWER OF THE SCALING LAWS, TOO

That’s not all. Some of Anthropic’s early investors in January 2023, during the very first round of Anthropic’s financing, also recognized the power of the scaling laws in improving AI models. Yasmin Ravazi, who studied engineering at the University of Toronto (where Geoffrey Hinton teaches), reportedly wrote 2 internal memos recommending that her investment firm Spark Capital invest in Anthropic. Spark Capital did.

As Clay Fisher, another partner at Spark Capital, wrote yesterday during Anthropic’s IPO: “She [Razavi] wrote two memos to our team on scaling laws, the power of generality, access to compute, and why the market for intelligence would be vastly larger than anyone imagined. Rereading them today, it’s eerie how much she got right. Deeply grateful for her courage, intellect, and conviction.”

Razavi’s recognition, in 2023, of the effectiveness of the scaling laws in improving AI models paid off. According to Dealroom, “Spark (Yasmin Razavi led the 2023 $450M round at $4.6B with a $75M check) has put in $777M across six rounds: about 18x, roughly $14B, around 1%.”

What about all the copyright suits v. anthropic?

The copyright suits against Anthropic didn’t appear to get any coverage yesterday on the day of its IPO. But, as Anthropic’s S1 indicated, there’s a risk of liability Anthropic faces if courts disagree with its position that AI training is a fair use. No one knows how the courts will rule in these and other AI cases.

But it’s important for the courts to understand why AI researchers and labs were doing what they were doing. I’ve written extensively on the origin of AI training, especially on the critical role of scaling to AI research and advancement. This history of the scaling laws is too important for courts to ignore. And the historical record of what the AI researchers were doing and why they were doing it is too extensive.

Leave a Reply