Fascinating research paper refuting the notion that scaling of datasets has diminishing returns in the performance of large language models. While that might be true for simple “single-step” tasks, it is not for more complex “long horizon” tasks, according to the following researchers (Akshit Sinha, Arvindh Arun, Shashwat Goel, Steffen Staab, Jonas Geiping), who posted … Continue reading Paper on The Illusion of Diminishing Returns [of scaling]