The Doctoral Programme in Information Engineering and Computer Science (IECS) organizes the fifth edition of the PI Stories, a series of seminars aimed at providing the opportunity for PhD students to learn the success stories of some of the most talented researchers in the world.
Abstract
Three years ago, a small, scrappy team inside Meta set out to do something that many thought impossible: train and release a state-of-the-art large-language model that would remain open-weights, code and research notes alike. In building Llama-2 we had to relearn, and sometimes unlearn, the “settled science” of LLMs: from data curation and scaling laws to RLHF. In this talk I will retrace that journey, showing how revisiting first principles unlocked surprising performance gains.
About the Speaker
Thomas Scialom is a Senior Staff Research Scientist at Meta AI, where he led the development of Llama-2 and spearheaded post-training for Llama-3. His research spans large-scale language modelling, tool-augmented reasoning and sustainable training pipelines. Before joining Meta, Thomas earned a PhD from Sorbonne University, focusing on reinforcement-learning methods for natural-language generation. He has published widely, contributed to projects such as Code Llama, Galactica and Toolformer, and frequently advises startups pushing the frontiers of generative AI.