In an upcoming edition of our team seminars, we are excited to host Lihu Chen, a Research Associate at Imperial College London. The seminar will take place on October 7th at 11:00 AM in Room Henri Poincaré (Ground Floor, Alan Turing Building, Palaiseau).
Title:
What is the role of small models in the LLM era
Abstract:
Large Language Models (LLMs) have made significant progress in advancing artificial general intelligence (AGI), leading to the development of increasingly large models such as GPT-4 and LLaMA-405B. However, scaling up model sizes results in exponentially higher computational costs and energy consumption, which makes these models impractical for academic researchers and businesses with limited resources. At the same time, Small Models (SMs) are frequently used in practical settings, although their significance is currently underestimated. This raises important questions about the role of small models in the era of LLMs, a topic that has received limited attention in prior research.
In this talk, I will discuss the relationship between LLMs and SMs from two perspectives: collaboration and competition. Specifically, I will present our recent study to showcase the two relationships: 1) how to use small models to estimate LLM confidence scores (collaboration); and 2) small models for short text representation (competition).
Bio:
Lihu Chen is a Research Associate at Imperial College London. His research focuses on natural language processing and large language models, in particular on developing efficient, reliable, and open-source models and tools, with an emphasis on information extraction and biomedical applications.
