Building fair and robust networks in the age of b^2 scale models
Bryan A. Plummer, Boston University
10.30am 11th Mar 2024
Abstract
In a time where models with parameter counts in the billions that are trained on billions of samples (b^2) are becoming commonplace, many questions arise on how we should use these powerful models in downstream applications. In this talk, I will highlight three topics that are becoming increasingly important in this new age. First, I will explore how we can use large generative models to build fairer datasets that we can use to help ensure that models are less biased in their predictions, while also avoiding potential biases being introduced by using generated data. Second, I will delve into my lab’s recent work that aims to connect two disparate research directions that address label noise. By reformulating these methods in a new framework, we find that we can boost performance in even very noisy datasets that often arise in datasets in the wild. Third, to defend against some of the negative effect of large language models being used for misinformation, we will explore methods that aim to localize machine generated text in documents that is created via a mix of human and machine text, even in cases where current methods fail such as majority human-written documents.