Harnessing Game-Theoretic Optimization for Adversarial, Hierarchical, and Scalable Machine Learning Models

Speaker:

Dr. LU Songtao
Senior Research Scientist
Mathematics and Theoretical Computer Science Department
IBM Thomas J. Watson Research Center

Abstract:
As machine learning continues to permeate our daily lives with the deployment of large-scale foundational models across diverse domains, we are witnessing an unprecedented era of data collection and exploration through smart devices. This abundance of data holds the potential to bring groundbreaking advancements across numerous industries and disciplines. However, effectively leveraging and safeguarding this wealth of data requires increasingly advanced mathematical techniques.

My research is centered on designing computationally efficient methods backed by theory to drive adversarial, hierarchical, and scalable machine learning models. In this talk, I will delve into my recent work on developing gradient-based optimization algorithms tailored to address game theory-related machine learning problems. Unlike traditional theories focused on convex/concave problems, my focus lies in nonconvex zero-sum games and Stackelberg games, which are essential for tackling nonconvex objective functions prevalent in neural network training. These advancements not only offer theoretical insights into stabilizing iterative numerical algorithms but also provide more generalizable solutions for downstream learning tasks. I will demonstrate the practical significance of these algorithms in addressing real-world machine learning challenges, including adversarial attacks, data hyper-cleaning, and automatic speech recognition. Furthermore, I will highlight the broader impact of the proposed learning framework on emerging problems, such as multilingual multitask learning, reinforcement learning with human feedback, and multi-agent RL.

Biography:
Songtao Lu is a Senior Research Scientist in the Mathematics and Theoretical Computer Science Department at the IBM Thomas J. Watson Research Center in Yorktown Heights, NY, USA. Additionally, he serves as a principal investigator at the MIT-IBM Watson AI Lab in Cambridge, MA, USA. He obtained his Ph.D. from the Department of Electrical and Computer Engineering at Iowa State University in 2018 and held a Post-Doctoral Associate position at the University of Minnesota Twin Cities from 2018 to 2019. His research primarily focuses on foundational machine learning models and algorithms, with applications in trustworthy learning, meta-learning, and distributed learning. He received the Best Paper Runner-Up Award at UAI in 2022, an Outstanding Paper Award from FL-NeurIPS in 2022, an IBM Entrepreneur Award in 2023, and an IBM Outstanding Research Accomplishment Award. Furthermore, he has multiple papers selected for oral/spotlight/long oral presentations at prestigious machine learning conferences, including ICML, NeurIPS, ICLR, AAAI, and UAI.

Enquiries:

Mr. WONG O-Bong (obong@cse.cuhk.edu.hk)

Ms. FUNG Wing Chi Mary (maryfung@cse.cuhk.edu.hk)