Disentangled Representation from Generative Networks

Speaker:
Dr. LIU Sifei

Abstract:

Disentangled representation in computer vision refers to encoding visual data into distinct, independent factors. These representations are critical for enhancing interpretability, improving generalization across tasks, and enabling controlled manipulation of specific visual attributes. Learning disentangled representation is challenging, primarily because obtaining ground-truth factorizations is often elusive.

In this talk, I will discuss our latest efforts to extract disentangled representations from GANs and diffusion models, for both 2D images and 3D textured shapes. I will demonstrate how, in the absence of annotations, our approaches can discern and extract fine-grained structural information, such as correspondence maps, in a self-supervised manner. Building on this space, I will introduce our work on a generalizable network designed for controlled generation and editing in a feed-forward paradigm. Additionally, I will spotlight our recent exploration into generating hand-object interactions, leveraging the disentanglement of layout and content through image diffusion models.

Biography:

Dr. LIU Sifei is a staff-level Senior Research Scientist at NVIDIA, where she is part of the LPR team led by Jan Kautz. Her work primarily revolves around the development of generalizable visual representation and data-efficiency learning for images, videos, and 3D contents. Prior to this, she pursued her Ph.D. at the VLLAB, under the guidance of Ming-Hsuan Yang. Sifei had received several prestigious awards and recognitions. In 2013, she was honored with the Baidu Graduate Fellowship. This was followed by the NVIDIA Pioneering Research Award in 2017, and the Rising Star EECS accolade in 2019. Additionally, she was nominated for the VentureBeat Women in AI Award in 2020.

Enquiries: Ms Anna Wong (annawong@cse.cuhk.edu.hk)