Learning and Memorization
|Title:||Learning and Memorization|
|Date:||May 09, 2019 (Thursday)|
|Time:||10:30 am - 11:30 am|
|Venue:||Room 121, 1/F, Ho Sin-Hang Engineering Building, The Chinese University of Hong Kong, Shatin, N.T.|
|Speaker:|| Dr. Alan MISHCHENKO
In the machine learning research community, it is generally believed that there is a tension between memorization and generalization. In this work, we examine to what extent this tension exists, by exploring if it is possible to generalize by memorizing alone. Although direct memorization with a lookup table obviously does not generalize, we find that introducing depth in the form of a network of support-limited lookup tables leads to generalization that is significantly above chance and closer to those obtained by standard learning algorithms on several tasks derived from MNIST and CIFAR-10. Furthermore, we demonstrate through a series of empirical results that our approach allows for a smooth tradeoff between memorization and generalization and exhibits some of the most salient characteristics of neural networks: depth improves performance; random data can be memorized and yet there is generalization on real data; and memorizing random data is harder in a certain sense than memorizing real data. The extreme simplicity of the algorithm and potential connections with generalization theory point to several interesting directions for future research.
Alan graduated with M.S. from Moscow Institute of Physics and Technology (Moscow, Russia) in 1993 and received his Ph.D. from Glushkov Institute of Cybernetics (Kiev, Ukraine) in 1997. In 2002, Alan joined the EECS Department at University of California, Berkeley, where he is currently a full researcher. His research is in computationally efficient logic synthesis and formal verification.
Enquiries: Ms. Shirley Lau at tel. 3943 8439
For more information, please refer to http://www.cse.cuhk.edu.hk/en/events