EE Seminar: Low Rank Models: From Signal Processing to Deep Learning Theory
Electrical Engineering Systems Seminar
Speaker: Dana Wizner
Ph.D. student under the supervision of Prof. Raja Giryes
Monday, 28th October 2024, at 15:00
ZOOM Seminar
Low Rank Models: From Signal Processing to Deep Learning Theory
Abstract
Recently, our world has entered the age of “Big Data.” We are now facing the challenge, and opportunity, of processing massive amounts of data while being able to uncover the information in it, buried as low-dimensional structures. To this end, we would like to explore the world of low-rank and sparse models, with an emphasis on theoretic aspects of real-world applications.
In this seminar, we study a variety of problems where a low rank or sparse prior arises naturally, and helps us to better explain observed phenomena or to simplify the common models. First, we study joint blind deconvolution and demixing, cast as a separable low-rank retrieval problem. Next, we explain the vulnerability of neural networks to universal and transferable adversarial attacks through the framework of sparsity. In another line of research we bridge between two important concepts: the Neural Tangent Kernel (NTK), which captures the evolution of deep neural networks (DNNs) during training, and the Neural Collapse (NC) phenomenon, which refers to the emergence of symmetry in DNNs, and prove the emergence of NC in DNNs with block-structured NTK. Finally, we study diffusion models as "correlation machines", where we analyze linear diffusion, and show a connection to the spiked covariance model and the power iteration method.
השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום בצ'ט של שם מלא + מספר ת.ז.
