CS 201: (Un)Conventional Regularization for Efficient Large Scale Machine Learning, LORENZO ROSASCO, University Of Genova – MIT

Speaker: Lorenzo Rosasco
Affiliation: University Of Genova - MIT

ABSTRACT: Regularization is classically designed by penalizing or imposing explicit constraints to an empirical objective function. This approach can be derived from different perspectives and has optimal statistical guarantees. However, it postpones computational considerations to a separate analysis. In large scale scenarios, considering independently statistical and numerical aspects often leads to prohibitive computational requirements.  It is then natural to ask whether different regularization principles exist or can be derived to encompass at once both statistical and computational aspects. In this talk, we will present several ideas in this direction, showing how procedures typically developed to perform efficient computations can often be seen as a form implicit regularization. We will discuss how iterative optimization of an empirical objective leads to regularization, and analyze the effect of acceleration, preconditioning and stochastic approximations. We will further discuss the regularization effect of sketching/subsampling methods by drawing a connection to classical regularization with projection methods common in applied mathematics. We will show how these forms of implicit regularization can obtain optimal statistical guarantees, with dramatically reduced computational properties. BIO: Lorenzo Rosasco is associate professor at University of Genova and Visiting professor at the Massachusetts Institute of Technology (MIT). He is also a researcher at the Italian Technological Institute (IIT), where he coordinates the joint IIT-MIT Laboratory for Computational and Statistical Learning. He received his PhD in 2006 from the University of Genova, after being a visiting student at the Center for Biological and Computational Learning at MIT, the Toyota Technological Institute at Chicago (TTI-Chicago) and the Johann Radon Institute for Computational and Applied Mathematics. Between 2006 and 2009 he has been a postdoctoral fellow at the Center for Biological and Computational Learning at MIT. His research focuses on theory and algorithms for machine learning.

Hosted by Professor Stefano Soatto

Date/Time:
Date(s) - Feb 08, 2018
4:15 pm - 5:45 pm

Location:
3400 Boelter Hall
420 Westwood Plaza Los Angeles California 90095