CS 201: The Unpaved Path of Deploying Reliable and Human-Centered Machine Learning Systems, BESMIRA NUSHI, Microsoft Research

Speaker: Besmira Nushi
Affiliation: Microsoft Research


As Machine Learning systems are increasingly becoming part of user-facing applications, their reliability and robustness are key to building and maintaining trust with users. While advances in learning are continuously improving model performance in expectation and in isolation, there is an emergent need for identifying, understanding, and mitigating cases where models may fail in unexpected ways and therefore break human trust or dependencies with other larger software ecosystems. Current development infrastructures and methodologies often designed with traditional software in mind, still provide very little support to enable practitioners debug and troubleshoot systems over time.

This talk will share some of the latest progress we have made on building interactive tools for engineering teams that enable ML developers to analyze and understand errors of learning models and systems prior to deployment and updates. The talk will cover case studies that show how these tools can be used to efficiently identify errors, compare different model versions, and train models which remain backward compatible with their previous versions. In this context, we will also share some ongoing work on integrating interpretability offerings with error analysis for guiding debugging experiences.


Besmira Nushi is a researcher in the Adaptive Systems and Interaction group at Microsoft Research. Her work lies at the intersection of human and machine intelligence and enables building Machine Learning systems that are reliable and can augment human productivity. In this realm, her interests span across two main directions: (1) Troubleshooting and Failure Analysis for ML systems for accelerating the software development lifecycle of intelligent systems and (2) Human-AI Collaboration for enhancing human capabilities while solving complex tasks. Prior to Microsoft Research, Besmira completed her PhD at ETH Zurich where she worked on designing methods and algorithms for optimizing the cost and quality of data collection processes for training Machine Learning models.

 Hosted by Professor Baharan Mirzasoleiman

Date(s) - Jan 21, 2021
4:00 pm - 5:45 pm

Zoom Webinar
404 Westwood Plaza Los Angeles
Map Unavailable