ABSTRACT:

Computer vision research has been revolutionized by a relatively straightforward recipe: obtain annotated data, and apply modern classification or regression techniques, as appropriate. This recipe has solved commercially valuable problems and built fame and fortune for many. But do we really believe that animals have vision because the proprietor issued some early owners of an eyeball with a gold standard dataset? What do we do if we don’t have, or can’t get, appropriately labelled data?

Intrinsic images are maps of surface properties. A classical problem is to recover an intrinsic image, typically a map of surface lightness, from an image. The decline of this topic has a consequence – mostly, we don’t understand and can’t mitigate the effects of lighting.

I will show the results of simple experiments that suggest that very good modern depth and normal predictors are strongly sensitive to lighting – if you relight a scene in a reasonable way, the reported depth will change. This is intolerable. To fix this problem, we need to be able to produce many different lightings of the same scene. I will describe a method to do so. First, one learns a method to estimate albedo from images without any labelled training data (which turns out to perform well under traditional evaluations). Then, one forces an image generator to produce many different images that have the same albedo — with care, these are relightings of the same scene. Finally, a GAN inverter allows us to apply the process to real images. I will show some interim results suggesting that learned relightings might genuinely improve estimates of depth, normal and albedo.

All this leads us into the question: what do image generators know? I will show strong evidence that the answer is: surprisingly little, and that not geometry.

 BIO:

David Alexander Forsyth is currently the Fulton-Watson-Copp chair in computer science at U. Illinois at Urbana-Champaign, where he moved from U.C Berkeley, where he was also a full professor. D.A. Forsyth has occupied the Fulton-Watson-Copp chair in Computer Science at the University of Illinois since 2014. Has published over 200 papers on computer vision, computer graphics and machine learning. Has served as program co-chair or general co-chair for vision conferences on many occasions. Has received an IEEE technical achievement award for 2005 for his research. Became an IEEE Fellow in 2009, and an ACM Fellow in 2014. His textbook,”Computer Vision: A Modern Approach” (joint with J. Ponce and published by Prentice Hall) was widely adopted as a course text. His recent textbook, “Probability and Statistics for Computer Science”, is in the top quartile of Springer computer science chapter downloads. A further textbook “Applied Machine Learning” has just appeared in print. D.A. Forsyth has served two terms as Editor in Chief, IEEE TPAMI. And serves on a number of scientific advisory boards, and has an active practice as an expert witness.

 Hosted by Professor Bolei Zhou