I'm interested in the motion of the visual world. Motion is critical for intelligent systems. As Gibson nicely put it "We see in order to move and we move in order to see". Motion informs us of changes in the world, its structure, physical properties, etc. In our lab we study what motion tells us about the world and how to estimate it from video. 


These concepts often fall in the following Computer Vision problem buckets:  


  • Optical Flow 
  • Depth and Structure from Motion 
  • Instance Segmentation in Video 
  • Object Tracking 
  • Temporal Representations 
  • Video Representations 
  • Action Recognition
  • Vision and Simulated Environments 
  • Active Vision 


These problems often require a fundamental ability to learn. When we have lots of high quality labeled data, visual problems may not be so daunting. When we have only some supervision, like it's the case of videos, learning becomes harder. These learning challenges broadly fall in the following Machine Learning areas: 


  • Deep Learning, in particular CNNs
  • Adversarial Learning  
  • Unsupervised or Weakly Supervised Learning 
  • Reinforcement Learning 


Below is recent work organized by category, and my full list of publications can be found here


Optical Flow and Scene Semantics

image12

Sevilla-Lara, Sun, Jampani and Black, "Optical Flow with Semantic Segmentation and Localized Layers", in CVPR 2016


Wulff, Sevilla-Lara, Black, "Optical Flow in Mostly Rigid Scenes", in CVPR 2017

What does Motion tell us about the World?

image13

I co-organized the "What is Flow For?" Workshop at ECCV '18 , see summary here


"Only Time can Tell: Discovering Temporal Data for Temporal Modeling" (coming soon)

Novel Motion Representations

image14

Shou, Yan, Kalantidis, Sevilla-Lara, et al. "DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition" in CVPR 2019.


Sevilla-Lara, Liao, Guney, Jampani, Geiger and Black, "On the Integration of Optical Flow and Action Recognition", in GCPR 2018 

Collaborators

Over the years I have been very fortunate to work together and learn from a big group of collaborators: