We address the character identification problem in movies and television videos: assigning names to faces on the screen. Most prior work on person recognition in videoPresented By Timothee Cour, INRIA - WILLOW Project,Benjamin Sapp Akash Nagle Ben Taskar,University of Pennsylvania
assumes some supervised data such as screenplay or handlabeled faces. In this paper, our only source of ‘supervision’ are the dialog cues: first, second and third person
references (such as “I’m Jack”, “Hey, Jack!” and “Jack left”). While this kind of supervision is sparse and indirect, we exploit multiple modalities and their interactions (appearance, dialog, mouth movement, synchrony, continuityediting cues) to effectively resolve identities through local temporal grouping followed by global weakly supervised recognition. We propose a novel temporal grouping model that partitions face tracks across multiple shots while respecting appearance, geometric and film-editing cues and constraints. In this model, states represent partitions of the k most recent face tracks, and transitions represent compatibility of consecutive partitions. We present dynamic programming inference and discriminative learning for the model.
The individual face tracks are subsequently assigned a name by learning a classifier from partial label constraints. The weakly supervised classifier incorporates multiple-instance constraints from dialog cues as well as soft grouping constraints from our temporal grouping. We evaluate both the temporal grouping and final character naming on several hours of TV and movies.
Reference : http://www.seas.upenn.edu
Tags : Talking Pictures: Temporal Grouping and Dialog-Supervised Person Recognition PDF, PPT, Seminar report
You do not have the required permissions to download the files attached to this post. You must LOGIN or REGISTER to download these files.