MDN Online conference: Extracting Mise-en-Scene and Emotional Metadata from Video Content

The Metadata Developer Network (MDN) conference is coming back, virtually, from May 30 to June 1. We are excited that our CTO, Paolo Cremonesi will be one of the speakers.

This conference, organized by the European Broadcasting Union (EBU), is the annual meeting point for developers and experts working on Metadata and Artificial Intelligence in media. It is a great way to expose your views and challenges to the community, engage in lively discussions and learn from your peers.

As mentioned before, Paolo Cremonesi will speak during the event about “Extracting Mise-en-Scene and Emotional Metadata from Video Content”, on May 30 from 3:30 PM CEST. He will describe a multi-modal content-based recommender system that replaces traditional metadata with emotional descriptors automatically extracted from the visual and audio channels of a video.

Emotional descriptors improve over traditional metadata in terms of both richness (it is possible to extract hundreds of meaningful features covering various modalities) and quality (emotional features are consistent across different systems and immune to human errors). These types of descriptors are created by integrating deep visual features, audio features, and mise-en-scène features, i.e., the design aspects of movie-making influencing aesthetic and style. We believe that the preferences of users on movies can be well described in terms of these emotional descriptors.

During his speech, Paolo will also present the results of a number of user studies where we evaluate the quality of recommendations with emotional descriptors against metadata-based baselines. Our results shed light on the accuracy and beyond-accuracy performance of audio, visual, and textual features in content-based movie recommender systems. Moreover, our recommender system opens new opportunities in the design of new user interfaces able to offer a personalized way to search for interesting movies through the analysis of film styles rather than using the traditional classifications of movies based on explicit attributes such as genre and cast.