PICASSO: automated soundtrack suggestion for multi-modal data

We demonstrate PICASSO, a novel approach to soundtrack recommendation. Given text, video, or image documents, PICASSO selects the best fitting music pieces, out of a given set of files, for instance, a user's personal mp3 collection. This task, commonly referred to as soundtrack suggestion, is non-trivial as it requires a lot of human attention and a good deal of experience, with master pieces distinguished, e.g., with the Academy Award for Best Original Score. We put forward PICASSO to solve this task in a fully automated way. We address the problem by extracting the required information, in form of music/screenshot samples, from available contemporary movies, making the training set easily obtainable. The training set is further extended with information acquired from movie scripts and subtitles, giving us a richer description of the action and atmosphere expressed in a particular movie scene. Although the number of applications for this approach is very large, we focus on two selected applications. First, we consider recommendation of the soundtrack for the slide show generation based on the given set of images. Second, we consider recommending a soundtrack as the background music for given audio books.