Affective prediction in photographic images using probabilistic affective model

With increasing the importance of affective computing, it becomes necessary to retrieve and process images according to human affects or preference. However, judging such affective qualities of images is a highly subjective task. In spite of the lack of firm rules, certain features in images are believed to be more related than certain others. In this paper, we suggest predicting certain affective features include in an image using color composition that constitutes the scene. Using such a feature is inspired from Kobayashi's color scale that studies the relation between colors/color compositions and human's affects. Thus, we propose a Probabilistic Affective Model (PAM) to estimate the probabilities that an image is related to certain affective features. For this, we segment an image using mean-shift clustering algorithm, and extract more important regions, which are called seed regions, based on their properties. Thereafter, we find the dominant color compositions among those seed regions and their neighboring regions. Finally, from such color compositions, we infer the numerical ratings for some affective features. To assess the effectiveness of our PAM, we compared its results with 52 users' affective judgments. It was tested with online photo images, then the results show our PAM produced the recall of 85.22% and the precision of 78.16% on average. Potential applications include content-based image retrieval and design of web page interfaces.