DE102020132977A1 - METHOD AND APPARATUS FOR COMBINING IMAGES - Google Patents

Abstract

The invention relates to a method for combining images, the method having the following steps: a) recording more than one image with a camera (190), each image containing at least one section of a front view including the face (205, 215 ) of one or more persons (200, 210); b) a method for image analysis, comprising: (i) applying an algorithm for segmenting images (130) to segment at least two image sections each having a respective face ( 205, 215) or display a front view including the face (205, 215) of the recorded persons (200, 210); (ii) applying a face detection algorithm (140) to detect the face (205, 215) of each person ( 200, 210) in the segmented images or in the captured images and to provide corresponding face recognition data;(iii) applying a facial emotion recognition algorithm (150 ), based on the face recognition data, to recognize a corresponding emotion expressed by each of the detected faces (205, 215) and to provide corresponding facial emotion data; (iv) evaluating the facial emotion data to determine for each recognized facial emotion, whether it satisfies a predefined selection criterion;c) selecting those segmented images whose corresponding facial emotion data meet the predefined selection criterion; andd) composing a combined image comprising the selected segmented images.

Description

The present invention relates to the field of image processing. In particular, the present invention relates to a method and apparatus for combining images.

Furthermore, the invention relates to a computer program and an image processing device.

In the field of image processing, there is often a need to detect one or more objects, to distinguish between different objects, or to distinguish between different configurations of an object based on image data representing such one or more such objects.

It is known in the prior art to compose a generic image from multiple images taken at different time steps by manually selecting specific areas in the images and combining these selected areas to compose the generic image based thereon. There are also known segmentation algorithms, face recognition algorithms and emotion recognition algorithms that can be applied to the images. Furthermore, it is known to replace faces in group portraits. In this case, a user must manually select the face to be replaced.

Known methods of compositing images require a user to select areas of the image to be combined or replaced.

The object of the present invention is to provide a method for automatically creating an image of a group of people from a number of recorded images.

A solution to this problem is provided by the teaching of the independent claims. Various preferred embodiments of the present invention are provided by the teachings of the dependent claims.

• - Aufnehmen von mehr als einem Bild mit einer Kamera, wobei jedes Bild wenigstens einen Ausschnitt einer Frontansicht einschließlich des Gesichts von einer oder von mehreren Personen darstellt;
• - ein Verfahren zur Bildanalyse, aufweisend:
  • o Anwenden eines Algorithmus zum Segmentieren von Bildern, um wenigstens zwei Bildabschnitte zu segmentieren, die jeweils ein entsprechendes Gesicht oder eine Frontansicht einschließlich des Gesichts der aufgenommenen Personen darstellen;
  • o Anwenden eines Algorithmus zum Detektieren von Gesichtern, um das Gesicht jeder Person in den segmentierten Bildern oder in den aufgenommenen Bildern zu erkennen und um entsprechende Gesichtserkennungsdaten bereitzustellen;
  • o Anwenden eines Algorithmus zum Erkennen von Gesichtsemotionen, auf der Grundlage der Gesichtserkennungsdaten, um eine entsprechende Emotion zu erkennen, die von jedem der detektierten Gesichter ausgedrückt wird, und um entsprechende Gesichtsemotionsdaten bereitzustellen;
  • o Auswerten der Gesichtsemotionsdaten, um für jede erkannte Gesichtsemotion zu ermitteln, ob sie ein vordefiniertes Auswahlkriterium erfüllt;
• - Auswählen derjenigen segmentierten Bilder, deren entsprechende Gesichtsemotionsdaten das vordefinierte Auswahlkriterium erfüllen; und
• - Zusammensetzen eines kombinierten Bildes, das die ausgewählten segmentierten Bilder aufweist.

A first aspect of the invention relates to a method, in particular a computer-implemented method, for combining images. The procedure shows:

• - taking more than one picture with a camera, each picture representing at least a portion of a front view including the face of one or more persons;
• - a method for image analysis, comprising:
  • o applying an image segmentation algorithm to segment at least two image portions each representing a corresponding face or front view including the face of the captured subjects;
  • o Applying a face detection algorithm to detect the face of each person in the segmented images or in the captured images and to provide corresponding face detection data;
  • o applying a facial emotion recognition algorithm, based on the face recognition data, to recognize a corresponding emotion expressed by each of the detected faces and to provide corresponding facial emotion data;
  • o evaluating the facial emotion data to determine for each recognized facial emotion whether it meets a predefined selection criterion;
• - selecting those segmented images whose corresponding facial emotion data meet the predefined selection criterion; and
• - Composing a combined image comprising the selected segmented images.

The invention describes a method that creates a composite image from segmented images. A visually appealing image of a number of people, in particular of two people, is thereby created. Furthermore, an algorithm for detecting faces is applied to the segmented images or the captured images. The algorithm for detecting faces includes recognizing the face, which may include facial components (e.g., eyes and nose) or orienting features from the face area. Furthermore, it can involve the extraction of various spatial and time-dependent features from the facial components. The face detection algorithm provides the face detection data. The algorithm for detecting emotions determines the facial expression. This determination can be based on the extracted features, by comparing them to face categories in a database and providing face emotion data. The database may be a publicly available facial emotion recognition database, such as Compound emotion (CE), Binghamton University 3D Facial Expression (BU-3DFE) or The Extended Cohn-Kanade Dataset (CK+).

After the facial emotion data of the emotion of a face of a segmented image has been recognized and categorized, it can be evaluated based on the predefined selection criterion whether it should be used for a newly composed image. This evaluation involves comparing the facial emotion data of the categorized emotion with the predefined selection criterion. If the criterion is met, the segmented image is selected. A composite image is created from the selected images; H. put together.

Accordingly, the method according to the first aspect automatically provides a composite image of multiple persons based on previously captured individual images, the composite image comprising a composition of selected individual (separate) images or portions thereof. The provided method ensures that in the composite image each of the people shows an emotion or expression that corresponds to the predefined selection criterion.

In the following, certain preferred embodiments of the method are described, which can be combined with each other or with other aspects of the present invention in any way, unless such a combination is expressly excluded or not technically possible.

In some embodiments, the image analysis method is repeated multiple times until a segmented image has been selected for each of the people represented in the captured images. This ensures that an image is automatically assembled in which each person has an expression or emotion that meets the predefined selection criteria.

In some embodiments, the predefined selection criterion is met and a segmented image is selected accordingly if the facial emotion data provided, which correspond to the detected emotion of the same face, have changed for a specific number of consecutive images. For example, if one of the subjects in a picture suddenly changes their emotion from one picture to the next, this could indicate an emotion of surprise. In this way, a predefined selection criterion related to surprise emotions can be applied to the captured images, and segmented images can be selected when a recomposed image is to represent people showing surprise emotions. The specific number can be any number greater than one.

The face detection algorithm can provide facial recognition data from a list of previously recognized faces, for example in a database, and the facial emotion recognition algorithm can, as described above, provide facial emotion data from a list of the recognized emotions for each face and their respective probabilities in Provide reference to the face categories of the databases. In this way, the persons recorded can be tracked from image to image and the emotion probabilities can also be tracked over time. This information can be used to select a segmented image depending on a particular change in emotion, which can be, for example, a sudden emotion, such as laughing at a joke. Another situation may be when an emotion probability crosses a certain threshold and persists for some time - for example, happiness or contentment. Furthermore, there may be two or more emotion probabilities that exceed a threshold, or where one follows another - for example, surprise/disgust followed by laughter.

In some embodiments, the predefined selection criterion is met and accordingly a segmented image is selected if the provided facial emotion data corresponding to a classified emotional state relates to a positive emotional state. In this way, all of the people in the composite image present a positive emotional state, particularly a contented expression, an expression of joy, particularly a smiling or laughing expression, or an expression of happiness.

In some embodiments, the predefined selection criterion has more than one sub-criterion, all of which must be met in order for a segmented image to be selected. In particular, the selection criterion can be defined based on a specific combination of specific recognized emotions serving as sub-criteria. There may also be the information regarding the emotion probability determined over time, so for each person the algorithm tracking these sub-criteria could be different. This can also be used to eliminate personality type bias. For example, extroverts might display happiness most of the time. So for them, the thresholds would be higher so that an image would only be selected if they were very surprised or happy. These images can also be saved separately for later use. Additional images can be saved, for example 10 images before and 10 images after an event, to maximize the likelihood that a 'funny' or appealing image was captured. This makes for a more targeted selection of segmented images, and the recomposed image will have more similar expressions from each person.

In some embodiments, images of people seated in a car, particularly people seated in the front seats of the car, are captured with a wide-angle camera to create a composite image. In this case, the camera may be mounted on the instrument panel facing the people, who in this case are the driver and a passenger. Due to the wide-angle focal length, the camera can capture both people at the same time. While the driver and passenger are in the car, their facial expressions are recorded on the camera. These facial expressions can result from a conversation between them, from a telephone call or from a radio program. At a certain moment, the camera can record the driver with a pleasant emotion, but the passenger with a serious emotion. At a later time, the camera can record the passenger with a pleasant emotion but the driver with a serious emotion. This would result in two images, in both images the driver and passenger are shown with different expressions. The method provided ensures that a recomposed image can be assembled in which both the driver and the passenger appear with a similar emotion or expression.

In some embodiments, a background image is captured separately, which is then combined with the pixels representing the respective people in the corresponding selected images. This can be useful in situations where the background changes between the captured images, for example the images of two people sitting in a car as described above and the car is moving. In this situation, the background seen through the side and rear windows will change over time. In order to have a homogeneous background in the finished composite image, it can be advantageous if the background is recorded separately, for example before the two occupants get into the car. Later, this image is then assembled from the two images using only the segmented pixels of the people. In this way, the transition area in the image between the image of the people and the image of the separate background can also be homogeneous when the images are stitched together.

Furthermore, it is possible to combine the respective different backgrounds of two segmented images, for example backgrounds that relate to the surroundings of the driver or the passenger, with an unsharp mask around the segmented images, so that no overlay artifacts arise. In this case, a separate image of a background is not needed.

It is also possible to select a single background from one of the two segmented images, for example a background of the image showing the driver and/or a background of the image showing a passenger to represent the main background. For example, the image of the two images that provides more background information can be identified, i. H. most of the pixels that belong to the background but not to the main subject of the image, e.g. the people. Another criterion would be to choose the image that has the most information in certain predefined areas, for example the image where the windows are not obscured by either occupant.

Another criterion for selecting an image to be used for the background may be to select the image in which an occupant in the car moves the most between two captured images; People from the other image who are not moving can be included in this image with less loss of information. The segmented image of the moving occupant is then integrated into this main background and any missing information is filled in using a so-called "smart fill" algorithm known in the art, since the two segmentation maps are known from the two images .

Furthermore, a background image can be recorded continuously, which changes with the lighting conditions and as the people in the car move. An adaptive background subtraction method can be used to create a background model and updated in time - for example, to fill in missing parts when the characters move, or to blend them in time so that there are no hard highlights on objects. The current state of adaptive backgrounds is saved each time the algorithm detects a pleasant emotion and can be used as a background replacement in any of the ways mentioned above. The backgrounds can also be saved periodically and presented to the user as an option in a manual composition interface.

Criteria can also be defined to select which of the above processes to use, for example the exposure time, also known as shutter speed in photography, at which the images are captured. For images captured within a short exposure time, the methods of merging the segmented images including the background or selecting the one from the two images that has more information related to the background can be used. Another method would be to take a segmented image from a series in which one person moves the most between two images. Then the person who is not moving can be copied into the image from another segmented image with less loss of information.

For images captured with a long exposure time or for a longer period of time such as 10 seconds or more, it is preferable to capture the background separately or to use the segmented image that contains the most information in certain predefined areas. In the situation where images of people in a car are taken, the predefined areas refer to areas where the windows are not obscured by any of the people in the car.

For images that have significant differences between the respective backgrounds, the method described above, in which the background image is captured separately, can be used. The "significant differences" metric can be defined as the sum of the differences between the non-person related pixels of the two images being blended (i.e. background pixels) exceeding a certain threshold. Furthermore, the method in which the background images are continuously recorded can be used.

It is also possible to take pictures in which only the subjects' faces are in focus and the background is blurred by adjusting the camera's settings, such as the aperture. In the same way, it is possible to capture an image where both the subjects' faces and the background are sharp.

In some embodiments, the face detection algorithm and the facial emotion detection algorithm are based on a machine learning algorithm. In this way, a single algorithm can handle face detection and facial emotion detection as described above. Such a machine learning algorithm can be a so-called end-to-end convolutional neural network (CNN). Other technologies are also available, such as what is known as long short-term memory (LSTM), which can be used for time-dependent features. For reasons of efficiency, the analysis steps described above can be combined into a network (a so-called multi-task network), whereby several outputs are generated. This is accomplished by appending multiple parallel layers (called heads) to the end of a network to extract a key feature. The multi-task web may have a "face bounding box" head, an "emotional analysis" head, and a "segmentation" head.

In some embodiments, the captured images are displayed on a graphical interface that is configured to be operated by a user such that the user can manually select the individual images to be composited to yield the composite image. The graphical interface can be displayed on a computer screen where the user can see the images to choose from. The interface may also include a feature such that the images are presented on a timeline so that the user can see the change in emotion or expression of the faces at different times. Another option could be that this presentation of the images on a timeline is customized for each person appearing in the image, while the images of the people are displayed at the same time on a virtually composite common image. It is also conceivable that all persons who are to appear in the final image can be viewed, but that the images of each person can be selectively exchanged. This enables the user to find the composition of the images of people that suits him or her the. The change over time can be continuous (i.e. the entire recording is used) or discrete (i.e. only the most interesting images are used, based on a scoring algorithm that selects important images based on people's emotions).

In some embodiments, two or more of the captured images are part of a video, i. H. they each define a single image of such a video. The method can be applied in such a way that each image in a sequence is evaluated as described above. A predefined criterion is then applied to the sequence as a whole. For example, a predefined criterion such as laughter is applied as a criterion to all individual images and if all individual images have one person laughing, the predefined criterion would be met for the entire sequence.

A second aspect of the invention is directed to a computer program which, when executed on a computer, causes the computer to carry out the method according to the first aspect.

A third aspect of the invention is directed to an image processing device for combining images of more than one person, the image processing device being configured to carry out the method according to the first aspect. The device can have a suitable image sensor, for example an image sensor that is integrated in a digital camera, for recording the images. The sensor can be connected to a memory and processing unit integrated in the camera or connected to an external computer. The processing unit can have a processor for processing images. The processing unit is configured to perform the process of image analysis, selection of the segmented images whose facial emotion data of the detected emotions meet the predefined criterion, and composing a composite image based on selected segmented images according to the method of the first aspect. The device may include a graphical interface configured to display the captured, selected, and/or composite images.

The invention can be used as an entertainment solution in the automotive field, for example.

• 1 zeigt schematisch ein Blockdiagramm einer beispielhaften Ausführungsform eines Verfahrens zum Kombinieren von Bildern.
• 2 zeigt schematisch einen beispielhaften Prozess zum Aufnehmen und Analysieren eines Bildes von zwei Personen.
• 3 zeigt schematisch einen beispielhaften Prozess des Zusammensetzens eines Bildes von zwei Personen.
• 4 zeigt schematisch eine grafische Schnittstelle.

The various embodiments and advantages described above in connection with the first aspect of the present invention apply similarly to the other aspects of the invention. Further advantages, features and applications of the present invention are presented in the following detailed description and the attached figures, in which:

• 1 FIG. 12 schematically shows a block diagram of an exemplary embodiment of a method for combining images.
• 2 FIG. 12 schematically shows an exemplary process for capturing and analyzing an image of two people.
• 3 FIG. 12 schematically shows an exemplary process of compositing an image of two people.
• 4 shows a graphical interface schematically.

In the figures, the same reference numbers are used for the same or for corresponding elements of the invention described here.

DETAILED DESCRIPTION OF EMBODIMENTS

1 FIG. 1 schematically shows a block diagram 100 of an exemplary embodiment of a method for combining images. A driver 210 and a passenger 200 sit side by side on the seats of a car, for example, the front seats. A wide-angle camera 190 is placed in front of them and captures images of their front view, particularly their faces.

The images are then further processed by an image processing processor 120 . The processor for processing images 120 is contained in an electronic control unit (ECU) 110 . The ECU 110 also contains an image memory 160 in which the processed data can be stored temporarily or permanently. In a first step, the image processing processor 120 executes a segmentation algorithm 130 and a face detection algorithm 140 . The algorithm for detecting faces 140 includes recognizing the image of the face image and facial components (e.g., eyes and nose) or orienting features from the face area. It also includes extracting various spatial and time dependent features from the face components. The emotion detection algorithm 150 is applied to the extracted information of the face detection algorithm 140 . The emotion detection algorithm 150 determines the facial expression based on the extracted features by matching them with face categories databases, which may be a publicly available facial emotion recognition database, such as Compound emotion (CE), Binghamton University 3D Facial Expression (BU-3DFE), or The Extended Cohn-Kanade Dataset (CK+). This determined classified facial emotion data of the emotion recognition algorithm 150 is then transferred to the image memory 160 .

In a further step, a segmentation algorithm 130 is executed by the image processing processor 120 and applied to the images captured by the wide-angle camera 190 . The segmentation algorithm 130 splits the captured image into two separate images, one image representing the driver 210 and the other image representing the passenger 200 . These two segmented images are then transferred to the image memory 160.

It is possible that on a timeline that the face detection algorithm 140 and the emotion detection algorithm 150 are applied to the segmented image or to the originally captured image and the segmentation algorithm 130 is applied to the captured image after the Emotions of the people in the picture have been recognized.

The recorded data in the image memory 160 includes the segmented images and the corresponding information regarding the emotion detected, i. H. the facial emotion data, in each segmented image. This process of capturing images of both the driver 210 and the passenger 200, segmenting the image, applying the face detection algorithm 140, applying the emotion detection algorithm 150 to the extracted face detection algorithm data 140 and transferring the results, i. H. the segmented images and the corresponding detected emotions, into the image memory 160 is repeated several times. In this way, the image memory 160 contains a number of segmented images of the driver 210 and the passenger 200 and the corresponding information about the detected emotion of each segmented image. This information can then be used to decide which segmented images to stitch together to form a composite driver 210 and passenger 200 image. Appropriate segmented images can be selected and assembled based on a predefined criterion related to the detected emotion. For example, if the criterion is that both the driver 210 and the passenger 200 should have a pleasant facial expression, the segmented images that meet this criterion are selected. This evaluation and stitching of the segmented images is determined by an image stitching algorithm 170 executed by the image processing processor 120 . The newly composited image is then transmitted to a user interface 180 where the user can decide whether to keep the composited image. It is also possible that the user can view all segmented images or a pre-selection of segmented images and manually select which segmented images to stitch together.

2 FIG. 1 schematically shows a method for capturing and analyzing an image of two people 200, 210 sitting in a car. In a first step, an image of a driver 210 and a passenger 200 is captured, for example with a wide-angle camera 190, which captures the front views of the driver 210 and the passenger 200 seated in the front seats of a car, see the schematic drawing above in 2 . In the next step, a segmentation algorithm 130 is applied to this image, so that the driver 210 and the passenger 200 belong to different segments. Furthermore, a face detection algorithm 140 is applied to the captured image so that the face 205 of the passenger 200 and the face 215 of the driver 210 are displayed with a rectangle enclosing the face, respectively. It may be possible for the face detection algorithm 140 to be applied to the segmented images. On the basis of the extracted data obtained by the face detection algorithm 140, an emotion recognition algorithm 150 provides characterizing information allowing to determine the emotion of the faces, for example in relation to joy, laughter, surprise and others. The algorithm can assign scores to the emotions of each face; for example, passenger 200's emotions may be rated as happy=0.7, laughing=0.4, and sad=0.15. Based on these ratings, the algorithm can conclude that the passenger's 200 overall emotion is positive or happy. In another example, the driver's 210 emotions may be rated such that happy=1.0, laugh=0.9, surprise=0.4, and sad=0.05. Based on these ratings, the algorithm can conclude that the overall emotion of the driver 210 is positive or happy. On the other hand, if images with sad expressions are to be selected, those images would not be selected. In the event that the criterion is that images with To select faces representing an emotion of surprise, the image of the driver 210 could be selected. In the example of 2 the face 205 of the passenger 200 refers to a smiling expression, while the face 215 of the driver 210 refers to a laughing expression.

3 shows schematically an exemplary process of compositing an image of two people based on the analysis presented in 2 is described. Two images are recorded, one at a first point in time at time t1 and another at a different, for example later point in time at time t2. Both images represent the passenger 200 and the driver 210 in the front seats of a car. The faces and their expressions are shown schematically. At time t1, the passenger's 200 face shows a surprised expression and the driver's 210 face shows a smiling or neutral expression, see the schematic drawing at the top left of FIG 3 . By applying the segmentation algorithm 130, the images are segmented with respect to the individual persons 200, 210, respectively. By applying the information provided as in 1 described from the face detection algorithm 140 and the emotion detection algorithm 150, the images can be categorized in terms of the expressions of the faces 205, 215 of the driver 210 and the passenger 200, respectively. The same procedure is applied to the image captured at time t2. In this image, the face 205 of the passenger 200 shows a neutral expression and the face 215 of the driver 210 shows a laughing expression, see the schematic drawing at the top right of FIG 3 . This image is also segmented and the terms are categorized. In this example, the default is that one of the people in the recomposed image has a surprised expression and the other person has a laughing expression. Accordingly, the two segmented images in which both the passenger 200 and the driver 210 were selected and stitched together using an image stitching unit.

If two segmented images from originally recorded images are combined into a new image, one has to consider the backgrounds of these segmented images. In particular, in the described situation where two people are photographed sitting in a car, the background may change over time as the car moves. Because the rear window 220 and the side windows 230 in the car show views from outside the car that change as the car moves. In order to get a composite image that represents a homogeneous background, it is necessary to consider what the options are for compositing in relation to the background.

One possibility would be to capture the background image of the interior of the car including the views through the windows 220, 230 separately, for example before the two occupants get into the car. This background image is then combined with the pixels of the people represented in the corresponding selected images. In this way, the transition area on the image between the image of the people and the image of the separate background can also be homogeneous when the images are stitched together. If the images of the people 200, 210 are taken against a background that differs from the separate background, the transition area in the image between the people and the separate background may be less homogeneous when the images of the people and the separate background are stitched together .

Another possibility is that from two images of the driver 200 and the passenger 210, both backgrounds are stitched together using an unsharp mask around the people 200, 210 so that there are no overlay artifacts. It is also possible to select an image that is the main background from either of the images of the driver 200 and the front passenger 210 (occupants). Here, for example, it can be advantageous to identify that segmented image of the two images that provides more background information, i.e. the most pixels that do not belong to either person 200, 210. Another criterion would be to choose the image that has the most information in certain predefined areas, for example the image where the windows are not obscured by either occupant.

4 Figure 12 shows schematically a graphical interface that can be viewed and/or used by a user. It shows in the different drawings a), b), c) and d) the driver 210 and the passenger 200 with different facial expressions or a different combination of facial expressions at different times. To switch between the drawings, a user can move a scroll bar 240 along an axis as indicated in drawing a). If an expression of one of the persons 200, 210 meets the specifications, this can be selected. Once an image that meets the specifications is selected for both people, the process of segmentation and assembly as shown in the 1-3 described, performed automatically to create a new image.

After this semi-automatic selection of images, the new image is then automatically assembled according to the method described. Furthermore, the user interface could also have the option of selecting a degree of blurring of the background. It might also be possible to select an image suitable for the background and then composite it with the pixels of the people from the selected images.

While the foregoing has described at least one exemplary embodiment of the present invention, it should be appreciated that a large number of variations exist thereto. It should also be noted that the exemplary embodiments described are merely non-limiting examples of how the present invention may be practiced and that the devices and methods described herein are not intended to limit the scope, application, or configuration. Rather, the foregoing description will provide those skilled in the art with guidance for implementing at least one exemplary embodiment of the invention, it being understood that various changes in the operation and arrangement of elements of the exemplary embodiment may be made without departing from departing from the subject matter defined in the appended claims and their legal equivalents.

Reference List

100

block diagram

110

Electronic control unit (ECU)

120

Processor for processing images

130

segmentation algorithm

140

Algorithm for detecting faces

150

Algorithm for detecting emotions

160

image storage

170

Image compositing unit

180

user interface

190

wide angle camera

200

passenger

205

passenger's face

210

driver

215

face of the driver

220

rear window

230

sidewindow

t1

time at first time

t2

time at the second point in time

Claims (14)

A method for combining images, comprising: a) recording more than one image with a camera (190), each image representing at least a section of a front view including the face (205, 215) of one or more persons (200, 210); b) a method for image analysis, comprising: (i) applying an algorithm for segmenting images (130) to segment at least two image sections each containing a corresponding face (205, 215) or a front view including the face (205, 215) of the persons (200, 210) recorded; represent; (ii) applying a face detection algorithm (140) to detect the face (205, 215) of each person (200, 210) in the segmented images or in the captured images and to provide corresponding face detection data; (iii) applying a facial emotion recognition algorithm (150), based on the face recognition data, to recognize a corresponding emotion expressed by each of the detected faces (205, 215) and to provide corresponding facial emotion data; (iv) evaluating the facial emotion data to determine for each recognized facial emotion whether it satisfies a predefined selection criterion; c) selecting those segmented images whose corresponding facial emotion data meet the predefined selection criterion; and d) composing a combined image comprising the selected segmented images. procedure after claim 1 , wherein the method for image analysis is repeated until a segmented image has been selected for each of the persons (200, 210) represented in the recorded images. procedure after claim 1 or 2 , wherein the predefined selection criterion is met and a segmented image is selected if the data provided, which correspond to the recognized emotion of the same face (205, 215), have changed for a certain number of consecutive images. Procedure according to one of Claims 1 until 3 , where the predefined selection criterion is met and selection of a segmented image occurs when the provided data corresponding to the emotional state relates to a positive emotional state. Procedure according to one of Claims 1 until 4 , where the predefined selection criterion has more than one sub-criterion, all of which must be met in order for a segmented image to be selected. Procedure according to one of Claims 1 until 5 wherein images of people (200, 210) seated in a car are captured with a wide-angle camera to create a composite image. Procedure according to one of Claims 1 until 6 wherein a background image is captured separately and then combined with the pixels of the people (200, 210) represented in the corresponding selected images to create a composite image. Procedure according to one of Claims 1 until 6 , where the background of the composite image is the segmented image that has the most pixels related to the background. Procedure according to one of Claims 1 until 8th , where an unsharp mask is applied to the selected segmented images before they are assembled into a composite image. Procedure according to one of Claims 1 until 9 , where the face detection algorithm and the facial emotion detection algorithm are based on a machine learning algorithm. Procedure according to one of Claims 1 until 10 wherein the captured images are displayed on a graphical interface configured to be operated by a user such that the user can manually select the images to be composited. Procedure according to one of Claims 1 until 11 , where two or more of the captured images are part of a video captured video. A computer program which, when run on a computer, causes the computer to carry out the method according to any one of the preceding claims. Image processing apparatus for combining images of more than one person (200, 210), the image processing apparatus being configured to perform the method of any one of Claims 1 until 12 executes

Images