KR102111762B1 - Apparatus and method for collecting voice - Google Patents

Abstract

The voice collection device according to an embodiment includes an analysis unit for analyzing a face of a person included in an image, and a section of voice to be extracted among the voices included in the image based on the results analyzed by the analysis unit. The result of the analysis is the selection unit for selecting, an extraction unit for extracting a voice corresponding to a section selected as an object of extraction by the selection unit from the audio included in the image, and an audio extracted by the extraction unit. It includes a storage unit for storing in correspondence with the information on the person derived from.

Description

Voice collection device and method {APPARATUS AND METHOD FOR COLLECTING VOICE}

The present invention relates to a voice collection device and method.

Recently, various media services for media content have been introduced. Viewers can be provided with various information related to media content through media services. For example, the viewer may be provided with information on various objects such as people, places, background music, or products appearing in media content.

Among them, information about the characters appearing in the media content will be described. Information about the person may include, for example, profile information of the person or information on the media content itself in which the person appears.

In addition, information about the time when the person appears in the media content may be included in the information about the person. Using this point of view information, it is possible to provide a service that moves the play point of the corresponding media content to the point of time when the person appears.

On the other hand, in order to provide viewers with information on the above-mentioned persons related to media content, information such as who is the person appearing in the media content, where is the person appearing in the media content, and what is the profile of the person appearing? It must be acquired and inserted into the media content. Here, in the method of acquiring such information, for example, a producer or editor of media content directly analyzes the media content and obtains information, or a person included in the media content by applying a video analysis technique such as deep learning to the media content. After identifying cognition, there are various methods, such as a method of obtaining more specific information on the identified person on the web.

Korean Patent Registration Publication, No. 10-1855241 (Registration on April 30, 2018)

The method of extracting information about a person appearing in media content includes analyzing a video constituting the media content, analyzing a voice constituting the media content, or analyzing both a video and a voice constituting the media content. There are methods.

Here, in the case of a method of analyzing voice constituting media content or analyzing both voice and video, a voice database for each person prepared in advance may be used.

Accordingly, an object of the present invention is to provide a technique for collecting voices of characters appearing in each of these media contents from various media contents.

In addition, by using the collected voice, information about a person appearing in a predetermined media content is provided.

However, the problem to be solved of the present invention is not limited to those mentioned above, and another problem not to be solved can be clearly understood by a person having ordinary knowledge to which the present invention belongs from the following description. will be.

The voice collection device according to an embodiment includes an analysis unit for analyzing a face of a person included in an image, and a section of voice to be extracted among the voices included in the image based on the results analyzed by the analysis unit. The result of the analysis is the selection unit for selecting, an extraction unit for extracting a voice corresponding to a section selected as an object of extraction by the selection unit from the audio included in the image, and an audio extracted by the extraction unit. It includes a storage unit for storing in correspondence with the information on the person derived from.

The voice collection method according to an embodiment is performed by a voice collection device, wherein the face of a person included in the image is analyzed, and the result analyzed in the analysis step is extracted from the voice included in the image. Selecting a section of the voice to be, and extracting the voice corresponding to the section selected in the selecting step from the voice included in the image, and the extracted voice in the extracting step, the analyzed result And matching and storing information on the person derived from.

A computer program stored in a computer-readable recording medium according to an embodiment includes analyzing a face of a person included in an image, and analyzing the result analyzed in the analyzing step, the voice to be extracted from the voice included in the image The step of selecting the section, extracting the voice corresponding to the section selected in the selecting step from the voice included in the image, and extracting the voice extracted in the extraction step from the analyzed result It is programmed to perform the step of storing in association with the information about the person.

According to an embodiment, voices for each person may be obtained in the form of a database together with information about the person. In the past, it was not easy to separately collect the voices of each person, but according to an embodiment, as long as an image is given, the voices of each person can be automatically or mechanically collected from the corresponding image and databased. . In addition, not only the voice itself of each person but also the unique information characterizing the voice of each person can be obtained in the form of a database together with information about the person.

The voice of the databased person can be utilized in a process of identifying a person using only the voice later, or a process of identifying a person by combining voice and video, and in this case, the accuracy or speed of person identification can be improved. . For example, even if the person in the image is molded or dressed, or even if face recognition alone cannot identify the person (if the person is looking back or running fast, or the size of the person's face in the picture is not suitable for identification) If it is as small as, etc.), when using the database for the voice of a person according to an embodiment, information about which person appears in the corresponding video or in which part of the corresponding video is accurately and quickly obtained Can be.

1 shows a screen in which an image is being played in an image playback device.
2 illustrates a system to which a voice collection device according to an embodiment is applied.
FIG. 3 shows the configuration of the speech collection device shown in FIG. 2.
FIG. 4 exemplarily shows an image, a plurality of scenes constituting the image, and a section of audio collected among the voices included in the image.
5 exemplarily shows a procedure of a voice collection method according to an embodiment.
FIG. 6 shows the procedure of the speech collection method illustrated in FIG. 5 in more detail.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to completely inform the person having the scope of the invention, and the present invention is only defined by the scope of the claims.

In describing embodiments of the present invention, when it is determined that a detailed description of known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to a user's or operator's intention or practice. Therefore, the definition should be made based on the contents throughout this specification.

1 shows a screen in which an image is being played in the image reproducing apparatus 10. First, hereinafter, an image refers to one of 'media contents', and may be a form in which sound (voice) is combined with a still image or a moving image.

Referring to FIG. 1, an image reproduced by the image reproducing apparatus 10 includes at least one object 20 or 30. Meta-data is provided to each of the objects 20 and 30. Looking at metadata for example, for the object 20 as a person, there may be information such as the person's name, gender, date of birth, when the person appears in the video, or information about other video in which the person appears in addition to the video. Can be. On the other hand, for the object 30 as an object, there may be a name of the object, a price, or a place for sale. However, the type of the meta data or the type of the object to which the meta data is assigned is not limited to those described above.

2 illustrates a system to which the voice collection device 100 according to an embodiment is applied. However, since FIG. 2 is only an example, the speech collection device 100 is not interpreted as being limited to the system illustrated in FIG. 2.

Referring to FIG. 2, the image providing server 50 stores an image. Examples of such videos may include dramas, news, show programs, or movies, but are not limited thereto. The video providing server 50 may be a broadcasting device provided or installed in a broadcasting station or the like.

The video providing server 50 provides video to the audio collection device 100. These images provided to the voice collection device 100 are used in the voice collection device 100 to construct a voice database for a person.

Even after the voice database for the person is completed in the voice collection device 100, the image providing server 50 provides the image to the voice collection device 100. In this case, the purpose of providing the image from the image providing server 50 to the voice collection apparatus 100 may be to obtain information about a person appearing in the image from the voice collection apparatus 100, but is not limited thereto. It does not work.

The video reproducing apparatus 10 is a device for reproducing an image to which meta data about a person is assigned. The video reproducing apparatus 10 may be, for example, a TV, a computer, or a smart device, but is not limited thereto.

The video reproduced by the video reproducing device 10 is that the video reproducing device 10 is provided from the video providing server 50 or the video reproducing device 10 is directly provided from the audio collection device 100. Can be.

The voice collection device 100 will be described. The voice collection device 100 receives various images from the video providing server 50 to identify who is the person appearing in the video, extracts the voice of the identified person, and extracts the extracted information about the person. Save in the form of a database with. That is, the voice collection device 100 collects the voice of a person and processes it in a database form.

In addition, when a plurality of voices for the same person are gathered, the voice collection device 100 extracts unique information for identifying the voice by applying various techniques such as a voice fingerprint to the plurality of voices, and then extracts the unique information. It can be stored in the aforementioned database.

In addition, when a video is given, the voice collection device 100 extracts metadata such as who is the person appearing in the video, and then provides the metadata to the video providing server 50 or meta data to the video. After granting this, the video itself may be returned to the video providing server 50. Hereinafter, the voice collection device 100 will be described in more detail.

FIG. 3 shows a configuration of the voice collection device 100 according to an embodiment, but the one shown in FIG. 3 is merely exemplary. The voice collection device 100 may be implemented in a PC or a server.

Referring to FIG. 3 together with FIG. 2, the voice collection device 100 includes a storage unit 120, an analysis unit 130, a selection unit 140 and an extraction unit 150, and according to an embodiment, the communication unit ( 110), at least one of a unique information extraction unit 160 and a person discrimination unit 170 may be included. In addition, although not shown in FIG. 3, various other configurations necessary for the implementation of the speech collection device 100 may be included in the speech collection device 100.

First, the communication unit 110 may be implemented as a wired / wireless communication module that transmits and receives data. The voice collection device 100 may transmit and receive data to and from the video providing server 50 or the video reproducing device 10 through the communication unit 110.

The storage unit 120 may be implemented as a memory for storing data. In the storage unit 120, voices for each of a plurality of characters are stored in the form of a database together with information about the corresponding people, and furthermore, unique information extracted from the voices of each character may be stored. The information stored is not limited to this.

Meanwhile, the analysis unit 130, the selection unit 140, the extraction unit 150, the unique information extraction unit 160, and the person discrimination unit 170 shown in FIG. 3 are each programmed to perform the functions described below. It can be implemented by the memory that stores the instructions and the microprocessor that executes these instructions.

First, the analysis unit 130 will be described. The analysis unit 130 analyzes the image. Here, the image to be analyzed by the analysis unit 130 may be provided by the audio collection device 100 from the image providing server 50 through the communication unit 110, but is not limited thereto.

The analysis unit 130 may divide an image to be analyzed into a plurality of still images, that is, a plurality of scenes.

Also, the analysis unit 130 may extract a feature vector from each of the divided scenes. Using the extracted feature vector, the analysis unit 130 may analyze whether each face includes a face, and if the face is included, how many faces are included. In addition, the analysis unit 130 may obtain information about who the person corresponding to each face is by using an image identification algorithm such as a convolutional neural network, and when the person is identified, the corresponding face is the next scene. You can also trace whether it appears continuously in.

In addition, the analysis unit 130 may analyze whether there is movement on the lips of the face included in each image using the aforementioned feature vector, and if there is lip movement, whether the degree of such movement exceeds a predetermined criterion Can be analyzed. Here, the lip movement can be discriminated in consideration of the feature vectors of each of a predetermined number of scenes arranged back and forth when based on the corresponding scene. In addition, the analysis unit 130 analyzes the direction (eg, whether it is facing the front, etc.) of the face included in each image on the screen where the image is displayed using the above-described feature vector. can do.

Here, the process of dividing an image into a plurality of scenes itself, the process of extracting a feature vector from each scene itself, and the process of identifying or analyzing a person corresponding to a face using the feature vector itself are well-known techniques. The description will be omitted.

In addition, the process in which the analysis unit 130 divides the image into a plurality of scenes or extracts a feature vector from each of the divided scenes is only one exemplary process for performing image analysis. That is, according to an embodiment, the above-described analysis process may be performed without dividing the image into a plurality of scenes, or although the image is divided into a plurality of scenes, the above-described analysis process without extracting feature vectors from each of these scenes Can be performed.

The selection unit 140 selects at least one section of the voice to be extracted among the voices included in the image, based on the results analyzed by the analysis unit 130. Here, the term 'voice section' refers to a section between the start point and the end point, which is defined as the start point and end point of the extraction. The length of the speech section may have various values. For example, the selection unit 140 corresponds to a first audio section corresponding to a scene from 1 minute 10 seconds to 1 minute 15 seconds in a video having a running time of 2 hours, and a scene from 1 minute 30 seconds to 1 minute 40 seconds. A plurality of audio sections can be selected, such as a second audio section, and a third audio section corresponding to a scene from 3 minutes 30 seconds to 3 minutes 45 seconds. However, the number of voice sections is three, the start point and end point of each voice section, and the length of the voice section is 5 seconds, 10 seconds, 15 seconds, etc., and is merely exemplary. In addition, depending on the embodiment, the length of the voice section may be 0.1 seconds or 0.01 seconds, or 1 minute or 10 minutes.

When the selection unit 140 selects a section of the voice to be extracted, the following criteria may be used, but these criteria are only exemplary.

First, the selection unit 140 may consider the result of how many faces are included in each scene, analyzed by the analysis unit 130.

As a result of the analysis, the scene containing one face will be described first. In this case, if the corresponding scene corresponds to any one of the following examples, the selection unit 140 may select an audio section corresponding to the corresponding scene as an object of extraction, but if the corresponding scene does not correspond to any one example, the selected scene may be selected. It can be excluded from selection.

Example 1: Scenes with movements above a certain standard on the lips of the face shown in the scene

2nd example: A scene in which the face faces the front of the screen where the scene is displayed while there is a movement of a predetermined criterion on the lips of the face shown in the scene

3rd example: A scene in which the faces appearing in the scene corresponding to the second example above appear in succession in subsequent scenes, and the lips have a movement of a predetermined standard or higher (in the third example, the face does not face the front of the screen) May be selected as an object)

On the other hand, let's take a look at the scene where multiple faces appear. In this case, the selection unit 140 examines whether there is a movement of a predetermined criterion or more on only one of the lips included in each of the plurality of faces shown in the corresponding scene. If there is more than a predetermined movement on two or more lips, the selection unit 140 excludes the corresponding scene from selection. However, if there is a movement of more than a predetermined criterion on only one lip, the selection unit 140 may select a scene to be extracted based on the three examples described above, referring to the scene where one face appears. .

The extraction unit 150 considers the section of the voice selected as the object of extraction by the selection unit 140, and extracts a voice corresponding to the selected section of the voice from the voice included in the image. Here, since the extraction unit 150 extracts a voice corresponding to a predetermined section from the voice included in the image itself, a description thereof will be omitted.

The storage unit 120 stores the voice extracted by the extraction unit 150 in the form of a database by correlating the extracted voice with information about the person who issued the voice. Here, the information on the person who emitted the extracted voice may be provided from the analysis unit 130.

That is, according to an embodiment, voices for each person may be obtained in the form of a database together with information about the person. In the past, it was not easy to separately collect the voices of each person, but according to an embodiment, as long as an image is given, the voices of each person can be automatically or mechanically collected from the corresponding image and databased. .

The voice of the databased person can be utilized in a process of identifying a person using only the voice later, or a process of identifying a person by combining voice and video, and in this case, the accuracy or speed of person identification can be improved. . For example, even if the person in the image is molded or dressed, or even if face recognition alone cannot identify the person (if the person is looking back or running fast, or the size of the person's face in the picture is not suitable for identification) If it is as small as, etc.), when using the database for the voice of a person according to an embodiment, information about which person appears in the corresponding video or in which part of the corresponding video is accurately and quickly obtained Can be.

Meanwhile, FIG. 4 is a section of a voice selected as an extraction target among images that are analysis targets, a plurality of scenes divided from these images, and voices corresponding to each of the plurality of scenes, which are the results of applying the technology according to an embodiment. Conceptually.

As described above, the analysis unit 130 divides the image 131 into a plurality of scenes, and FIG. 4 shows scenes of parts a1 to a8 that are temporally adjacent among the plurality of scenes. The selection unit 140 selects a section of the voice to be extracted among the voices included in the image 131 based on the analysis results for each of the scenes a1 to a8, and the selected section is the scene It is a section b1 of a speech corresponding to a1, a section b2 of a speech corresponding to scene a3, a section b3 of a speech corresponding to scenes a4 to a6, and a section b4 of a speech corresponding to scene a8.

Here, the sections b2 and b3 are separated from each other even though they are sections for scenes a3, a4 to a6 adjacent to each other. This means that, for example, the person appearing in scene a3 corresponding to section b2 is different from the person appearing from scenes a4 to a6 corresponding to section b3, so that the voice for section b2 and the voice for section b3 must be distinguished from each other. It may mean.

Referring back to FIG. 3, the unique information extracting unit 160 extracts unique information about the voice of each person by using a plurality of voices of each person stored in the storage unit 120. At this time, a known technique such as a voice fingerprint may be used, but is not limited thereto. The extracted unique information is matched to each person and stored in the storage unit 120.

That is, according to an embodiment, not only the voice itself of each person but also the unique information characterizing the voice of each person can be obtained in the form of a database together with information about the person.

On the other hand, the voice collection device 100 according to an embodiment may collect a voice of a person or unique information extracted from the voice and provide it in the form of a database. In addition, the following functions may be provided. For example, if an arbitrary image is provided from the image providing server 50, after extracting metadata such as information about a person appearing in the image, the metadata itself is provided to the image providing server 50 or the corresponding image It may be given to the metadata to be returned to the image providing server 50. This will be described with reference to the person discrimination unit 170 shown in FIG. 3.

The person discrimination unit 170 may be learned by machine learning or deep learning. The input data at the time of learning is voice to the storage unit 120, and the correct answer data may be information about a person stored in the storage unit 120 to correspond to the input data voice. After learning is completed, when a predetermined image is input to the person discrimination unit 170, the person discrimination unit 170 may output a person who appears in the corresponding image.

Here, since the process of learning by the person discrimination unit 170 itself or the process of operating after learning is completed is a technique known in the field of machine learning or deep learning, a description thereof will be omitted.

5 illustrates a procedure of a voice collection method according to an embodiment. The voice collection method illustrated in FIG. 5 may be performed by the voice collection device 100 described above.

Meanwhile, the procedure illustrated in FIG. 5 is merely exemplary, and thus the spirit of the present invention is not limited to that illustrated in FIG. 5, and the procedure may be performed in a different order from that illustrated in FIG. 5 according to an embodiment. , At least one of the procedures illustrated in FIG. 5 may not be performed, and a procedure not illustrated in FIG. 5 may be additionally performed.

First, a step (S100) of analyzing the face of a person included in the image is performed.

Thereafter, based on the result analyzed in step S100, a step (S200) of selecting a section of the voice to be extracted among the voices included in the image is performed.

Thereafter, a step S300 of extracting a voice corresponding to the section selected in step S200 from the voice included in the image is performed.

Thereafter, a step (S400) of storing the voice extracted in step S300 in association with information about the person derived from the result analyzed in step S100 is performed.

Hereinafter, the voice collection method illustrated in FIG. 5 will be described with reference to FIG. 6, which shows one embodiment, which is further subdivided.

Referring to FIG. 6 together with FIGS. 1 to 5, the voice collection device 100 receives an image to be analyzed from the image providing server 50 through the communication unit 110 (S1000).

The analysis unit 130 divides the image provided in step S1000 into a plurality of still images, that is, a plurality of scenes (S1050).

In addition, although not shown in FIG. 6, the analysis unit 130 extracts a feature vector from each scene divided in step S1050.

The analysis unit 130 may analyze whether a face is included in each scene or how many faces are included if each face is included based on the feature vector extracted for each scene (S1100). In addition, the following information may be further analyzed in step S1100. For example, information about a person's face included in each scene, whether there is movement on the lips of the face included in each scene, and if there is lip movement, whether the degree of such movement exceeds a predetermined criterion, each Information on which direction the face included in the image faces (eg, facing the front) on the screen on which the image is displayed may be analyzed.

In step S1150, the selection unit 140 selects any one scene not previously selected from the plurality of scenes divided in step S1050.

Hereinafter, a case where one face is included in the scene selected in step S1150 will be described (S1200). If the scene selected in step S1150 corresponds to any one of the following examples, the selection unit 140 may select a section of speech corresponding to the scene as an extraction target. However, if the scene selected in step S1150 does not correspond to any one example, the scene may be excluded from selection, and thereafter, the process moves to step S1400.

Example 1: Scenes with movements above a certain standard on the lips of the face shown in the scene

Example 2 (Step S1210): A scene in which the face faces the front of the screen on which the scene is displayed while there is a movement of more than a predetermined standard on the lips of the face shown in the scene

3rd example: A scene in which the faces appearing in the scene corresponding to the second example above appear in succession in subsequent scenes, and the lips have a movement of a predetermined standard or higher (in the third example, the face does not face the front of the screen) May be selected as an object)

Here, although the step S1210 illustrated in FIG. 6 is shown to correspond only to the second example and not to the first example and the third example, the spirit of the present invention is not limited to that illustrated in FIG. 6. For example, in step S1210 according to an embodiment, a scene for the first example or the third example may be selected unlike the one illustrated in FIG. 6.

Unlike this, the scene in which multiple faces appear will be described (S1300). The selection unit 140 examines whether there is a movement of a predetermined criterion or more on only one of the lips included in each of the plurality of faces shown in the corresponding scene. If there is more than a predetermined movement on two or more lips, the selection unit 140 excludes the corresponding scene from selection. However, if there is a movement above a predetermined criterion on only one lip, the selection unit 140 selects a scene to be extracted based on the above-mentioned three examples referring to the scene where one face appears (S1310). ), The extracting unit 150 extracts a section of audio corresponding to the selected scene from the audio included in the image (S1320).

The voice extracted in step S1220 or step S1320 by the extracting unit 150 is stored in the storage unit 120 along with information of a person who matches it (S1230 or S1330).

After step S1230 or S1330, the selection unit 140 looks at whether the scene not selected in step S1150 is among a plurality of scenes divided in step S1050. If any, the flow goes to step S1150. However, if not, the unique information extraction unit 160 extracts unique information for identifying each person based on the voice of each person stored in the storage unit 120 (S1500).

That is, according to an embodiment, voices for each person may be obtained in the form of a database together with information about the person. In the past, it was not easy to separately collect the voices of each person, but according to an embodiment, as long as an image is given, the voices of each person can be automatically or mechanically collected from the corresponding image and databased. .

The voice of the databased person can be utilized in a process of identifying a person using only the voice later, or a process of identifying a person by combining voice and video, and in this case, the accuracy or speed of person identification can be improved. . For example, even if the person in the image is molded or dressed, or even if face recognition alone cannot identify the person (if the person is looking back or running fast, or the size of the person's face in the picture is not suitable for identification) If it is as small as, etc.), when using the database for the voice of a person according to an embodiment, information about which person appears in the corresponding video or in which part of the corresponding video is accurately and quickly obtained Can be.

On the other hand, in the case of the voice collection method illustrated in FIGS. 5 and 6, it may be implemented based on substantially the same technical idea as the voice collection device 100 illustrated in FIG. 3. The portions described above will use the description portion for the voice collection device 100 described with reference to FIGS. 2 to 4.

Meanwhile, each step included in the method for collecting voice according to the above-described embodiment may be implemented in a computer-readable recording medium recording a computer program programmed to perform such steps.

Meanwhile, each step included in the method for collecting voice according to the above-described embodiment may be implemented in a computer-readable recording medium recording a computer program programmed to perform such steps.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and variations without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

According to an embodiment, the voice of each person may be obtained in the form of a database together with information about the person. In the past, it was not easy to separately collect the voices of each person, but according to an embodiment, as long as an image is given, the voices of each person can be automatically or mechanically collected from the corresponding image and databased. .

100: voice collection device

Claims (10)

An analysis unit analyzing faces of at least one person included in the image;
A selection unit for selecting a section of the voice to be extracted among the voices of the at least one person included in the image based on the results analyzed by the analysis unit;
An extraction unit for extracting a voice corresponding to a section selected as an object of extraction by the selection unit from the audio included in the image;
And a storage unit for storing the voice extracted by the extraction unit in association with information on the person derived from the analyzed result,
The analysis unit,
Analyzing whether the lips included in the face of the at least one person included in the image moves above a predetermined criterion,
The selection unit,
When the lips included in the faces of a plurality of persons in the first scene of the image move above a predetermined criterion, the first scene is not selected as a section of the voice to be extracted.
Voice collection device. According to claim 1,
The analysis unit,
Analyzing the movement of the lips included in the face
Voice collection device. According to claim 2,
The analysis unit,
On the screen where the image is displayed, the direction of the face is additionally analyzed.
Voice collection device. The method of claim 3,
The selection unit,
According to the analysis of the analysis unit, a section in which the direction the face faces is analyzed as being front to the screen is selected as a section of speech to be extracted.
Voice collection device. According to claim 1,
The selection unit,
The selection process is performed on a scene in which the number of faces in the scene included in the image is one.
Voice collection device. According to claim 1,
The selection unit,
When one of the lips included in each of the faces of the plurality of persons appearing in the second scene of the image moves above the predetermined reference, performing the selection process for the second scene
Voice collection device. According to claim 1,
The person stored in the storage unit is input data, and information about the person stored in the storage unit is used as correct answer data so as to correspond to the input data, and a person corresponding to the predetermined input is input when a predetermined voice is input. Further comprising a person discrimination unit for outputting information about
Voice collection device. According to claim 1,
Further comprising a unique information extraction unit for extracting the unique information from the voice stored in the storage unit, and matching the extracted unique information with information about the person corresponding to the extracted voice of the unique information
Voice collection device. A speech collection method performed by a speech collection device,
Analyzing a face of at least one person included in the image;
Selecting a section of the voice to be extracted among the voices of the at least one person included in the image based on the analyzed result in the analyzing step;
Extracting a voice corresponding to the selected section from the voice included in the video;
And storing the extracted voice in association with information on a person derived from the analyzed result, and storing the extracted voice.
The step of analyzing the face of the person,
And analyzing whether the lips included in the face of the at least one person included in the image move above a predetermined criterion,
The step of selecting the section of the voice,
When the lips included in the faces of a plurality of persons in the first scene of the image move above a predetermined criterion, the first scene is not selected as a section of the voice to be extracted.
How to collect voice. A computer program stored on a computer-readable recording medium,
The computer program,
Analyzing a face of at least one person included in the image;
Selecting a section of the voice to be extracted among the voices of the at least one person included in the image based on the analyzed result in the analyzing step;
Extracting a voice corresponding to the selected section from the voice included in the video;
And storing the extracted voice in association with information on a person derived from the analyzed result, and storing the extracted voice.
The step of analyzing the face of the person,
And analyzing whether the lips included in the face of the at least one person included in the image move above a predetermined criterion,
The step of selecting the section of the voice,
When the lips included in the faces of the plurality of persons in the first scene of the image move above a predetermined criterion, the processor may perform a method of not selecting the first scene as a section of voice to be extracted. Containing commands for,
Computer program.

Images