JPWO2016056408A1 - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

The image processing apparatus (4) includes, for each image included in the image group acquired in time series, an area detection unit (451) that detects an invalid area other than an effective area useful for observation in the image, and an image group A plurality of target images that are targets for calculating the similarity from the images included in the image are set, and for each of the plurality of target images, the target image and the images included in the image group are time-sequential to the target image. A similarity calculation unit (452) that calculates a similarity between adjacent images adjacent to the image, and an image selection unit (453) that selects a representative image from the plurality of target images based on the similarity for each of the plurality of target images. With. The similarity calculation unit (452) is configured to obtain a gap between the effective area excluding the invalid area in one image of the target image or the adjacent image and the area corresponding to the effective area in the other image of the target image or the adjacent image. To calculate the similarity.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

Conventionally, an image summarization process is performed in which an image group including a plurality of images acquired in time series is acquired, and a part of the images is extracted from the image group and summarized into an image group having a smaller number than the original image group. An image extracting device (image processing device) that performs such processing is known (see, for example, Patent Document 1).
In the image processing apparatus described in Patent Document 1, images at positions where the scene changes are selected from the image group as representative images, and the image group is summarized into a predetermined number of representative images.
Then, the user can grasp the contents of the entire original image group in a short time by observing a predetermined number of representative images included in the image group after the image summarization process.

JP 2009-5020 A

However, in the image summarization process described in Patent Document 1, since the condition for selecting the representative image is a change in the scene, the invalid area other than the effective area useful for observation is included in the selected number of representative images. There is a problem that an image including many areas may be included.
For example, when the image summarization process is performed on an in-vivo image group introduced into a subject and captured by a capsule endoscope that images the subject, a predetermined number of representatives selected There is a possibility that an in-vivo image including many ineffective areas unnecessary for observation of bubbles, residues and the like is included in the image, not an effective area useful for observation of the mucous membrane or the like in the subject.

  The present invention has been made in view of the above, and provides an image processing apparatus, an image processing method, and an image processing program that can select more images including many effective regions useful for observation as representative images. There is to do.

  In order to solve the above-described problems and achieve the object, the image processing apparatus according to the present invention provides, for each image included in an image group acquired in time series, a region other than an effective region useful for observation in the image. An area detection unit for detecting an invalid area, and a plurality of target images for which similarity is calculated from images included in the image group are set, and the target image and the image group are set for each of the plurality of target images. A plurality of targets based on a similarity calculation unit that calculates a similarity with an adjacent image that is adjacent to the target image in time series among the included images, and the similarity for each of the plurality of target images An image selection unit that selects a representative image from the image, and the similarity calculation unit includes the effective region excluding the invalid region in one of the target image and the adjacent image, and the target Image or adjacent image And calculates the similarity between a region corresponding to the effective area of any in the other images of the.

  The image processing apparatus according to the present invention includes, for each image included in an image group acquired in time series, an area detection unit that detects an invalid area other than an effective area useful for observation in the image, and the image A plurality of target images for which similarity is calculated from images included in the group are set, and for each of the plurality of target images, the target image and the image included in the image group are compared with the target image. A similarity calculation unit that calculates the similarity between adjacent images that are adjacent in series, and an image selection unit that selects a representative image from the plurality of target images based on the similarity for each of the plurality of target images. The similarity calculation unit, when the target image and the adjacent image are overlapped, the effective area excluding the invalid area in the target image and the invalid area in the adjacent image Effective area And calculates the similarity between the overlapping regions when.

  The image processing apparatus according to the present invention is the image processing apparatus according to the above invention, wherein the similarity calculation unit sets, as the target image, an image including the effective area excluding the invalid area among images included in the image group. It is characterized by doing.

  In the image processing apparatus according to the present invention as set forth in the invention described above, the image selection unit selects a predetermined number of the representative images from the plurality of target images in ascending order of similarity.

  The image processing method according to the present invention is an image processing method performed by an image processing apparatus. For each image included in an image group acquired in time series, an invalid area other than an effective area useful for observation in the image. A plurality of target images for which similarity is calculated from images included in the image group, and each target image is included in the target image and the image group. Based on the similarity calculation step for calculating the similarity between adjacent images of the target image in time series among the images, and based on the similarity for each of the plurality of target images, An image selection step of selecting a representative image, and in the similarity calculation step, the effective region excluding the invalid region in either one of the target image or the adjacent image, and Serial and calculates the degree of similarity between the target image or the region corresponding to the effective area of any in the other image of the adjacent images.

  The image processing method according to the present invention is an image processing method performed by an image processing apparatus. For each image included in an image group acquired in time series, an invalid area other than an effective area useful for observation in the image. A plurality of target images for which similarity is calculated from images included in the image group, and each target image is included in the target image and the image group. Based on the similarity calculation step for calculating the similarity between adjacent images of the target image in time series among the images, and based on the similarity for each of the plurality of target images, An image selection step of selecting a representative image, and in the similarity calculation step, when the target image and the adjacent image are superimposed, the effective area excluding the invalid area in the target image Characterized in that the said effective region excluding the invalid area in range and said adjacent images to calculate the similarity between the overlapping area of each other.

  An image processing program according to the present invention causes an image processing apparatus to execute the above-described image processing method.

The image processing apparatus according to the first aspect of the present invention sets a plurality of target images that are targets for calculating similarity from images included in an image group acquired in time series, and for each of the plurality of target images, An effective area excluding the invalid area in either one of the target image or the adjacent image, and an effective area in the one image in the other image of the target image or the adjacent image The similarity is calculated between the corresponding regions. In other words, the image processing apparatus sets the calculation area for calculating the similarity as an effective area in the target image or the adjacent image, and calculates the similarity between the same calculation areas in the target image and the adjacent image. Then, the image processing apparatus selects, for example, a target image that has a similarity with a relatively low similarity value as a representative image from the plurality of target images.
As described above, according to the image processing device according to the first aspect of the present invention, an image including a large number of effective regions useful for observation is reduced by reducing the contribution to the similarity calculation processing of invalid regions. As a result, it is possible to select more.

The image processing apparatus according to the second aspect of the present invention sets a plurality of target images that are targets for calculating similarity from images included in an image group acquired in time series, and for each of the plurality of target images, When the target image and the adjacent image are overlapped, the similarity is calculated between the regions where the effective regions overlap each other. Then, the image processing apparatus selects, for example, a target image that has a similarity with a relatively low similarity value as a representative image from the plurality of target images.
As described above, according to the image processing apparatus according to the second aspect of the present invention, the similarity is calculated between the effective regions, so that the contribution to the similarity calculation processing of the invalid region can be eliminated. Therefore, according to the image processing device according to the second aspect of the present invention, as in the image processing device according to the first aspect of the present invention described above, more images including many effective regions useful for observation are used as representative images. There is an effect that it can be selected.

Since the image processing method according to the present invention is a method performed by the above-described image processing apparatus, the same effect as the above-described image processing apparatus can be obtained.
Since the image processing program according to the present invention is a program executed by the above-described image processing apparatus, the same effect as that of the above-described image processing apparatus can be obtained.

FIG. 1 is a schematic diagram showing an endoscope system according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing the image processing apparatus shown in FIG. FIG. 3 is a flowchart showing the operation (image processing method) of the image processing apparatus shown in FIG. FIG. 4 is a diagram for explaining the image processing method shown in FIG. FIG. 5A is a diagram for explaining step S5 shown in FIG. FIG. 5B is a diagram for explaining step S5 shown in FIG. FIG. 6A is a diagram for explaining step S5 according to Embodiment 2 of the present invention. FIG. 6B is a diagram for explaining step S5 according to Embodiment 2 of the present invention. FIG. 7A is a diagram for explaining step S5 according to Embodiment 3 of the present invention. FIG. 7B is a diagram for explaining step S5 according to Embodiment 3 of the present invention. FIG. 8A is a diagram for explaining step S5 according to Embodiment 4 of the present invention. FIG. 8B is a diagram for explaining step S5 according to Embodiment 4 of the present invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of an image processing device, an image processing method, and an image processing program according to the invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

(Embodiment 1)
[Schematic configuration of endoscope system]
FIG. 1 is a schematic diagram showing an endoscope system according to Embodiment 1 of the present invention.
The endoscope system 1 is a system that acquires an in-vivo image inside a subject 100 using a swallowable capsule endoscope 2 and causes a doctor or the like to observe the in-vivo image.
As shown in FIG. 1, the endoscope system 1 includes a receiving device 3, an image processing device 4, and a portable recording medium 5 in addition to the capsule endoscope 2.
The recording medium 5 is a portable recording medium for transferring data between the receiving device 3 and the image processing device 4, and is configured to be detachable from the receiving device 3 and the image processing device 4. Yes.

The capsule endoscope 2 is a capsule endoscope device that is formed in a size that can be introduced into the organ of the subject 100, and is introduced into the organ of the subject 100 by oral ingestion or the like, and is peristaltic. In-vivo images are sequentially taken while moving inside the organ by means of, for example. Then, the capsule endoscope 2 sequentially transmits image data generated by imaging.
The receiving device 3 includes a plurality of receiving antennas 3a to 3h, and receives image data from the capsule endoscope 2 inside the subject 100 via at least one of the plurality of receiving antennas 3a to 3h. Then, the receiving device 3 stores the received image data in the recording medium 5 inserted in the receiving device 3.
The receiving antennas 3a to 3h may be arranged on the body surface of the subject 100 as shown in FIG. 1 or may be arranged on a jacket worn by the subject 100. Further, the number of receiving antennas provided in the receiving device 3 may be one or more, and is not particularly limited to eight.

[Configuration of image processing apparatus]
FIG. 2 is a block diagram showing the image processing apparatus 4.
The image processing apparatus 4 is configured as a workstation that acquires image data in the subject 100 and displays an image corresponding to the acquired image data.
As shown in FIG. 2, the image processing apparatus 4 includes a reader / writer 41, a memory unit 42, an input unit 43, a display unit 44, and a control unit 45.

The reader / writer 41 has a function as an image acquisition unit that acquires image data to be processed from the outside.
Specifically, when the recording medium 5 is inserted into the reader / writer 41, the reader / writer 41 is controlled by the control unit 45 to store image data (stored in the capsule endoscope 2). A group of in-vivo images including a plurality of in-vivo images captured (acquired) in time series. Further, the reader / writer 41 transfers the captured in-vivo image group to the control unit 45. The in-vivo image group transferred to the control unit 45 is stored in the memory unit 42.

  The memory unit 42 stores the in-vivo image group transferred from the control unit 45. In addition, the memory unit 42 stores various programs (including an image processing program) executed by the control unit 45, information necessary for processing of the control unit 45, and the like.

  The input unit 43 is configured using a keyboard, a mouse, and the like, and accepts user operations.

  The display unit 44 is configured using a liquid crystal display or the like, and includes a display screen including in-vivo images under the control of the control unit 45 (for example, a display screen including a predetermined number of representative images selected by image summarization processing described later) ) Is displayed.

The control unit 45 is configured using a CPU (Central Processing Unit) or the like, reads a program (including an image processing program) stored in the memory unit 42, and controls the operation of the entire image processing apparatus 4 according to the program.
Hereinafter, as a function of the control unit 45, a function of executing “image summarization processing” which is a main part of the present invention will be mainly described.
As illustrated in FIG. 2, the control unit 45 includes an area detection unit 451, a similarity calculation unit 452, and an image selection unit 453.

The area detection unit 451 detects, for each in-vivo image included in the in-vivo image group stored in the memory unit 42, an invalid area other than the effective area useful for observation in the in-vivo image.
Specifically, the region detection unit 451 compares the feature value indicating the color information, frequency information, shape information, and the like that can be acquired from the in-vivo image with the second threshold value, and based on the comparison result, An invalid area other than an effective area useful for observation is detected.
Here, the effective area means an area where mucous membranes, blood vessels, and blood on the surface of the living body are reflected. On the other hand, the invalid region is a region other than the effective region, such as a region where residues or bubbles are reflected, a region where the deep part of the lumen is reflected (dark portion), a halation region (bright portion) that is specularly reflected from the surface of the subject, It means a region that becomes noise due to a poor communication state between the capsule endoscope 2 and the receiving device 3.
It should be noted that various known methods can be employed as the method for detecting the invalid area as described above (for example, JP 2007-313119 A, JP 2011-234931 A, and JP 2010-115413 A). Publication, Unexamined-Japanese-Patent No. 2012-16454, etc.).

The similarity calculation unit 452 sets a plurality of target images that are targets for calculating the similarity from the in-vivo images included in the in-vivo image group stored in the memory unit 42, and sets the target image for each of the plurality of target images. And the degree of similarity between the immediately preceding in-vivo image (hereinafter referred to as the immediately preceding adjacent image) in time series with respect to the target image.
Here, the similarity calculation unit 452 calculates the similarity between the target image and the immediately preceding adjacent image, the effective region excluding the invalid region detected by the region detection unit 451 in the target image, and the immediately preceding The degree of similarity is calculated with an area corresponding to the effective area of the target image in the adjacent image (having the same positional relationship as the effective area of the target image). In other words, the similarity calculation unit 452 sets the calculation area for calculating the similarity as an effective area in the target image, and calculates the similarity between the same calculation areas in the target image and the immediately preceding adjacent image.
In the first embodiment, the similarity calculation unit 452 calculates a normalized cross-correlation value as the similarity between the calculation regions in the target image and the immediately adjacent image. As a method of calculating the normalized cross-correlation value, various known methods can be employed (for example, International Publication No. 2012/117816).
Note that the similarity between the calculation regions in the target image and the immediately preceding adjacent image is not limited to the normalized cross-correlation value described above. For example, the motion vector change amount and the change in pixel value (luminance value or G component value) An amount or the like may be adopted as the similarity.

  The image selection unit 453 selects a predetermined number of representative images from the target image based on the similarity for each target image calculated by the similarity calculation unit 452.

[Operation of image processing apparatus]
Next, the operation (image processing method) of the above-described image processing apparatus 4 will be described.
FIG. 3 is a flowchart showing the operation (image processing method) of the image processing apparatus 4. FIG. 4 is a diagram for explaining the image processing method shown in FIG.
Specifically, FIG. 4A illustrates a state in which in-vivo images included in the in-vivo image group to be processed (in FIG. 4A, only the in-vivo images F1 to F14 are illustrated) are virtually arranged in time series. Is shown. FIGS. 4B and 4C show examples of invalid regions in the in-vivo image F12 (represented by hatching in FIG. 4A) included in the in-vivo image group. In FIG. 4B and FIG. 4C, the invalid area is expressed in white (in FIG. 4C, a state where there is no valid area is illustrated). FIG. 4D schematically shows that an in-vivo image set as a target image for calculating similarity is set as a candidate image as a representative image candidate when a predetermined condition is satisfied. It is a figure. FIG. 4E schematically shows that the in-vivo image set as the target image does not satisfy the predetermined condition, or that the in-vivo image not set as the target image is not set as the candidate image. FIG. FIG. 4F is a diagram schematically showing that a candidate image is selected as a representative image when a predetermined condition is satisfied.

In the following, the recording medium 5 is inserted into the reader / writer 41, the in-vivo image group stored in the recording medium 5 is taken in via the reader / writer 41, and the in-vivo image group is already stored in the memory unit 42. It shall be.
First, the control unit 45 reads all in-vivo images included in the in-vivo image group stored in the memory unit 42 one by one in time-series order (frame number order) (step S1).
Next, the region detection unit 451 detects an invalid region in the in-vivo image read in step S1 (step S2: region detection step).

Next, the similarity calculation unit 452 refers to the detection result in step S2, and determines whether or not an effective area is included in the in-vivo image (whether the entire image is not detected as an invalid area in step S2). (Step S3).
When it is determined that the in-vivo image includes an effective region (step S3: Yes), the similarity calculation unit 452 sets the in-vivo image as a target image for which the similarity is calculated (step S4). ).
For example, when the in-vivo image read in step S1 is the in-vivo image F12 (FIG. 4 (a)) and the effective area is included in the in-vivo image F12 as shown in FIG. 4 (b). The in-vivo image F12 is set as a target image.

On the other hand, when it is determined that the effective region is not included in the in-vivo image (step S3: No), the control unit 45 returns to step S1. Then, the control unit 45 executes the above-described process again for the next in-vivo image (in-vivo image F13 when step S3 is executed for in-vivo image F12).
For example, when the in-vivo image read in step S1 is the in-vivo image F12 (FIG. 4A), and the entire in-vivo image F12 is detected as an invalid area as shown in FIG. 4C, The in-vivo image F12 is a non-target image that is not a target for calculating the similarity.

After step S4, the similarity calculation unit 452 reads the in-vivo image (target image) read out in step S1 and the immediately adjacent image immediately before the target image (the target image is the in-vivo image F12). In some cases, the degree of similarity between the immediately adjacent image and the in-vivo image F11) is calculated (step S5). Then, the similarity calculation unit 452 stores the calculated similarity in the memory unit 42 in association with the target image.
Specifically, in step S5, the similarity calculation unit 452 calculates the similarity between the target image and the immediately adjacent image as described below.

5A and 5B are diagrams for explaining step S5.
Specifically, FIG. 5A is a diagram schematically showing the invalid area detected in step S2 and the valid area excluding the invalid area in the target image. FIG. 5B is a diagram schematically showing the invalid area detected in step S2 and the valid area excluding the invalid area in the immediately adjacent image. In FIG. 5A and FIG. 5B, the invalid area is expressed in white. In FIG. 5A and FIG. 5B, the calculation area for calculating the similarity is represented by a thick frame.
First, as shown in FIG. 5A, the similarity calculation unit 452 sets an effective area excluding the invalid area detected in step S2 as a calculation area in the target image.
Then, as shown in FIGS. 5A and 5B, the similarity calculation unit 452 calculates a similarity (normalized cross-correlation value) between the same calculation areas in the target image and the immediately preceding adjacent image.
Steps S4 and S5 described above correspond to the similarity calculation step according to the present invention.

  After step S5, the image selection unit 453 determines whether or not the similarity (normalized cross-correlation value) calculated in step S5 is less than the first threshold (step S6). In other words, the image selection unit 453 determines in step S6 whether or not the scene has been switched due to the transition from the immediately adjacent image to the target image.

If it is determined that the similarity between the target image and the immediately preceding adjacent image is less than the first threshold (step S6: Yes), the image selection unit 453 indicates a flag indicating that the candidate image is a candidate for a representative image. Are stored in association with the target image stored in the memory unit 42, and the target image is set as a candidate image (step S7).
For example, the in-vivo image read in step S1 is the in-vivo image F12 (FIG. 4A), and the similarity between the in-vivo image (target image) F12 and the in-vivo image (immediately adjacent image) F11 is the first. When it is less than one threshold (when similarity is low), the in-vivo image F12 is set as a candidate image as shown in FIG.

On the other hand, when it is determined that the degree of similarity between the target image and the immediately preceding adjacent image is greater than or equal to the first threshold (step S6: No), the control unit 45 returns to step S1. Then, the control unit 45 executes the above-described process again for the next in-vivo image (in-vivo image F13 when step S6 is executed with the in-vivo image F12 as the target image).
For example, the in-vivo image read in step S1 is the in-vivo image F12 (FIG. 4A), and the similarity between the in-vivo image (target image) F12 and the in-vivo image (immediately adjacent image) F11 is the first. When it is equal to or greater than one threshold (when similarity is high), the in-vivo image F12 is a non-candidate image that is not a candidate for a representative image, as shown in FIG. Note that the in-vivo image (FIG. 4C) in which the entire image is detected as an invalid area is also a non-candidate image.

After step S7, the control unit 45 determines whether or not steps S1 to S7 have been performed for all in-vivo images included in the in-vivo image group stored in the memory unit 42 (step S8).
If it is determined that not all the in-vivo images are implemented (step S8: No), the control unit 45 returns to step S1. And the control part 45 performs the process mentioned above about the remaining in-vivo images.
On the other hand, when it is determined that all in-vivo images have been performed (step S8: Yes), the image selection unit 453 converts the candidate images included in the in-vivo image group stored in the memory unit 42 into the candidate images. A predetermined number (for example, 2000) of representative images is selected in ascending order of the related similarity (step S9: image selection step).
For example, when 2000 or more candidate images exist in the in-vivo image group stored in the memory unit 42, the candidate images are represented in order from the lowest similarity as shown in FIG. Selected as an image.

The image processing apparatus 4 according to the first embodiment described above uses the calculation area for calculating the similarity as an effective area in the target image, and calculates the similarity between the same calculation area in the target image and the immediately preceding adjacent image. calculate. Then, the image processing apparatus 4 selects a target image having a similarity with a relatively low similarity value as a representative image from the plurality of target images.
As described above, according to the image processing device 4 according to the first embodiment, the representative image is an image including many effective regions useful for observation by reducing the contribution to the similarity calculation processing of the invalid region. As a result, it is possible to select more.

Further, the image processing device 4 according to the first embodiment does not set the in-vivo image in which the entire image is detected as the invalid area as the target image that is the target for calculating the similarity.
For this reason, by excluding the in-vivo image whose entire image is an invalid area as described above from the target image in advance, the target image is not selected as a representative image, and the processing load for calculating the similarity is further reduced. Can be reduced.

In addition, the image processing device 4 according to the first embodiment reduces the number of target images by setting the similarity to the first threshold (sets it as a candidate image), and makes a similarity from a plurality of candidate images. A predetermined number of representative images are selected in ascending order.
For this reason, it is possible to select as many representative images as planned by a simple process.

(Embodiment 2)
Next, a second embodiment of the present invention will be described.
In the following description, the same reference numerals are given to the same configurations and steps as those in the above-described first embodiment, and the detailed description thereof is omitted or simplified.
In Embodiment 1 described above, the calculation area for calculating the similarity is the effective area in the target image.
On the other hand, in the second embodiment, the calculation area for calculating the similarity is set as an effective area in the immediately adjacent image.
The configuration of the image processing apparatus according to the second embodiment is the same as that of the image processing apparatus 4 described in the first embodiment. The image processing method according to the second embodiment is the same as the image processing method described in the first embodiment described above, except for step S5.
Only step S5 according to the second embodiment will be described below.

[Step S5]
6A and 6B are diagrams for explaining step S5 according to Embodiment 2 of the present invention.
Specifically, FIG. 6A schematically corresponds to FIG. 5A and schematically shows an invalid area and an effective area in the target image. 6B corresponds to FIG. 5B and is a diagram schematically showing an invalid area and an effective area in the immediately adjacent image.
First, as shown in FIG. 6B, the similarity calculation unit 452 sets the effective area excluding the invalid area detected in step S2 in the immediately adjacent image as the calculation area.
Then, as shown in FIGS. 6A and 6B, the similarity calculation unit 452 calculates a similarity (normalized cross-correlation value) between the same calculation areas in the target image and the immediately preceding adjacent image.

  Even in the case where the effective region in the immediately adjacent image is configured as the calculation region as in the second embodiment described above, the same effect as in the first embodiment described above can be obtained.

(Embodiment 3)
Next, a third embodiment of the present invention will be described.
In the following description, the same reference numerals are given to the same configurations and steps as those in the above-described first embodiment, and the detailed description thereof is omitted or simplified.
In the first embodiment described above, the similarity between the target image and the immediately preceding adjacent image is calculated.
On the other hand, in the third embodiment, the degree of similarity between the target image and the in-vivo image immediately after the target image in time series (hereinafter referred to as the immediately adjacent image) is calculated.
The configuration of the image processing apparatus according to the third embodiment is the same as that of the image processing apparatus 4 described in the first embodiment. The image processing method according to the third embodiment is the same as the image processing method described in the first embodiment described above except for step S5.
Only step S5 according to the third embodiment will be described below.

[Step S5]
7A and 7B are diagrams for explaining step S5 according to Embodiment 3 of the present invention.
Specifically, FIG. 7A corresponds to FIG. 5A and is a diagram schematically showing an invalid area and an effective area in the target image. FIG. 7B is a diagram schematically showing the invalid area detected in step S2 and the valid area excluding the invalid area in the immediately adjacent image. In FIG. 7B, the invalid area is expressed in white as in FIG. 7A. In FIG. 7B, similarly to FIG. 7A, the calculation area for calculating the similarity is represented by a thick frame.
First, as shown in FIG. 7B, the similarity calculation unit 452 sets the effective area excluding the invalid area detected in step S2 in the immediately adjacent image as the calculation area.
Then, as shown in FIGS. 7A and 7B, the similarity calculation unit 452 calculates a similarity (normalized cross-correlation value) between the same calculation areas in the target image and the immediately adjacent image.

  As described above in the third embodiment, even when the effective area in the immediately adjacent image is used as the calculation area and the similarity of the target image with the immediately adjacent image is calculated, the above-described case is described above. The same effects as in the first embodiment are obtained.

(Modification of Embodiment 3)
In Embodiment 3 described above, the effective area in the immediately adjacent image is used as the calculation area. However, the present invention is not limited to this, and the effective area in the target image may be used as the calculation area as in Embodiment 1 described above.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described.
In the following description, the same reference numerals are given to the same configurations and steps as those in the above-described first embodiment, and the detailed description thereof is omitted or simplified.
In Embodiment 1 described above, the calculation area for calculating the similarity is the effective area in the target image.
On the other hand, in the fourth embodiment, when the target image and the immediately preceding adjacent image are superimposed, an area where the effective area of the target image and the effective area of the immediately preceding adjacent image overlap is calculated. The calculation area.
The configuration of the image processing apparatus according to the fourth embodiment is the same as that of the image processing apparatus 4 described in the first embodiment. The image processing method according to the fourth embodiment is the same as the image processing method described in the first embodiment described above, except for step S5.
Only step S5 according to the fourth embodiment will be described below.

[Step S5]
8A and 8B are diagrams for explaining step S5 according to Embodiment 4 of the present invention.
Specifically, FIG. 8A schematically corresponds to FIG. 5A and schematically shows an invalid area and an effective area in the target image. FIG. 8B corresponds to FIG. 5B and is a diagram schematically showing an invalid area and an effective area in the immediately adjacent image.
First, as shown in FIGS. 8A and 8B, the similarity calculation unit 452 includes an effective area excluding the invalid area detected in step S2 in the target image when the target image is superimposed on the immediately preceding adjacent image, A region where the effective region excluding the invalid region detected in step S2 in the immediately preceding adjacent image overlaps is defined as a calculation region. In other words, the similarity calculation unit 452 calculates an area in which the position of the target image and the immediately preceding adjacent image in each image has the same positional relationship between the effective area of the target image and the effective area of the immediately preceding adjacent image. This is an area.
Then, as shown in FIGS. 8A and 8B, the similarity calculation unit 452 calculates a similarity (normalized cross-correlation value) between the same calculation regions in the target image and the immediately preceding adjacent image.

According to the fourth embodiment described above, there are the following effects in addition to the effects similar to those of the first embodiment.
In the fourth embodiment, the calculation area for calculating the similarity is an area in which the effective areas overlap when the target image and the immediately preceding adjacent image are overlapped, and the calculation area for the target image and the immediately adjacent image is the same. The similarity is calculated between the same calculation areas. Then, a target image having a similarity with a relatively low similarity is selected as a representative image from the plurality of target images.
As described above, according to the fourth embodiment, since the similarity is calculated between the effective areas, the contribution to the similarity calculation processing of the invalid area can be excluded. Therefore, according to the fourth embodiment, as in the above-described embodiment, there is an effect that more images including many effective regions useful for observation can be selected as representative images.

(Modification of Embodiment 4)
In Embodiment 4 described above, the similarity between the target image and the immediately preceding adjacent image is calculated. However, the present invention is not limited to this, and as in Embodiment 3 described above, between the target image and the immediately adjacent image. You may calculate the similarity in.

(Other embodiments)
Up to this point, the mode for carrying out the present invention has been described. However, the present invention should not be limited only by the above-described first to fourth embodiments.
In Embodiments 1 to 4 described above, the image summarization process is performed on the in-vivo image group captured by the capsule endoscope 2, but the present invention is not limited to this, and the image group acquired in time series If so, the image summarization process may be executed for other image groups.

In the first to fourth embodiments described above, the image processing apparatus 4 acquires the in-vivo image group captured in time series by the capsule endoscope 2 using the recording medium 5 and the reader / writer 41. Not limited to this.
For example, the in-vivo image group is stored in advance in a separately installed server. The image processing apparatus 4 is provided with a communication unit that communicates with the server. The image processing apparatus 4 may acquire the in-vivo image group by communicating with the server using the communication unit.
That is, the communication unit has a function as an image acquisition unit that acquires image data to be processed from the outside.

  In Embodiments 1 to 4 described above, the similarity is compared with the first threshold, and the candidate image is set from the target image based on the comparison result. However, the present invention is not limited to this, and from all the target images, A predetermined number of representative images may be selected in ascending order of similarity. That is, steps S6 and S7 may be omitted.

  In Embodiments 1 to 4 described above, the in-vivo image in which the entire image is detected as an invalid region is excluded from the target for calculating the similarity, but the present invention is not limited to this, and the in-vivo image is also set as the target image All in-vivo images may be set as target images), and a high similarity value (“1” if normalized cross-correlation value) may be set as the similarity of the in-vivo images.

The processing flow is not limited to the processing order in the flowcharts described in the first to fourth embodiments, and may be changed within a consistent range.
Furthermore, the processing algorithm described using the flowcharts in this specification can be described as a program. Such a program may be recorded on a recording unit inside the computer, or may be recorded on a computer-readable recording medium. Recording of the program in the recording unit or recording medium may be performed when the computer or recording medium is shipped as a product, or may be performed by downloading via a communication network.

DESCRIPTION OF SYMBOLS 1 Endoscope system 2 Capsule-type endoscope 3 Receiving device 3a-3h Receiving antenna 4 Image processing device 5 Recording medium 41 Reader / writer 42 Memory unit 43 Input unit 44 Display unit 45 Control unit 100 Subject 451 Area detection unit 452 Similar Degree calculation unit 453 Image selection unit F1-F14 In-vivo image

Claims (7)

For each image included in the image group acquired in time series, an area detection unit that detects an invalid area other than an effective area useful for observation in the image, and
A plurality of target images for which similarity is calculated from images included in the image group are set, and the target image and the target image among the images included in the image group are set for each of the plurality of target images. A similarity calculation unit for calculating the similarity between adjacent images adjacent in time series,
An image selection unit that selects a representative image from the plurality of target images based on the similarity for each of the plurality of target images;
The similarity calculation unit includes:
The effective area excluding the invalid area in either one of the target image or the adjacent image, and the area corresponding to the effective area in the other image of the target image or the adjacent image The similarity is calculated between the image processing apparatus and the image processing apparatus. For each image included in the image group acquired in time series, an area detection unit that detects an invalid area other than an effective area useful for observation in the image, and
A plurality of target images for which similarity is calculated from images included in the image group are set, and the target image and the target image among the images included in the image group are set for each of the plurality of target images. A similarity calculation unit for calculating the similarity between adjacent images adjacent in time series,
An image selection unit that selects a representative image from the plurality of target images based on the similarity for each of the plurality of target images;
The similarity calculation unit includes:
When the target image and the adjacent image are overlapped, the effective area excluding the invalid area in the target image and the effective area excluding the invalid area in the adjacent image are overlapped with each other. An image processing apparatus that calculates the similarity. The similarity calculation unit includes:
The image processing apparatus according to claim 1, wherein an image including the effective area excluding the invalid area among images included in the image group is set as the target image. The image selection unit
The image processing apparatus according to claim 1, wherein a predetermined number of the representative images are selected from the plurality of target images in descending order of the similarity. In the image processing method performed by the image processing apparatus,
For each image included in the image group acquired in time series, an area detection step for detecting an invalid area other than an effective area useful for observation in the image; and
A plurality of target images for which similarity is calculated from images included in the image group are set, and the target image and the target image among the images included in the image group are set for each of the plurality of target images. A similarity calculation step for calculating the similarity between adjacent images adjacent in time series,
An image selection step of selecting a representative image from the plurality of target images based on the similarity for each of the plurality of target images;
In the similarity calculation step,
The effective area excluding the invalid area in either one of the target image or the adjacent image, and the area corresponding to the effective area in the other image of the target image or the adjacent image An image processing method characterized in that the similarity is calculated between and. In the image processing method performed by the image processing apparatus,
For each image included in the image group acquired in time series, an area detection step for detecting an invalid area other than an effective area useful for observation in the image; and
A plurality of target images for which similarity is calculated from images included in the image group are set, and the target image and the target image among the images included in the image group are set for each of the plurality of target images. A similarity calculation step for calculating the similarity between adjacent images adjacent in time series,
An image selection step of selecting a representative image from the plurality of target images based on the similarity for each of the plurality of target images;
In the similarity calculation step,
When the target image and the adjacent image are overlapped, the effective area excluding the invalid area in the target image and the effective area excluding the invalid area in the adjacent image are overlapped with each other. An image processing method characterized by calculating the similarity. An image processing program causing an image processing apparatus to execute the image processing method according to claim 5.

Images