JP6825058B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The disclosed techniques relate to image processing devices, image processing methods and programs.

It is known that findings (morphological abnormalities) are automatically extracted from a tomographic image of the anterior segment of the eye and the contents of the abnormalities are displayed (Patent Document 1).

JP-A-2010-268916

However, how to display the contents of the anomaly and the tomographic image in association with each other is not disclosed, and there is a problem that the doctor cannot easily grasp the relationship between the contents of the anomaly and the tomographic image. ..

The disclosed technique was made in view of the above problems, and one of the purposes is to make it possible to easily grasp the relationship between the findings and the tomographic image.

In addition to the above-mentioned purpose, other of the disclosed techniques, it is an action and effect derived by each configuration shown in the embodiment for carrying out the invention described later, and the action and effect which cannot be obtained by the conventional technique can be obtained. It can be positioned as one of the purposes.

One of the disclosed image processing devices is
A detection means for detecting an abnormal part of the eye to be inspected by analyzing a tomographic image of the eye to be inspected.
An acquisition means for acquiring a similar case image related to a similar case of the detected abnormal site by using the finding information regarding the findings of the detected abnormal site and the position information regarding the position of the detected abnormal site.
A display control means for displaying the tomographic image and the acquired similar case image on the display means is provided.

According to one of the disclosed techniques, it is possible to easily grasp the relationship between the findings and the tomographic image.

It is a figure which shows an example of the functional structure of a medical diagnosis support device. It is a figure which shows an example of the hardware composition of the medical diagnosis support device. It is a flowchart which shows an example of the processing of a medical diagnosis support apparatus. It is a figure which shows an example of image processing. It is a figure which shows an example of the table information. It is a figure which shows an example of image processing and identification of a position. It is a figure which shows an example of the display of the display part in Embodiment 1. FIG. It is a figure which shows an example of the display of the display part in Embodiment 1. FIG. It is a figure which shows an example of the functional structure of a medical diagnosis support device. It is a flowchart which shows an example of the processing of a medical diagnosis support apparatus. It is a figure which shows an example of the display of the display part in Embodiment 2. It is a flowchart which shows an example of the processing of a medical diagnosis support apparatus. It is a flowchart which shows an example of the processing of a medical diagnosis support apparatus. It is a figure which shows an example of the display of the display part in Embodiment 3. It is a figure which shows an example of the imaging method of an optical interference tomography. It is a figure which shows an example of the display of the display part in Embodiment 4. It is a figure which shows an example of the display of the display part in Embodiment 4. It is a figure which shows an example of the functional structure of a medical diagnosis support device. It is a flowchart which shows an example of the processing of a medical diagnosis support apparatus. It is a figure for demonstrating the estimation process in Embodiment 5. It is a figure which shows an example of the display of the display part in Embodiment 5.

Hereinafter, the image processing apparatus according to the present embodiment will be described with reference to the attached drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations disclosed in the present specification and drawings.

<Embodiment 1>
The diagnostic support device (image processing device) in the first embodiment acquires a medical image to be diagnosed, and presents support information on the medical image using the image processing result of the acquired medical image to obtain the medical image. Diagnostic support for.

In the present embodiment, a case where image processing is performed on an optical interference tomographic image (hereinafter, simply referred to as a tomographic image) in the ophthalmic region, findings are identified based on the image processing result, and the information is presented on the tomographic image. This will be described as an example. In the present embodiment, the tomographic image is a two-dimensional image of the fundus taken in the direction passing through the optic nerve head and the fovea centralis (macula). Hereinafter, an image taken in the direction passing through the optic nerve head and the fovea centralis may be referred to as an H scan image. Of course, the target of image processing is not limited to this. For example, the tomographic image may be an image containing only one of the optic nerve head and the macula, or an image not including the optic disc and the macula. In addition, the findings and the like shown below are merely examples for explaining the processing process of the diagnostic support device. In the embodiment shown below, image processing for a tomographic image of the fundus of the eye will be mainly described as a medical image, but the image processing shown below may be applied to a tomographic image of the anterior eye.

FIG. 1 is a diagram showing an example of the functional configuration of the diagnostic support device 100 according to the first embodiment. The diagnostic support device 100 is, for example, a desktop PC, a notebook PC, or a tablet PC. The diagnostic support device (image processing device) 100 in the present embodiment is connected to the optical interference tomographic meter 200 via a wire or wirelessly. The optical coherence tomography 200 comprises, for example, SD (Spectral-Domain) -OCT, SS (Swept-Source) -OCT or PS (Polarization-Sensitive) -OCT. Since the optical interference tomometer 200 is a known device, detailed description thereof will be omitted. The optical interference tomogram 200 acquires a tomographic image (that is, a medical image) showing a tomographic fault of the retina by utilizing the interference between the return light and the reference light obtained by irradiating the fundus with near-infrared light. Then, the optical interference tomographic meter 200 transmits the acquired medical image and data related to imaging information (image type, imaging site, imaging size, scan pattern, etc.) to the diagnostic support device 100 via communication such as USB or LAN. Output to. The imaging information includes, for example, information such as "OCT" for the image type, "macula" for the imaging region, "12 mm" for the imaging size, and "line scan" for the scanning pattern. In another example, the imaging information includes information such as the image type is "OCT", the imaging site is "optic disc", the imaging size is "6 mm", and the scan pattern is "3D scan". The shooting information is not limited to this, and all the information described above is not always necessary. The type of image may be included not only in OCT but also in the information including the type of OCT such as SD-OCT, SS-OCT and PS-OCT. Further, the unit of the shooting size may be an angle instead of a length. The type of image is a type of medical image according to modality, such as an OCT image (tomographic image obtained by an optical coherence tomography), a simple X-ray image, or a CT image.

The diagnosis support device 100 includes a medical image acquisition unit 102, a storage unit 104, an analysis information acquisition unit 106, an image processing unit 108, a determination unit 110, a position acquisition unit 112, a display control unit 116, and a display unit 118.

The medical image acquisition unit 102 acquires the medical image transmitted from the optical interference tomography meter 200 to the diagnosis support device 100 and the data related to the imaging information of the medical image. The medical image and the photographing information of the medical image may be stored in the storage unit 104 and read out from the storage unit 104 in response to a request from the user. Alternatively, an external storage device (not shown) such as an SSD, HDD, FDD, USB memory, CD drive, DVD drive, MO drive, or ZIP drive is connected to the diagnostic support device 100, and the medical image acquisition unit 102 is connected to the external storage device from the external storage device. You may try to get the information of. In the present embodiment, the medical image acquisition unit 102 acquires data related to a tomographic image and imaging information of the tomographic image from the optical interference tomographic meter 200. That is, the medical image acquisition unit 102 corresponds to an example of an acquisition means for acquiring a tomographic image of the eye to be inspected.

The medical image acquisition unit 102 outputs the acquired medical image to the image processing unit 108 and the display control unit 116, and outputs the data related to the imaging information of the captured medical image to the analysis information acquisition unit 106.

The storage unit 104 contains, for example, first table information including a plurality of candidate findings to be given to the image to be diagnosed for each part of the medical image to be diagnosed, and image processing corresponding to each candidate for the findings. The second table information including the parameters of is stored. This image processing parameter is a parameter used in image processing for identifying each of a plurality of candidate findings. Details of the data structure of the first table information and the second table information will be described later with reference to FIGS. 5 (a) and 5 (b).

The analysis information acquisition unit 106 acquires analysis information (image processing parameters) from the storage unit 104 based on the information on the type and site of the medical image acquired by the medical image acquisition unit 102. Specifically, the analysis information acquisition unit 106 acquires a plurality of candidate findings corresponding to the type and site of the medical image received from the medical image acquisition unit 102 from the first table information stored in the storage unit 104. To do. Further, the analysis information acquisition unit 106 acquires image processing parameters corresponding to each candidate from the second table information stored in the storage unit 104 based on the acquired candidates for the plurality of findings. Then, the analysis information acquisition unit 106 outputs the candidates for a plurality of findings to the determination unit 110 and the image processing parameters to the image processing unit 108.

The image processing unit 108 performs image processing on the medical image based on the medical image acquired by the medical image acquisition unit 102 and the image processing parameters acquired by the analysis information acquisition unit 106. The image processing unit 108 outputs the result of image processing to the determination unit 110 and the position acquisition unit 112.

The determination unit 110 finds which of the plurality of candidate findings exists in the medical image based on the plurality of candidate findings acquired by the analysis information acquisition unit 106 and the result of image processing executed by the image processing unit 108. Determine whether or not to do so. That is, the determination unit 110 determines which finding candidate is the finding corresponding to the medical image. The determination unit 110 outputs the determination result to the position acquisition unit 112.

The position acquisition unit 112 positions the lesion (abnormal site) corresponding to the finding in the medical image based on the result of the image processing executed by the image processing unit 108 and the candidate findings determined to be applicable by the determination unit 110. Get the information that indicates. The acquisition of information indicating the position will be described later. The position acquisition unit 112 outputs the information indicating the acquired position to the display control unit 116 in association with the candidate for the finding determined to be applicable by the determination unit 110.

The display control unit 116 displays based on the medical image acquired by the medical image acquisition unit 102 and the information indicating the position in the medical image associated with the candidate for the finding determined to be applicable acquired by the position acquisition unit 112. Control. The position in the medical image associated with the candidate for the finding is the position in the medical image of the lesion (abnormal site) corresponding to the finding. Specifically, the display control unit 116 displays the medical image on the display unit 118, and superimposes and displays the candidate findings determined to be applicable on the displayed medical image. The display control unit 116 may display a display (marker or the like described later) indicating the position of the finding on the display unit 118.

The display unit 118 displays the display content controlled by the display control unit 116. Although the display unit 118 is included in the diagnosis support device 100, the present invention is not limited to this, and the display unit 118 may be separate from the diagnosis support device 100.

At least a part of each part of the diagnostic support device 100 shown in FIG. 1 may be realized as an independent device. Further, it may be realized as software that realizes each function. In this embodiment, it is assumed that each part is realized by software.

2A and 2B are diagrams showing an example of the hardware configuration of the diagnostic support device 100. The CPU 1001 mainly controls the operation of each component. The main memory 1002 stores a control program executed by the CPU 1001 and provides a work area when the program is executed by the CPU 1001. The magnetic disk 1003 stores programs for realizing various application software including an operating system (OS), device drivers for peripheral devices, and programs for performing processing described later. When the CPU 1001 executes a program stored in the main memory 1002 and the magnetic disk 1003, the function (software) of the diagnostic support device 100 shown in FIG. 1 and the processing in the flowchart described later are realized.

The display memory 1004 temporarily stores display data. The monitor 1005 is an example of the display unit 118, for example, a CRT monitor, a liquid crystal monitor, or the like, and displays an image, a text, or the like based on the data from the display memory 1004. The mouse 1006 and the keyboard 1007 each perform pointing input and character input by the user. Each of the above components is communicably connected to each other by a common bus 1008.

The monitor 1005, mouse 1006 and keyboard 1007 in FIG. 2A may be replaced with the touch panel monitor 1015 as shown in FIG. 2B. This is, for example, a touch panel liquid crystal or the like, which displays images, texts, and the like based on data from the display memory 1004, and accepts pointing input and character input by the user. In addition to the touch panel 1015, a mouse 1006 and a keyboard 1007 may be provided. When selecting a patient in a device provided with a touch panel 1015, the operator taps one of the plurality of patient information (patient IDs) displayed on the display unit 118 to temporarily select the desired patient information. Will be done. Then, after the tentative selection is made, the operator swipes to select the patient information. The position on the touch panel on which the swipe is performed may be different from the position where the desired patient information is displayed. The CPU 1001 detects the above tap and swipe operations and selects desired patient information. That is, the CPU 1001 provisionally selects the one patient information when one of the plurality of patient information displayed in the display means is tapped, and when the swipe is performed after the tap. This is an example of a selection means for selecting the above-mentioned patient information. When the patient information is selected, the medical image acquisition means 102 acquires a tomographic image corresponding to the patient information selected by the selection means. By adopting such a patient information selection method, it is possible to easily select patient information according to the intention of the operator from a plurality of patient information displayed in small characters. For example, when the patient information is selected by double tapping, the problem that the same place cannot be tapped and the desired patient information cannot be quickly selected is solved by the above-mentioned method of selecting patient information.

Next, an example of the processing performed by the diagnosis support device 100 will be described with reference to the flowchart of FIG. 3A. FIG. 3A is a flowchart showing an example of processing performed by the diagnosis support device 100. In the present embodiment, the process shown in FIG. 3 is realized by the CPU 1001 executing a program that realizes the functions of each part stored in the main memory 1002.

In step S3000, the medical image acquisition unit 102 acquires the medical image transmitted from the optical interference tomogram 200, the information regarding the type of the captured medical image, and the information regarding the part of the medical image. In the example of the present embodiment, since the medical image is a tomographic image (H scan image) taken by the optical coherence tomography 200, the medical image acquisition unit 102 uses "OCT" as information regarding the type of the medical image, and the medical image. Acquire the "yellow spot" as information about the site. Information on the site of the medical image is not limited to the "macula".

In step S3010, the analysis information acquisition unit 106 corresponds to a plurality of corresponding information from the first table information stored in the storage unit 104 based on the information regarding the type of the medical image acquired in step S3000 and the information regarding the part of the medical image. Get candidates for findings.

FIG. 5A is a diagram showing an example of the first table information stored in the storage unit 104. In this table information, information on the type of image is associated with the portion of the fundus of the eye to be inspected, and a plurality of candidates for findings are associated with the information on the type of image and stored. In the case of the example of this embodiment, since the site is the "macular hole" and the image type is "OCT", the analysis information acquisition unit 106 has a plurality of foveal cyst-like holes, fibrous cyst-like holes, macular holes, and the like. Get candidates for findings. This is because the findings that can be judged differ depending on the part and the type of image. It should be noted that the number of candidate findings associated with the type of image does not necessarily have to be multiple. For example, when the parts are classified more finely, there may be one candidate for the finding associated with the type of image. Further, in the example of FIG. 5A, the type of image may be omitted and the findings may be associated with the portion. That is, the first table information is not limited to the example shown in FIG. 5A.

In step S3020, the analysis information acquisition unit 106 is required to identify each of the plurality of finding candidates from the second table information of the storage unit 104 based on the plurality of finding candidates acquired in step S3010. Get image processing parameters. In the present embodiment, the analysis information acquisition unit 106 uses information on the type of medical image acquired in step S3000 in addition to the information on candidate findings in order to acquire the parameters of image processing.

FIG. 5B is a diagram showing an example of the second table information stored in the storage unit 104. For example, when the finding is "fibrous layer cyst-like hole" and the image type is "OCT", the image is "X: 25% -75%, Z: 30% -70%, pixel value: 10 or less". The processing parameters are linked. The analysis information acquisition unit 106 acquires such image processing parameters for each of the plurality of findings candidates acquired in step S3010. Here, Z: 30% -70% indicates, for example, the retinal region in a tomographic image.

In the case of the example of this embodiment, the type of medical image is "OCT", and "foveal cyst-like foramen" is acquired as a candidate for findings in step S3010. Therefore, the analysis information acquisition unit 106 acquires the image processing parameter "X: 40% -60%, Z: 30% -70%, pixel value: 10 or less" from the second table information.

In the example of FIG. 5B, the image type may be omitted and the image processing parameter may be associated with the finding. That is, the second table information is not limited to the example shown in FIG. 5 (b).

In step S3030, the image processing unit 108 performs image processing on the medical image based on the medical image acquired in step S3000 and the image processing parameters acquired in step S3020. In the following example, it is assumed that the tomographic image is a monochrome image with 256 gradations and the low reflection region corresponding to the background is black (pixel value is 0).

The image processing will be described with reference to the flowchart of FIG. 3B and FIG. FIG. 3B is a flowchart detailing an example of the process of step S3030. FIG. 4A is a diagram showing an example of a tomographic image. In FIG. 4A, 400 is a tomographic image, 401 is the retinal layer region, 402 is the macula, 403 is the upper limiting membrane (ILM) or nerve fiber layer (NFL), and 404 is the retinal pigment epithelium (RPE). .. Further, in FIG. 4A, 405 is the posterior vitreous detachment, 406 is the lid, 410 is the background region, 420 is the optic disc region, 421 is the optic disc head, and 430 is the shaded region above the retinal layer. Shown. As shown in FIG. 4A, the tomographic image includes a region showing the retina and a background region 410 (a region other than the retina).

In step S3031, the image processing unit 108 removes noise from the tomographic image. In OCT imaging, in line scanning, cross scanning, or the like, the optical coherence tomography 200 tracks the eyes and photographs spatially identical locations a plurality of times. Then, a process of removing noise from the tomographic image is performed by aligning a plurality of tomographic images taken at the same location and performing an averaging process on the corresponding plurality of pixels among the tomographic images. Here, for example, it is assumed that the tomographic image alignment and averaging process has already been performed by the image processing unit 108. As described above, the image processing unit 108 aligns the tomographic images taken a plurality of times and performs an averaging process to remove random noise. However, depending on the condition of the eye to be inspected at the time of imaging (poor fixation), the similarity between the tomographic images at the time of alignment may decrease. May not be possible. In that case, the noise level of the image also differs depending on the difference in the number of tomographic images used for averaging. For example, the histogram of the region 410 of FIG. 4 (a) is shown in FIGS. 4 (b) and 4 (c). FIG. 4B is a histogram 411 when the number of averaging processes is small (for example, 1 to 3), and FIG. 4C is a histogram when the number of averaging processes is large (for example, 10 or more). This is an example of histogram 412. The black portions in FIGS. 4 (b) and 4 (c) show the histogram of the region 410. The vertical axis of FIGS. 4B and 4C shows the frequency, and the horizontal axis shows the brightness. As shown in the figure, the noise differs depending on the number of tomographic images used in the averaging process. Therefore, the image processing unit 108 changes the noise removal parameter according to the noise level. As a method for removing noise from a tomographic image, first, the image processing unit 108 estimates noise from a region 410 (at least one of the vitreous body and the sclera) corresponding to the background of the inner retina layer of the tomographic image. Then, the image processing unit 108 changes the filter size of the median filter, Gaussian filter, etc. used for noise removal according to the amount of noise. The image processing unit 108 increases the filter size when there is a lot of noise, and decreases the filter size when there is little noise. As a result, blurring of the image due to noise removal can be reduced, and when there is a lot of noise, noise can be removed. Further, it is desirable that the upper limit of the filter size is determined by the physical size per pixel. For example, the image range corresponding to the pixels of the image differs between the case where the 1 mm range is photographed with 100 A scans and the case where the image is photographed with 50 A scans. That is, when the parameter of the noise reduction filter in the X direction is set to 10, the physical size per pixel is equivalent to 0.1 mm for those photographed with 100 pixels, but is equivalent to 0.2 mm for those photographed with 50 pixels. Therefore, adjust the parameters so that noise is removed within the same range.

In step S3032, the image processing unit 108 normalizes the density of the tomographic image. This is a process of matching the brightness levels of the reference tomographic image and the tomographic image to be processed as much as possible. As a result, the difference in the brightness of the image for each eye to be inspected is absorbed. As a method of shading correction, there is a method of matching the histograms of the reference image and the target image, or a method of matching the shading levels using a shading conversion table. Further, in the case of a tomographic image having a white background (pixel value is 255), the image processing unit 108 performs black-and-white inversion. Although the case where the background of the tomographic image is black in this image processing is described, it is not necessarily limited to this. The case where the background of the tomographic image is white may be considered as a reference. In that case, if the threshold value of the image processing parameter is 10 or less, the pixel value is 245 or more.

In step S3033, the image processing unit 108 detects the optic nerve head. This is because the optic disc region is removed and treated in the case of macular disease, and the optic disc is extracted in the case of a disease related to the optic disc. Here, as an example, a case of providing diagnostic support for macular disease will be described. In that case, the image processing unit 108 excludes the optic disc region from the tomographic image. As shown in FIG. 4A, the optic nerve head has a large concave shape in the tomographic image. Therefore, the image processing unit 108 detects a large recessed region from the tomographic image. Alternatively, if there is an SLO image or a fundus photograph (fundus image), the image processing unit 108 detects a circular optic disc from the image, and associates the detected optic disc with the imaging range of the tomographic image. The optic disc may be detected from the tomographic image. In the case of a horizontal scan of the center of the macula, the left eye has the optic disc on the left side, and the right eye has the optic disc on the right side, so the end of the tomographic image (the optic disc side) is simply May be an invalid area. In the case of a vertical scan of the center of the macula, the optic nerve head is not visible, so it is not necessary to perform the process of this step.

In step S3034, the image processing unit 108 detects the retinal region. Specifically, the image processing unit 108 detects ILM and RPE from the tomographic image, and detects a region sandwiched between these layer boundaries as a retinal region. The image processing unit 108 is not limited to the region sandwiched between the ILM and the RPE, and the image processing unit 108 extends the region sandwiched between one of the ILM and the NFL and one of the RPE and the inner segment outer segment boundary (IS / OS) of the photoreceptor to the retinal region. It may be detected as.

Further, since the retinal region is brighter than the background region, the image processing unit 108 binarizes the retinal region by using a discriminant analysis method or the like, and detects the bright region from the image. Then, the image processing unit 108 performs morphological conversion (Closing or Opening) to connect bright regions and remove noise portions. The image processing unit 108 obtains a center line for the region, and above the center line, above the region (for example, the upper end) is the initial boundary line of the ILM, and below the center line, below the region (for example, the lower end). Set the initial border of RPE in. Then, the image processing unit 108 detects the layer boundary by applying a contour line extraction algorithm such as Snakes to the boundary line. In Snakes, define the shape energy _{E shape} and image energy _{E Image,} minimizing energy by repetitive calculations. The shape energy E _shape is defined so that the smoother the _shape of the retinal layer, the smaller the energy, and the image energy E _Image is defined so that the stronger the edge strength, the smaller the energy. Therefore, the image processing unit 108 finds a place where the energy is minimized while moving all the detection points of the boundary line set by the initial value, so that the initial boundary line is a layer boundary (strong edge) and smooth. It is finally output as a shape. That is, the image processing unit 108 corresponds to an example of an extraction means for extracting a layer boundary from a tomographic image. In the above example, the image processing unit 108 extracts ILM (inner limiting membrane) and RPE (retinal pigment epithelial layer boundary), but the present invention is not limited to this. For example, the image processing unit 108 may extract at least one of the inner limiting membrane and the nerve fiber layer boundary and at least one of the retinal pigment epithelial layer boundary and the photoreceptor inner segment outer segment boundary.

Normally, in Snakes, the layer boundary is detected as a smooth boundary line, so that the ILM boundary line is detected as a smooth boundary line even when a hole is formed in the retinal layer by a macular hole. When measuring the network thickness, the image processing unit 108 detects the vitreous body region in step S3037, which will be described later, based on the ILM boundary line detected here because the measurement is performed based on the boundary line information. The ILM boundary line may be corrected. In addition, various known methods may be used for detecting the layer boundary.

In step S3035, the image processing unit 108 normalizes the retinal position in the depth direction (Z direction). The normalization of the retina position aligns the position in the depth direction of the retina with the position of the retina in the reference image using the center line obtained in step S3034 so as to match the preset reference image. This is done to make it easier to identify the position when the finding candidate is detected and to secure a place for displaying the finding on the tomographic image.

In step S3036, the image processing unit 108 detects the candidate region of the macula. The macula has a small concave shape in the tomographic image. For example, the depression of the macula is smaller than the depression of the optic disc. Therefore, the image processing unit 108 detects a small recessed region from the tomographic image. Alternatively, if there is an SLO image or a fundus photograph (fundus image), the image processing unit 108 detects the macula from the image and associates the detected macula with the imaging range of the tom image to match the macula from the tom image. May be detected. Alternatively, when the image is taken at the center of the macula, the macula is near the center of the tomographic image, so the image processing unit 108 may simply detect the vicinity of the center of the tomographic image as the macula. Alternatively, the image processing unit 108 may detect the macula portion with reference to the position of the optic nerve head detected in step S3033. Since the ratio of the optic disc diameter (DD) to the distance (DM) from the center of the papilla to the fovea centralis of the macula is usually 2.4 to 3.0, the image processing unit 108 uses that ratio to approximate the macula region. May be specified.

In step S3037, the image processing unit 108 detects the vitreous region (hatched portion 430 in FIG. 4A). The image processing unit 108 detects the shallow side (upper side) in the Z direction as the vitreous region in the tomographic image with reference to the retinal region (or ILM) detected in step S3034. Further, the image processing unit 108 may perform a thinning process in the vitreous region to extract 405 as a posterior vitreous peeling candidate (see FIG. 5B). The extraction of the posterior vitreous peeling candidate 405 may be performed in step S3038. Here, a case where the image processing unit 108 corrects the ILM after detecting the vitreous body existing region 430 will be described. If the macular hole is perforated by the macular hole, the ILM border detected as a smooth line crosses the vitreous region 430. Since the vitreous region 430 is divided by the ILM boundary line, the ILM boundary line may be rediscovered at that location.

Next, in step S3038, the image processing unit 108 performs image processing (region extraction of lesion candidates) based on the image processing parameters corresponding to the plurality of findings candidates. Here, the image processing parameters "X: 25% -75%, Z: 30% -70%, pixel value: 10 or less" of the "fibrous layer cyst-like pore" will be described as an example. In this case, for the tomographic image normalized by the preprocessing, the image processing unit 108 follows the above image processing parameters, and the X coordinate of the tomographic image is 25% -75% and the Z coordinate is 30% -70%. Pixels whose pixel value is 10 or less with respect to the range are extracted. Such processing is performed for each of the plurality of findings candidates. Further, in the present embodiment, there is only one tomographic image, but when there are a plurality of tomographic images, the above processing is performed for each of the plurality of tomographic images.

Return to step S3040 in FIG. 3A. In step S3040, the determination unit 110 determines whether or not each of the plurality of findings candidates is applicable based on the plurality of findings candidates acquired in step S3020 and the result of the image processing executed in step S3030 (to the tomographic image). Whether it is an existing finding) is determined.

For example, for the "fibrous layer cyst-like hole", the determination unit 110 sets adjacent pixels in the extracted pixels as one region, and sets the aspect ratio (short side / long side) to each region. calculate. Then, the determination unit 110 determines that, for example, a region having an aspect ratio of 0.7 or less corresponds to a “fibrous layer cyst-like hole”. Alternatively, in the case of "rear vitreous peeling", the determination unit 110 "rear vitreous" when the length of the line extracted by the thinning process is equal to or greater than the threshold value (for example, 20% of the length in the X direction of the image). It is judged that it corresponds to "peeling". That is, the image processing unit 108 and the determination unit 110 correspond to an example of the detection means for detecting an abnormal portion of the eye to be inspected by performing structural analysis on the tomographic image acquired by the acquisition means.

These determination criteria may be stored as a part of the second table information, or may be configured such that the storage unit 104 has a third table information (not shown) in which the determination criteria are stored. ..

Further, in step S3038, the region whose Z coordinate is 30% -70% is the processing target, but the region to be processed may be determined using the detection result of the layer boundary. For example, the region where the Z coordinate is 30% -70% may be replaced with a region sandwiched between the ILM and the RPE (an example of the first region). That is, in order to detect the region of the fibrous cyst-like pore (an example of the first abnormal site), the image processing unit 108 has a region where the X coordinate of the tomographic image is 25% -75% and is sandwiched between ILM and RPE. A pixel having a pixel value of 10 or less may be extracted. Further, as shown in the second table of FIG. 5B, the image processing unit 108 is based on the ILM which is the detection result of the layer boundary in order to detect the region of the posterior vitreous detachment (an example of the second abnormal region). The thinning process is performed in the upper region (an example of the second region). That is, the image processing unit 108 and the determination unit 110 apply structural analysis of different algorithms to the first region and the second region in the tomographic image defined by the layer boundary (for example, ILM) extracted by the extraction means. Therefore, the first abnormal part is detected from the first region, and the second abnormal part different from the first abnormal part is detected from the second region.

The region sandwiched by one of the inner limiting membrane and the nerve fiber layer boundary and one of the retinal pigment epithelial layer boundary and the photoreceptor inner segment outer segment boundary may be the first region. Further, the region on the vitreous side of one of the inner limiting membrane and the nerve fiber layer boundary may be used as the second region. Then, the first abnormal portion and the second abnormal portion may be generated by applying the structural analysis of different algorithms to each of the first region and the second region defined as described above.

FIG. 6 is a diagram showing an example of the image processing result in step S3030 and an example of the determination result in step S3040. The RPE layer 610 and ILM620 shown in FIG. 6 are layer boundaries extracted by the process of step S3030 with respect to the tomographic image 600.

The corresponding region 630 of the fibrous cyst-like foramen and the non-corresponding region 635 of the fibrous cyst-like foramen indicate the regions extracted by the image processing parameters corresponding to the “fibrous layer cyst-like foramen” in step S3030. As shown in FIG. 6, since the aspect ratio of the corresponding region 630 of the fibrous layer cyst-like hole is 0.7 or less (that is, horizontally long), the determination unit 110 determines that the corresponding region 630 is a “fibrous layer cyst-like hole”. It is judged that it corresponds to. On the other hand, in the non-corresponding region 635 of the fibrous layer cyst-like hole, since the aspect ratio exceeds 0.7, the determination unit 110 determines that it does not correspond to the “fibrous layer cyst-like hole”. Similarly, the rear vitreous peeling corresponding portion 640 shows a line extracted by the image processing parameter corresponding to the “rear vitreous peeling” in step S3030. In the case of this example, since the length of the line in the X direction exceeds 20% of the length of the image in the X direction, which is the threshold value, the determination unit 110 determines that it corresponds to "rear vitreous detachment". Such a determination is made for each of the plurality of findings candidates. In the example of FIG. 6, in addition to the above-mentioned findings, the macular hole 650 and the lid 660 are determined as applicable findings. In this embodiment, the findings determined by the computer (diagnosis support device 100) as described above are referred to as CAD findings.

In step S3050, the position acquisition unit 112 indicates information (of the findings) indicating the position of the medical image corresponding to the findings, based on the result of the image processing executed in step S3030 and the candidate findings determined to be applicable in step S3040. Information indicating the position) is acquired. The information indicating the position may be any information as long as the position can be specified, but in the present embodiment, the position acquisition unit 112 is rectangular information including an image area determined to correspond to the finding. (For example, the four coordinates of the vertices) are acquired as information indicating the position of the finding. In other words, the position acquisition unit 112 acquires information on the rectangle circumscribing the image area determined to correspond to the finding (for example, the coordinates of the four vertices of the rectangle) as information indicating the position of the finding. The information acquired by the position acquisition unit 112 is not limited to the information of the rectangle that includes the image area or circumscribes the image area, and may have a shape other than the rectangle as long as the image area is included.

In the example shown in FIG. 6, the position acquisition unit 112 is determined to have fibrous layer cyst-like hole position information 632 and “posterior vitreous detachment” with respect to the region determined to be “fibrous layer cyst-like hole” in step S3040. The posterior vitreous peeling position information 642 is acquired for the region. Here, the fibrous cyst-like pore position information 632 is the position information of a rectangular region circumscribing the corresponding region 630 of the fibrous cyst-like foramen, and the posterior vitreous detachment position information 642 circumscribes the corresponding region 640 of the posterior vitreous detachment. This is the position information of the rectangular area to be used.

The position acquisition unit 112 acquires information indicating the position of such a finding for each of the determined candidates for the plurality of findings. In the example shown in FIG. 6, macular hole position information 652 and lid position information 662 are also acquired. Information indicating the position of the same finding may be acquired for each area, priority is given to each area, and information indicating the position of the area having the highest priority indicates the position of the finding. It may be acquired as information. The priority can be determined by, for example, at least one of the size of the rectangular area, the distance from the center of the image, and the distance from the edge of the image. For example, when there are a plurality of the same findings in the tomographic image, priority may be given to the findings having a large rectangular area, and information indicating only the positions of the findings having a large rectangular area among the same findings may be acquired.

In step S3060, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116.

Specifically, using the information indicating the position in the tomographic image of the findings determined to be applicable in step S3040 acquired in step S3050, the information of the findings determined to be applicable on the tomographic image acquired in step S3000. Is superimposed and displayed. More specifically, the display control unit 116 displays the tomographic image on the display unit 118, and superimposes the findings on the tomographic image as a document or a word. That is, the display control unit 116 corresponds to an example of the display control means that superimposes the findings of the abnormal portion detected by the detection means on the tomographic image as a document or a word and displays them on the display means. Further, the display control unit 116 displays the findings at a position on the tomographic image corresponding to the position of the image region (abnormal portion) determined to correspond to the findings.

In the present embodiment, the display control unit 116 determines the position of the superimposed display on the tomographic image according to the following display rules for each applicable finding. (1) Do not superimpose on the information indicating the position of other findings determined to be applicable. (2) The character information of the findings is displayed below the RPE layer in the Z direction or above the ILM in the Z direction. That is, the textual information of the findings is not superimposed on the retina. (3) When the information indicating the position of the findings determined to be applicable is between the RPE layer and the ILM in the Z direction, the position of the findings indicated by the information A marker (for example, an arrow) is added in the vicinity. Of course, the display rules are not limited to the above example. For example, condition (2) is displayed in a region other than the region where the textual information of the findings is sandwiched between one of the inner limiting membrane and the nerve fiber layer boundary and one of the retinal pigment epithelial layer boundary and the photoreceptor inner segment outer segment boundary. You may change to the condition. Further, the condition (2) may be changed to a condition that at least a part of the character information of the findings is displayed below the RPE layer in the Z direction or above the ILM in the Z direction. Even if the conditions are changed in this way, not all of the character information is superimposed on the tomographic image, and it is possible to prevent deterioration of the visibility of the tomographic image. In addition to displaying the marker under the condition (3), the character information of the findings is displayed in the region below the RPE layer in the Z direction and the region above the ILM in the Z direction, whichever is closer to the image region corresponding to the corresponding finding. May be displayed. By doing so, it is possible to prevent a decrease in visibility of the tomographic image due to the marker 712 shown in FIG. 7 becoming unnecessarily large.

FIG. 7 shows an example of the display content displayed by the display unit 118. The display content 700 displayed by the display unit 118 includes case information 701, a tomographic image 710 of the target case, findings 711 superimposed on the tomographic image, marker 712, and a fundus image corresponding to the tomographic image of the target case. 715, it has a horizontal arrow 717 that is superimposed and displayed on the fundus image 715. The horizontal arrow 717 indicates the acquisition position of the tomographic image 710 on the fundus image. Further, it has an analysis processing result image 720, a mode 730, display information 740, an option 750, and an image processing parameter 760.

The display control unit 116 superimposes and displays the character information of the finding 711 on the tomographic image based on the above-mentioned display rule and the information indicating the position of the finding 711. For example, regarding the "fibrous layer cyst-like foramen", since the information indicating the position of the fibrous layer cyst-like foramen region 713 is between the RPE layer and the ILM in the Z direction, the finding 711 is located below the RPE layer in the Z direction. Character information is displayed. In the present embodiment, since the region 713 of the fibrous layer cyst-like pore is closer to the region below the RPE layer in the Z direction than the region above the ILM in the Z direction, the word "fibrous layer cyst-like pore" is referred to as Z from the RPE layer. It is displayed in the area below the direction. Further, a marker 712 is added in the vicinity of the information indicating the position of the finding 711. That is, the marker 712 is displayed in the vicinity of the region 713 of the fibrous cyst-like foramen, which is an abnormal site. Regarding "rear vitreous detachment", since the information indicating the position of "rear vitreous detachment" is above the ILM in the Z direction, the position of "rear vitreous detachment" is prevented so that the character information does not overlap with the ILM. The character information of the finding 711 is displayed in the vicinity of the information indicating. In the example shown in FIG. 7, the character information of the finding 711 is displayed so that the character information does not overlap the detected vitreous peeled portion. The character information of the finding 711 may be displayed so that at least a part of the character information does not overlap the detected vitreous peeled portion.

Similar to the above processing, the character information of the finding 711 is displayed in the vicinity of the abnormal portion for the “macular hole” and the “lid”. The textual information of the displayed findings 711 may be a word such as "rear vitreous detachment" or a document such as "rear vitreous detachment".

The analysis processing result image 720 is an image showing the above-mentioned image analysis result. The analysis processing result image 720 shows the vitreous region 721 (hatched portion), the lid 722, the fibrous layer cyst-like hole 723, the posterior vitreous detachment 724, the ILM725, the retinal layer region 726, the RPE727, and the subchoroidal region 728. .. These display the analysis results of the image analysis for the tomographic image that was the basis for outputting the findings 711. In the present embodiment, an example of detecting a lid 722, a fibrous layer cyst-like hole 723, and a posterior vitreous detachment 724 as a lesion (abnormal site) is shown, but the present invention is not limited to this. If there are epiretinal membranes, retinal atrophy, vitiligo, etc., show the results of detecting them. The analysis processing result image 720 may or may not be displayed.

Mode 730 indicates a display mode. In this embodiment, a mode for displaying findings on a tomographic image is shown as a diagnostic image mode 731. In the embodiments (FIGS. 11 and 14) described later, an interface in which a plurality of modes can be selected is displayed as the mode 730.

The display information 740 is a user interface for selecting information to be displayed on the tomographic image. A user such as a doctor can switch the information displayed on the tomographic image by, for example, clicking or tapping the displayed user interface. As a user interface, a diagnostic image display 741 for turning on / off the superimposed display of all information such as a doctor's finding, a CAD finding, and a handwritten memo as a result of layer detection on the tomographic image 710 is displayed on the display unit 118. Further, as a user interface, a layer detection display 742 that switches the display ON / OFF of the layer detection result is displayed on the display unit 118 in order to individually switch the information to be superimposed and displayed. In addition, a doctor's finding display 743 for switching ON / OFF of the display of the findings confirmed by the doctor is displayed on the display unit 118. Further, as a user interface, a CAD finding display 744 for switching the display ON / OFF of the findings detected by CAD and a handwritten memo display 745 for switching the display ON / OFF of the handwritten memo are displayed on the display unit 118. The user interface for designating the display information 740 is, for example, a button or a check box.

The findings include not only CAD findings but also findings confirmed by a doctor. The CAD findings indicate the findings determined by the computer, and the findings by the doctor shall be the confirmed (approved) findings of the computer-output findings or the modified findings by the CAD. The findings may be newly added by the doctor. Regarding the determination of CAD findings, it is possible to confirm the CAD findings by an operator such as a doctor selecting an approval check box or the like (not shown). In that case, it is possible to determine a plurality of CAD findings at once. If the PC device has a mouse, the CAD findings can be confirmed by clicking the check box and selecting the CAD findings. For example, check boxes are displayed near each of Findings 711. Further, in the case of a touch panel monitor, it is possible to confirm the CAD findings by selecting the CAD findings by tapping the screen. In the case of batch confirmation, the findings may be collectively confirmed by double-clicking or double-tapping on the tomographic image 711 in which the finding 711 does not exist, instead of the check box. As a method for correcting CAD findings, a predetermined finding 711 is selected by tapping or clicking the finding 711 displayed on the display unit 118, and a doctor or the like uses a keyboard 1007 or the like for the selected finding 711. It is possible to make corrections.

Alternatively, the findings 711 may be individually selected and confirmed may be selected. In that case, the menu may be displayed by right-clicking on the findings and "confirm" included in the menu may be selected, or may be confirmed by double-clicking or double-tapping on the findings 711. Further, the finding 711 may be tentatively selected by touching, and may be confirmed by sliding gesture (swipe) on the touch panel monitor in the tentatively selected state. In the case of correction of the finding, when the finding 711 is selected, another finding name may be displayed in the vicinity of the finding 711 as a list box, and the finding may be confirmed by selecting an arbitrary finding from the list box. Further, in the case of deleting unnecessary findings, it is possible to delete the findings by selecting the findings 711 and selecting the deletion from the menu, or by dragging the findings 711 to the outside of the tomographic image 710. That is, the display control unit 116 can detect the operation on the finding 711 and delete the finding 711 from the display screen.

In addition, both the batch confirmation and individual confirmation of the findings shall have a confirmation cancellation function. In the case of a check box, uncheck the check box. Also, in the case of double-click or double-tap, it is possible to cancel by performing the same operation again.

Since the findings include the findings confirmed by the doctor, it is desirable to display the characters in a different color from the findings determined by the CAD (diagnosis support device 100). For example, CAD findings are red and doctors' findings are blue. In this way, it is possible to easily grasp which findings are due to CAD or doctors at first glance. It should be noted that the display is not limited to the color of the characters, and may be displayed so that the difference between the CAD findings and the findings by the doctor can be understood, such as adding an icon, enclosing the word with a rectangle, adding a shadow to the character, and changing the font. For example, when the display control unit 116 accepts the doctor's correction in which the CAD findings are input via the keyboard 1007 or the like, the display control unit 116 may display the findings 711 in different display forms before and after the correction by the doctor. .. For example, before and after the correction, the character color of the finding 711 is changed by the display control unit 116. That is, the display control unit 116 corresponds to an example of a reception means displayed on the display means by the operator and accepting corrections to the recorded findings, and causes the display means to display the corrected findings in a display form different from that before the correction.

The timing at which the display control unit 116 changes the character color may be immediately after the change is made by the doctor, or at the timing when the modification is confirmed by selecting an approval check box or the like (not shown). There may be.

Further, the display control unit 116 may change the color of the characters of the finding 711 and display it on the display unit 118 according to the type of the finding. In that case, the icon, font, etc. are changed between the findings by CAD and the findings by the doctor, and the same color is used for the same findings. For example, the macular hole is light blue, the lid is purple, etc., the CAD findings are italic, and the doctor's findings are bold.

In addition to the findings, any comment can be entered as a handwritten memo. The difference between handwritten memo and finding input is shown below. A selection unit for selecting a handwritten memo input or a finding input (not shown) may be provided at the time of character input, and the CPU 1001 may determine whether the handwritten memo input or the finding input is based on the selection by a doctor or the like. Alternatively, in the case of finding input, the word of the finding is selected from the pre-registered finding dictionary and added, and the free input is a handwritten memo. Alternatively, the character analysis of the input word is performed, and it is determined that the word registered in the finding dictionary is a finding. Alternatively, the findings obtained by modifying the findings output by the CAD or the findings output by the CAD are confirmed by the doctor, and the free input is a handwritten memo.

Option 750 is a user interface for processing tomographic images or displaying CAD information. A user such as a doctor can execute processing on a tomographic image by the image processing unit 108 or display CAD information by the display control unit 116 by, for example, clicking or tapping the displayed user interface. As option 750, a user interface of black-and-white inverted display 751, tilt correction display 752, noise removal 753, density correction 754, and CAD details 755 is displayed. The black-and-white inverted display 751 that specifies the black-and-white inversion of the display image is a display in which the display color of the tomographic image is black in the low reflection area and white in the high reflection area, or white in the low reflection area and black in the high reflection area. It is a user interface for switching between. The tilt correction display 752 is a user interface for correcting the tilt of the retinal layer. This corrects the tilt of the retinal layer to be displayed based on the image processing parameter 760. The noise reduction 753 is a user interface for removing noise from a tomographic image based on the image processing parameter 760. The density correction 754 is a user interface for correcting the WW / WL of the tomographic image. The CAD detail 755 is a user interface for selecting ON / OFF of the display of the analysis processing result image 720 and the image processing parameter 760. That is, the analysis processing result image 720 and the image processing parameter 760 are displayed on the display unit 118 by the display control unit 116 when the CAD detail 755 is selected. Further, the analysis processing result image 720 may display the intermediate result during the execution of the analysis processing at any time. In the column of the image processing parameter 760, the item 761 of the parameter required for the image processing and the numerical value 762 thereof are displayed, and the processing is executed based on the numerical value displayed here. The numerical value 762 can be edited using a mouse 1006, a keyboard 1007, or the like.

FIG. 8 shows another display example in this embodiment. In FIG. 8, the location of the analysis processing result image 720 and the image processing parameter 760 in FIG. 7 is different, and the analysis data 820 and the analysis value 860 are displayed. Other than that, the display is the same as the example shown in FIG. These displays are switched by turning on / off the CAD details 755. In the diagnostic image mode 731, ON of CAD detail 755 is set as the default state, but OFF of CAD detail 755 may be set as the default state and display of analysis data 820 and image analysis value 860 may be set as default by a setting (not shown). The analysis data 820 shows a statistical normal eye database 821 and a layer thickness graph 822 of the displayed tomographic image 710. The normal eye database 821 contains, for example, a range of "outside the upper limit of normal value" (99% or more), a range of "upper boundary" (95% -99%), and a range of "normal value" (5%-). 95%), the range of "lower boundary line" (1% -5%), and the range of "outside the lower limit of normal value" (1% or less) are color-coded. The layer thickness graph 822 can be seen with respect to the range in which the thickness of the retinal layer deviates from the normal by comparing with the normal eye database 821. The image analysis value 860 is a result of measuring the size and number of the determined lesions by the image processing in step S3030 and the determination result of the finding candidate in step S3040.

According to the present embodiment, the diagnostic support device 100 acquires image processing parameters related to deriving the corresponding finding candidates and finding candidates based on the information of the site of the medical image. Then, image processing is performed based on the acquired image processing parameters, the determination and position of the corresponding findings are specified, and these results are presented. By doing so, the doctor can confirm the findings determined to be applicable on the medical image, their positions, and the analysis results. As a result, the doctor can view the state of the medical image currently being targeted and the diagnosis name, so that the burden related to the diagnosis can be reduced.

More specifically, according to the present embodiment, since the findings are displayed on the tomographic image, the doctor or the like can easily grasp the relationship between the tomographic image and the findings. Further, since the findings are displayed corresponding to the positions of the abnormal parts in the tomographic image, the doctor or the like can easily grasp the relationship between the abnormal parts and the findings, and a quick diagnosis becomes possible.

Further, since the findings are displayed so as not to overlap the retinal region in the tomographic image as much as possible, the findings can be displayed on the tomographic image while preventing deterioration of the visibility of the tomographic image itself.

<Embodiment 2>
In the first embodiment, a medical image to be diagnosed is acquired, support information such as CAD findings is presented on the medical image using the image processing result, and an analysis image and analysis data of the image processing are presented. In the present embodiment, a case where the support information is presented on the medical image and the result of the similar case (similar image) is presented will be described.

An example of the functional configuration of the diagnostic support device 900 according to the second embodiment will be described with reference to FIG. In the diagnosis support device 900, a search unit 914 and a database 300 are functionally added to FIG. The database 300 is communicably connected to the diagnostic support device 900 via wire or wireless. The database 300 stores a medical image and a finding and a diagnosis name corresponding to the image in association with each other. That is, the database 300 corresponds to an example of a storage means for storing each of a plurality of tomographic images in association with the findings.

In addition, patient information such as gender and age is associated with the medical image stored in the database 300. Then, in response to the request from the diagnosis support device 900, the database 300 outputs the medical image and findings corresponding to the request to the diagnosis support device 900. The database 300 may be configured to be stored in the storage unit 104 inside the diagnosis support device 900. The search unit 914 requests information on an image similar to the target medical image from the database 300, using the candidate findings determined by the determination unit 110 as a key. Then, the search unit 914 receives the information requested from the database 300. The search unit 914 outputs the information of the received image to the display control unit 116 together with the similarity with the target medical image. The hardware configuration of the diagnostic support device 900 is the same as that of FIG. 2 in the first embodiment. That is, when the CPU 1001 executes the program stored in the main memory 1002 and the magnetic disk 1003, the functions (software) of the diagnostic support device 900 of the present embodiment and the processing in the flowchart are realized.

FIG. 10 shows a flowchart illustrating an example of processing performed by the diagnosis support device 900. It is basically the same as the flow of FIG. 3 in the first embodiment, but a part of the processing is different from the first embodiment. Specifically, the search S10060 for similar images and the information display S10070 are different from the flow of FIG. Hereinafter, with reference to the flowchart of FIG. 10 and an example of the display contents of FIG. 11, the processing performed by the diagnostic support device 900 in the present embodiment will be described only for the differences from the first embodiment.

In the flowchart of FIG. 10, the processes of steps S3000 to S3050 are the same as those of the first embodiment, and thus the description thereof will be omitted.

In step S10060, the search unit 914 acquires a similar image from the database 300 using the candidate findings determined to be applicable in step S3040 as a key. That is, the search unit 914 corresponds to an example of a similar image acquisition means that acquires a tomographic image associated with a finding that matches the finding of the abnormal portion detected by the detection means from the storage means.

The search key is "posterior vitreous detachment" or "macular hole" or the like. The number of findings used as the search key may be multiple or one. It is assumed that the findings stored in the database 300 are, for example, findings specified by the doctor in advance for the case, and the database 300 also includes information regarding the position of the findings specified by the doctor. The information on the findings contained in the database 300 may be information on literature such as a textbook instead of the information specified in advance by the doctor, or the doctor may specify the cases collected for the database. You may use the information of the findings.

As a method of acquiring similar images by the search unit 914, only those whose search keys match exactly may be acquired from the database 300, or similar images whose search keys partially match may be acquired. For example, when a plurality of findings are used as a search key, the search unit 914 may acquire a similar image that matches only a part of the plurality of findings. Alternatively, the similarity may be calculated by some method, and a specified number of images may be acquired in descending order of similarity. In the present embodiment, it is assumed that only the images whose search keys match exactly are acquired. The method of using the similarity will be described in a modified example described later.

In step S10070, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116. Specifically, using the information indicating the position of the findings determined to be applicable in step S3040 in the medical image acquired in step S3050, the information of the findings determined to be applicable on the medical image acquired in step S3000. Is superimposed and displayed. The rules for displaying the finding information are the same as those in the first embodiment. Further, the display control unit 116 displays the similar images and the information of the diagnosis name associated with the similar images in descending order of the degree of similarity based on the information of the similar images acquired in step S10060. In the present embodiment, since only the images whose search keys exactly match in step S10060 are acquired, the patient information related to the medical image acquired in step S3000 is displayed from similar images of the same sex and similar ages. .. Alternatively, if the configuration is such that information on lifestyle habits can be acquired in addition to shooting information, lifestyle-related items may be added to the search key. For example, the search may be performed using information such as the presence or absence of diabetes and the presence or absence of smoking. In addition to these, information that is close to the shooting date may be displayed first. Further, in the present embodiment, information on similar images and diagnosis names associated therewith can be managed in a list, the information in the list can be displayed, and one case in the list can be compared with the medical image acquired in step S3000. Display in a simple form. The diagnosis support device 900 may select one case to be displayed in a comparable form, or the diagnosis support device 900 may accept the selection by user input. For example, it can be realized by preparing a list box that stores a list of similar cases as a user interface and having one case selected from the list box.

In the present embodiment, the display control unit 116 determines as one case in which a similar image having the closest shooting date is displayed in an initial contrastable form, and thereafter, a contrastable form according to a user input selection. The cases to be displayed in shall be determined. The display method is not limited to this. For example, a plurality of similar images and information on the diagnosis name associated therewith may be displayed in a form that can be compared with the medical image acquired in step S3000, or all of them may be displayed in a form that can be compared. ..

FIG. 11 shows an example of the display contents displayed by the display unit 118. The display content 700 displayed by the display unit 118 is based on the diagram described with reference to FIG. 7 of the first embodiment. The difference from the first embodiment is that the similar case mode 1132 is displayed as a user interface. When a doctor or the like selects a similar case mode by tapping or clicking the similar case mode 1132, the display control unit 116 displays a similar case list (that is, a list in which similar images are displayed in descending order of similarity) 1160. Then, the display control unit 116 sets the tomographic image 1120, the finding 1121, the diagnosis name 1122, and the fundus corresponding to the tomographic image of one case in the similar case list (here, the image having the closest shooting date as described above). The image 1125 is displayed on the display unit 118. When similar case mode 1132 is selected, option 750 adds case search 1155, a user interface that performs case search. When an operator such as a doctor taps or clicks the case search 1155, a case search similar to the target tomographic image 710 is executed. In the present embodiment, an example in which the user interface for executing the case search is different from the similar case mode 1132 is shown, but the case search may be executed at the same time as selecting the similar case mode 1132.

In the tomographic images 710 and 1120 on which the findings are superimposed and displayed, the information of the findings 711 and 1121 determined to be applicable based on the information indicating the position of the findings is superimposed and displayed in accordance with the display rules described above. As described above, as a result of selection by user input, when it is determined as a case to display other cases in the similar case list in a comparable form, the display content of the display unit 118 is controlled by the display control unit 116. .. Specifically, the display control unit 116 displays the tomographic image 1120 in which the findings are superimposed and displayed, the diagnosis name 1122, and the fundus image 1125 corresponding to the tomographic image corresponding to the cases selected in the similar case list 1160. To display.

According to the present embodiment, the diagnostic support device 900 acquires image processing parameters related to deriving the corresponding finding candidates and finding candidates based on the information of the site of the medical image. Then, image processing is performed based on the acquired image processing parameters, the determination and position of the corresponding findings are specified, similar images are searched based on the findings determined to be applicable, and these results are presented. By doing so, the doctor can confirm the findings determined to be applicable on the medical image, their positions, and the diagnosis name associated with the similar image. As a result, the doctor can browse the diagnosis name of the case similar to the state of the medical image currently targeted, so that the burden related to the diagnosis can be reduced.

In addition, in the default state, an image most similar to the currently targeted medical image is displayed in a comparable form, so that an operator such as a doctor can save time to select an image suitable for the contrast. Is.

The information stored in the database 300 is not limited to the above example. For example, the database 300 may store a medical image, a finding and a diagnosis name corresponding to the image, a position of the finding in a tomographic image, a diagnosis name, and a finding in association with each other. In this case, the search unit 914 can search for similar images by using the position of the finding as a search key in addition to the finding. For example, the search unit 914 prepares a database of tomographic images (similar images) in which the difference between the tomographic images to which the findings match and the positions of the findings match or the target tomographic images and the tomographic image in the database 300 are within a predetermined threshold value. It may be obtained from 300. That is, the database 300 may store the findings and the positions of the findings in the tomographic image in association with each of the plurality of tomographic images. In addition, the search unit 914, which is an example of the similar image acquisition means, stores the findings of the abnormal portion detected by the detection means, the findings matching the position of the findings, and the tomographic image associated with the position of the findings from the storage means. get.

By doing so, it is possible to acquire a similar image in which the findings and the positions of the findings match, so that it is possible to acquire a tomographic image showing symptoms closer to the target tomographic image.

(Modification example 1)
In the above embodiment, the findings are used as a search key to search for similar cases with matching findings. However, the method for searching for similar cases is not limited to this. For example, similar cases may be searched based on the amount of image features. This process will be described with reference to the flowchart of FIG. FIG. 12 is a refinement of the process of step S10060 in the flowchart of FIG.

In step S10061, as a global feature extraction, the image processing unit 108 extracts the thickness feature of the retinal layer from the tomographic image. For the thickness of the retinal layer, the difference in the Z direction between the boundary line between ILM and RPE is defined as the thickness of the retinal layer. As a region for calculating the thickness of the retinal layer, the image processing unit 108 extracts a feature amount from each of the central region of the macula, the left and right regions excluding the center of the macula, and the entire region. As a method of region division, the X direction of the tomographic image may be simply divided into three, or five divisions may be made so that the left and right regions are 2/5 each and the macula region is 1/5 of the center. It is desirable to remove the optic disc region from the region when calculating the thickness. The thickness feature in each region shall be one of average thickness, maximum thickness, minimum thickness, dispersion or standard deviation. The image processing unit 108 creates a global feature vector from these features. Thus, the global feature represents the shape feature of the entire retina.

In step S10026, as local feature extraction, the image processing unit 108 calculates the HOG feature in the local region of the inner layer of the retina. The HOG feature is a feature amount that calculates the gradient direction and gradient intensity of brightness in a local region and creates a histogram with the horizontal axis as the gradient direction and the vertical axis as the gradient intensity. If the inner layer of the retina is normal, the inner layer of the retina has a layered structure, so that the gradient directions of the brightness are aligned in the local region, but when the layer structure is disturbed by the lesion, the gradient direction of the brightness is also disturbed. Thus, local features represent the alignment of the inner layers of the retina. It is desirable that the region for calculating the HOG feature is not the entire tomographic image of OCT but the retinal region shown in step S3034 of the first embodiment. Since the local feature amount obtained here has several thousand dimensions, the dimensions are reduced in the image representation described later.

In step S10063, the image processing unit 108 converts the local feature amount obtained in step S10026 into an image representation by Bag of Features. Bag of Features is a method of classifying images based on the appearance frequency of image features registered in a dictionary (codebook) of image features. At the time of learning, local features are extracted from all the learning images, the extracted local features are clustered in the feature space, and the center of gravity of each cluster is set as a visual word (codebook). K-means is used for clustering at this time. At the time of image representation in step S10063, the image processing unit 108 votes the local feature amount obtained in step S10062 to the closest Visual word in the Visual word created at the time of learning. As a result, the image processing unit 108 converts the local feature amount into a local feature vector that counts the appearance frequency of the visual word.

In the search in step S10064, the search unit 914 searches for similar images using the global feature vector obtained in step S10061 and the local feature vector obtained in step S10063. As a search method, the similarity is calculated for each of the global feature vector and the local feature vector, and the total similarity is used as the search result. As the similarity, for example, cosine similarity can be used. Assuming that the target image feature vector is Vt and the image feature vector stored in the database 300 is Vs, the cosine similarity is expressed by the following equation.

この類似度は１に近いほど二つのベクトルが類似していることを示し、０に近ければ類似していないことを示す。

The closer this similarity is to 1, the more similar the two vectors are, and the closer to 0, the less similar they are.

The method of calculating the similarity is not limited to this, and the deviation pattern similarity may be calculated. The deviation from the average value in the group to which each element belongs is defined as the similarity, and the maximum value is 1. That is, the deviation pattern similarity is a similarity that reflects the distribution of data in the feature space. When such similarity is used, it is desirable to display it in the case list in descending order of similarity in step S10070. In this way, it is possible to search for similar cases based on image features instead of searching based on findings.

The database 300 may store each of the plurality of tomographic images in association with information based on the thickness of a predetermined layer of the tomographic image and information based on the brightness of the retina included in the tomographic image. Here, the information based on the thickness of a predetermined layer of the tomographic image is, for example, a global feature vector, and the information based on the brightness of the retina included in the tomographic image is, for example, a local feature vector. When the database 300 stores the above-mentioned information, the search unit 914 finds the abnormal portion detected by the detection means, the finding of the abnormal portion corresponding to the information based on the layer thickness and the information based on the brightness. A tomographic image associated with the layer thickness-based information and the brightness-based information is acquired from the storage means. It should be noted that this modification can be applied not only to the second embodiment but also to other embodiments.

(Modification 2)
In the second embodiment, in step S3060, the search unit 914 uses the candidate for the finding determined to be applicable in step S3040 as the search key, and acquires only the image (case) in which the search key completely matches from the database 300. However, the search key may acquire an image with a partial match, or the image may be acquired by using a method of calculating the similarity of the search key.

As the similarity, for example, a cosine similarity for calculating the similarity of vectors can be used. Hereinafter, a specific description will be given. First, the image processing unit 108 creates a finding vector using the plurality of finding candidates acquired in step S3010 and the finding finding candidates determined to be applicable in step S3040. The finding vector can be generated by considering each finding candidate as an element of the vector and giving 1 when the finding in the target image corresponds (exists) and 0 when it does not. For example, when there are 20 candidates for a plurality of findings, it becomes a 20-dimensional finding vector. Next, the image processing unit 108 generates the generated finding vector and the finding vector of the medical image stored in the database 300, and the search unit 914 calculates the similarity between these vectors. Alternatively, the finding vector may be stored in the database 300 in advance.

Assuming that the finding vector of the target medical image is Vt and the finding vector of the medical image stored in the database 300 is Vs, the cosine similarity is represented by the formula 1 described in the modification 1.

Note that this modification can be applied not only to the second embodiment but also to other embodiments, and a similar case is searched for using both the similarity of the image features and the similarity of the findings shown in the first embodiment. You may try to do it. Specifically, since each similarity is normalized by 1, the final similarity may be simply the sum of the similarity, or the image feature similarity and the finding similarity may be used. The final similarity may be calculated by multiplying each weight and adding weights.

(Modification 3)
In the second embodiment, the doctor identifies the corresponding findings and the positions of the findings of the cases stored in the database 300 in advance. However, the findings and the like may be specified by other methods. For example, the cases stored in the database 300 may be processed in steps S3010 to S3050, and the corresponding findings and the position information of the findings may be associated with a medical image. Further, after performing the process of step S10070, the case may be saved in the database.

(Modification example 4)
In the second embodiment, the search unit 914 searched for similar cases from the cases stored in the database 300. However, it is not always necessary to search for similar cases from the database 300 connected to the outside. For example, case information having information equivalent to that of the database 300 may be stored in the storage unit 104, and the search unit 914 may search for similar cases from the case information stored in the storage unit 104. In this case, the case information stored in the storage unit 104 may be replaced according to various conditions such as the user, the installation location, and the time. Similarly, the first table information and the second table information may be replaced depending on the conditions.

<Embodiment 3>
In the second embodiment, a medical image to be diagnosed is acquired, support information is presented on the medical image using the image processing result, and an analysis image of the image processing and a similar case are presented. In the present embodiment, a case where support information such as CAD findings is presented on a medical image and information that serves as a guideline for diagnosis is presented will be described. The guideline in the present embodiment is information that supports the diagnosis, and is a diagnostic manual showing the diagnostic procedure, an atlas, a point of interpretation, an image of the same modality, and an image of a different modality (for example, a fundus photograph). , FA image, IA image), etc., including information such as sentences and figures. That is, the guideline is information indicating the diagnostic procedure of the disease. In addition, a plurality of guidelines are stored in the database 300. That is, the database 300 stores information indicating a procedure for diagnosing a plurality of diseases. The guideline may be an electronic book showing a diagnostic procedure.

The functional configuration of the diagnostic support device 900 in this embodiment is the same as that in FIG. 9 in the second embodiment. The database 300 stores medical images and guidelines and findings corresponding to the images in association with each other. That is, the database 300 stores the findings in association with each of the information indicating the diagnostic procedure of the plurality of diseases. Then, in response to the request from the diagnosis support device 900, the database 300 outputs the guideline and the findings corresponding to the request to the diagnosis support device 900. The database 300 may be configured to be stored in the storage unit 104 inside the diagnosis support device 900. The search unit 914 requests a guideline corresponding to the target medical image from the database 300 using the candidate of the finding determined to be applicable by the determination unit 110 as a search key. Then, the search unit 914 receives (acquires) the guideline from the database 300. That is, the search unit 914 corresponds to an example of an information acquisition means that acquires information indicating at least one disease diagnosis procedure from information indicating a plurality of disease diagnosis procedures based on the findings of the abnormal site detected by the detection means. To do. Specifically, the search unit 914, which is an example of the information acquisition means, acquires information indicating a disease diagnosis procedure associated with the findings of the abnormal site detected by the detection means.

Further, the search unit 914 outputs the received guideline to the display control unit 116 together with the degree of relevance to the medical image targeted. Then, the display control unit 116 causes the display unit 118 to display the guideline. That is, the display control unit 116 causes the display means to display information indicating the diagnosis procedure of the disease acquired by the information acquisition means.

The hardware configuration of the diagnostic support device 900 is the same as that of FIG. 2 in the first embodiment. That is, when the CPU 1001 executes the program stored in the main memory 1002 and the magnetic disk 1003, the functions (software) of the diagnostic support device 900 of the present embodiment and the processing in the flowchart are realized. Further, FIG. 13 shows a flowchart illustrating an example of processing performed by the diagnosis support device 900. It is basically the same as the first embodiment, but a part of the treatment is different from the first embodiment. Specifically, the guideline search S13060 and the information display S13070 are different from the flow of FIG. Hereinafter, an example of the processing performed by the diagnostic support device 900 in the present embodiment will be described only with reference to the flowchart of FIG. 13 and an example of the display contents of FIG. 14, and only the differences from the first and second embodiments.

In the flowchart of FIG. 13, the processes of steps S3000 to S3050 are the same as those of the first and second embodiments.

In step S13060, the search unit 914 acquires a guideline from the database 300 using the candidate of the finding determined to be applicable in step S3040 as a search key. Search keys are "posterior vitreous detachment", "macular hole" and the like. The number of findings used as the search key may be plural or one. It is assumed that the findings stored in the database 300 are, for example, findings specified by the doctor in advance for the case, and the database 300 also includes information regarding the position of the findings specified by the doctor. The information on the findings contained in the database 300 may be information on literature such as a textbook instead of the information specified in advance by the doctor, or the doctor may specify the cases collected for the database. You may use the information of the findings.

As for the method of obtaining the guideline, only the search key that exactly matches may be obtained, the one that partially matches may be obtained, or the guideline that partially matches the search key may be obtained. May be. For example, when a plurality of findings are used as a search key, the search unit 914 may acquire a guideline that matches only a part of the plurality of findings. Alternatively, the similarity may be calculated by some method, and a specified number of guidelines may be obtained in descending order of similarity. The method using the similarity can be calculated by the degree of coincidence of the feature vectors as described in the modified example of the second embodiment. In this embodiment, the degree of similarity will be described as the degree of relevance.

In step S13070, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116. This will be described with reference to FIG.

FIG. 14 shows an example of the display contents displayed by the display unit 118. The display content 700 displayed by the display unit 118 is based on the diagram described with reference to FIG. 7 of the first embodiment. The difference from the first embodiment is that the guideline mode 1433 is displayed as a user interface for selecting the guideline mode. When the operator selects the guideline mode by tapping or clicking the guideline mode 1433, the display control unit 116 displays the guideline list (that is, the list displayed in descending order of the relevance of the guideline) 1460. Further, the display control unit 116 displays a tomographic image 1420 and findings 1421 of one case in the guideline list, its diagnosis name 1422, an interpretation point 1424, and a fundus image 1425 and a fundus photograph 1426 corresponding to the tomographic image. Displayed in unit 118. The interpretation point 1424 contains, for example, information on what kind of procedure is desirable for interpretation. The guideline list 1460 includes the case ID, diagnosis, and relevance. The degree of relevance is equivalent to the degree of similarity, and in this example, the degree of relevance (similarity) is shown on a 5-point scale. For example, since the similarity is normalized by 1, those with a similarity of 0.9 or more are 5, those with a similarity of 0.8-0.9 are 4, and those with a similarity of 0.6-0.8. The one with a similarity of 0.4-0.6 is 2, the one with a similarity of 0.4 or less is 1, and so on. Note that this numerical value is an example, and the similarity may be displayed as it is, or may be evaluated on a 10-point scale instead of a 5-point scale. Further, when displaying using the similarity, those having a certain degree of similarity or higher may be displayed, and those having a low correlation may not be displayed in the list.

If guideline mode 1433 is selected, option 750 adds guideline search 1455, a user interface that performs guideline search. Then, by tapping or clicking the guideline search 1455, the guideline search related to the target tomographic image 710 is executed. In the present embodiment, an example in which the user interface for executing the guideline search is different from the guideline mode 1433 is shown, but the guideline search may be executed at the same time as the guideline mode 1433 is selected.

In the tomographic image 710 on which the findings are superimposed and displayed, the information of the findings 711 determined to be applicable based on the information indicating the position of the findings is superimposed and displayed in accordance with the above-mentioned display rule. As described above, when another case in the guideline list is selected as a result of the selection by the user input, the display content of the display unit 118 is controlled by the display control unit 116. Specifically, the display control unit 116 corresponds to the tomographic image 1420 on which the findings are superimposed and displayed, the diagnosis name 1422, the interpretation point 1424, and the image corresponding to the case selected in the guideline list 1160. The fundus image 1425 and the fundus photograph 1426 are displayed on the display unit 118.

According to the present embodiment, the diagnostic support device 900 acquires image processing parameters related to deriving the corresponding finding candidates and finding candidates based on the information of the site of the medical image. Then, image processing is performed based on the acquired image processing parameters, the determination and position of the corresponding findings are specified, the guideline is searched based on the findings determined to be applicable, and these results are presented. By doing so, the doctor confirms the findings and their positions determined to be applicable on the medical image, the diagnosis name associated with the guideline, the image taken with other modality, and the interpretation point for interpretation. be able to. As a result, the doctor can browse the diagnosis name of the case similar to the state of the medical image currently targeted, so that the burden related to the diagnosis can be reduced.

As described in the second embodiment, the position of the finding may be used for searching for similar images.

(Modification example 1)
In the third embodiment, the guideline is searched by matching the findings using the findings as a search key. However, the guideline search method is not limited to this. For example, a guideline search may be performed by selecting only arbitrary findings from a plurality of findings. When the guideline search is performed with the doctor's finding display 743 turned on and the CAD finding display 744 turned off, the guideline may be searched using only the findings confirmed by the doctor. .. Further, when the doctor finding display 743 is turned off and the CAD finding display 744 is turned on and the guideline search is executed, the guideline may be searched using only the CAD findings. Further, when only an arbitrary finding is selected from the findings 711 displayed on the tomographic image and the guideline search is executed, the guideline search may be executed using the selected arbitrary finding. Note that this modification can be applied not only to the third embodiment but also to other embodiments.

<Embodiment 4>
In the first to third embodiments, support information such as CAD findings is presented using one captured tomographic image. On the other hand, in the inspection using the optical interference tomography, tomographic images may be taken and acquired from a plurality of directions in one inspection. For example, as shown in FIG. 15A, two tomographic images are acquired, one is a tomographic image passing through the optic disc and the fovea, and the other is a cross section orthogonal to the tomographic image passing through the optic disc and the fovea. In some cases. Further, as shown in FIG. 15B, a three-dimensional image may be taken by performing a raster scan (3D scan) on the fundus.

The diagnostic support device in the present embodiment acquires a plurality of medical images taken in a plurality of directions or at a plurality of positions, and presents support information on the medical image using the image processing result of the acquired medical image. Provide diagnostic support for the medical image.

The functional configuration of the diagnostic support device 900 in this embodiment is the same as that in FIG. 9 in the second embodiment. The hardware configuration of the diagnostic support device 900 is the same as that of FIG. 2 in the first embodiment. That is, when the CPU 1001 executes the program stored in the main memory 1002 and the magnetic disk 1003, the functions (software) of the diagnostic support device 900 of the present embodiment and the processing in the flowchart are realized. Further, the flowchart for explaining an example of the processing performed by the diagnosis support device 900 is the same as that of any one of FIGS. 3, 10, and 13. In the present embodiment, a case of searching for the similar image of FIG. 10 will be described. Hereinafter, with reference to the flowchart of FIG. 10, the entire processing performed by the diagnostic support device 900 in the present embodiment will be described only with respect to the differences from the first to third embodiments. In this embodiment, a case of acquiring two tomographic images as shown in FIG. 15A and an example of acquiring a three-dimensional tomographic image shown in FIG. 15B will be described. That is, this is an example in which the medical image acquisition unit 102 acquires a plurality of tomographic images having different positions in the eye to be inspected.

First, the case of the cross scan of FIG. 15A will be described. Hereinafter, the tomographic image passing through the optic nerve head and the fovea centralis is referred to as an H scan image, and the tomographic image passing through the fovea centralis and orthogonal to the H scan image is referred to as a V scan image.

In the process of step S3000, the medical image acquisition unit 102 includes a plurality of medical images taken by investigating light from a plurality of directions transmitted from the optical interference tomography meter 200, and information on the types of the taken medical images. Obtain information about the part of the medical image. That is, the medical image acquisition unit 102 acquires a plurality of tomographic images obtained by scanning the measurement light in different directions with respect to the eye to be inspected. In this embodiment, the different directions are orthogonal to each other.

It should be noted that the information regarding the type of the photographed medical image and the information regarding the part of the medical image may be acquired as one for the entire plurality of medical images, or may be acquired for each of the medical images. Hereinafter, it will be described as assuming that information on the type of the medical image and information on the part of the medical image are acquired for the entire plurality of medical images.

Since the process of step S3010 is the same as the process in the first embodiment, the description thereof will be omitted.

The processing of steps S3020 to S3050 is the same as that of the first embodiment except that the processing is performed on each of the plurality of medical images, and thus the description thereof will be omitted.

In step S10060, the search unit 914 acquires a similar image from the database 300 using the candidate for the finding determined to be applicable in step S3040 as a search key. In the present embodiment, a similar image is acquired from the database 300 using a candidate for a finding determined to be applicable in one medical image (that is, a medical image taken from a certain direction) among a plurality of medical images as a search key. To do. In this embodiment, a similar image is acquired using the findings obtained from the H scan image. In the present embodiment, the image to be searched is only the medical image taken from the same direction as the direction in which the medical image to be searched is taken. That is, the search unit 914, which corresponds to an example of the similar image acquisition means, is obtained by scanning the measurement light in the same direction as the scanning direction of the measurement light when obtaining the tomographic image in which the abnormal portion is detected by the detection means. The tomographic image is acquired from the storage means.

Further, as in the second embodiment, it is assumed that only the images whose search keys match exactly are acquired as similar images. If there are medical images corresponding to similar images and taken from a direction different from the similar images, they are collectively acquired as one similar image.

In step S10070, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116. Specifically, at least one of the plurality of medical images is displayed under the same control as in the second embodiment. Further, the display control unit 116 displays similar images and information on the diagnosis name associated with them in descending order of similarity, as in the second embodiment.

Further, also in the present embodiment, as in the second embodiment, similar images and information on the diagnosis name associated therewith are managed in a list, the information in the list is displayed, and one case in the list is displayed in step S3000. Display in a form that can be compared with multiple medical images acquired in. That is, when similar images have a plurality of medical images taken from different directions, at least one of them is displayed in a comparable form. As the image to be displayed at this time, an image taken from the same direction as the plurality of medical images acquired in step S3000 is displayed. Images taken from different directions may be displayed.

In the present embodiment, among the plurality of medical images acquired in step S3000 and the plurality of medical images of similar images determined as one case displayed in a comparable form, one image is displayed. However, the image to be displayed shall be determined by the diagnostic support device 900 or selection by user input. Selection by user input can be realized by using a user interface such as a button.

FIG. 16 shows an example of the display contents displayed by the display unit 118. The display content 700 displayed by the display unit 118 includes a tomographic image 710 of the target case and a fundus image 715 corresponding to the tomographic image of the target case.

An arrow 717 indicating the direction in which the tomographic image was taken is superimposed and displayed on the fundus image 715. The horizontal arrow 717 corresponds to the H-scan image, and the vertical arrow 1619 corresponds to the V-scan image. In addition, the display content 700 has a similar case list 1160. Further, the display content 700 includes a tomographic image 1120 and its diagnosis name 1122 of one determined case in the similar case list, and a fundus image 1125 and a horizontal arrow corresponding to the tomographic image, as in the case of the target case. It has 1127 and a vertical arrow 1639. It should be noted that the superimposed display of the findings relating to the tomographic images 710 and 1120 and the explanation of the user's selection of the similar case list are the same as those in the second embodiment and will be omitted.

In the initial state of the tomographic image 1120 (that is, before the selection is made by user input), the display control unit 116 displays the tomographic image in the direction in which the tomographic image 710 including the findings used for the search in step S10060 was taken. It shall be displayed in. That is, when the tomographic image 710 is an H scan image, the tomographic image 1120 is also an H scan image. In the present embodiment, as described above, one of a plurality of medical images can be displayed by selection by user input. Specifically, the horizontal arrows 717 and 1127 and the vertical arrows 1619 and 1639 superimposed on the fundus images 715 and 1125 are selected by, for example, clicking or tapping the mouse. When the vertical arrows 1619 and 1639 are selected, the display control unit 116 causes the display unit 118 to display the medical image (V-scan image) taken in the direction corresponding to the clicked arrow as the tomographic images 710 and 1120. In this case, it is desirable to display the currently selected arrow so as to distinguish it from other arrows. In the example of FIG. 16, the arrow displayed by the solid line indicates that the arrow is selected, and the arrow of the alternate long and short dash line indicates that it is not selected. By selecting either one of the vertical arrows 1619 and 1639, both the tomographic images 710 and 1120 may be linked and changed to a V-scan image.

It should be noted that only the common findings of the V-scan image and the H-scan image may be displayed on each tomographic image. That is, the display control unit 116 may display the findings of the abnormal portion common to the plurality of tomographic images on the display means based on the plurality of abnormal portions obtained from the plurality of tomographic images.

Next, the case of the three-dimensional scan shown in FIG. 15B will be described. Hereinafter, with reference to the flowchart of FIG. 10, an example of the processing performed by the diagnostic support device 900 in the present embodiment will be described only with respect to the difference from the second embodiment. In the three-dimensional scan, the following processing will be described assuming that the alignment processing between a plurality of tomographic images is performed by the image processing unit 108 or the alignment means (not shown).

Since the process of step S3010 is the same as the process in the first embodiment, the description thereof will be omitted.

In step S3020, the same processing as in the other embodiments is performed for each of the plurality of medical images, but the image processing parameters stored in the second table information include parameters in the Y direction as spatial parameters. May include. For example, since the macular hole is generated in the macula, when the image is taken at the center of the macula, the macula exists near the center on both the X and Y axes. Therefore, the range and area of X: 40-60% and Y: 40-60% of the imaging range can be limited, and the image processing parameters are applied only to the tomographic images included in the range.

In step S3030, the abnormal portion of the eye to be inspected is detected by performing structural analysis on a plurality of tomographic images acquired by raster scan. That is, the image processing described in the other embodiment is applied to each of the plurality of tomographic images.

In step S3030, image processing may be performed three-dimensionally. Luminance normalization is performed not for each of a plurality of tomographic images, but for histogram adjustment for the entire plurality of tomographic images. Further, various filters used for image processing may be defined in three dimensions.

Similarly, in step S3040, the finding is determined three-dimensionally. For example, the determination itself may be performed for each of the plurality of tomographic images, and then re-judgment may be performed between the regions of adjacent finding candidates in the adjacent tomographic images, or the lesion candidates detected in step 3030 may be used. On the other hand, the candidate findings may be determined based on the size ratio of the three-dimensionally connected regions and the like.

In step S3050, the position acquisition unit 112 acquires information indicating the position of the finding in the medical image based on the result of the image processing executed in step S3030 and the candidate of the finding determined to be applicable in step S3040. The information indicating the position may be any information as long as the position can be specified, but in the present embodiment, the information indicating the position of a rectangular parallelepiped or a cube adjacent to the image processing area determined to correspond to the findings. Get as.

In step S10060, the search unit 914 acquires a similar image from the database 300 using the candidate for the finding determined to be applicable in step S3040 as a search key. In the present embodiment, the search unit 914 acquires a similar image from the database 300 by using the candidate of the finding determined to be applicable in the plurality of medical images as a search key. Similar to the second embodiment, it is assumed that an image having a completely matching search key or a high degree of similarity is acquired as a similar image. If there are images taken in different directions or a plurality of images taken in the same direction in similar images having similar findings and image features, the search unit 914 collectively acquires them as one similar image. To do.

In step S10070, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116. Specifically, at least one of the plurality of medical images is displayed under the same control as in the second embodiment. Further, the display control unit 116 displays similar images and information on the diagnosis name associated with them in descending order of similarity, as in the second embodiment.

Further, also in the present embodiment, as in the second embodiment, the information of similar images and the diagnosis name associated therewith is managed in a list, the information in the list is displayed, and one case in the list is displayed in step S3000. Display in a form that can be compared with the acquired multiple medical images. In the present embodiment, of the plurality of medical images acquired in step S3000 and the plurality of medical images of similar images determined as one case to be displayed in a comparable form, one image is displayed. However, the image to be displayed shall be determined by the diagnostic support device 900 or selection by user input. Selection by user input can be realized by using a user interface such as a button.

FIG. 17 shows an example of the display content displayed by the display unit 118. The display content 700 displayed by the display unit 118 includes a tomographic image 710 of the target case and a fundus image 715 corresponding to the tomographic image of the target case. The fundus image 715 is, for example, an integrated image generated by integrating a plurality of tomographic images acquired by the medical image acquisition unit 102 in the depth direction. The medical image acquisition unit 102 may generate an integrated image. Since the fundus image 715 is an image generated based on the tomographic image, it is possible to make the position of the findings of the tomographic image correspond to the position on the fundus image 715. Further, the fundus image 715 may be an image taken by SLO or a fundus camera. In this case, the medical image acquisition means 102 acquires a fundus image from an SLO or a fundus camera (not shown). That is, the medical image acquisition unit 102 acquires the fundus image of the eye to be inspected. By aligning the fundus image obtained by the SLO or the fundus camera with the integrated image generated based on the tomographic image, the findings of the tomographic image are associated with the fundus image 715 obtained by the SLO or the fundus camera. It is possible. Since it is possible to associate the findings of the fundus image with the fundus image 715 as described above, the display control unit 116 can display the findings on the tomographic image 715. That is, the display control means can further display the fundus image on the display means and also display the findings on the fundus image.

On the fundus image 715, an arrow 717, a finding 1711, and a marker 1712 indicating the position of the displayed tomographic image 710 on the fundus are superimposed and displayed. The marker 1712 is displayed so as to indicate the position of the center of gravity of the rectangular parallelepiped or cube indicating the region of the finding position acquired in step S3050.

In the present embodiment, the position to be superimposed and displayed on the fundus image 715 is determined according to the following display rules for each applicable finding. Of course, the display rules are not limited to the examples shown below. (1) Do not superimpose on the character information of other findings judged to be applicable. (2) For the information indicating the position of the finding determined to be applicable, a marker (for example, an arrow) is added in the vicinity of the information indicating the position of the finding. (3) When there are multiple identical findings, the representative location is displayed in the spatially close area. (4) If there are more than N same findings, the number to be displayed shall be N or less.

Regarding (3) of the above-mentioned rule, for example, a plurality of hard vitiligo may be present in a close region. In that case, items that are spatially close to each other are displayed together. Specifically, when five hard white spots are present in a circle having a diameter of 1 mm on the XY plane, only one finding is displayed together instead of displaying five findings on the fundus image. That is, the display control unit 116 has the same findings corresponding to the abnormal sites detected by the detection means from the different tomographic images, and the distance between the abnormal sites detected from the different tomographic images is less than the threshold value. In this case, the same finding is displayed on the fundus image as one finding. The findings may be summarized when the distance between the hard white spots is less than the threshold value, or the findings may be summarized when the distance between the plurality of tomographic images in which the hard white spots are detected is less than the threshold value.

Regarding (4) of the above-mentioned rule, for example, even if the ones spatially close to each other according to the above-mentioned rule (3) are put together, if they appear in a wide range, more findings than N numbers will be displayed. In that case, the number of the same findings displayed should be N or less. For example, N is 3. If it is more than 3, leave one that is close to the fovea centralis of the macula, or select one that is close to the fovea centralis of the macula, and the remaining two are spatially separated. You may leave. For example, the center is set as one place, and the remaining two places are selected one by one from the vertical direction. Findings near the fovea have a higher effect on visual function than other findings, and are therefore necessary information for doctors and the like.

The findings 1711 and the marker 1712 to be superimposed and displayed on the fundus image can be turned ON / OFF by the user interface of the information display 740 as described in the first embodiment. In addition, it is possible to turn on / off the display of the findings by the doctor and the findings by CAD. ON / OFF of these findings display may be changed according to the number of clicks. For example, "both are ON" ⇒ "tomographic image findings ON, fundus image findings OFF" ⇒ "tomographic image findings OFF, fundus image findings ON" ⇒ "both are OFF" and the like. Alternatively, the type of findings to be turned ON / OFF may be specified by a user interface (not shown). It is possible to confirm (approve) the CAD findings output by the computer, or to correct the CAD findings. These processes can be performed from either the findings 711 superimposed on the tomographic image 710 and the findings 1711 superimposed on the fundus image 715. Then, when any of the findings 711 and 1711 is confirmed (or modified), the corresponding findings are also determined (corrected) in conjunction with each other. That is, when the color of the characters of the findings is changed between the CAD findings and the doctor's findings, the display control unit 116 assumes that the "lid" of the findings 711 is also determined by determining the "lid" of the findings 1711. Change the text of both findings. Further, in the case of data taken in three dimensions, since the lesion has a three-dimensional spread, the same findings are displayed in a plurality of tomographic images. For example, if it is a "lid", it will be displayed on five adjacent tomographic images. Even in this case, if the fundus image is confirmed or one of the tomographic images is confirmed, the display control unit 116 displays on the display unit 118 with the character color that the other tomographic images are also confirmed. Let me.

The slice positions of a plurality of tomographic images can be changed by operating a slider bar (not shown) or an arrow 717. Alternatively, the position of the tomographic image may be changed by selecting the finding 1711. That is, by tapping or clicking the finding 1711 on the fundus image, the tomographic image including the finding 1711 is displayed. In that case, the display control unit 116 causes the display unit 118 to display a tomographic image corresponding to the position of the center of gravity of the rectangular parallelepiped or cube adjacent to the image processing area determined to correspond to the selected finding 1711. In this way, it is possible to quickly display the tomographic image to be noticed based on the fundus image and the findings.

In addition, the display content 700 has a similar case list 1160. Further, as in the case of the target case, the tomographic image 1120 and its diagnosis name 1122 of one determined case in the similar case list, the fundus image 1125 corresponding to the tomographic image, and the arrow 1127 indicating the position of the tomographic image 1210. And has findings 1721. The superimposition display of the findings relating to the superposed tomographic images 710 and 1120 and the explanation of the selection by the user of the similar case list will be omitted because they are the same as those in the second embodiment.

In the present embodiment, as described above, one of a plurality of medical images can be displayed by selection by user input.

According to the present embodiment, when the medical image consists of a plurality of images, not only the image taken from the direction to be searched but also the image taken from another direction can be confirmed by a simple operation. can do. That is, since the findings that spread over a wide area can be easily confirmed, the burden on the doctor's diagnosis can be reduced.

(Modification example 1)
In the fourth embodiment, in step S10060, the search unit 914 uses a search key as a search key for a candidate for a finding that is determined to be applicable to one of the plurality of medical images, and the database 300 targets only the medical images taken from the same direction. A similar image was obtained from. However, it is not necessary to use only the findings determined to be applicable in one medical image. For example, a search key may be created for each of two or more or all medical images among a plurality of medical images, and similar images may be acquired in all directions. For example, consider a case where a plurality of medical images of a search source are an H scan image and a V scan image. In this case, the search unit 914 has an H-scan image similar to the H-scan image of the search source and a V-scan image similar to the V-scan image of the search source among the images existing in the database 300. As a similar image.

Further, in the fourth embodiment, only the medical image taken from the same direction as the direction in which the medical image as the search source was taken is targeted for acquisition from the database 300, but the medical image taken from another direction is obtained. May be the target to be acquired from the database 300. For example, when the image of the search source is an H scan image, the image of the V scan image existing in the database 300 may be acquired as a similar image. Further, the search unit 914 does not have to use only the findings determined to be applicable in one medical image. The search key may be obtained by taking AND or OR of findings determined to be applicable to two or more or all medical images out of a plurality of medical images.

<Embodiment 5>
In the fourth embodiment, a plurality of medical images taken from a plurality of directions are acquired, and support information such as CAD findings is presented on the medical image using the image processing result of the acquired medical image. In the present embodiment, support information is presented on the medical image using the image processing result of the acquired medical image, and diagnosis support for the medical image is performed by estimating and presenting the disease name (diagnosis name). An example of the functional configuration of the diagnostic support device 1800 according to the fifth embodiment will be described with reference to FIG. In the diagnosis support device 1800, the estimation unit 1811 is added to the functional configuration shown in FIG. The estimation unit 1811 realized by executing the program by the CPU 1001 estimates the disease name using the abnormal site detected from the tomographic image. Further, FIG. 19 shows a flowchart illustrating an example of processing performed by the diagnosis support device 1800. It is basically the same as the first embodiment, but a part of the treatment is different from the first embodiment. Specifically, step S19060 for estimating a disease name candidate and step S19070 for displaying information are different from the first embodiment. Hereinafter, with reference to an example of the display contents of FIG. 19 and FIGS. 20 and 21, the entire processing performed by the diagnostic support device 1800 in the present embodiment will be described only for the differences from the first to fourth embodiments. .. In this embodiment, the case of the cross scan as shown in FIG. 15A will be described. The present embodiment can be applied to other scan patterns.

Since the processing of steps S3000 to S3050 is the same as that of the fourth embodiment, the description thereof will be omitted.

In step S19060, the estimation unit 1811 estimates the disease name candidate based on the obtained findings. That is, the estimation unit 181 corresponds to an example of an estimation means for estimating the disease name based on the findings. The estimation unit 1811 estimates the disease name candidate using the positions of the candidate findings for the H scan image and the V scan image acquired in step S3050. That is, the estimation unit 1811 estimates the disease name candidate based on which finding was detected at which position on the fundus image. The identification of the location of the findings on the fundus image will be described with reference to FIG. FIG. 20 shows a plurality of examples of dividing a region with respect to a fundus image in FIGS. 20A to 20D. 20 (a) and 20 (b) are examples in which the area of the fundus image is divided into the ear side, the upper side, the nasal side, and the lower side with respect to the fovea centralis of the macula, depending on the distance from the fovea centralis of the macula. The sector is divided. For example, the diameter of the sector is 1 mm, 3 mm, and 10 mm. The numerical values are not limited to these values. FIG. 20A is an example of dividing a circular region based on the fovea centralis of the macula, and FIG. 20B is an example of dividing the entire region outside 3 mm from the fovea centralis.

Further, FIGS. 20 (c) and 20 (d) are examples of dividing the entire fundus image into a grid, and FIG. 20 (c) divides the fundus image into regions by horizontal lines and vertical lines in the shooting range of the fundus image. This is an example. Further, FIG. 20D is an example in which the fundus image is divided into regions by horizontal lines and vertical lines with respect to the straight line connecting the optic nerve head and the fovea centralis of the macula. As shown here, the fundus image is divided into arbitrary regions with reference to at least one of the macula and the optic disc.

Here, a case where the area of the fundus image is divided as shown in FIG. 20A will be described. For example, in the estimation unit 1811, when retinal thickening is detected on the upper side or the lower side of the V scan image and the foveal cyst-like hole is detected in the center, and the foveal cyst-like hole is detected in the H scan image, Estimate (detect) a candidate for the name of branch retinal vein occlusion. In this way, the estimation unit 1811 estimates the disease name based on the findings of the V-scan image and the findings of the H-scan image. That is, the estimation unit 1811 estimates the disease name based on the findings of the abnormal site obtained from each of the plurality of tomographic images.

The estimation unit 1811 may use a decision tree method such as Random forests or a discriminator such as SVM to estimate the disease name candidate. Alternatively, the estimation unit 1811 may use the similar case search based on the findings or image features to perform a search considering the spatial features in which the findings appear in the fundus image, and present the disease name associated with the search results. Good. In the case of cross scan, the spatial features are calculated by the feature vector of the center + upper + lower region and the feature vector of the center + nasal + ear region, respectively.

In step S19070, the display control unit 116 controls the content to be displayed on the display unit 118. Then, the display unit 118 displays the display content controlled by the display control unit 116. Specifically, at least one of the plurality of medical images is displayed under the same control as in the second embodiment. FIG. 21 shows an example of the display contents displayed by the display unit 118. The display content 700 displayed by the display unit 118 includes a fundus image 2110, findings 2111 and a marker 2112 of the target case, and a fundus image 2115 corresponding to the fundus image of the target case. An arrow 2117 indicating the position of the fundus image 2110 (H scan image) displayed on the fundus image 2115 is superimposed and displayed. Similarly, the V-scan has a tomographic image 2120, findings 2121 and markers 2122, and a fundus image 2115 corresponding to the tomographic image of the case of interest. An arrow 2119 indicating the position of the tomographic image 2120 (V-scan image) displayed is superimposed and displayed on the fundus image 2115. As shown in FIG. 21, a wide range of bleeding region 2116 can be seen in the lower part of the fundus image 2115. In addition, the display content 700 has a diagnosis name candidate mode 2134 and a diagnosis name candidate list 2160. By selecting the diagnosis name candidate mode by tapping or clicking the diagnosis name candidate mode 2134, the process of displaying the above-mentioned diagnosis name candidate is executed.

In this embodiment, an example of displaying an H scan image and a V scan image is shown, but the present invention is not limited to this. For example, a plurality of tomographic images may be displayed at a location at the upper part of the screen (for example, the location of the tomographic image 2110), and the tomographic images may be switched and displayed by a user interface (not shown). That is, one of the H scan image 2110 and the V scan image may be displayed, and the image to be displayed may be switched by the user interface. Then, as shown in the third embodiment, the diagnosis guideline corresponding to the diagnosis name candidate list may be displayed at the lower part of the screen. By switching the selection of the diagnosis name displayed in the diagnosis name candidate list, the diagnosis guideline is also switched.

According to the present embodiment, the findings are displayed and the diagnosis name candidates are displayed. Furthermore, since the guidelines for diagnosis related to it can be referred to, the burden of diagnosis by a doctor can be reduced. Further, according to the present embodiment, since the disease name is estimated based on the findings of the two orthogonal tomographic images, it is possible to estimate the disease name more accurately.

<Other Embodiments>
All of these functions, or some of them, may be located on the network. In this case, it may be a server / client type configuration in which only the display function or a part of the functions are operated locally. In addition, each of the above-described embodiments may be an independent embodiment, or any embodiment may be combined.

Although the examples have been described in detail above, the present invention can be implemented as, for example, a system, an apparatus, a method, a program, a recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or it may be applied to a device composed of one device. good.

Needless to say, the purpose of the disclosed technology is achieved by doing the following. That is, a recording medium (or storage medium) in which a software program code (computer program) that realizes the functions of the above-described embodiment is recorded is supplied to the system or device. Needless to say, the storage medium is a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the function of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

100 Diagnosis support device 102 Medical image acquisition unit 104 Analysis information acquisition unit 106 Storage unit 108 Image processing unit 110 Judgment unit 112 Position acquisition unit 116 Display control unit 118 Display unit 300 Database 914 Search unit

Claims (18)

A detection means for detecting an abnormal part of the eye to be inspected by analyzing a tomographic image of the eye to be inspected.
An acquisition means for acquiring a similar case image related to a similar case of the detected abnormal site by using the finding information regarding the findings of the detected abnormal site and the position information regarding the position of the detected abnormal site.
A display control means for displaying the tomographic image and the acquired similar case image on the display means, and
An image processing device characterized by comprising. The acquisition means is related to the searched similar case by searching for a similar case having a finding that matches the finding of the detected abnormal site and a position similar to the position of the detected abnormal site. The image processing apparatus according to claim 1, wherein the similar case image is acquired. The display control means displays the tomographic image on the tomographic image in a state where at least a part of the finding information does not overlap with the region indicating the detected abnormal portion in the tomographic image. The image processing apparatus according to claim 1 or 2, wherein the image processing apparatus is displayed on the image processing apparatus. By intends line analysis with respect to the subject's eye tomographic image, a detection unit configured to detect abnormalities of the eye to be examined,
By intends line analysis with respect to the tomographic image obtaining means for obtaining the similar case images relating to the similar case of the detected abnormalities,
The acquired similar case image is displayed on the display means, and at a position where at least a part of the finding information regarding the findings of the detected abnormal site does not overlap with the region indicating the detected abnormal site in the tomographic image. A display control means for displaying the tomographic image on the display means in a state where the finding information is superimposed, and
An image processing device characterized by comprising. The image processing according to claim 3 or 4, further comprising a determining means for determining the non-overlapping position in the tomographic image based on the position of the region indicating the detected abnormal site in the tomographic image. apparatus. Further provided with an extraction means for extracting the layer boundary from the tomographic image,
The detection means performs structural analysis of different algorithms for the first region and the second region in the tomographic image defined by the extracted layer boundary, thereby performing structural analysis from the first region to the second region. The image according to any one of claims 1 to 5, wherein one abnormal site is detected, and a second abnormal site different from the first abnormal site is detected from the second region. Processing equipment. The display control means is characterized in that, together with the tomographic image and the acquired similar case image, the fundus image corresponding to the tomographic image and the fundus image corresponding to the acquired similar case image are displayed. Item 6. The image processing apparatus according to any one of Items 1 to 6. The display control means displays the first information regarding scanning of the measurement light when obtaining the tomographic image on the fundus image corresponding to the tomographic image, and measures when obtaining the acquired similar case image. The image processing apparatus according to claim 7, wherein the second information regarding the scanning of light is displayed on the fundus image corresponding to the acquired similar case image. By analyzing the tomographic image of the eye to be inspected, an acquisition means for acquiring a similar case image related to a similar case in the abnormal part of the eye to be inspected, and
A display control means for displaying the tomographic image and the acquired similar case image on the display means is provided.
The display control means displays the tomographic image and the acquired similar case image, as well as the fundus image corresponding to the tomographic image and the fundus image corresponding to the acquired similar case image.
The display control means displays the first information regarding scanning of the measurement light when obtaining the tomographic image on the fundus image corresponding to the tomographic image, and measures when obtaining the acquired similar case image. An image processing device characterized in that a second information regarding light scanning is displayed on the fundus image corresponding to the acquired similar case image. The first information is information about at least one of the scanning position and the scanning direction of the measurement light when obtaining the tomographic image.
The image according to claim 8 or 9, wherein the second information is information regarding at least one of the scanning position and the scanning direction of the measurement light when obtaining the acquired similar case image. Processing equipment. 10. The acquisition means according to claim 10, wherein the acquisition means acquires the similar case image obtained by scanning the measurement light in the same direction as the scanning direction of the measurement light when obtaining the tomographic image. Image processing device. The image processing apparatus according to any one of claims 1 to 11, wherein the display control means further displays a diagnosis name of the similar case. The image processing apparatus according to any one of claims 1 to 12, wherein the display control means displays the tomographic image and the acquired similar case image side by side in a contrastable manner. The display control means is on the tomographic image other than the region sandwiched between one of the inner limiting membrane and the nerve fiber layer boundary and one of the retinal pigment epithelial layer boundary and the photoreceptor inner segment outer segment boundary included in the tomographic image. The image processing apparatus according to any one of claims 1 to 13, wherein the finding information regarding the finding of the abnormal portion is displayed in the area of. The process of detecting an abnormal part of the eye to be inspected by analyzing the tomographic image of the eye to be inspected, and
A step of acquiring a similar case image related to a similar case of the detected abnormal site by using the finding information regarding the findings of the detected abnormal site and the position information regarding the position of the detected abnormal site, and
A step of displaying the tomographic image and the acquired similar case image on the display means, and
An image processing method characterized by including. By intends line analysis with respect to the subject's eye tomographic image, and detecting abnormalities in the subject's eye,
By intends line analysis with respect to the tomographic image, a step of acquiring a similar case images relating to the similar case of the detected abnormalities,
The acquired similar case image is displayed on the display means, and at a position where at least a part of the finding information regarding the findings of the detected abnormal site does not overlap with the region indicating the detected abnormal site in the tomographic image. A process of displaying the tomographic image on the display means in a state where the finding information is superimposed, and
An image processing method characterized by including. A step of acquiring a similar case image related to a similar case of the abnormal part of the eye to be inspected by analyzing the tomographic image of the eye to be inspected.
Measurement when the tomographic image is obtained by displaying the tomographic image, the acquired similar case image, the fundus image corresponding to the tomographic image, and the fundus image corresponding to the acquired similar case image on a display means. The first information regarding the scanning of light is displayed on the fundus image corresponding to the tomographic image, and the second information regarding the scanning of the measured light when obtaining the acquired similar case image is obtained. The process of displaying on the fundus image corresponding to the similar case image, and
An image processing method characterized by including. A program comprising causing a computer to execute the image processing method according to any one of claims 15 to 17.

Images