JP2018201629A - Fundus image processing device - Google Patents

Abstract

To provide a fundus image processing device capable of accurately specifying an object region by automatically extracting a predetermined object region in a range including an optic nerve head, and correcting an interactive form.SOLUTION: A fundus image processing device includes: gray scale image conversion means 10 for converting a color fundus image to a gray scale image for which a pixel value is set according to the probability of an object region including an optic nerve head; closing processing means 20 for generating a smoothed gray scale image based on a circular structure element; extraction range definition means 30 for defining the lower limit of the pixel value including all pixels included in the object region; object region identification image generation means 40 for generating an object region identification image in which the object region and a non-object region can be identified visually to the pixels of the pixel value equal to or higher than the lower limit for each pixel of the smoothed gray scale image; and object region identification image correction means 50 for correcting the value of the pixels identified as the object region in an indicated region to a value identified as a non-object region.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a fundus image processing technique, and more particularly to a technique for identifying a predetermined target region in a fundus image and supporting a diagnosis of a disease.

  Diagnosis of glaucoma, which is a common optic nerve disease, is mainly performed by fundus examination and visual field examination. In the fundus examination, the optic disc in the fundus image (Disc, concave cup, rim Rim), PPA (β-PPA, peripapillary network) Choroidal atrophy) and NFLD (retinal optic nerve fiber layer defect) are diagnostic points. Above all, the C / D ratio (Cup / Disc ratio) and R / D ratio (Rim-to-Disc Ratio) of the optic nerve head are measured as indicators to quantitatively grasp the progress of glaucoma. ing.

  Regarding the Cup / Disc ratio, a method for quantitatively grasping the progression of glaucoma by calculating the Cup / Disc ratio from a plurality of fundus images taken in time series has been proposed (see Patent Document 1). . In addition, a method of automatically extracting the Disc region from the R component of the full-color fundus image and the Cup region from the G component has been proposed (see Patent Document 2).

Japanese Patent No. 5701494 Japanese Patent No. 3508112

  However, the technique described in Patent Document 1 uses a method of manually setting the Cup / Disc region of the optic nerve head, and there is a problem that a user's bias tends to enter the measurement value. In addition, in the technique described in Patent Document 2, blood vessels are extracted together in order to verify the validity of the extracted nipple region, and extracted using vertical blood vessels (blood vessels that branch up and down in the nipple). When the nipple area does not coincide with the nipple area, a method of manually setting the nipple area is used, and similarly, there is a problem that a user's bias is easily included in the measured value. Furthermore, although the technique described in Patent Document 2 supports the analysis of NFLD (retinal optic nerve fiber layer defect), there is a problem that it requires not only fundus images but also visual field examination data, and PPA (β-PPA, There is no support for analysis of peripapillary choroidal atrophy).

  Therefore, the present invention provides a fundus image processing apparatus capable of specifying a target region with high accuracy by automatically extracting a target region as a glaucoma diagnosis location and modifying the interactive format. Is an issue.

In order to solve the above problems, in the first aspect of the present invention,
Gray scale image conversion means for converting a color fundus image into a gray scale image set so that the pixels of the target region serving as a glaucoma diagnosis location are included in a high pixel value range;
Closing processing means for creating a smoothed grayscale image by performing a closing process on the grayscale image based on a circular structuring element of a predetermined size;
Extraction range defining means for defining a lower limit value of a pixel value including at least all pixels included in the target area in the smoothed grayscale image;
For each pixel of the smoothed grayscale image, a target region identification image that can visually identify the target region and a non-target region other than the target region with respect to a pixel having a pixel value equal to or greater than the lower limit value. Target area identification image generating means for generating
Based on the area designated for the target area identification image, the value of the pixel identified as the target area in the designated area is corrected to a value identified as a non-target area, and the correction target area A target area identification image correcting means for creating an identification image;
There is provided a fundus image processing apparatus characterized by comprising:

    According to the first aspect of the present invention, for a color fundus image, a grayscale image in which a higher value is set for a pixel having a higher probability of being a predetermined target region in a range including the optic papilla is obtained, and the grayscale image is obtained. For example, a lower limit value of pixel values including all pixels included in the target region is defined by performing a closing process based on a circular structuring element having a predetermined size. Generate a target area identification image for visually identifying the target area and the non-target area for the pixels having the above pixel values, and specify the specified area based on the specified area for the target area identification image. The pixels that are incorrectly assigned the target area attribute are corrected to the non-target area attributes, so it is possible to correct the attributes with little variation for each individual with a small workload. To become. For this reason, the operator only needs to roughly indicate the correction target portion in an area such as a rectangle, and can perform attribute correction with little individual difference.

  Further, according to the first aspect of the present invention, since the operator mistakenly designates a region including a pixel to which the attribute of the target region is given, for the target region identification image having two types of attributes, By recording region information instructed that all the pixels in the region have the attributes of the non-target region as teacher data, it can be used for automatic correction of the region using machine learning means.

In the second aspect of the present invention, as the target region, a region corresponding to at least one of the three types of Disc, which is the optic papilla, Cup, which is the optic nerve depression, and PPA, which is the peripapillary choroidal atrophy, is provided. Set,
The gray scale image conversion means, the closing processing means, the extraction range definition means, the target area identification image generation means, and the target area identification image correction means are executed for each target area, and a Disc area, a Cup area, a PPA It is characterized in that at least one type of correction target area identification image is created among the three types having the area as the target area.

  Further, in the third aspect of the present invention, as the target region, at least regions corresponding to a plurality of types of discs that are the optic papilla, Cup that is the optic nerve recess, and PPA that is the peripapillary choroidal atrophy are set, The gray scale image converting means, the closing processing means, the extraction range defining means, the target area identification image generating means, and the target area identification image correcting means are executed on the target area, and at least the Disc area, Cup area, PPA A plurality of types of correction target region identification images having the region as the target region are created.

  In the fourth aspect of the present invention, for the plurality of types of correction target region identification images, pixels having attributes of each target region are colored with different colors for each of the plurality of types, and the anatomy / histology of the eyeball Corresponding to the pixels of the target area of the different correction target area identification image for each correction target area identification image, based on the replacement priority order of the corresponding types of the plurality of corresponding areas determined in advance so as not to contradict each other. Identification image composition for creating a composition target area identification image by combining the plurality of types of correction target area identification images by replacing the pixel to be replaced with the color of the pixel of the target area of the different correction target area identification image of the different type The apparatus further comprises means.

  In the fifth aspect of the present invention, as the target region, at least regions corresponding to a plurality of types of discs that are the optic papilla, Cup that is the optic nerve recess, and PPA that is the peripapillary choroidal atrophy are set. The first correction target area identification image corresponds to the Disc area, the second correction target area identification image corresponds to the Cup area, and the third correction target area identification image corresponds to the PPA area. In the case of corresponding one, the identification image combining means sets a pixel corresponding to a pixel of the target area of the second correction target area identification image to the second correction target area with respect to the first correction target area identification image. When replacing with the color of the pixel in the target area of the identification image, if the corresponding pixel in the first correction target area identification image is a pixel in the non-target area, the replacement is not performed. When replacing the pixel corresponding to the pixel of the target area of the third correction target area identification image with the color of the pixel of the target area of the third correction target area identification image with respect to another image, the first correction target area If the corresponding pixel of the identification image is a pixel in the target area, it is not replaced.

  In the sixth aspect of the present invention, the number of pixels colored with the three kinds of colors is counted for each color of the composition target region identification image, and the number of pixels corresponding to the second color is set to the first number. Area ratio calculating means for calculating a first ratio obtained by dividing the number of pixels corresponding to the color and a second ratio obtained by dividing the number of pixels corresponding to the third color by the number of pixels corresponding to the first color; It is characterized by providing.

  Further, in the seventh aspect of the present invention, when the target region is a region corresponding to three types of disc, which is an optic papilla, Cup which is a optic nerve recess, and PPA which is a peripapillary choroidal atrophy, the gray The scale image converting means uses the R component of the color fundus image as the gray scale image when the target region is a Disc region, and the G component of the color fundus image as the gray scale image when the target region is a Cup region. When the target area is a PPA area, an image reflecting a value obtained by subtracting the B component from the R component of the color fundus image from the maximum value is set as the grayscale image. It is characterized by.

  In the eighth aspect of the present invention, the extraction range defining means further defines an upper limit value of pixel values including all pixels included in the target area, and the target area identification image generating means A target area identification image is generated for a pixel having a pixel value within a range from a lower limit value to the upper limit value, so that a target area and a non-target area other than the target area can be visually identified.

  In the ninth aspect of the present invention, the target area identification image generation unit may include, for each pixel of the smoothed grayscale image, when the pixel value of the pixel is included in a range equal to or greater than the lower limit value. A value obtained by multiplying a difference between a pixel value of a pixel and the lower limit value by a predetermined coefficient β of one or more and adding a predetermined offset value is set as one of RGB color components, and for other color components , By setting a value obtained by multiplying the pixel value of the pixel in the grayscale image by a predetermined coefficient γ less than 1 to set the color of the target region, and when the pixel value of the pixel is less than the lower limit value, The pixel value of the pixel in the gray scale image is set as a color of the non-target region by setting all the RGB color components, and the target region identification image is generated.

  In the tenth aspect of the present invention, the target area identification image generation means includes a pixel value of the pixel in the range from the lower limit value to the upper limit value for each pixel of the smoothed grayscale image. In this case, a value obtained by multiplying the difference between the pixel value of the pixel and the lower limit value by a predetermined one or more coefficient β and adding a predetermined offset value is set as one of the RGB color components, and the other color components are set. On the other hand, by setting a value obtained by multiplying the pixel value of the pixel in the grayscale image by a predetermined coefficient γ less than 1 as the color of the target region, the pixel value of the pixel is less than the lower limit value or If the upper limit value is exceeded, the pixel value of the pixel in the grayscale image is set as the color of the non-target region by setting all the RGB color components, and the target region identification image is generated. It is characterized by that.

  Further, in the eleventh aspect of the present invention, the target area identification image correcting means is configured to perform RGB of the pixel for the target area identification image with respect to a pixel in which the target area is set in the designated area. The pixel is corrected to the value of the non-target region by correcting a value obtained by dividing the minimum value of any one of the color components by the coefficient γ so as to be all the RGB color components. To do.

  In the twelfth aspect of the present invention, the target area identification image correcting means displays the correction target area identification image on a screen to indicate a correction target area, and the target area identification image correction means displays the correction target area identification image on the target area in the specified area. The included pixels are further corrected to the value of the non-target region, and the correction target region identification image is updated.

  In the thirteenth aspect of the present invention, the target region identification image correcting unit divides a designated region into a plurality of pixel blocks having a predetermined number of pixels, and each pixel in each of the divided pixel blocks. These pixel values are input to machine learning means, the attribute of the non-target region is input to the machine learning means as teacher data, and learning processing is performed on the machine learning means.

  In the fourteenth aspect of the present invention, the target area identification image correcting unit divides the target area identification image or the correction target area identification image into a plurality of pixel blocks having the predetermined number of pixels, and The pixel value of each pixel in each pixel block is input to the machine learning means, and the attribute given to the pixel in the pixel block is corrected to the attribute output from the machine learning means. The correction target area identification image is created or updated.

  In the fifteenth aspect of the present invention, the machine learning means includes, in the input layer, any two RGB color components of each pixel of the pixel block, and two colors when the pixel is included in the target region. Corresponding to two types of pixel values selected so as to have different components, the node includes twice as many nodes as the predetermined number of pixels, and the output layer includes 2 attributes of a target area and a non-target area. It is characterized by comprising a multilayer neural network having two nodes corresponding to the attributes of the type and having at least one intermediate layer having a predetermined number of nodes.

  In the sixteenth aspect of the present invention, the gray scale image conversion means, the closing processing means, the extraction range definition means, the target area identification image generation means, and the target area identification image correction means are realized. A first computer and a second computer having the machine learning means are connected via a network.

According to a seventeenth aspect of the present invention, there is provided a method in which a computer creates an image for identifying a target region that is a diagnostic site for glaucoma from a color fundus image,
Converting the color fundus image into a grayscale image that is set so that the pixel of the target region that is a glaucoma diagnosis location is included in the range of high pixel values;
The grayscale image is subjected to a closing process based on a circular structuring element of a predetermined size to create a smoothed grayscale image;
Defining a lower limit value of a pixel value including at least all pixels included in the target region in the smoothed grayscale image;
For each pixel of the smoothed grayscale image, a target region identification image that can visually identify the target region and a non-target region other than the target region with respect to a pixel having a pixel value equal to or greater than the lower limit value. Generating
Based on the area designated for the target area identification image, the value of the pixel identified as the target area in the designated area is corrected to a value identified as a non-target area, and the correction target area Creating an identification image;
A method for creating an image is provided.

  According to an eighteenth aspect of the present invention, there is provided a program for causing a computer to function as the fundus image processing apparatus.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to extract a target area | region used as the glaucoma diagnosis location automatically, and to specify a target area | region with high precision by modifying a dialog format.

1 is a hardware configuration diagram of a fundus image processing apparatus 100 according to an embodiment of the present invention. 1 is a functional block diagram illustrating a configuration of a fundus image processing apparatus according to an embodiment of the present invention. It is a flowchart which shows the process outline | summary of the fundus image processing apparatus according to an embodiment of the present invention. It is the figure which showed the two-dimensional color distribution of each site | part in a full-color fundus image corresponding to R component and G component. It is the figure which showed the two-dimensional color distribution of each site | part in a full-color fundus image corresponding to RB component and GB component. It is a flowchart which shows the detail of a closing process. It is a figure which shows an example of the circular structural element used by a closing process. FIG. 8 is a diagram showing a full-color target area identification image Pseg (x, y, c) displayed on the screen of the display unit 7 by the processing of the target area identification image correcting means 50. It is a figure which shows the pixel value according to attribute of a synthetic | combination object area | region identification image. It is an example of the image obtained by the fundus image processing apparatus according to the present embodiment. It is a functional block diagram which shows the structure of the fundus image processing apparatus according to the second embodiment. It is a figure which shows the detail of the machine learning means 67. FIG. It is a flowchart which shows the process outline | summary of the fundus image processing apparatus according to the second embodiment. It is a figure for demonstrating the machine learning in 2nd Embodiment.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.
<1. Device configuration>
FIG. 1 is a hardware configuration diagram of a fundus image processing apparatus according to the first embodiment of the present invention. The fundus image processing apparatus 100 according to the present embodiment can be realized by a general-purpose computer. As shown in FIG. 1, a CPU (Central Processing Unit) 1 and a RAM (Random Access Memory) that is a main memory of the computer. 2, a large-capacity storage device 3 such as a hard disk or flash memory for storing programs and data executed by the CPU 1, an instruction input I / F (interface) 4 such as a keyboard and a mouse, and a data storage medium A data input / output I / F (interface) 5 for data communication with an external device, a GPU (Graphical Processing Unit) 6 for speeding up display and machine learning, and a display unit 7 which is a display device such as a liquid crystal display And are connected to each other via a bus.

  FIG. 2 is a functional block diagram illustrating a configuration of the fundus image processing apparatus according to the present embodiment. In FIG. 2, 10 is a gray scale image conversion means, 20 is a closing processing means, 30 is an extraction range definition means, 40 is a target area identification image generation means, 50 is a target area identification image correction means, 60 is an identification image synthesis means, 65 is an area ratio calculation means, 70 is a fundus image storage means, and 80 is a processing data image storage means.

  The gray scale image conversion means 10, the closing processing means 20, the extraction range definition means 30, the target area identification image generation means 40, the target area identification image correction means 50, the identification image synthesis means 60, and the area ratio calculation means 65 are processed by the CPU 1. This is realized by executing a program stored in the storage device 3, and the GPU 6 is also used in some processes. The fundus image storage unit 70 is a storage unit that stores a full-color fundus image to be extracted from a blood vessel region, which is captured in full color using a visible light / light source type fundus camera, and is realized by the storage device 3. When the full-color fundus image is read by the fundus image processing apparatus and processed as it is, the RAM 2 serves as the fundus image storage unit 70.

  A full-color fundus image is image data recorded with three RGB components, and is obtained by photographing the fundus of a subject. In the present embodiment, RGB recorded with 8-bit 256 gradations for each color is used. The processing data storage unit 80 includes an image generated or corrected by the target region identification image generation unit 40, the target region identification image correction unit 50, the identification image synthesis unit 60, the area ratio calculated by the area ratio calculation unit 65, and the like. And is realized by the storage device 3.

  Each component shown in FIG. 2 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices as shown in FIG. That is, the computer executes the contents of each means according to a dedicated program. In this specification, a computer means a device that has an arithmetic processing unit such as a CPU and a GPU and can perform data processing. The computer includes not only a general-purpose computer such as a personal computer but also a tablet equipped with a GPU. Includes portable terminals.

  The storage device 3 shown in FIG. 1 is mounted with a dedicated program for causing the computer to function as a fundus image processing apparatus while operating the CPU 1 and using the GPU 6 as an auxiliary. By executing this dedicated program, the CPU 1 performs gray scale image conversion means 10, closing processing means 20, extraction range definition means 30, target area identification image generation means 40, target area identification image correction means 50, identification image synthesis. Functions as the means 60 and the area ratio calculation means 65 are realized. The storage device 3 not only functions as the fundus image storage unit 70 and the processing data storage unit 80 but also stores various data necessary for processing as the fundus image processing device.

<2. Processing action>
<2.1. Pretreatment>
First, a full-color fundus image to be processed is prepared. As a full-color fundus image, if there is an image file taken in full color by a digital fundus camera, it can be used as it is. In addition, if it is an old one recorded on a photographic medium by an analog fundus camera, the stored digital color negative / positive film, photographic paper, instant photo, etc. are read in full color with a scanner. Obtain a full-color fundus image file. Generally, a full-color fundus image is obtained by photographing in full color using a visible light / light source type fundus camera. The acquired full-color fundus image is stored in the fundus image storage means 70 of the fundus image processing apparatus. In this embodiment, R, G, B component 8-bit color images with 256 gradations are prepared as full-color fundus images.

<2.2. Process Overview>
Next, processing operations of the fundus image processing apparatus shown in FIGS. 1 and 2 will be described together with an image creation method for creating a correction target region identification image. FIG. 3 is a flowchart showing an outline of processing of the fundus image processing apparatus according to the embodiment of the present invention. As described above, the full-color fundus image to be processed is 8-bit 256-gradation image data for each RGB color. Therefore, a full-color fundus image having the number of pixels Xs in the x direction and the number of pixels Ys in the y direction is represented by Image (x, y , C) = 0 to 255 (x = 0,..., Xs-1; y = 0,..., Ys-1; c = 0, 1, 2).

  First, the gray scale image conversion means 10 performs gray scale conversion on the full-color fundus image to obtain a gray scale image (step S100). The target areas to be diagnosed for glaucoma are mainly the Disc area that is the optic papilla, the Cup area that is the optic nerve depression, and the PPA area that is the choroidal atrophy around the nipple. Disorder) and nipple bleeding. Among these, in the present embodiment, three types of regions are extracted as target regions: a Disc region that is an optic nerve head, a Cup region that is a optic nerve recess, and a PPA region that is a peripapillary choroidal atrophy. These are all target regions that are glaucoma diagnosis locations, and are target regions in a range including the optic papilla. In the present embodiment, the grayscale image is created using the optimum color component according to the characteristics of the three types of target areas targeted for extraction.

  FIG. 4 is a diagram showing the two-dimensional color distribution of each part in the full-color fundus image in association with the R component and the G component. In FIG. 4, the Disc region of the optic disc, the Cup region of the optic disc, the PPA region around the optic disc, the arterial blood vessel (Artery), the venous blood vessel (Vein), which are representative regions other than these five regions. A rough distribution of the region (Background) is shown. As shown in FIG. 4, in the Disc region and the Cup region, the difference from the other regions is clarified by the R component, and the difference between the Disc region and the Cup region is also clarified by the G component. However, the PPA region is difficult to distinguish from venous blood vessels in the R component, and difficult to distinguish from the background in the G component. The PPA region can be extracted to some extent by negatively inverting the R component, but the accuracy is not good.

  FIG. 5 is a diagram showing the two-dimensional color distribution of each part in the full-color fundus image in association with the RB component and the GB component. In FIG. 5 as well, as in FIG. 4, the disc region of the optic disc, the Cup region of the optic disc, the PPA region around the optic disc, the arterial blood vessel (Artery), the venous blood vessel (Vein), and the like. A rough distribution of regions (Background) other than 5 regions is shown. The RB component is a component obtained by subtracting the B component from the R component, and the GB component is a component obtained by subtracting the B component from the G component. As shown in FIG. 5, the PPA region is covered with the Cup region by the RB component, but is generally distinguishable from other regions, and the GB component is included in the region other than the five regions (Background). I can't distinguish.

  Using the characteristics of the target region as shown in FIGS. 4 and 5, the grayscale image conversion means 10 applies the following [Equation 1] to [Equation 1] to [Fully fundus image Image (x, y, c)]. Grayscale conversion is performed by executing processing according to Equation 3] to generate three types of grayscale images.

[Formula 1]
GrayD (x, y) = Image (x, y, 0)

  In the above [Equation 1], Image (x, y, 0) represents an R (red, red) component of the full-color fundus image. The grayscale image GrayD (x, y) is a gray that is set so that the pixel of the disc region of the optic disc is included in the range of high pixel values as the target region of the optic disc or the peripheral region of the optic disc. It is a scale image.

[Formula 2]
GrayC (x, y) = Image (x, y, 1)

  In the above [Equation 2], Image (x, y, 1) represents a G (green, green) component of the full-color fundus image. The grayscale image GrayC (x, y) is a gray that is set so that the pixel of the Cup region of the optic disc is included in the high pixel value range as the target region of the optic disc or the peripheral region of the optic disc. It is a scale image.

[Formula 3]
GrayP (x, y) = [255−Image (x, y, 0) + Image (x, y, 2)] × α
When GrayP (x, y)> 255, GrayP (x, y) = 0

  In the above [Equation 3], Image (x, y, 2) represents a component of B (blue, blue) in the full-color fundus image. Α is a real number of 1 or more, and preferably a real number of 10 or less. The grayscale image GrayP (x, y) is a gray that is set so that a pixel that is a PPA region is included in a relatively high pixel value range as a target region of the optic papilla or a target region around the optic papilla. It is a scale image. For the extraction of the PPA area, the pixel value is improved by negative inversion of the R component (Image (x, y, 0)) minus the B component (Image (x, y, 2)). Adjustment is performed so that the pixels are included in the range of high pixel values and the background pixels of the fundus region are included in the non-target region. Negative inversion means that the pixel value is subtracted from the maximum value (255). Therefore, in the case of the PPA region, the grayscale image GrayP (x, y) is obtained by subtracting a predetermined real value α of one or more from the value obtained by subtracting the B component from the R component from the maximum value (255). The image is multiplied. Furthermore, since most of the pixels with GrayP (x, y)> 255 are included outside the fundus region, GrayP (x, y) = 0 is set so that the outside of the fundus region is included in the non-target region. It has been fixed. Hereinafter, for GrayD (x, y), GrayC (x, y), and GrayP (x, y), the same processing is performed according to the corresponding target region, but processing performed in common is performed for these. Is shown as Gray (x, y).

  When extracting a target region from a gray scale image, a blood vessel portion that intersects the target region cannot be extracted, which hinders the area calculation of the target region described later. Therefore, the gray scale fundus image obtained by the gray scale image conversion means 10 is subjected to a closing process using a circular structuring element (step S200). Generally, the closing process means a process in which each of the expansion process and the contraction process is repeated the same number of times. The expansion process is a process for increasing the pixel value of the target pixel by replacing the pixel value of the target pixel with the maximum pixel value around the target pixel, and the contraction process is a process for increasing the pixel value of the target pixel. This is a process of increasing the number of pixels having a low pixel value by replacing it with the smallest peripheral pixel value.

  FIG. 6 is a flowchart showing details of the closing process by the circular structural element. First, the closing processing means 20 performs an expansion process using the designated structural element (step S210). In the expansion process in step S210, a circular structural element is used as the designated structural element. The circular structuring element is a (2R + 1) × (2R + 1) mask image in which a distance R pixels or less from the center is effective. In the present embodiment, the gray scale fundus images GrayD (x, y), GrayC (x, y), and GrayP (x, y) are all 1024 × 1024 pixels, and thus any case of Disc, Cup, and PPA extraction. In principle, R = 7. FIG. 7 is a diagram illustrating an example of a circular structural element in the case of R = 7. As shown in FIG. 7, when R = 7 is set, a circular structural element in the form of a 15 × 15 mask image in which a radius of 7 pixels, that is, a distance of 7 pixels or less from the center is effective is used. As shown in FIG. 7, among the 15 × 15 pixels, the pixel value at a distance of 7 pixels or less from the center is “1”, and the pixel value of a pixel exceeding the distance 7 from the other center is “0”. The circular structural element shown in FIG. 7 is M (u, v) = {0, 1} (u = −R,... 0,... R; v = −R,. R). Note that R indicates a valid radius, and R = 7 in the example of FIG. The radius R can be set as appropriate. However, when the grayscale fundus image Gray (x, y) is 1024 pixels × 1024 pixels, it is preferably about 7 ≦ R ≦ 10. As described above, in this embodiment, R = 7. u represents the horizontal axis in FIG. 7, and v represents the vertical axis in FIG. The closing processing means 20 uses the circular structuring element M (u, v) to perform the processing according to the above [Equation 1] to [Equation 3] to obtain a grayscale fundus image Gray (x, y). On the other hand, processing according to the following [Equation 4] is executed to obtain an expanded image Dray (x, y).

[Formula 4]
Dray (x, y) = MAX _{u = −R, R; v = −R, R} [M (u, v) × Gray (x + u, y + v)]

  In the above [Equation 4], MAX indicates the maximum value, and the suffix “u = −R, R; v = −R, R” of MAX indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Gray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the maximum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 7, the calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the circular structure element, and the maximum value is given as Dray (x, y). As the gray scale fundus image Gray (x, y) in [Equation 4], three types of gray scale fundus images Gray D (x, y), Gray C (x, y), and Gray P (x, y) are used, respectively. Three types of expanded images Dray (x, y), Dray C (x, y), and Dray P (x, y) corresponding to Cup and PPA are obtained.

  When the expansion image Dray (x, y) is obtained by executing the processing according to the above [Equation 4], this Dray (x, y) is replaced with Gray (x, y), and the above [Equation 4] is obtained. It is also possible to repeatedly execute the process. That is, the expansion process in step S210 may be repeatedly executed. The number of repetitions can be set as appropriate. However, in order to suppress image quality deterioration, it is normally set to once and not repeated.

  After the expansion processing is repeatedly executed and the expansion image Dray (x, y) is obtained, the closing processing means 20 performs the contraction processing using the designated structural element (step S220). In the contraction process in step S220, the circular structural element used for the expansion process is used as the designated structural element. That is, the circular structural element shown in FIG. 7 is used. The closing processing means 20 uses the circular structuring element M (u, v) to execute the processing according to the following [Equation 5] on the expanded image Dray (x, y), and the image Fray after contraction (X, y) is obtained.

[Formula 5]
Fray (x, y) = MIN _{u = −R, R; v = −R, R} [(255−M (u, v) × 254) × Dray (x + u, y + v)]

  In the above [Equation 5], MIN indicates that it takes the minimum value, and the subscript “u = −R, R; v = −R, R” of MIN indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Ray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the minimum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 7, the calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the circular structure element, and the minimum value is given as Ray (x, y). As the gray scale fundus image Dray (x, y) in [Equation 5], three types of dilated fundus images DrayD (x, y), Dray C (x, y), and Dray P (x, y) are used, respectively, and Disc, Cup , Three kinds of contracted images Fray (x, y) corresponding to the respective PPA are obtained.

  When the processing according to the above [Equation 5] is executed to obtain a contracted image Fray (x, y), this Fray (x, y) is replaced with Dray (x, y), and the above [Equation 5 ] Can be repeatedly executed. That is, the contraction process in step S220 may be repeatedly executed. The number of repetitions needs to be set to the same number as the number of repetitions of the expansion process in step S210. Normally, both expansion processes are set to one and no repetition is performed. The image Fray (x, y) after contraction is a smoothed gray scale image. As a result of the closing process, in the smoothed grayscale image Fray (x, y), the lack of the target region due to the blood vessel crossing portion is interpolated. Therefore, when a blood vessel crosses the target region in the full-color fundus image, only the target region can be clearly visually recognized by removing the blood vessel.

  If the smoothed grayscale image Fray (x, y) is obtained, then the extraction range defining means 30 defines the extraction range (step S300). Specifically, the extraction range is defined by obtaining an upper limit value and a lower limit value of the extraction range. The lower limit is essential, but the upper limit is not essential. First, a grayscale smoothed grayscale image Fray (x, y) (x = 0,..., Xs-1; y = 0,..., Ys-1) having a value of 0 to 2L-1. On the other hand, the density histogram Hist (va) (va = 0,..., 2L−1) is calculated for (x, y) that satisfies Fray (x, y)> 0. Typically, the smoothed grayscale image Fray (x, y) has 8 bits 256 gray levels for each pixel, and L = 128. Then, assuming that Tl is within the range of 0 ≦ Tl ≦ Vmax (Vmax = 254 (= 2 (L−1)) for Disc or Cup, Vmax = 128 (= L) for PPA), the histogram is The boundary (threshold value) to be divided into two is s (s = 1,..., Vmax), the process according to the following [Equation 6] is executed, and the discriminant value D (s) is calculated.

[Formula 6]
Sum1 (s) = Σ _{va = 1, s-1} Hist (va) va
Sum2 (s) = Σ _{va = s, Vmax} Hist (va) va
N1 (s) = Σ _{va = 1, s-1} Hist (va)
N2 (s) = Σ _{va = s, Vmax} Hist (va)
W (s) = (Sum1 (s) / N1 (s) −Sum2 (s) / N2 (s)) × (N1 (s) + N2 (s)) / (Sum1 (s) + Sum2 (s))
D (s) = N1 (s) N2 (s) W (s) ² (N1 (s) + N2 (s)) ²

  In the first formula of the above [Formula 6], the subscript “va = 1, s−1” of Σ indicates that the sum of va from 1 to s−1 is obtained. Similarly, in the second to fourth formulas, the sum is obtained within the range of subscripts. In the above [Expression 6], N1 (s) is the number of pixels on the lower gradation side than the gradation value (pixel value) s, and N2 (s) is a pixel on the higher gradation side than the gradation value (pixel value) s. It is a number. For example, when L = 256, the discriminant analysis process is executed using the range of va = 1 to 254. D (s) is calculated according to the discriminant that is the last equation of [Formula 6], and s that maximizes the discriminant value D (s) is selected as the lower limit threshold value Tl of the extraction range. The process of specifying the threshold value Tl in step S300 is substantially the same as a known discriminant analysis method.

  When extracting PPA, an upper limit threshold Th of the extraction range is also selected. Note that Th = 2L-1 is always set when the Disc or Cup is extracted. Assuming that Th is in the range of Tl <Th <2L-1, the boundary (threshold value) for dividing the histogram into two is s (s = Tl + 1,..., 2 (L-1)), and the following [ Processing according to Equation 7] is executed to calculate the discriminant value D (s).

[Formula 7]
Sum3 (s) = Σvb _{= 1, s-1} Hist (vb) vb
Sum4 (s) = Σvb _{= s, 2 (L-1)} Hist (vb) vb
N3 (s) = Σ _{vb = 1, s-1} Hist (vb)
N4 (s) = _{Σvb = s, 2 (L-1)} Hist (vb)
W2 (s) = (Sum3 (s) / N3 (s) −Sum4 (s) / N4 (s)) × (N3 (s) + N4 (s)) / (Sum3 (s) + Sum4 (s))
D3 (s) = N3 (s) N4 (s) W2 (s) ² (N3 (s) + N4 (s)) ²

  In the first equation of [Expression 7], the subscript “vb = 1, s−1” of Σ indicates that the sum of vb from 1 to s−1 is obtained. Similarly, in the second to fourth formulas, the sum is obtained within the range of subscripts. In the above [Expression 7], N3 (s) is the number of pixels on the lower gradation side than the gradation value (pixel value) s, and N4 (s) is a pixel on the higher gradation side than the gradation value (pixel value) s. It is a number. For example, when L = 128, the discriminant analysis process is executed using the range of vb = 1 to 254.

  D2 (s) is calculated according to the discriminant that is the last equation of the above [Formula 7], and s that maximizes the discriminant value D2 (s) is selected as the upper limit threshold Th of the extraction range. The process of specifying the threshold value Th in step S300 is substantially the same as a known discriminant analysis method. As a result, the extraction range is defined as the lower limit value Tl to the upper limit value Th.

  When the threshold values Tl and Th are obtained and the extraction range is defined, the target area identification image generation unit 40 generates a target area identification image. For this purpose, first, the target area identification image generating means 40 performs contrast correction (step S400). Specifically, the range from the threshold value Tl to Th is extracted as the target region, and the range from Tl (0 ≦ Tl <2L−1) to Th (Tl <Th ≦ 2L−1) is expanded to L gradation. Process. As described above, different values for Tl and Th are set for each extraction target area (Disc, Cup, PPA). Also, an adjustment parameter of Tm (Tl <Tm ≦ 2L-1) is added. Usually, Tm = Th, but when the image after contrast correction is dark, the value of Tm is decreased and the image after contrast correction becomes brighter. If saturated, increase the value of Tm. Then, by executing the processing according to the following [Equation 8], when L = 128, a gradation corrected image Cray (x, y) after contrast correction having positive and negative pixel values of −128 to 127 is obtained. calculate.

[Formula 8]
If Fray (x, y) = Th is not
Cray (x, y) = [Fray (x, y) −Tl] [Th-Fray (x, y)] × (L−1) / [(Tm−Tl) | Th-Fray (x, y) | ]
If Fray (x, y) = Th,
Cray (x, y) = [Fray (x, y) −Tl] × (L−1) / (Tm−Tl)
However, when Cray (x, y)> L−1, Cray (x, y) = L−1, and when Cray (x, y) <− L, Cray (x, y) = − L. .

  Next, the target area identification image generation means 40 performs pseudo color coloring to generate a target area identification image (step S500). The pseudo color means a pseudo color that characterizes the target area, although it is different from the actual color of the target area. By coloring the target area with the pseudo color, the target area and the non-target area can be visually identified in the target area identification image. Specifically, each pixel value Pseg (x, y, c) of the target area identification image of RGB 256 (= 2L) gradation is calculated by executing processing according to the following [Equation 9]. Prior to the processing of [Formula 9], Cr = 0, Cg = 2L-1, and Cb = 0 are set. This setting is the same for the other [mathematical formulas] described below. By setting Cr = 0, Cg = 2L-1, and Cb = 0, the target region can be characterized by the G component. That is, in this embodiment, coloring is performed with the G component as a pseudo color.

[Formula 9]
When Cray (x, y) ≧ 0 (target area)
When Cr> 0, Pseg (x, y, 0) = Cray (x, y) × β + L
When Cr = 0, Pseg (x, y, 0) = Gray (x, y) × γ
When Cg> 0, Pseg (x, y, 1) = Cray (x, y) × β + L
When Cg = 0, Pseg (x, y, 1) = Gray (x, y) × γ
When Cb> 0, Pseg (x, y, 2) = Cray (x, y) × β + L
When Cb = 0, Pseg (x, y, 2) = Gray (x, y) × γ
When Cray (x, y) <0 (non-target area)
Pseg (x, y, 0) = Pseg (x, y, 1) = Pseg (x, y, 2) = Gray (x, y)

  In [Equation 9], L is an offset value, and in the case of 256 gradations, L = 128 and Cg = 255. In [Formula 9], β is a coefficient having a value of 1 or more, and in this embodiment, β = 1.0. In [Equation 9], γ is a coefficient having a value less than 1, and in this embodiment, γ = ½. The target area identification image Pseg (x, y, c) gives the attribute of the target area to pixels having a pixel value equal to or higher than the lower limit value Tl in the smoothed grayscale image Fray (x, y), and other pixels. Is a target region identification image in which the attribute of the non-target region is given and the two kinds of assigned attributes can be visually identified. Here, a pixel having a pixel value greater than or equal to the lower limit value Tl in the smoothed grayscale image Ray (x, y) is the same as a pixel having a value greater than or equal to 0 in the gradation correction image Cray (x, y). Cray (x, y) is the pixel value of a pixel when the pixel value of the pixel is included in the range from the lower limit value Tl to the upper limit value Th for each pixel of the smoothed grayscale image Ray (x, y). And the lower limit value Tl. By multiplying this Cray (x, y) by one or more coefficient β and adding a predetermined offset value L, the color of the target region is set as one of the RGB color components, so that the target region in the target region identification image Is obtained. In addition, the attribute assignment does not mean that a special value indicating the attribute is assigned to the pixel, but indicates that the pixel value of a predetermined color component satisfies a predetermined condition. For example, in the example of [Equation 9], the G component of the target area identification image Pseg (x, y, c) is multiplied by β (= 1.0) to the value of the gradation corrected image Cray (x, y), Since it is a value added by L, it always becomes a value greater than or equal to L, whereas the R and B components of Pseg (x, y, c) are γ () in the value of the grayscale image Gray (x, y). Since the value is always less than L, the attribute of the target area is given when the value of the G component is larger than the values of the R and B components. become. On the other hand, when the values of all the R, G, and B components of the target area identification image Pseg (x, y, c) are the same, the attribute of the non-target area is given.

  In [Expression 9], not only Cg but also a large value can be set for Cr or Cb, so that the target region can be characterized by an R component or a B component. As Pseg (x, y, c), Pseg D (x, y, c) corresponding to the Disc region, Pseg C (x, y, c) corresponding to the Cup region, Pseg P (x, y, c) corresponding to the PPA region There are three types of c). Hereinafter, when simply indicated as Pseg (x, y, c), Pseg D (x, y, c), Pseg C (x, y, c), and Pseg P (x, y, c) are representative. To do.

  Once the target area identification image Pseg (x, y, c) is obtained, the target area identification image correcting means 50 then performs an overextraction area (originally a non-target area but a pixel set as the target area) The target area identification image is corrected by deleting the area (step S600). Specifically, first, the target area identification image Pseg (x, y, c) is displayed on the display unit 7. Then, the target area identification image correcting means 50 prompts the designated area to be superimposed on the displayed target area identification image Pseg (x, y, c). The shape of the designated region can be any shape, and can be a circle, an ellipse, or a polygon, but in the present embodiment, it is a rectangle. The rectangular designation area can be changed in position, inclination, and size by inputting an instruction from the mouse or touch panel via the instruction input I / F 4.

  The user should correct in the target area identification image Pseg (x, y, c), that is, an arbitrary position, inclination, and size so as to include a portion that seems to be excessively extracted as the target area. Specify the designated area with. At this time, a non-target area may be included in the designated area. Then, the target area identification image correcting means 50 displays an outer frame for specifying the designated area on the target area identification image Pseg (x, y, c). In the present embodiment, a rectangular frame is used as the outer frame for specifying the designated area. FIG. 8 is a diagram showing a full-color target area identification image Pseg (x, y, c) displayed on the screen of the display unit 7 by the processing of the target area identification image correcting means 50. FIG. 8A shows before correction, and FIG. 8B shows after correction. Then, the user looks at the blood vessel in the designated area, and gives an instruction to correct the attribute of the non-target area from the mouse, the touch panel, or the like via the instruction input I / F 4. When a correction instruction is issued, each correction target pixel in the specified area is processed by the target area identification image correction means 50 according to the following [Formula 10] and [Formula 11]. Will be corrected. First, the target area identification image correcting unit 50 calculates the background pixel value Vg by executing the process according to [Formula 10].

[Formula 10]
When Pseg (x, y, 1)> Pseg (x, y, 0), Vg = Pseg (x, y, 0) / γ
Other than the above (Pseg (x, y, 0) = Pseg (x, y, 1) = Pseg (x, y, 2)), Vg = Pseg (x, y, 0)

  Using the background pixel value Vg calculated by the above [Equation 10], the target area identification image Pseg (x, y, c) is corrected by executing processing according to the following [Equation 11], A target area identification image Qseg (x, y, c) is obtained.

[Formula 11]
Qseg (x, y, 0) = Qseg (x, y, 1) = Qseg (x, y, 2) = Vg

  By executing the processing of [Equation 10] and [Equation 11], the target area identification image correcting means 50 applies to the target area identification image with respect to the pixels in which the target area is set in the designated area. The pixel is corrected to the value of the non-target region by correcting the value obtained by dividing the minimum value of any one of the RGB color components of the pixel by the coefficient γ so that all the RGB color components are obtained. An identification image is created. By executing the processing according to [Equation 10] and [Equation 11], the correction target region identification image Qseg (x, y, c) has QsegD (x, y, c), Cup region corresponding to the Disc region. Three types of correction target area identification images QsegC (x, y, c) corresponding to, and QsegP (x, y, c) corresponding to the PPA area are obtained. In each equation, when “Qseg (x, y, c)” is simply indicated, QsegD (x, y, c), QsegC (x, y, c), and QsegP (x, y, c) are used. The process performed in common is shown. Qseg (x, y, c) has the same format as Pseg (x, y, c), and each RGB has 8 bits and 256 gradations.

  By executing the processing according to [Equation 10] and [Equation 11], the target area identification image as shown in FIG. 8A is corrected as shown in FIG. 8B. As is clear from a comparison between FIG. 8A and FIG. 8B, the target region that is excessively extracted in the generated target region identification image is clearly deleted in the correction target region identification image.

  Next, the identification image synthesizing unit 60 synthesizes three types of correction target area identification images after correction (step S700). Specifically, the synthesis is performed in the order of PPA, Disc, and Cup, the region where PPA and Disc overlap is replaced with Disc, and only the region where Cup overlaps with Disc is overlapped with Disc, and the region where PPA and Cup overlap is PPA. And the Cup single region is deleted. More specifically, first, the correction target region identification image QsegD (x, y, c) corresponding to the Disc region (x = 0,..., Xs-1; y = 0,..., Ys-1). On the other hand, processing according to the following [Equation 12] is executed to calculate the reference value Vd and the background pixel value Vg. In the following [Equation 12], it is assumed that Cr = 0, Cg = 2L-1, and Cb = 0 are set and the target region is characterized by the G component. When characterizing with other color components, the setting is different.

[Formula 12]
When QsegD (x, y, 1)> QsegD (x, y, 0), Vd = QsegD (x, y, 1), Vg = 2 QsegD (x, y, 0)
Other than the above (QsegD (x, y, 0) = QsegD (x, y, 1) = QsegD (x, y, 2)), Vd = 0, Vg = QsegD (x, y, 0)

  Next, for the correction target region identification image QsegC (x, y, c) (x = 0,..., Xs-1; y = 0,..., Ys-1) corresponding to the Cup region, The process according to the following [Formula 13] is executed to calculate the reference value Vc.

[Formula 13]
When QsegC (x, y, 1)> QsegC (x, y, 0), Vc = QsegC (x, y, 1)
Other than the above (QsegC (x, y, 0) = QsegC (x, y, 1) = QsegC (x, y, 2)), Vc = 0

  Further, for the correction target area identification image QsegP (x, y, c) (x = 0,..., Xs−1; y = 0,..., Ys−1) corresponding to the PPA area, The process according to [Formula 14] is executed to calculate the reference value Vp.

[Formula 14]
When QsegP (x, y, 1)> QsegP (x, y, 0), Vp = QsegP (x, y, 1)
Other than the above (QsegP (x, y, 0) = QsegP (x, y, 1) = QsegP (x, y, 2)), Vp = 0

  Disc area is Green (Vg / 2, Vd, Vg / 2), Cup area is Yellow (Vc, Vd, Vg / 2), PPA area is Blue (Vg / 2, Vg / 2, Vp), and background area is Gray. When the colors are classified by (Vg, Vg, Vg), if the reference value Vd> 0 (the value of Vp is not limited), the processing according to the following [Equation 15] is executed, and the composition target region colored in pseudo color The identification image Rseg (x, y, c) (x = 0,..., Xs−1; y = 0,..., Ys−1) is calculated.

[Formula 15]
When Vc> 0, Rseg (x, y, 0) = Vc, Rseg (x, y, 1) = Vd, Rseg (x, y, 2) = Vg / 2
When Vc = 0, Rseg (x, y, 0) = Vg / 2, Rseg (x, y, 1) = Vd, Rseg (x, y, 2) = Vg / 2

  When the reference value Vd = 0 (the value of Vc does not matter), processing according to the following [Equation 16] is executed, and the composition target region identification image Rseg (x, y, c) colored with pseudo color is executed. ) (X = 0,..., Xs-1; y = 0,..., Ys-1).

[Formula 16]
When Vp> 0, Rseg (x, y, 0) = Vg / 2, Rseg (x, y, 1) = Vg / 2, Rseg (x, y, 2) = Vp
When Vp = 0, Rseg (x, y, 0) = Rseg (x, y, 1) = Rseg (x, y, 2) = Vg

  As a result of executing the processing according to [Equation 15] and [Equation 16], in the synthesis target region identification image Rseg (x, y, c) obtained, the Disc region, Cup region, PPA region, and other four attributes are as follows: Are set as values shown in FIG. Based on the calculated reference values Vd, Vc, Vp, and background pixel value Vg, the Disc area is displayed in Green, the Cup area is Yellow, the PPA area is Blue, and other areas are displayed in gray (pseudo color). The That is, in the synthesis target region identification image Rseg (x, y, c), each target region is expressed by a pseudo color that characterizes each target region, although it is different from the actual color. When changing the attribute, the RGB components are exchanged, and the values of Vd, Vc, Vp, and Vg are retained without being changed even after the attribute correction. Rseg (x, y, c) has the same format as Pseg (x, y, c) and Qseg (x, y, c), and each RGB has 8 bits and 256 gradations.

  Eventually, by executing the processing of [Equation 12] to [Equation 15], the identification image synthesizing means 60 is preliminarily determined so as to be colored in different colors for a plurality of types and consistent with the anatomy / histology of the eyeball. Based on the replacement priority order of the plurality of types of corresponding areas, the pixels corresponding to the pixels of the target area of the different correction target area identification image are different from each other. By substituting with the color of the pixel of the target area of the correction target area identification image, a composite target area identification image is created by combining a plurality of types of correction target area identification images. More specifically, the identification image synthesizing unit 60 colors the pixels having the attributes of each target region with three types of colors with respect to the three types of correction target region identification images, thereby obtaining the first correction target region identification image. The pixel corresponding to the pixel of the target region of the second correction target region identification image is replaced with the color of the pixel of the target region of the second correction target region identification image, and the replaced first correction target region identification The pixel corresponding to the pixel of the target area of the third correction target area identification image is replaced with the color of the pixel of the target area of the third correction target area identification image with respect to the image, and the first correction target area identification image By adding a modification to the image using the second correction target region identification image and the third correction target region identification image, a composite target region identification image is created by combining the three types of correction target region identification images. .

  In the present embodiment, the first correction target area identification image is QsegD (x, y, c) corresponding to the Disc area, and the second correction target area identification image is QsegC (x, y, c) corresponding to the Cup area. c), and the third correction target area identification image is QsegP (x, y, c) corresponding to the PPA area, and the identification image synthesis means 60 performs the second correction on the first correction target area identification image. When the pixel corresponding to the pixel of the target area of the second correction target area identification image is replaced with the color of the pixel of the target area of the second correction target area identification image, the corresponding pixel of the first correction target area identification image is not In the case of a pixel in the target area, the pixel corresponding to the pixel in the target area of the third correction target area identification image is replaced with the third correction target area with respect to the replaced first correction target area identification image. Of the target area of the identification image In case of replacing the corresponding pixel of the first correction target region identification image that is substituted in the case of pixels in the target region so that no substitution.

  As in the present embodiment, the first correction target area identification image corresponds to the Disc area, the second correction target area identification image corresponds to the Cup area, and the third correction target area identification image corresponds to the PPA area, as described above. When the process is performed, the Cup area is displayed only in the Disc area, and the basis of the priority at the time of synthesis that does not result in a histologically contradictory form such as the presence of the PPA area in the Disc area is the anatomy of the eyeball. Based on academic and histological findings. The Cup region is the same optic nerve tissue as the Disc region and is a partially depressed region, and therefore exists in the Disc region. Therefore, the Cup region overlaps with the Disc region and has a higher priority than the Disc region. The PPA region is a region that is partially atrophied in a retinal tissue different from that of the Disc region (a state in which cells have died and have lost the light receiving function), and is located outside the Disc region. Accordingly, the PPA area and the Disc area do not overlap. If they overlap, it is determined that the PPA area is erroneously detected, and the Disc area is given priority. Since the PPA area has a significantly lower contrast than the Disc area, the Disc area is given priority because it may be erroneously detected. From the above, the three priorities are PPA <Disc <Cup, and based on this, a composition target area identification image is created.

  After the compositing target area identification image is created, the area ratio calculating unit 65 then counts the number of pixels colored with three kinds of colors for the compositing target area identification image for each color to obtain the second color. A first ratio obtained by dividing the number of corresponding pixels by the number of pixels corresponding to the first color, and a second ratio obtained by dividing the number of pixels corresponding to the third color by the number of pixels corresponding to the first color. Calculate (step S800). Since the first ratio and the second ratio are the ratio of the number of pixels, they correspond to the area ratio on the image. Here, the first color, the second color, and the third color respectively correspond to the three types of colors of the target regions. In step S800, processing according to the following [Equation 17] is executed to calculate a Cup-to-Disc area ratio that is a first ratio and a PPA-to-Disc area ratio that is a second ratio. .

[Formula 17]
Cup-to-Disc area ratio =
{ _{ΣRseg (x, y, 0)> (L-1)} (Rseg (x, y, 0) −L) / (L−1)} / { _{ΣRseg (x, y, 1)> (L− 1)} (Rseg (x, y, 1) -L) / (L-1)}
PPA-to-Disc area ratio =
{ _{ΣRseg (x, y, 2)> (L-1)} (Rseg (x, y, 2) −L) / (L−1)} / { _{ΣRseg (x, y, 1)> (L− 1)} (Rseg (x, y, 1) -L) / (L-1)}

  FIG. 10 is an example of an image obtained by the fundus image processing apparatus according to the present embodiment. Among these, FIG. 10A shows a full-color fundus image obtained by imaging. FIGS. 10B, 10C, and 10D show the target area identification with the Disc area as the target area after the target area identification image correcting unit 50 performs the deletion process of the excessive extraction area in step S600. An image, a target area identification image having a Cup area as a target area, and a target area identification image having a PPA area as a target area are shown. FIG. 10E shows a synthesis target area identification image after the pseudo color image synthesis process of step S700 is performed by the identification image synthesis means 60.

  As shown in FIG. 10B, in the target area identification image in which the Disc area is the target area, the Disc area that is the target area and the non-target area that are two types of attributes can be visually distinguished from each other. It is displayed. As shown in FIG. 10 (c), in the target area identification image with the Cup area as the target area, the two types of attributes, that is, the Cup area that is the target area and the non-target area, can be visually distinguished from each other. Is displayed. As shown in FIG. 10D, in the target area identification image with the PPA area as the target area, two types of attributes, the PPA area that is the target area and the non-target area, can be visually identified from each other. Is displayed. As shown in FIG. 10E, in the synthesis target area identification image, the three target areas, the Disc area, the Cup area, the PPA area, and the non-target area are displayed in colors that are visually identifiable. Yes.

  As a result of the area ratio calculation means 65 performing the degree of progress calculation process of step S800 on the composition target region identification image shown in FIG. 10E, the Cup-to-Disc area ratio is 0.43 ( The PPA-to-Disc area ratio was 0.15 (denoted as P / D = 15 in the figure).

<3. Second Embodiment>
Next, a second embodiment will be described. FIG. 11 is a functional block diagram illustrating a configuration of a fundus image processing apparatus according to the second embodiment. In FIG. 11, the same means as those in FIG. In FIG. 11, 67 is a machine learning means.

  The machine learning means 67 is realized by causing the CPU 1 to load a part of the program stored in the storage device 3 into the GPU 6 and causing the GPU 6 to mainly execute a series of learning processing and determination processing. The machine learning means 67 has a function of securing predetermined areas of the storage device 3 and the GPU 6 and storing predetermined information.

  FIG. 12 is a diagram showing an example in which the machine learning means 67 is realized by a known multilayer neural network (four layers in FIG. 12). The multilayer neural network of the machine learning means 67 includes an input layer 67a, two intermediate layers 67b and 67c, and an output layer 67d. The input layer 67a includes as many nodes Na corresponding to each pixel of the pixel block as twice the number of pixels of the pixel block. This is because it consists of two color components of RGB of each pixel of the pixel block, and when the pixel is included in the target area, it corresponds to two types of pixel values selected so that the two color components are different. is there. Of the three color components of RGB, the remaining one color component has the same value as one of the two selected color components, so input to the multilayer neural network is not performed (input the three color components as they are). This increases the number of nodes in the input layer, increasing the processing load and reducing the accuracy of machine learning). When the size of the pixel block is 8 × 8 pixels, 128 nodes Na are provided. Although at least one intermediate layer is required, it is usually composed of a plurality of layers. In FIG. 12, it is composed of two layers 67b and 67c. Each intermediate layer 67b and 67c includes a predetermined number of nodes Nb and Nc. The output layer 67d includes two nodes Nd corresponding to two types of attributes including a target area and a non-target area. The number of nodes Na in the input layer 67a and the number of nodes Nd in the output layer 67d are inevitably determined based on the input / output specifications as described above, but the number of intermediate layers and the number of nodes in each intermediate layer are determined. The number has no fundamental basis for determination, and is adjusted by trial and error while evaluating the accuracy of machine learning. In general, each of the intermediate layer 67b and the intermediate layer 67c is provided with a larger number of nodes Nb and Nc than the number of nodes Nd in the output layer 67d and smaller than the number of nodes Na in the input layer 67a. The number of Nb and node Nc is often set to be different.

  In the example of FIG. 12, the number of nodes Nb is set to 32, and the number of nodes Nc is set to 4. The number of nodes Na in the input layer 67a is Ni (= 128), the number of nodes Nb in the first intermediate layer 67b is Nj (= 32), and the number of nodes Nc in the second intermediate layer 67c is Nk (= 4). The number of nodes Nd in the output layer 67d is Nl (= 2). Between each node Na of the input layer 67a and each node Nb of the first intermediate layer 67b, Ni × Nj links Lab are extended, and each node Nb of the first intermediate layer 67b is connected to the second intermediate layer 67b. Nj × Nk links Lbc are extended between the nodes Nc of the layer 67c, and Nk × Nl links between the nodes Nc of the second intermediate layer 67c and the nodes Nd of the output layer 67d. A link Lbc is stretched. The link Lab between the i-th (i = 1,..., Ni) node Na (i) and the j-th (j = 1,..., Nj) node Nb (j) And the link Lbc between the j-th node Nb (j) and the k-th (k = 1,..., Nk) node Nc (k) is Lbc (j, k), and the k-th node Nc and l Assuming that the link Lcd with the node Nd (l) of the th (l = 1,..., Nl) is Lcd (k, l), Lab (i, j), Lbc (j, k), Lcd (k, In 1), real-valued weights Wab (i, j), Wbc (j, k), and Wcd (k, l) each having an independent positive / negative sign are set, and the GPU 6 is used by a learning process described later. All weights are automatically calculated and set (in the current mainstream backpropagation algorithm, the weights are 0-1 And it is limited to the real value of). These weights Wab (i, j), Wbc (j, k), Wcd (k, l) are recorded in predetermined areas of the storage device 3 and the GPU 6. In the above description, the link Lab (i, j) and the weight Wab (i, j) are distinguished from each other. However, in the neural network theory, the weight Wab (i) is actually used for machine learning and machine judgment calculation. , J), the expression “link” is not used and the expression “weight between nodes” is often used.

  In the second embodiment, machine learning means 67 is provided, and machine learning is performed based on the information corrected by the target region identification image correction means 50, so that the correction instruction is not necessarily required, and the optic nerve head or This is different from the first embodiment in that peripheral attributes can be set. FIG. 13 is a flowchart illustrating a processing outline of the fundus image processing apparatus according to the second embodiment. From step S100 to step S500, processing is performed in the same manner as in the first embodiment, and a target area identification image Pseg (x, y, c) is generated.

  When the target area identification image Pseg (x, y, c) is generated, it is determined whether or not to use machine determination (step S650). As for whether to use machine determination, various modes such as a mode according to a preset setting and a mode based on the number of executions of machine learning can be used. Even when the machine determination is used, correction can be made to the automatically corrected target area identification image.

  If it is determined in step S650 that the machine determination is not used, the target region identification image is corrected by deleting the excessively extracted region based on an instruction from the user as in the first embodiment (step S600). . When the target area identification image is corrected, the correction information is stored (step S900). Specifically, when the target area identification image is corrected in step S600, a pixel block of a predetermined size is set from the area specified by the user, and pixel block information that is information on the pixels included in each pixel block And the corrected attribute for each pixel block are stored in association with each other and used for machine learning. Actually, since the corrected attribute is limited to the non-target region, only the position information of the pixel block may be stored. FIG. 14A is a diagram illustrating a state of machine learning accompanying correction of a target area identification image. When the target area identification image is corrected in step S600, a pixel block having a predetermined size is set in step S900. The size of the pixel block can be set as appropriate, but can be set to a size of 8 × 8 pixels, for example. Then, the pixel block information, which is information on the pixels included in each pixel block, and the corrected attribute for each pixel block are stored in association with each other.

  In parallel with step S900, post-processing is performed as in the first embodiment (step S800). Each time the process of step S700 is performed, the machine learning means learns based on the pixel block information and the corrected attribute of the pixel block. That is, what kind of pixel block is modified to what attribute is regarded as teacher data in machine learning, and supervised learning is performed. Specifically, the machine learning means gives the pixel value of each pixel of the pixel block to the corresponding node Na of the input layer 67a, and supplies it to the node Nd of the output layer 67d corresponding to the attribute instructed to correct the pixel block. The maximum weight value +1.0 is given, and 0 is given to the other nodes Nd of the output layer 67d, and a learning algorithm based on a known back propagation method (error back propagation method) is executed by the GPU 6. Then, when the pixel value of each pixel of the pixel block is input to the corresponding node Na of the input layer 67a, the node Nd of the output layer 67d corresponding to the modified attribute of the pixel block outputs the maximum output. The weights of all the links connecting the nodes of each layer shown in 12 are automatically changed. That is, when the pattern of each pixel of the pixel block is given to the input layer, learning is automatically performed so that the attribute weight instructed to the pixel block calculated in the output layer is increased. Since the correspondence between the pixel block and the attribute of the pixel block is not learned / stored, when the learning progresses, even if the same pixel block is given to the input layer, the pixel block calculated in the output layer is instructed. The weight of the attribute that has been changed may be smaller than the other attribute in some cases, and a result different from that in the learning may be calculated. When such a problem occurs frequently, the configuration of the neural network is inappropriate, so the number of nodes and the number of layers in the intermediate layer are reviewed. Each time the target region image is corrected based on the instruction in step S700, the machine learning means is shown in FIG. 12 based on the correspondence relationship between the pixel value feature of the pixel block and the attribute instructed to the pixel block. The weights Wab (i, j), Wbc (j, k), and Wcd (k, l) of all links connecting the nodes of each layer are calculated and updated. That is, by repeating the process of step S900, the machine learning means 67 causes all link weights Wab (i, j) based on the correspondence relationship between the pixel block of the target area identification image before correction and the attribute after correction. ), Learning proceeds so that Wbc (j, k) and Wcd (k, l) are close to optimum values.

  On the other hand, if it is determined in step S650 that the machine determination is to be used, the target area identification image is corrected based on the machine determination (step S950). FIG. 14B is a diagram illustrating a state in which attribute correction is automated using the result of machine determination. In step S950, a pixel block is set for the target area identification image generated in step S600. For example, the pixel block is set so as to divide the entire target area identification image. The size of the pixel block can be set as appropriate, but needs to be set to the same size as in machine learning. When a pixel block is set for the entire target area identification image, the pixel block is extracted one by one, and the pixel value of each pixel of the pixel block is input to the node Na corresponding to the input layer 67a of the machine learning means 67. The machine learning means converts the multi-layer neural network to the GPU 6 based on the weights Wab (i, j), Wbc (j, k), and Wcd (k, l) of all the links connecting the nodes of each layer that are already stored. To run. Then, the attribute of each pixel of the pixel block in the target area identification image is modified so that the attribute corresponds to the node Nd that has output the maximum output among the nodes Nd of the output layer 67d. As described above, after the machine learning means 67 has learned, it is possible to automatically set an attribute for each pixel in the target region identification image by machine determination without human intervention. When the target area identification image set automatically is visually confirmed on the screen by the user and there is at least one place where inappropriate setting is made, the same as in the first embodiment Based on the instruction from the user, the target area identification image is corrected by deleting the overextracted area (step S600), the correction information is stored (step S900), and used again for machine learning. By repeating such an operation for various color fundus images, it is almost unnecessary for the user to correct the target area identification image.

<5. Modified example>
The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above embodiment, a full color image of 8 bits for each RGB is used as a color fundus image obtained by photographing. However, the present invention is not limited to this, and various color images can be used. For example, in recent years, the gradation of commercial digital cameras has been expanded to 10 bits or more, and it has become possible to acquire full-color images such as 1024 gradations for each color, using color images with a larger number of gradations. May be. A color image having a smaller number of gradations may be used.

  The machine learning means preferably shares or exchanges data via a communication medium such as a computer network. The machine learning means may exist on the cloud server. Learning efficiency can be improved by setting it as the said structure. For example, when the machine learning means 67 shown in FIG. 11 is installed on the cloud server, other means such as the target area identification image generation means 40 and the target area identification image correction means 50 are connected to the cloud server via the network. Installed on one or more computers that can be directly operated by a user. That is, the gray scale image conversion means 10, the closing processing means 20, the extraction range definition means 30, the target area identification image generation means 40, the first computer that realizes the target area identification image correction means 50, machine learning The second computer having the means 67 is connected via the network.

  In the above embodiment, the target region of the optic papilla or the target region around the optic papilla is a disc region that is the optic papilla, a cup region that is a optic nerve recess, or a PPA region that is a reticulochoroidal atrophy around the nipple. Although three types are set, it is not necessary to set all of them as target regions, and at least one type may be set as target regions. By setting at least one type as a target region, at least one type of correction target region identification image can be created. Moreover, you may make it set other than these three types.

1 ... CPU (Central Processing Unit)
2 ... RAM (Random Access Memory)
3 ... Storage device 4 ... Instruction input I / F
5. Data input / output I / F
6 ... GPU (Graphical Processing Unit)
DESCRIPTION OF SYMBOLS 7 ... Display part 10 ... Gray scale image conversion means 20 ... Closing processing means 30 ... Extraction range definition means 40 ... Target area identification image generation means 50 ... Target area identification image correction means 60: Identification image composition means 65 ... Area ratio calculation means 67 ... Machine learning means 70 ... Fundus image storage means 80 ... Processing data storage means 100 ... Fundus image processing apparatus

Claims (18)

Gray scale image conversion means for converting a color fundus image into a gray scale image set so that the pixels of the target region serving as a glaucoma diagnosis location are included in a high pixel value range;
Closing processing means for creating a smoothed grayscale image by performing a closing process on the grayscale image based on a circular structuring element of a predetermined size;
Extraction range defining means for defining a lower limit value of a pixel value including at least all pixels included in the target area in the smoothed grayscale image;
For each pixel of the smoothed grayscale image, a target region identification image that can visually identify the target region and a non-target region other than the target region with respect to a pixel having a pixel value equal to or greater than the lower limit value. Target area identification image generating means for generating
Based on the area designated for the target area identification image, the value of the pixel identified as the target area in the designated area is corrected to a value identified as a non-target area, and the correction target area A target area identification image correcting means for creating an identification image;
A fundus image processing apparatus comprising: As the target region, a region corresponding to at least one of the three types of Disc, which is the optic disc, Cup, which is the optic nerve depression, and PPA, which is the perinipple reticulochoroidal atrophy,
The gray scale image conversion means, the closing processing means, the extraction range definition means, the target area identification image generation means, and the target area identification image correction means are executed for each target area, and a Disc area, a Cup area, a PPA The fundus image processing apparatus according to claim 1, wherein at least one type of correction target region identification image is created among three types in which the region is the target region. As the target region, at least a disc corresponding to the optic nerve head, a cup corresponding to the optic nerve depression, a region corresponding to a plurality of types of PPA that is a choroidal atrophy of the nipple,
The gray scale image conversion means, the closing processing means, the extraction range definition means, the target area identification image generation means, and the target area identification image correction means are executed for each target area, and at least a Disc area, a Cup area, The fundus image processing apparatus according to claim 1, wherein a plurality of types of correction target area identification images having a PPA area as the target area are created.   For the plurality of types of correction target region identification images, the pixels having the attributes of each target region are colored with different colors for each of the plurality of types, and are determined in advance so as not to contradict the anatomy / histology of the eyeball Based on the replacement priority order of a plurality of types of corresponding regions, the pixels corresponding to the pixels of the target region of different types of other correction target region identification images for each correction target region identification image are changed to the other types of other corrections. The apparatus further comprises identification image synthesis means for creating a synthesis target area identification image obtained by synthesizing the plurality of types of correction target area identification images by replacing with a pixel color of the target area of the target area identification image. Item 4. A fundus image processing apparatus according to Item 3. As the target area, at least three areas corresponding to Disc that is the optic nerve head, Cup that is the optic nerve depression, and PPA that is the peripapillary choroidal atrophy are set, and the first correction target area identification image is the Disc. When the second correction target area identification image corresponds to the Cup area, and the third correction target area identification image corresponds to the PPA area,
The identification image synthesis means includes
When replacing the pixel corresponding to the pixel of the target area of the second correction target area identification image with the color of the pixel of the target area of the second correction target area identification image with respect to the first correction target area identification image, If the corresponding pixel of the first correction target region identification image is a non-target region pixel, do not replace it,
When replacing the pixel corresponding to the pixel of the target area of the third correction target area identification image with the color of the pixel of the target area of the third correction target area identification image with respect to the first correction target area identification image, The fundus image processing apparatus according to claim 4, wherein when the corresponding pixel of the first correction target region identification image is a pixel of the target region, the fundus image processing device is not replaced.   The number of pixels colored with the three kinds of colors is counted by color for the composition target region identification image, and the number of pixels corresponding to the second color is divided by the number of pixels corresponding to the first color. 6. The apparatus according to claim 5, further comprising an area ratio calculating means for calculating a ratio of 1 and a second ratio obtained by dividing the number of pixels corresponding to the third color by the number of pixels corresponding to the first color. The fundus image processing apparatus described. When the target area is an area corresponding to three types of disc, which is the optic nerve head, Cup which is the optic nerve depression, and PPA which is the periapapillary choroidal atrophy,
The gray scale image conversion means uses the R component of the color fundus image as the gray scale image when the target region is a Disc region, and converts the G component of the color fundus image as the gray region when the target region is a Cup region. When a scale image is used and the target area is a PPA area, an image reflecting a value obtained by subtracting the maximum value from the value obtained by subtracting the B component from the R component of the color fundus image is set as the gray scale image. The fundus image processing apparatus according to any one of claims 1 to 6, wherein the fundus image processing apparatus is configured as described above. The extraction range defining means further defines an upper limit value of pixel values including all pixels included in the target region;
The target area identification image generating means is a target area identification that allows visually identifying a target area and a non-target area other than the target area with respect to a pixel having a pixel value in a range from the lower limit value to the upper limit value. The fundus image processing apparatus according to any one of claims 1 to 7, wherein an image is generated. The target area identification image generating means
For each pixel of the smoothed grayscale image, when the pixel value of the pixel is included in the range equal to or higher than the lower limit value, a difference between the pixel value of the pixel and the lower limit value is a predetermined one or more coefficient β A value obtained by multiplying by a predetermined offset value is set as one of RGB color components, and for the other color components, the pixel value of the pixel in the grayscale image is less than a predetermined coefficient γ Is set as the color of the target area by setting a value multiplied by, and when the pixel value of the pixel is less than the lower limit value, the pixel value in the grayscale image of the pixel is set as all RGB color components The fundus image processing apparatus according to claim 1, wherein the fundus image processing apparatus is configured to generate the target area identification image by setting the color of the non-target area by using the method. The target area identification image generating means
For each pixel of the smoothed grayscale image, when the pixel value of the pixel falls within the range from the lower limit value to the upper limit value, the difference between the pixel value of the pixel and the lower limit value is a predetermined one or more A value obtained by multiplying the coefficient β by a predetermined offset value is set as one of the RGB color components, and for the other color components, the pixel value of the pixel in the grayscale image is less than the predetermined 1 If the pixel value of the pixel is less than the lower limit value or exceeds the upper limit value, the pixel value of the pixel in the grayscale image is set to RGB. The fundus image processing apparatus according to claim 8, wherein the fundus image processing apparatus is configured to generate the target area identification image by setting the color of the non-target area by setting all the color components. The target area identification image correction means includes
For the target area identification image, for a pixel in which the target area is set in the designated area, a value obtained by dividing the minimum value of any color component of RGB of the pixel by the coefficient γ is RGB The fundus image processing apparatus according to claim 9 or 10, wherein the pixel is corrected to a value of a non-target region by correcting to be all color components. The target area identification image correction means includes
The correction target area identification image is displayed on the screen to indicate the correction target area, and the pixels included in the target area in the specified area are further corrected to the value of the non-target area, and the correction The fundus image processing apparatus according to any one of claims 1 to 11, wherein the target area identification image is updated. The target area identification image correction means includes
The designated area is divided into a plurality of pixel blocks having a predetermined number of pixels, the pixel values of the pixels in the divided pixel blocks are input to machine learning means, and the attribute of the non-target area is instructed. The fundus image processing apparatus according to any one of claims 1 to 12, wherein the fundus image processing apparatus inputs data as data to the machine learning means and performs learning processing on the machine learning means. The target area identification image correction means includes
The target area identification image or the correction target area identification image is divided into a plurality of pixel blocks having the predetermined number of pixels, and pixel values of the pixels in the divided pixel blocks are input to the machine learning unit. The correction target region identification image is created or updated by correcting an attribute given to the pixel in the pixel block to an attribute output from the machine learning means. The fundus image processing apparatus according to 13. The machine learning means includes
When the input layer is composed of any two color components of RGB of each pixel of the pixel block, and the pixel is included in the target region, it corresponds to two types of pixel values selected so that the two color components are different. And having twice as many nodes as the predetermined number of pixels,
The output layer is provided with two nodes corresponding to two types of attributes consisting of attributes of the target area and attributes of the non-target area,
15. The fundus image processing apparatus according to claim 13, wherein the fundus image processing apparatus is configured by a multi-layer neural network having at least one intermediate layer having a predetermined number of nodes.   A first computer for realizing the gray scale image converting means, the closing processing means, the extraction range defining means, the target area identification image generating means, and the target area identification image correcting means; and the machine learning means. The fundus image processing apparatus according to any one of claims 13 to 15, wherein the second computer including the second computer is connected via a network. A method in which a computer creates an image that identifies a target region that is a diagnostic site for glaucoma from a color fundus image,
Converting the color fundus image into a grayscale image that is set so that the pixel of the target region that is a glaucoma diagnosis location is included in the range of high pixel values;
The grayscale image is subjected to a closing process based on a circular structuring element of a predetermined size to create a smoothed grayscale image;
Defining a lower limit value of a pixel value including at least all pixels included in the target region in the smoothed grayscale image;
For each pixel of the smoothed grayscale image, a target region identification image that can visually identify the target region and a non-target region other than the target region with respect to a pixel having a pixel value equal to or greater than the lower limit value. Generating
Based on the area designated for the target area identification image, the value of the pixel identified as the target area in the designated area is corrected to a value identified as a non-target area, and the correction target area Creating an identification image;
A method of creating an image characterized by comprising:   A program for causing a computer to function as the fundus image processing apparatus according to any one of claims 1 to 16.