JP2016168134A - Fundus image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a fundus image processing device capable of performing accurate matching processing of a fundus image.SOLUTION: A fundus image processing device includes: blood vessel highlighting image conversion means 30 for highlighting a blood vessel candidate area after converting each of a plurality of fundus images to a gray scale format, and creating blood vessel highlighting images; blood vessel highlighting image matching means 40 for setting one of a plurality of the blood vessel highlighting images to a reference blood vessel highlighting image, setting each of the blood vessel highlighting images other than the reference blood vessel highlighting image to a calibration blood vessel highlighting image, and performing calibration to match a magnification ratio, an angle, and a position of each calibration blood vessel highlighting image with those of the reference blood vessel highlighting image; and matching index calculation means 50 for calculating a matching index between each of the calibration blood vessel highlighting images after the calibration by the blood vessel highlighting image matching means, and the reference blood vessel highlighting image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a fundus image processing technique, and more particularly to a technique for obtaining consistency of a plurality of fundus images taken in the past.

  Popular myopia in ophthalmological diseases is known as severe myopia, a severe illness that causes posterior polar tissue degeneration, eye axis elongation, strong myopia, and sometimes blindness. . Since this disease gradually progresses over a long time span of 20 years or more, it is important to observe a long-term course. However, medical images taken in the ophthalmology field more than 20 years ago include only fundus photographs. Even if fundus photographs remain, they are not digital digital fundus cameras, but films and polaroids (registered trademarks). ) Since the recording medium is an analog recording medium such as a photograph, the photographing conditions are completely different, and comparison and matching are difficult.

  On the other hand, a technique has been developed that enables alignment of a plurality of fundus images having an opening of about several tens of years in the same patient and enables comparison of dimensions of various fundus sites (see Patent Document 1). ). For alignment, the center of the optic nerve head (hereinafter simply referred to as “papillae”) and blood vessels are assumed on the premise that the running form of the main blood vessels of the human retina will not change throughout life even if various ophthalmic diseases occur. The magnification, angle, and position are corrected based on the standard. However, since the method of adjusting the magnification, angle, and position while observing the overlay image in a pseudo color on the screen by performing gray scale conversion on the two types of fundus images to be collated, depending on the fundus image, Alignment work can be difficult. Especially in fundus photographs with bleeding, even if contrast correction is applied, the blood vessels are unclear and it may be difficult to align the blood vessels with the reference.

  In addition, with the three-dimensional deformation of the eyeball shape such as severe myopia, some blood vessels may change, or new blood vessels may appear as the disease progresses. In this case, the pseudo-color overlay image can only be qualitatively evaluated, and quantitative analysis is difficult.

 As a method for accurately analyzing changes in fundus blood vessels, a method is conceivable in which a fluorescent agent is administered to a patient, blood vessels are fluorescent with infrared rays, blood vessel images are drawn, and blood vessel images are compared (see Patent Document 2). However, in the case of analyzing a new image alone, there is basically no fundus image captured in this way in the past. I can't.

JP 2013-85583 A JP 2001-137192 A

  Therefore, an object of the present invention is to provide a fundus image processing apparatus capable of performing a highly accurate fundus image alignment process and quantitatively presenting the degree of alignment after the alignment process.

In order to solve the above problems, in the first aspect of the present invention,
A blood vessel-enhanced image conversion means for emphasizing a blood vessel candidate region and creating a blood vessel-enhanced image after converting each of a plurality of fundus images into a grayscale format;
One of the plurality of blood vessel emphasized images is set as a reference blood vessel emphasized image, and each of the blood vessel emphasized images other than the reference blood vessel emphasized image is set as a corrected blood vessel emphasized image. A blood vessel-enhanced image matching means for performing correction for matching the blood vessel-enhanced image with the magnification, angle and position;
Consistency index calculation means for calculating a consistency index between each corrected blood vessel emphasized image after correction by the blood vessel emphasized image matching means and the reference blood vessel emphasized image;
There is provided a fundus image processing apparatus characterized by comprising:

  According to the first aspect of the present invention, each of a plurality of fundus images is converted to a gray scale format, a blood vessel candidate image is created by emphasizing a blood vessel candidate region, and one of the plurality of blood vessel emphasized images is selected. Correction is set to the reference blood vessel enhancement image, each of the blood vessel enhancement images other than the reference blood vessel enhancement image is set to the correction blood vessel enhancement image, and the correction, blood vessel enhancement image, magnification, angle, and position of each correction blood vessel enhancement image are matched. Thus, the consistency index between each corrected blood vessel emphasized image after correction and the reference blood vessel emphasized image is calculated, so that highly accurate matching processing can be performed. After matching processing, quantitative analysis of changes in blood vessel running can be realized by comparing a plurality of images with different imaging timings.

In the second aspect of the present invention,
Image digitizing means for digitizing a fundus image recorded on a photographic recording medium among a plurality of fundus images;
A fundus region corresponding to the fundus is extracted from the digitized fundus image and subjected to a scaling process at the same vertical and horizontal magnification so as to have a predetermined number of pixels, and pixels other than the fundus region are deleted or Fundus image pre-processing means for converting into a fundus image with a uniform number of pixels while adding,
The blood vessel emphasized image conversion means performs processing on the fundus image after the number of pixels is unified.

  According to the second aspect of the present invention, a photographic recording medium (positive / negative film, photographic paper, etc.) among a plurality of fundus images is stored in the form of an instant print such as a 35 mm slide for presentation at a conference or Polaroid (registered trademark). Digitized fundus images are extracted, and the fundus region corresponding to the fundus is extracted from the digitized fundus image, and the vertical and horizontal directions are the same so that the predetermined number of pixels is obtained. Since scaling processing is performed at a magnification, pixels other than the fundus area are deleted or added, and converted to a fundus image with a uniform number of pixels, so that various forms of images are optimally suited for comparison. It becomes possible to unify.

  In the third aspect of the present invention, the fundus image pre-processing unit recognizes a nipple region whose luminance is larger than a predetermined value in the fundus region, and the nipple region is located above the fundus region or A predetermined angle unit with respect to the fundus image so that the fundus image is positioned leftward when the fundus image corresponds to the left eye and is positioned rightward when the fundus image corresponds to the right eye. It is characterized by being rotated at (for example, 90 degrees).

  According to the third aspect of the present invention, in the pre-processing, when recognizing a papilla region whose luminance is larger than a predetermined value in the fundus region and the papilla region is located above or below the fundus region, When the fundus image corresponds to the left eye, it is rotated to the left, and when the fundus image corresponds to the right eye, it is rotated by a predetermined angle unit (for example, 90 degrees) so as to be positioned to the right. Therefore, even if the shooting angle and digitization scan direction vary, the direction of the fundus image can be aligned to the direction observed from the opposite side of the human right and left eyeballs, which has a big impact on comparison The position of the image can be accurately adjusted based on the nipple.

  In the fourth aspect of the present invention, the fundus image pre-processing unit makes the rectangular area circumscribing the fundus area have a predetermined number of pixels Sy in the height direction of the rectangular area. Is subjected to scaling processing so that W × Sy / H, and the width W ′ of the rectangular area after scaling processing is W ′ < In the case of Sx, background pixels are added to both ends of the rectangular area so that the width of the rectangular area is Sx, and in the case of W ′> Sx, the rectangular area is set so that the width of the rectangular area is Sx. The background pixels are deleted from both ends of the.

  According to the fourth aspect of the present invention, during the preprocessing, the number of pixels H in the height direction of the rectangular region circumscribing the fundus region is set to the predetermined number of pixels Sy, and the pixels in the width direction of the rectangular region If the magnification W is applied to the number W so that W × Sy / H, and the width W ′ of the rectangular area after the magnification processing is W ′ <Sx with respect to the predetermined number of pixels Sx, the rectangular area A background pixel is added to both ends of the rectangular area so that the width of the rectangular area becomes Sx. When W ′> Sx, background pixels are deleted from both ends of the rectangular area so that the width of the rectangular area becomes Sx. Therefore, it is possible to easily and highly accurately correct the corrected blood vessel emphasized image so that the magnification is mainly matched with the reference blood vessel emphasized image.

In the fifth aspect of the present invention, the blood vessel emphasized image conversion means includes
A gray scale conversion unit that performs a predetermined calculation on each color component for a fundus image in a color format and converts the pixel value to be smaller as the first color component is larger and converts the pixel value to a gray scale format;
Linear component enhancement means for performing an opening process on the converted fundus image in the gray scale format using a predetermined structural element to create a linear component enhanced image in which the luminance of the blood vessel candidate region is equal to or higher than a certain level; ,
In the linear component emphasized image, the pixel value satisfying the predetermined condition is replaced so as to be near the minimum value of the gradation of the converted blood vessel emphasized image, and the linear value that is equal to or greater than the pixel value satisfying the predetermined condition Pixel gradation conversion means for correcting the gradation of the pixel so that the pixel value of the component-enhanced image is in the range from the value near the minimum value of the gradation to the maximum value, and creating a blood vessel-enhanced image;
It is characterized by having.

  According to the fifth aspect of the present invention, when the color image fundus image is converted into the blood vessel emphasized image, predetermined calculation is performed on each color component, and the larger the first color component, the smaller the pixel value. A linear component in which the luminance of the blood vessel candidate region is set to a certain level or more by performing an opening process on the converted fundus image in the gray scale format using a predetermined structural element. Create an emphasized image, replace the pixel value satisfying the predetermined condition in the linear component emphasized image so that it is close to the minimum value of the gradation of the converted blood vessel emphasized image, and more than the pixel value satisfying the predetermined condition Since the pixel gradation of the linear component enhanced image is corrected so that the pixel gradation is within the range from the value near the minimum gradation value to the maximum value, the blood vessel enhanced image is created. As an image with extracted regions Tube emphasized image is obtained, it is possible to automatically extract the vessel regions while maintaining the vessel fine shape such as a blood vessel diameter from the fundus image.

  In the sixth aspect of the present invention, the blood vessel emphasized image matching unit assigns one of the three color components of RGB to the reference blood vessel emphasized image, and assigns the color assigned to the reference blood vessel emphasized image. A color component different from a component is assigned to the corrected blood vessel enhancement image, and the reference blood vessel enhancement image pseudo-colored with each color component and the correction blood vessel enhancement image are synthesized to create a color composite image. To do.

  According to the sixth aspect of the present invention, when matching the blood vessel emphasized image, any one of the three color components of RGB is assigned to the reference blood vessel emphasized image, and the color component assigned to the reference blood vessel emphasized image Different color components are assigned to the corrected blood vessel emphasizing image, and the reference blood vessel emphasizing image and the corrected blood vessel emphasizing image that are artificially colored with each color component are synthesized to create a color composite image. It is possible to display the degree of consistency between the images while clearly distinguishing the blood vessel candidate regions of both images.

  Further, in the seventh aspect of the present invention, the consistency index calculation means calculates a sum of pixel values of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image as C1, When C2 is set and C3 is the sum of geometric mean values of the pixel values of the pixels having pixel values exceeding the threshold at the same position in both the reference blood vessel emphasized image and the corrected blood vessel emphasized image, C1 ≦ If C2, a value proportional to C3 / C1 is calculated as a consistency index, and if C1> C2, a value proportional to C3 / C2 is calculated as a consistency index.

  According to the seventh aspect of the present invention, when the consistency index is calculated, the sum of pixel values of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image is set as C1 and C2. When the sum of the geometric mean values of the pixel values having the pixel values exceeding the threshold value at the same position in both the reference blood vessel emphasized image and the corrected blood vessel emphasized image is C3, if C1 ≦ C2, C3 / Since a value proportional to C1 is calculated as a consistency index, a value proportional to C3 / C2 if C1> C2 is calculated as a consistency index. Even if pixels in other regions are excessively extracted, it is possible to quantitatively obtain a highly accurate consistency index without affecting the consistency index.

  In the eighth aspect of the present invention, the consistency index calculating means calculates the number of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image as C1 ′ and C2 ′. When the number of pixels having the pixel value exceeding the threshold at the same position in both the reference blood vessel enhanced image and the corrected blood vessel enhanced image is C3 ′, C3 ′ / C1 if C1 ′ ≦ C2 ′. A value proportional to 'is calculated as a consistency index if C1'> C2 ', and a value proportional to C3' / C2 'is calculated.

  According to the eighth aspect of the present invention, when calculating the consistency index, the number of pixels having a pixel value exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image is C1 ′, C2 ′, A value proportional to C3 ′ / C1 ′ if C1 ′ ≦ C2 ′, where C3 ′ is the number of pixels having pixel values exceeding the threshold value at the same position in both the reference blood vessel enhanced image and the corrected blood vessel enhanced image. If C1 ′> C2 ′, a value proportional to C3 ′ / C2 ′ is calculated as the consistency index. Therefore, even if the pixel in the region other than the blood vessel is present in the blood vessel emphasized image having the larger pixel value sum, Even if it is extracted excessively, it is possible to obtain a consistent index with high accuracy quantitatively without affecting the consistency index.

  In the ninth aspect of the present invention, the consistency index calculating means sets C1a and C2a as the number of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image. If C1a ≦ C2a, the correlation coefficient between the C1a pixel group in the corrected blood vessel enhancement image corresponding to the C1a pixel group having a pixel value exceeding the threshold in the reference blood vessel enhancement image is matched. If C1a> C2a is calculated as a sex index, the C2a pixel group having a pixel value exceeding the threshold value in the corrected blood vessel emphasized image and the C2a pixel group in the reference blood vessel emphasized image corresponding to each other The number of relationships is calculated as a consistency index.

  According to the ninth aspect of the present invention, when C1a and C2a are the numbers of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image, if C1a ≦ C2a, A correlation coefficient between C1a pixel groups having pixel values exceeding the threshold value in the reference blood vessel enhanced image and C1a pixel groups corresponding to the corrected blood vessel enhanced image is calculated as a consistency index, and C1a> In the case of C2a, the correlation coefficient between the C2a pixel group in the reference blood vessel emphasized image corresponding to the C2a pixel group having a pixel value exceeding the threshold in the corrected blood vessel emphasized image is calculated as a consistency index. As a result, even if pixels in regions other than blood vessels are extracted excessively in the blood vessel-enhanced image with the larger sum of pixel values, the consistency index is not affected. , It is possible to obtain high integrity indicator of quantitative accuracy.

  According to the present invention, it is possible to perform fundus image alignment processing with high accuracy.

1 is a hardware configuration diagram of a fundus image processing apparatus 100 according to an embodiment of the present invention. 1 is a functional block diagram illustrating a configuration of a fundus image processing apparatus according to an embodiment of the present invention. It is a flowchart which shows the process outline | summary of the fundus image processing apparatus according to an embodiment of the present invention. It is a figure which shows the mode of the change of the color fundus image by pre-processing. It is a flowchart which shows the detail of the matching process of a reference blood vessel emphasis image and a correction blood vessel emphasis image. It is a figure which shows the mode of a change of the correction | amendment blood vessel emphasis image by a matching process. It is a figure which shows an example of the overlay display of a reference blood vessel emphasis image and a correction blood vessel emphasis image. 3 is a functional block diagram showing a detailed configuration of a blood vessel emphasized image conversion means 30. FIG. 4 is a flowchart showing an outline of processing of a blood vessel emphasized image conversion means 30. It is a flowchart which shows the detail of the 1st method of the linear component emphasis process of step S700. It is a flowchart which shows the detail of the opening process by the circular structure element of step S710. It is a figure which shows an example of the circular structure element used with the 1st method of a linear component emphasis process. It is a flowchart which shows the detail of the 2nd method of the linear component emphasis process of step S700. It is a flowchart which shows the detail of the opening process by the linear structure element of step S750. It is a figure which shows an example of the linear structure element used with the 2nd method of linear component emphasis processing. It is a figure which shows the specific pixel value of the linear structure element in the case of direction d = 1. It is a figure which shows the specific pixel value of the linear structure element in the direction of d = 2. It is a figure which shows the image conversion example using the linear component emphasis of the 1st method. It is a figure which shows the image conversion example using the linear component emphasis of the 2nd method. It is a figure which shows the example before the matching process by a prior art. It is a figure which shows the example after the matching process by a prior art. It is a figure which shows the example before the alignment process by this invention. It is a figure which shows the example after the alignment process by this invention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.
<1. Device configuration>
FIG. 1 is a hardware configuration diagram of a fundus image processing apparatus according to an embodiment of the present invention. The fundus image processing apparatus 100 according to the present embodiment can be realized by a general-purpose computer. As shown in FIG. 1, a CPU (Central Processing Unit) 1 and a RAM (Random Access Memory) that is a main memory of the computer. 2, a large-capacity storage device 3 such as a hard disk or flash memory for storing programs and data executed by the CPU 1, a key input I / F (interface) 4 such as a keyboard and a mouse, a 3D printer and data storage A data input / output I / F (interface) 5 for data communication with an external device such as a medium and a display unit 6 which is a display device such as a liquid crystal display are provided and are connected to each other via a bus.

  The image scanner 7 is a general-purpose image scanner, and is a digitizing device that reads an image recorded from a film, a photographic paper, an instant photograph, or the like and acquires it as digital image data. The image scanner 7 is connected to the data input / output I / F 5, and the image data output from the image scanner 7 is input from the data input / output I / F 5. As the image scanner 7, various types such as a flat bed type, a 35 mm slide only, and other types specialized for radiation diagnostic films can be used.

  FIG. 2 is a functional block diagram illustrating a configuration of the fundus image processing apparatus according to the present embodiment. In FIG. 2, 10 is an image digitizing means, 20 is a fundus image preprocessing means, 30 is a blood vessel emphasized image converting means, 40 is a blood vessel emphasized image matching means, and 50 is a consistency index calculating means.

  The image digitizing means 10 performs a process of digitizing a fundus image recorded on a photographic recording medium such as a film among a plurality of fundus images and creating color digital image data. The fundus image pre-processing means 20 extracts a range including the fundus area corresponding to the fundus from the digitized fundus image, and performs scaling processing at the same vertical and horizontal magnifications so as to obtain a predetermined number of pixels. Then, a process of converting to a fundus image with a uniform number of pixels is performed while deleting or adding pixels other than the fundus region. The blood vessel emphasized image conversion means 30 performs processing for emphasizing a blood vessel candidate region and creating a blood vessel emphasized image after converting each of the converted plurality of fundus image data into a gray scale format. The blood vessel emphasized image matching unit 40 sets one of a plurality of blood vessel emphasized images taken for the same patient at different times as a reference blood vessel emphasized image, and sets each of the blood vessel emphasized images other than the reference blood vessel emphasized image. The corrected blood vessel emphasized image is set, and correction is performed for each corrected blood vessel emphasized image so that the reference blood vessel emphasized image and the magnification, angle, and position are matched. At this time, a blood vessel emphasized image based on a fundus image recently taken with a digital camera is set as the reference blood vessel emphasized image. The reason is that many of the images taken in the past have unknown shooting conditions, and especially when taken with an analog camera, the vertical and horizontal scales are not always equal. The consistency index calculating unit 50 performs a process of calculating a consistency index between each corrected blood vessel emphasized image corrected by the blood vessel emphasized image matching unit 40 and the reference blood vessel emphasized image.

  In the image digitizing means 10, fundus image preprocessing means 20, blood vessel emphasized image converting means 30, blood vessel emphasized image matching means 40, and consistency index calculating means 50, the CPU 1 executes a program stored in the storage device 3. Is realized.

  Each component shown in FIG. 2 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices as shown in FIG. That is, the computer executes the contents of each means according to a dedicated program. Note that in this specification, a computer means an apparatus having an arithmetic processing unit such as a CPU and capable of data processing, and includes not only a general-purpose computer such as a personal computer but also a portable terminal such as a tablet.

  A dedicated program for operating the CPU 1 and causing the computer to function as a fundus image processing apparatus is installed in the storage device 3 illustrated in FIG. By executing this dedicated program, the CPU 1 functions as the image digitizing means 10, fundus image preprocessing means 20, blood vessel emphasized image converting means 30, blood vessel emphasized image matching means 40, and consistency index calculating means 50. Will be realized. The storage device 3 stores various data necessary for processing as a fundus image processing device.

<2. Process Overview>
Next, the processing operation of the fundus image processing apparatus shown in FIGS. 1 and 2 will be described. FIG. 3 is a flowchart showing an outline of processing of the fundus image processing apparatus according to the embodiment of the present invention. First, the image digitizing means 10 digitizes the fundus image obtained by photographing to obtain a digital color fundus image (step S100). The color fundus image is not necessarily full color, but is preferably full color.

  Specifically, a fundus image existing in the form of a film is read by a scanner or the like, and the image digitizing means 10 digitizes it. In the present embodiment, a full-color image having 8 bits and 256 gradations for each of R, G, and B components is prepared as a color fundus image. Digital color fundus images obtained by photographing in full color using a visible / light source fundus camera are read in parallel with digitization. That is, a color fundus image, which is image data, is input via the data input / output I / F 5 and stored in the storage device 3.

  Next, the fundus image preprocessing unit 20 performs preprocessing on the color fundus image (step S200). Specifically, a fundus region corresponding to the fundus is extracted from the color fundus image obtained as a digital image, and is changed at the same vertical and horizontal magnification so that a color fundus image having a predetermined number of pixels is obtained. A double process is performed to perform conversion to a color fundus image with a uniform size (= number of vertical and horizontal pixels) while deleting or adding background pixels.

  FIG. 4 is a diagram illustrating a change in the color fundus image due to the preprocessing. FIG. 4A shows a color fundus image after digitization by scanning or the like. In the example of FIG. 4A, the image size is 5905 pixels wide and 4032 pixels high. The fundus image pre-processing unit 20 first identifies a brighter, substantially elliptical nipple within a substantially circular fundus region. In reality, the fundus area is clear because of the large difference in pixel value from the black part of the background, but the difference in pixel value between the nipple and other parts in the fundus area is relatively clear in healthy subjects. Yes, but not so much if you have a condition with inflammation or bleeding. Therefore, a test is performed in advance and an assumed threshold value is set. Then, the fundus oculi image preprocessing means 20 identifies a set portion of pixels having a pixel value larger than a preset threshold value as a nipple. A portion surrounded by a black circle in the upper part of FIG. 4A is a portion specified as a teat.

  The fundus image preprocessing means 20 identifies the relative position of the identified nipple in the fundus area. Various methods can be used as a method for specifying the relative position. For example, when the coordinates of the center of the nipple divide the fundus region into four regions, upper, right, lower, and left. It can be specified by determining which position it belongs to. In rare cases, the image is taken at the center angle of the nipple. In this case, since it is located near the center, it cannot be determined whether it is classified into any of the above four areas only by the position of the nipple. It is also necessary to consider the direction of the blood vessels. That is, if the direction in which the main blood vessel is protruding is downward, leftward, upward, and rightward, it can be specified that the nipple position is in four regions, that is, upward, rightward, downward, and leftward, respectively.

  When the position of the nipple belongs above or below the fundus region, the fundus image preprocessing means 20 rotates the color fundus image (by 90 degrees. At this time, the color fundus image is obtained by photographing the right eye in advance. Information on whether there is a left eye or a photograph of the left eye is also recorded, so if it corresponds to the right eye, it corresponds to the left eye so that the nipple is located on the right side. In some cases, the nipple is rotated 90 degrees so that the nipple is positioned to the left, that is, when the nipple is at the upper left eye, it is rotated 90 degrees (counterclockwise) and the nipple position is at the upper right eye. 4 is rotated by -90 degrees (clockwise), is rotated by -90 degrees when the position of the nipple is the lower eye and the left eye, and is rotated by 90 degrees when the position of the nipple is the lower eye and the right eye. Is a color fundus image of the right eye, and the nipple is in the eye as shown in FIG. Since it is located above the bottom area, it is rotated 90 degrees rightward (clockwise), resulting in a color fundus image in which the nipple is located to the right as shown in FIG. Since only the rotation is performed, the size of the color fundus image after the rotation is 4032 pixels wide and 5905 pixels long.

  If the position of the nipple is close to the left end or right end and is the left eye or right eye, respectively, there is no need to rotate. That is, after digitization, a color fundus image may be obtained in a state as shown in FIG. 4B, and in this case, rotation is not performed.

  Next, the fundus image preprocessing unit 20 performs trimming of the color fundus image. Specifically, the pixels in the background area are deleted so that the entire image has a minimum rectangular shape including the entire fundus area. Since the pixel value of the fundus region is high and the pixel value of the background region is a low value close to “0” in principle, the threshold value can be easily set. Therefore, the fundus image preprocessing unit 20 clearly specifies the fundus region and the background region, and deletes the pixels in the background region so as to have a minimum rectangular shape including all the fundus regions. As a result of the trimming, a color fundus image as shown in FIG. 4C is obtained. Since the pixels in the background region are deleted, the size of the color fundus image after trimming (the number of horizontal pixels W × the number of vertical pixels H) is 3545 pixels wide and 3789 pixels long. When scanning an instant photo, a portion of the print support (usually white) may appear on the edge of the background, or a note such as the patient ID on the edge of the film may appear on the background. Pixels higher than the pixel value of the fundus region may be mixed into the region. In such a case, the background area in which pixels higher than the pixel value of the fundus area exist manually is partially deleted beforehand.

  Next, the fundus image preprocessing unit 20 enlarges or reduces the image so that the number of pixels in the vertical direction of the color fundus image becomes a predetermined value. Color fundus images include those obtained by scanning a film and digitizing it, and those obtained by taking images with various models. Therefore, here, in order to unify the size of the color fundus image, enlargement or reduction is performed so that a predetermined number of pixels of horizontal Sx × vertical Sy is obtained. The horizontal pixel number Sx and the vertical pixel number Sy can be set as appropriate, but in this embodiment, Sx = 1280 and Sy = 1024. The reason for setting Sy = 1024 is that the number of scanning lines of recent digital cameras for the fundus is often set to 1024 to 1600, and the reason why it is horizontally long as Sx = 1280 is as follows. Since the fundus image is usually circular, the fundus image can be basically square, but the upper and lower ends of the fundus are not important for diagnosis because the blood vessels are distributed in the left and right directions starting from the nipple. In many cases, the photo is taken horizontally up. Also, as will be described later, when performing color synthesis of two blood vessel emphasized images, they are often shifted in the horizontal direction and overlapped. From such experience, the pixel configuration was set to be slightly horizontally long. When enlarging a color fundus image, a process of interpolating pixels is performed, and when reducing an image, a process of thinning out pixels is performed. In the example of FIG. 4, the number of pixels in the vertical direction of the image shown in FIG. 4C is 3789 pixels, and the set number of vertical pixels is Sy = 1024 pixels. Reduces the color fundus image. Naturally, the image is reduced while maintaining the aspect ratio of the image. As a result of the reduction of the color fundus image, a color fundus image as shown in FIG. 4D is obtained. Since the color fundus image is reduced, the size of the color fundus image after reduction is 958 (W ′ = W × Sy / H) pixels and 1024 (Sy) pixels.

  As shown in FIG. 4D, since the number of pixels in the vertical direction of the image has been changed to the predetermined value Sy, the number of pixels in the horizontal direction is also changed to the predetermined value Sx. For this reason, the fundus image pre-processing unit 20 adds background pixels so that the number of pixels in the horizontal direction of the color fundus image becomes the predetermined value Sx. If the background pixel value is the same as the existing background pixel value, the processing can be easily performed. As a result of the addition of the background, a color fundus image as shown in FIG. 4E is obtained. Since the background is added, the size of the color fundus image after processing is 1280 (Sx) pixels in the horizontal direction and 1024 (Sy) pixels in the vertical direction. In the example of FIG. 4E, since W ′ <Sx, the background pixel is added. However, when W ′> Sx, the number of pixels in the horizontal direction of the color fundus image is Sx. The background pixels are deleted from both ends of the color fundus image.

  When the preprocessing of step S200 by the fundus image preprocessing unit 20 is completed, the blood vessel emphasized image conversion unit 30 then converts the color fundus image into a blood vessel emphasized image (step S300). Details of the conversion process into the blood vessel emphasized image will be described later.

  If a blood vessel emphasized image is obtained by the blood vessel emphasized image conversion processing of step S300 by the blood vessel emphasized image converting means 30, then the blood vessel emphasized image matching means 40 performs alignment of the blood vessel emphasized image (step S400). The alignment of the blood vessel emphasized image is a process of changing the magnification, the angle, and the position so that one image is a reference image and the other image is matched with the reference image. By matching images taken under different conditions, it becomes easy to diagnose secular changes.

  First, the blood vessel emphasized image matching unit 40 sets one of the blood vessel emphasized images obtained by the blood vessel emphasized image conversion unit 30 as a reference blood vessel emphasized image and the other as a corrected blood vessel emphasized image. Various methods can be used as a method for determining the reference blood vessel emphasized image. For example, the reference image enhanced image having the latest date and time when the image was taken can be used. Then, the reference blood vessel enhanced image and the plurality of corrected blood vessel enhanced images are matched. FIG. 5 is a flowchart showing details of the matching process between the reference blood vessel enhanced image and the corrected blood vessel enhanced image.

  In order to match the corrected blood vessel emphasized image with the reference blood vessel emphasized image, first, the blood vessel emphasized image matching unit 40 performs magnification correction of the corrected blood vessel emphasized image (step S401). There are various specific methods for magnification correction, and the method shown in Patent Document 1 can be used. The magnification correction is basically performed by setting different magnification correction parameters in the X direction and the Y direction. The reason is that if the optical axis of the eyeball and camera is tilted during shooting (in many cases, it is tilted to the left or right even in a natural state with high myopia), the image is recorded vertically (when tilted to the left or right) or landscape Therefore, there is a possibility that the aspect ratio of the pixels of an image taken with an analog camera in the past is not necessarily one-to-one like an image taken with a recent digital camera. That is, as an automatic method, first, the number of pixels in the horizontal direction and the number of pixels in the vertical direction of the blood vessel region of each of the reference blood vessel emphasized image and the corrected blood vessel emphasized image are obtained, and the ratio of both images is determined as the magnification in the X direction and the Y direction. Obtained as a correction parameter. Then, the magnification of the corrected blood vessel emphasized image is corrected according to the magnification correction parameters in the X direction and the Y direction. However, in this method, there is a significant difference in the extraction ratio of the blood vessel region between the reference blood vessel enhanced image and the corrected blood vessel enhanced image, or there is a significant angular difference between the reference blood vessel enhanced image and the corrected blood vessel enhanced image. Therefore, an appropriate magnification correction parameter cannot be obtained, and high-accuracy correction cannot be performed. Therefore, the corrected blood vessel emphasized image and the corrected blood vessel emphasized image may be displayed side by side on the display unit 6 and interactively performed by the user giving an instruction using the key input I / F 4. As an interactive method, the method shown in Patent Document 1 can be used.

  Next, the blood vessel emphasized image matching unit 40 performs angle correction of the corrected blood vessel emphasized image (step S402). Although there are various specific methods for angle correction, the method shown in Patent Document 1 can be used. Specifically, the blood vessel emphasized image matching means 40 obtains the length of the blood vessel region of the reference blood vessel emphasized image and the corrected blood vessel emphasized image after the magnification correction, respectively, and determines the angle for matching the length directions of both images. Obtained as an angle correction parameter. Then, the angle of the corrected blood vessel emphasized image is corrected according to the angle correction parameter. However, in this method, when there is a significant difference in the extraction ratio of the blood vessel region between the reference blood vessel emphasized image and the corrected blood vessel emphasized image, an appropriate angle correction parameter cannot be obtained and high accuracy correction cannot be performed. Therefore, the reference blood vessel emphasized image and the corrected blood vessel emphasized image may be displayed side by side on the display unit 6 and interactively performed by the user giving an instruction using the key input I / F 4. As an interactive method, the method shown in Patent Document 1 can be used.

  Next, the blood vessel emphasized image matching unit 40 identifies feature points of the reference blood vessel emphasized image (step S403). Various techniques can be used as a technique for specifying the feature points. For example, when performing automatically, the blood vessel emphasizing image matching unit 40 recognizes an area formed by pixels that are equal to or greater than a predetermined value as a nipple, and specifies the center of gravity of the nipple as a feature point. The center of gravity of the nipple can be determined, for example, as a coordinate that is an average value of the x coordinate value and the y coordinate value of the pixels in the nipple region. However, since the nipple area changes from a perfect circle to an ellipse compared to blood vessels, etc., as the disease progresses, the center of the nipple area (the method of taking the center of the maximum value from the minimum coordinate value) over time It often shifts and the center of gravity shifts. Therefore, as described below, it is possible to use a method that uses a branching portion, a crossing portion, a bending portion, or the like of a blood vessel in the nipple region as a feature point that is not easily affected by the deformation of the nipple region.

  The reference blood vessel emphasized image is displayed on the display unit 6, and the user can perform an interactive operation by giving an instruction using the key input I / F 4. In this case, the blood vessel emphasized image matching means 40 displays the reference blood vessel emphasized image on the display unit 6 and features one characteristic point (branch portion, crossover portion, bent portion, etc.) of a relatively dark blood vessel in the nipple. A message is displayed to instruct the user to instruct as a point, and the user is instructed to specify one point designated by the user as a feature point.

  Next, the blood vessel emphasized image matching unit 40 identifies feature points of the corrected blood vessel emphasized image (step S404). This can be performed in the same manner as the processing for the reference blood vessel emphasized image in step S403. When interactively performing, the reference blood vessel emphasized image and the corrected blood vessel emphasized image are simultaneously displayed on the display unit 6, and feature points (characteristic points of blood vessels in the nipple region (branch portion, crossover portion, bent portion) are displayed for each. Etc.)) may be specified.

  After the feature points of the reference blood vessel emphasized image and the corrected blood vessel emphasized image are specified, the blood vessel emphasized image matching unit 40 next performs position correction of the corrected blood vessel emphasized image based on both feature points (step S405). . Specifically, the blood vessel enhancement image matching unit 40 calculates the difference between the relative coordinates in the same image of the feature points of the reference blood vessel enhancement image and the relative coordinates in the same image of the feature points of the corrected blood vessel enhancement image, Obtained as a correction parameter. Then, according to the position correction parameter, a process of moving the entire corrected blood vessel emphasized image within the region of the original corrected blood vessel emphasized image of Sx × Sy is performed. For example, when moving to the right by dx pixels in the X-axis direction and moving upward by dy pixels in the Y-axis direction, the dx × Sy pixel region that protrudes to the right end is deleted, and instead the left dx × Sy pixel region Since it becomes blank, the background pixel is filled, and the area of the Sx × dy pixel that protrudes to the upper end is deleted. Instead, the area of the lower Sx × dy pixel becomes blank, and the background pixel is filled.

  FIG. 6 is a diagram illustrating a change in the corrected blood vessel emphasized image by the matching process. FIG. 6 shows the results of the processing from steps S401 to S405 in FIG. As shown in the upper part of FIG. 6, there is no change in the reference blood vessel emphasized image, but as shown in the lower part of FIG. 6, magnification, angle, and position are corrected for the corrected blood vessel emphasized image.

  After the correction of the corrected blood vessel emphasized image, the blood vessel emphasized image matching unit 40 displays the reference blood vessel emphasized image and the corrected blood vessel emphasized image in a superimposed state (step S406). That is, the reference blood vessel emphasized image and the corrected blood vessel emphasized image are displayed in an overlay manner. Specifically, one of the RGB primary colors (any one or a plurality of components) is assigned to each of the reference blood vessel enhancement image and the corrected blood vessel enhancement image, which are grayscale images, and pseudo color conversion is performed to perform one color conversion. A color composite image is created, and this color composite image is displayed on the display unit 6 as an overlay image. Unlike the natural colors provided in the original fundus image and blood vessel enhancement image, this color composite image is an artificially colored pseudo color for the convenience of display identification. Here, the reference blood vessel emphasized image is assigned to the R (red) component, and the corrected blood vessel emphasized image is assigned to the G (green) component.

  The reference blood vessel enhanced image and the corrected blood vessel enhanced image, which are gray scale images, are assigned to Uray1 (x, y), Uray2 (x, y) (x = 0,..., Xs-1, y = 0,. Ys-1), and the color composite image, which is a full-color image, is ImageS (x, y, c) (c = 0, 1, 2), and the blood vessel emphasized image matching unit 40 follows the following [Equation 1]. By executing the process, a color composite image ImageS (x, y, c) is calculated.

[Formula 1]
ImageS (x, y, 0) = r1 × Uray1 (x, y) + r2 × Uray2 (x, y)
ImageS (x, y, 1) = g1 × Uray1 (x, y) + g2 × Uray2 (x, y)
ImageS (x, y, 2) = b1 × Uray1 (x, y) + b2 × Uray2 (x, y)
However, 0 ≦ r1, g1, b1, r2, g2, b2 ≦ 1, r1 + r2 ≦ 1, g1 + g2 ≦ 1, b1 + b2 ≦ 1

  The coefficients r1, g1, b1, r2, g2, and b2 in the above [Equation 1] are all real numbers set in advance. In the above example, r1 = 1.0, g1 = 0.0, b1 = 0.0, r2 = 0.0, g2 = 1.0, b2 = 0.0, but r1 + r2 ≦ 1, g1 + g2 ≦ 1, b1 + b2 ≦ 1, if r1 = 1.0, g1 = 1.0, b1 = 0.0, r2 = 0.0, g2 = 0.0, b2 = 1.0 2 components of RGB may be assigned to the image, such as r1 = 0.9, g1 = 0.5, b1 = 0.1, r2 = 0.1, g2 = 0.5, b2 = 0.9 Alternatively, RGB three components may be assigned to both images.

  The color composite image created by the blood vessel emphasized image matching means 40 is displayed on the display unit 6. FIG. 7 is a diagram illustrating an example of overlay display using a color composite image. By using the reference blood vessel emphasized image and the corrected blood vessel emphasized image shown on the left side of the drawing, the color composite image shown on the left side of the drawing is displayed by performing the matching process in step S400. As described above, since the reference blood vessel emphasized image is assigned to R (red) and the corrected blood vessel emphasized image is assigned to G (green), the blood vessel candidate region of the reference blood vessel emphasized image is the color composite image that is a full color image. Red, the blood vessel candidate region of the corrected blood vessel enhancement image is displayed in green. In FIG. 7, in the color composite image, the red portion is indicated by dark shading and the green portion is indicated by light shading.

  Next, it is determined whether or not the consistency is acceptable (OK). Specifically, as a result of the display, if the user confirms and determines that the consistency is high, the key input I / F 4 is used to give an instruction to pass. When an instruction to pass is given, the blood vessel enhancement image matching unit 40 outputs the magnification correction parameter, the angle correction parameter, and the position correction parameter of the corrected blood vessel enhancement image at this time. On the other hand, if an instruction indicating that it is not passed is given, the blood vessel emphasized image matching unit 40 returns to step S401 and repeats the process for the corrected blood vessel emphasized image.

  The matching processing shown in FIG. 5 is executed for all the corrected blood vessel emphasized images, and the magnification correction parameter, the angle correction parameter, and the position correction parameter are acquired for each corrected blood vessel emphasized image and correspond to each corrected blood vessel emphasized image. In addition, it is stored in the RAM 2 or the storage device 3.

  When the magnification correction parameter, the angle correction parameter, and the position correction parameter are obtained by the blood vessel emphasized image matching process in step S400 by the blood vessel emphasized image matching means 40, the consistency index calculating means 50 next calculates the consistency index. (Step S500). As the consistency index, a pixel value matching ratio, a pixel matching ratio, and a correlation coefficient are calculated. The calculation of the consistency index is performed in a state where each corrected blood vessel enhanced image is corrected using the magnification correction parameter, the angle correction parameter, and the position correction parameter.

  First, the pixel value matching ratio is calculated. As in the above [Equation 1], the reference blood vessel emphasized image and the corrected blood vessel emphasized image are respectively represented as Uray1 (x, y), Uray2 (x, y) (x = 0,..., Xs-1, y = 0,. .., Ys-1), and the consistency index calculation means 50 executes processing according to the following [Equation 2], so that the blood vessel candidate regions in the reference blood vessel enhanced image and the corrected blood vessel enhanced image are respectively Pixel value sums C1 and C2 are calculated.

[Formula 2]
C1 = Σ _{Uray1 (x, y)> SA} Uray1 (x, y) / 255
C2 = _{ΣUray2 (x, y)> SA} Uray2 (x, y) / 255

  In the above [Equation 2], the subscripts “Uray1 (x, y)> SA” and “Uray2 (x, y)> SA” of Σ are the threshold values of the reference blood vessel emphasized image and the corrected blood vessel emphasized image. It shows that the sum of pixel values (values obtained by normalizing pixel values from 0 to 1) is obtained for all pixels exceeding the value SA. In the above [Equation 2], SA is a threshold value common to the reference blood vessel emphasized image and the corrected blood vessel emphasized image. Pixels having a value larger than this threshold value are designated as blood vessel candidate regions, and pixel value sums C1 and C2 are calculated. Calculated.

  Subsequently, the consistency index calculating unit 50 uses the pixel value sum C1 or C2 of the blood vessel candidate region, the reference blood vessel emphasized image Uray1 (x, y), or the corrected blood vessel emphasized image Uray2 (x, y) as follows: The pixel value matching ratio R1 is calculated by executing the processing according to Equation 3].

[Formula 3]
When C1 ≦ C2 R1 = { _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA} {Uray1 (x, y) × Uray2 (x, y)} ^1/2 / 255} × 100 / C1
When C1> C2 R1 = { _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA} {Uray1 (x, y) × Uray2 (x, y)} ^1/2 / 255} × 100 / C2

By executing the processing according to the above [Equation 3], Uray1 (x, y)> SA in the range of x = 0,..., Xs-1, y = 0,. The pixel value matching ratio R1 is calculated as the ratio of the sum of the geometric mean values of the pixel values satisfying Uray2 (x, y)> SA to the pixel value sum C1 or C2 of the smaller blood vessel candidate region. In the above [Equation 3], “ _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA} Uray1 (x, y)”, “ _{ΣUray1 (x, y)>SA; Uray2 (x, y )> SA} Uray2 (x, y) ”indicates the sum C3 of the geometric mean values of the pixel values at the same position as the pixel having the pixel value exceeding the threshold value SA in both the reference blood vessel enhanced image and the corrected blood vessel enhanced image. ing. In this case, the pixel value matching ratio R1 is a value proportional to C3 / C1 or C3 / C2.

  In the processing according to the above [Equation 2] and [Equation 3], the pixel value sums C1 and C2 are used, but instead of the pixel value sums C1 and C2, the number of pixels C1 ′ and C2 ′ is used. The pixel matching ratio R1 ′ is calculated by executing the process according to [Formula 4].

[Formula 4]
When C1 ′ ≦ C2 ′ R1 ′ = { _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA1} } × 100 / C1 ′
When C1 ′> C2 ′ R1 ′ = { _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA1} } × 100 / C2 ′

In the above [Equation 4], the pixel numbers C1 ′ and C2 ′ correspond to the areas of the blood vessel candidate regions of the reference blood vessel enhanced image and the corrected blood vessel enhanced image, respectively. In this case, “ _{ΣUray1 (x, y)>SA; Uray2 (x, y)> SA1} ” is a pixel having a pixel value exceeding the threshold value SA in both the reference blood vessel enhanced image and the corrected blood vessel enhanced image. The number of pixels C3 ′ at the same position is shown. At this time, the pixel matching ratio R1 ′ is a value proportional to C3 ′ / C1 ′ or C3 ′ / C2 ′. It is also possible to calculate not only one of the pixel value matching ratio R1 and the pixel matching ratio R1 ′ but also both the pixel value matching ratio R1 and the pixel matching ratio R1 ′.

  After the pixel value matching ratio R1 is calculated, the consistency index calculating unit 50 calculates a correlation coefficient. The correlation coefficient is an index indicating the strength of the relationship between two array elements. Here, it is used to determine the strength of similarity between the reference blood vessel emphasized image and the corrected blood vessel emphasized image in the blood vessel candidate region. First, the number of pixels C1a and C2a corresponding to the area of the blood vessel candidate region in the reference blood vessel emphasized image and the corrected blood vessel emphasized image is calculated by executing processing according to the following [Equation 5].

[Formula 5]
C1a = Σ _{Uray1 (x, y)> SA} 1
C2a = Σ _{Uray2 (x, y)> SA} 1

  As shown in the above [Formula 5], the pixel numbers C1a and C2a simply counted the number of pixels satisfying “Uray1 (x, y)> SA” and “Uray2 (x, y)> SA”, respectively. Is. Next, the consistency index calculation unit 50 calculates the pixel value averages A1 and A2 of the blood vessel candidate regions in the reference blood vessel emphasized image and the corrected blood vessel emphasized image by executing processing according to the following [Equation 6]. To do.

[Formula 6]
When C1a ≦ C2a A1 = Σ _{Uray1 (x, y)> SA} Uray1 (x, y) / C1a
A2 = _{ΣUray1 (x, y)> SA} Uray2 (x, y) / C1a
When C1a> C2a A1 = Σ _{Uray2 (x, y)> SA} Uray1 (x, y) / C2a
A2 = _{ΣUray2 (x, y)> SA} Uray2 (x, y) / C2a

  The pixel value average A1 calculated by the processing according to [Formula 6] is for the blood vessel candidate region in the reference blood vessel emphasized image, but the pixel value average A2 is the same as the blood vessel candidate region in the reference blood vessel emphasized image. This is for the region of the corrected blood vessel enhancement image that is the position. Then, the consistency index calculating means 50 calculates the correlation coefficient R2 by executing processing according to the following [Equation 7] using the pixel value averages A1 and A2.

[Formula 7]
R2 = { _{ΣUray1 (x, y)> SA} (Uray1 (x, y) −A1) × (Uray2 (x, y) −A2)} × 100 / [{ _{ΣUray1 (x, y)> SA} (Uray1 (X, y) _-A1 ) ² } ^1/2 * {[Sigma] _{Uray1 (x, y)> SA} (Uray2 (x, y) -A2) ² } ^1/2 ]

  As a result of the processing according to [Equation 7], the correlation coefficient between the pixel group of the reference blood vessel emphasized image and the pixel group of the corrected blood vessel emphasized image in the blood vessel candidate region in the reference blood vessel emphasized image is obtained as the correlation coefficient R2. It is done. The larger the value of the correlation coefficient R2, the higher the consistency between the reference blood vessel enhanced image and the corrected blood vessel enhanced image.

  In the above [Equation 7], the subscript “Uray1 (x, y)> SA” of Σ indicates that the sum is obtained for all pixels in the reference blood vessel emphasized image whose pixel value exceeds the threshold value SA. . The correlation coefficient R2 is obtained by dividing the covariance of the pixel groups Uray1 (x, y) and Uray2 (x, y) of the reference blood vessel enhanced image in the blood vessel candidate region by the respective standard deviations. In general, the processing load is high when dividing the square root of the denominator twice as in the calculation of [Expression 7]. For this reason, in the field of image processing, there is a habit of calling only a numerator product-sum operation as a “correlation coefficient” and R2 shown in the above [Equation 7] as a “normalized correlation coefficient”. .

  As the determination of consistency, conditions can be set by combining one or more and three or less consistency indices among the consistency indices of the pixel value matching ratio R1, the pixel matching ratio R1 ′, and the correlation coefficient R2. . A predetermined threshold value is set for each consistency index, and when a predetermined condition is satisfied, it can be determined that the reference blood vessel enhanced image and the corrected blood vessel enhanced image are matched. For example, if any one consistency index satisfies the condition, it may be determined that the condition is consistent, or if all three consistency indices satisfy the condition, You may judge.

<3. Details of Conversion to Blood Vessel-Enhanced Image in Step S300>
Next, details of the conversion process into the blood vessel emphasized image in step S300 will be described. FIG. 8 is a functional block diagram showing a detailed configuration of the blood vessel emphasized image converting means 30. In FIG. 8, 31 is a gray scale conversion means, 32 is a linear component enhancement means, and 33 is a pixel gradation conversion means.

  The gray scale conversion means 31 performs a process for converting the color fundus image into a gray scale fundus image obtained by performing a predetermined calculation on the RGB three primary color components and performing negative / positive inversion so that the blood vessel candidate region has high luminance. The linear component enhancement means 32 performs an opening process on the grayscale fundus image using a predetermined structural element to create a linear component enhanced image in which the luminance of all the blood vessel candidate regions is equal to or higher than a certain level. Do. The pixel gradation conversion means 33 replaces a pixel value satisfying a predetermined condition in the linear component emphasized image so as to become a value near the minimum gradation value of the converted blood vessel emphasized image, and satisfies the predetermined condition. A blood vessel emphasized image (Uray) is created by correcting the pixel gradation so that the pixel value of the linear component enhanced image that is equal to or greater than the pixel value is in the range from the value near the minimum gradation value to the maximum value. Process.

  Next, details of the conversion process into the blood vessel emphasized image in step S300 by the blood vessel emphasized image converting means 30 will be described. FIG. 9 is a flowchart showing a processing outline of the blood vessel emphasized image converting means 30. First, the gray scale conversion means 31 performs gray scale conversion on the color fundus image to obtain a gray scale fundus image (step S600).

  As described above, the color fundus image to be processed is 8-bit, 256-gradation image data for each color. Therefore, the restricted image having the number of pixels in the x direction Xs and the number of pixels in the y direction Ys is assumed to be Image (x, y, c) = 0 to 255 (x = 0,..., Xs-1; y = 0,..., Ys-1; c = 0, 1, 2).

  When converting to a gray scale image, conventionally, only the Green component is used for the color fundus image Image (x, y, c). That is, it is known that the conversion method using Gr (x, y) = Image (x, y, 1) is the best (see Patent Document 1). The reason is that the three primary color filters used in the fundus camera have a lower resolution of the Red and Blue components than the Green component, and the outline of the blood vessel tends to be unclear in these color separation images. However, as a result of analyzing many fundus camera images, the inventor of the present application has found that the Green component is clear near the center where the illumination light is sufficiently hit, but the Blue component is rather than the Green component in the periphery where the illumination light is weak. In contrast to the Green component, the inversion phenomenon was found in which the brightness of the blood vessel part was higher than that of the surroundings (the brightness of the blood vessel part was lower than that of the surroundings in the entire fundus image). Therefore, this embodiment proposes a method of performing gray scale conversion by executing processing according to the following [Equation 8]. However, [Equation 8] is not applied to all color fundus images, and the conventional method of converting by Gr (x, y) = Image (x, y, 1) is more appropriate. Since there are not a few fundus images, it is necessary to select these appropriately for each given fundus image. The gray scale conversion means 31 performs gray scale conversion by executing processing according to the following [Equation 8].

[Formula 8]
Gr (x, y) = [Image (x, y, 1) × (255−Image (x, y, 2))] ^1/2

  In the above [Equation 8], Image (x, y, 1) represents a component of G (Green, green) in the color fundus image, and Image (x, y, 2) represents in the color fundus image. The component of B (Blue, blue) is shown. Therefore, in the above [Equation 8], the geometrical average value of the negative and positive inversions of the G component and B component of each pixel is obtained to obtain a grayscale image Gr (x, y). Further, the gray scale conversion means 31 performs a process according to the following [Equation 9] on the gray scale image Gr (x, y), thereby performing negative / positive inversion, and gray scale fundus image Gray (x , Y).

[Formula 9]
Gray (x, y) = 255-Gr (x, y)

  By executing the processing according to the above [Equation 9] and performing negative / positive inversion, the pixel value of the blood vessel candidate region whose luminance is lower than that of the surrounding is converted to a high state. Here, negative / positive reversal means a process of converting a smaller original pixel value to a smaller value.

  Once the grayscale fundus image is obtained, the linear component enhancement means 32 then performs linear component enhancement on the grayscale fundus image using top hat transformation to obtain a linear component enhanced image ( Step S700). Since there are two methods for the top hat conversion in step S700, the details will be described later.

  After the linear component enhanced image is obtained, the pixel gradation conversion unit 33 performs gradation conversion (contrast correction) on the linear component enhanced image to obtain a blood vessel enhanced image (step S800). Specifically, first, the pixel gradation converting unit 33 performs frequency distribution of the pixel value h for all Xs × Ys pixels of the linear component emphasized image Tray (x, y) having a value of 0 to 255. H (h) (h = 0,..., 255) is obtained. Then, the minimum value hmin that satisfies the condition shown in the following [Equation 10] is obtained.

[Formula 10]
Σ _{h = 0, hmin} H (h) ≧ (Xs × Ys) × α

  In the above [Equation 10], the subscript “h = 0, hmin” of Σ indicates that h is the sum of 0 to hmin. Therefore, the condition shown in the above [Expression 10] indicates that, of the total number of pixels (Xs × Ys), pixels with a ratio α × 100% have pixel values equal to or less than the pixel value hmin. As the ratio α, it is necessary to give the ratio of the region other than the blood vessel included in the fundus image, and this ratio varies for each acquired fundus image, but on average 0.7 to 0.9, typically As a typical example, 0.8 can be set.

  Once the minimum value hmin that satisfies the condition shown in [Equation 10] is obtained, the pixel gradation conversion means 33 then selects x = 0,..., Xs−1; y = 0,. The processing according to the following [Formula 11] is executed for all the pixels of −1 to perform contrast correction.

[Formula 11]
Uray (x, y) = {Tray (x, y) −hmin} × 255 × β / (255−hmin)

  In the above [Formula 11], β is a luminance scaling value. The luminance scaling value β depends on the set ratio α and can be arbitrarily set. However, when α = 0.8, it is preferably 18.0 to 22.0, particularly preferably 20 .0. In Uray (x, y), “0”, which is the minimum value of the gradation, indicates the outside of the blood vessel candidate region, and the blood vessel candidate region is “1” or more and “255” or less. As the minimum value of the blood vessel candidate region, a value near the minimum value of the gradation is set. In the present embodiment, the minimum gradation value + 1 is set as the value near the minimum gradation value, but it may be more than that. Of course, in order to increase the contrast of the blood vessel candidate region, it is preferable that the minimum gradation value is +1.

<3.1. Linear component enhancement>
The linear component enhancement is performed using the top hat transformation. The top hat conversion generally means a process of subtracting the opening image obtained by the opening process from the original image, and is used for making light source unevenness uniform at the time of photographing. Next, each of the two top hat conversion methods in step S700 will be described in detail.

<3.1.1. First Method>
FIG. 10 is a flowchart showing the processing operation of the first method of linear component enhancement in step S700. First, the linear component emphasizing unit 32 performs an opening process using a circular structuring element on the grayscale fundus image obtained by the grayscale conversion unit 31 (step S710). In general, the opening process means a process in which each of the contraction process and the expansion process is repeated the same number of times. When white is targeted in a binary image of white and black, the shrinking process is a process of reducing white pixels by replacing black if there are black pixels around the pixel of interest. This is a process for increasing white pixels by replacing white pixels if there are white pixels around the pixels.

  FIG. 11 is a flowchart showing details of the opening process by the circular structural element. First, the linear component emphasizing means 32 performs contraction processing using the designated structural element (step S711). In the contraction process in step S711, a circular structural element is used as the designated structural element.

  FIG. 12 is a diagram illustrating an example of a circular structural element used in the first method. As shown in FIG. 12, the first method uses a circular structure element in the form of a 15 × 15 mask image in which a radius of 7 pixels, that is, a distance of 7 pixels or less from the center is effective. As shown in FIG. 12, among the 15 × 15 pixels, the pixel value at a distance of 7 pixels or less from the center is “1”, and the pixel value of a pixel exceeding the distance 7 from the other center is “0”.

  The circular structural elements shown in FIG. 12 are M (u, v) = {0, 1} (u = −R,... 0,... R; v = −R,. R). Note that R indicates a valid radius, and R = 7 in the example of FIG. The radius R can be set as appropriate, but for a gray scale fundus image Gray (x, y) of 1024 pixels, R = 7 is preferable. U represents the horizontal axis in FIG. 12, and v represents the vertical axis in FIG. The linear component emphasizing means 32 uses a circular structuring element M (u, v) and performs grayscale fundus image Gray (x, y) obtained by executing the process according to [Formula 9]. Then, processing according to the following [Equation 12] is executed to obtain a contracted image Eray (x, y).

[Formula 12]
Eray (x, y) = MIN _{u = −R, R; v = −R, R} [(255−M (u, v) × 254) × (Gray (x + u, y + v) +1) −1]

  In the above [Equation 12], MIN indicates the minimum value, and the subscript “u = −R, R; v = −R, R” of MIN indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Gray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the minimum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 12, the calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the circular structure element, and the minimum value is given as Ray (x, y).

  When the contracted image Ray (x, y) is obtained by executing the processing according to the above [Formula 12], the Ray (x, y) is replaced with Gray (x, y), and the above [Formula 12] is obtained. It is also possible to repeatedly execute the process. That is, the contraction process in step S711 may be repeatedly executed. The number of repetitions can be set as appropriate. However, in order to suppress image quality deterioration, it is normally set to once and not repeated.

  After the contraction process is repeatedly executed and the contracted image Eray (x, y) is obtained, the linear component emphasizing unit 32 performs the expansion process using the designated structural element (step S712). In the expansion process in step S712, the circular structural element used for the contraction process is used as the designated structural element. That is, the circular structural element shown in FIG. 12 is used. The linear component emphasizing means 32 performs processing according to the following [Equation 13] on the contracted image Eray (x, y) using the circular structural element M (u, v), and after expansion, An image Fray (x, y) is obtained.

[Formula 13]
Fray (x, y) = MAX _{u = −R, R; v = −R, R} [M (u, v) × Eray (x + u, y + v)]

  In the above [Equation 13], MAX indicates that it takes the maximum value, and the subscript “u = −R, R; v = −R, R” of MAX indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Ray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the maximum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 12, the calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the circular structure element, and the maximum value is given as Fray (x, y).

  When the expanded image Ray (x, y) is obtained by executing the processing according to the above [Formula 13], this Ray (x, y) is replaced with the Ray (x, y), and the above [Formula 13 ] Can be repeatedly executed. That is, the expansion process in step S712 may be repeatedly executed. The number of repetitions needs to be set to the same number as the number of repetitions of the contraction process in step S711. Normally, the contraction process is set to one and no repetition is performed.

  The expanded image Fray (x, y) obtained by repeatedly executing the contraction process a predetermined number of times becomes an opening image in which fine granular noise is deleted and a feature pattern such as a linear component is emphasized. However, low frequency components such as light source unevenness remain at this stage. If this opening image is obtained, then the linear component enhancement unit 32 performs a subtraction process from the grayscale fundus image (step S720). As a result, low-frequency components such as light source unevenness included in the grayscale fundus image and the opening image are canceled, and the feature pattern emphasized in the opening image mainly remains. Specifically, the linear component emphasizing unit 32 executes processing according to the following [Equation 14] using the original grayscale fundus image Gray (x, y) and the opening image Ray (x, y). Then, a linear component emphasized image Ray (x, y) is obtained.

[Formula 14]
Ray (x, y) = Gray (x, y) −Fray (x, y) +128

  As shown in the above [Equation 14], the linear component-enhanced image Ray (x, y) is the original grayscale fundus image Gray (x, y) and each pixel (x = y) of the opening image Ray (x, y). 0,..., Xs-1; y = 0,..., Ys-1), and a difference calculation is performed, and an offset of 128 is added. The integer value to which the offset is added is set to a half value of the gradation value 256 of the image so that the pixel value falls within the range of 0 to 255. This offset addition is performed so that the pixel value falls within the range of 0 to 255 because the pixel value is concentrated in the vicinity of 0 and a part of the pixels has a negative value when the difference calculation is simply performed.

<3.1.2. Second method>
FIG. 13 is a flowchart showing the processing operation of the second method of linear component enhancement in step S700. First, the linear component emphasizing unit 32 performs an opening process using a linear structural element on the grayscale fundus image obtained by the grayscale conversion unit 31 (step S750).

  FIG. 14 is a flowchart showing details of the opening process by the linear structure element. First, the linear component emphasizing means 32 performs an opening process using eight types of linear structural elements (step S751). In the opening process in step S751, eight types of linear structural elements are used as designated structural elements. FIG. 15 is a diagram illustrating an example of a linear structural element used in the second method. As shown in FIG. 15, in the second method, a linear structural element in the form of a 15 × 15 mask image in which a radius of 7 pixels, that is, a distance of 7 pixels or less from the center is effective is used.

  Since the linear structural element shown in FIG. 15 serves as a mask, it is a binary image. Therefore, in FIG. 15, the value of the pixel indicated as “0” is “0”, but the value of the pixel indicated as a numerical value other than “0” is “1” according to each of the eight types of directions d. ”And“ 0 ”in the other direction. Numbers other than “0” in FIG. 15 indicate directions. “1” is the horizontal direction (the horizontal direction in the drawing), and thereafter the direction changes at intervals of 22.5 degrees as the number increases by one. “2”, “4”, “6”, and “8” are arranged in adjacent sections because columns of the same number cannot be arranged on a straight line because of the lattice arrangement of pixels. Three-digit numbers such as “234” and “678” are respectively in the three directions “2” “3” “4” and “6” “7” “8”. It shows that the pixel value is “1”. Note that “9” indicates the center pixel, and the value of the center pixel is always “1”.

  As shown in FIG. 15, the linear structuring element is twice as long as the radius R of the circular structuring element, has a pixel width of 1 pixel, and is arranged at least in the vicinity of straight lines with an interval of 22.5 degrees, and has eight directions. Is defined. Actually, it is preferable that the continuous pixels to which the pixel value “1” is given in each direction be a straight line, but it is not always a straight line when the total number of pixels of the linear structural element is small. Therefore, in the example of FIG. 15, the direction indicated by “2”, “4”, “6”, and “8” is not a straight line, but the pixel having the pixel value “1” is in the vicinity of a straight line having an interval of 22.5 degrees. Will be placed.

  The actually used linear structural element is obtained by adding a condition of a radius of 7 pixels from the center. That is, the logical product (AND) of the linear structural element shown in FIG. 15 and the circular structural element shown in FIG. 16 is obtained. For example, when the direction d = 1, a binary mask image as shown in FIG. 16 is obtained. When the direction d = 2, a binary mask image as shown in FIG. 17 is obtained.

  The linear structural elements shown in FIG. 15 are M (d, u, v) = {0, 1} (d = 1,..., 8; u = −R,..., 0,. V = −R,..., 0,. Note that R indicates a valid radius, and R = 7 in the example of FIG. U represents the horizontal axis in FIG. 15, and v represents the vertical axis in FIG. The linear component emphasizing means 32 uses the linear structural element M (d, u, v) to the gray scale fundus image Gray (x, y) obtained by executing the process according to [Formula 9]. On the other hand, processing according to the following [Equation 15] is executed to obtain a contracted image Eray (x, y).

[Formula 15]
Eray (d, x, y) = MIN _{u = −R, R; v = −R, R} [(255−M (d, u, v) × 254) × (Gray (x + u, y + v) +1) −1] ]

  In the above [Expression 15], MIN indicates the minimum value, and the subscript “u = −R, R; v = −R, R” of MIN indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Gray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the minimum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 15, calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the linear structure element, and the minimum value is given as Eray (x, y).

  When the contracted image Eray (d, x, y) is obtained by executing the processing according to [Equation 15], this Eray (d, x, y) is replaced with Gray (x, y), and the above [ It is also possible to repeatedly execute the processing according to Equation 15]. That is, similar to the contraction process in step S711 of the first method, it may be repeatedly executed. The number of repetitions can be set as appropriate. However, it is normally set once and not repeated in order to suppress image quality deterioration. By performing for each of the eight directions d, eight kinds of contracted images Eray (d, x, y) of d = 1 to 8 are obtained.

  After the contraction process is repeatedly executed and eight kinds of contracted images Eray (d, x, y) are obtained, the linear component emphasizing means 32 then uses the linear structural element used for the contraction process as the designated structural element. The expansion process is performed. Specifically, using the linear structural element M (d, u, v), the contracted image Eray (d, x, y) is subjected to processing according to the following [Equation 16], and after expansion Image Fdray (d, x, y) is obtained.

[Formula 16]
Fdray (d, x, y) = MAX _{u = −R, R; v = −R, R} [M (d, u, v) × Edray (d, x + u, y + v)]

  In the above [Expression 16], MAX indicates the maximum value, and the subscript “u = −R, R; v = −R, R” of MAX indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Edward (d, x, y) of “x = R,..., Xs−R−1; y = R,. R, R; v = −R, R ″ is converted into the maximum pixel value of M (d, u, v) = 1 in the adjacent pixels. Therefore, in the example of FIG. 15, the calculation is performed using 225 pixels as (2R + 1) × (2R + 1) pixels of the linear structure element, and the maximum value is given as Fdray (d, x, y).

  When the expanded image Fdray (d, x, y) is obtained by executing the processing according to the above [Equation 16], this Fdray (d, x, y) is replaced with Edray (d, x, y). Thus, the processing according to the above [Equation 16] can be repeatedly executed. That is, similar to the expansion process in step S712, it may be repeatedly executed. The number of repetitions needs to be the same as the number of repetitions of the contraction process. Normally, the contraction process is set to one and no repetition is performed. By performing for each of the eight directions d, eight types of expanded images Fdray (d, x, y) of d = 1 to 8 are obtained. The image obtained as a result of the contraction and expansion is obtained as a linear opening image Fdray (d, x, y).

  Next, the linear component emphasizing means 32 performs a process of creating a maximum value image of the eight types of linear opening images (step S752). Specifically, processing according to the following [Equation 17] is executed to obtain a maximum value image Lray (x, y).

[Formula 17]
Lray (x, y) = MAX _{d = 1,8} Fdray (d, x, y)

  In the above [Equation 17], MAX indicates that it takes the maximum value, and the subscript “d = 1, 8” of MAX is in all eight types of linear opening images Fdray (d, x, y). It shows performing an operation. That is, for each pixel (x, y), a process of acquiring the maximum value of eight types of linear opening images Fdray (d, x, y) is performed. As a result, a representative linear opening image Lray (x, y) that is an image of the maximum value is obtained.

  Next, the linear component emphasizing unit 32 performs an opening process using a circular structuring element on the representative linear opening image Lray (x, y) obtained in step S750 (step S760). Specifically, the same processing as step S710 in the first method is performed. First, the linear component emphasizing unit 32 performs contraction processing using a circular structural element that is a designated structural element. Also in the contraction process in step S760, the circular structural element M (u, v) shown in FIG. 12 is used. The linear component emphasizing means 32 performs processing according to the following [Equation 18] on the linear component emphasized image Lray (x, y) using the circular structural element M (u, v), A contracted image ELray (x, y) is obtained.

[Formula 18]
ELray (x, y) = MIN _{u = −R, R; v = −R, R} [(255−M (u, v) × 254) × (Lray (x + u, y + v) +1) −1]

  In the above [Formula 18], MIN indicates that it takes the minimum value, and the subscript “u = −R, R; v = −R, R” of MIN indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel Lray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the minimum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 12, calculation is performed using 225 pixels as (2R + 1) × (2R + 1) pixels of the circular structure element, and the minimum value is given as ELray (x, y).

  When the contracted image ELray (x, y) is obtained by executing the processing according to the above [Equation 18], this ELray (x, y) is replaced with Lray (x, y), and the above [Equation 18] is obtained. It is also possible to repeatedly execute the process. That is, the contraction process may be repeatedly executed. The number of repetitions can be set as appropriate. However, in general, it is set to once to suppress image quality deterioration and is not repeated.

  When the contraction process is repeatedly executed and the contraction image ELray (x, y) is obtained, the linear component emphasizing unit 32 then performs the expansion process using the circular structural element that is the designated structural element. Also in the expansion process in step S760, the circular structural element M (u, v) shown in FIG. 12 used for the contraction process is used. The linear component emphasizing means 32 performs processing according to the following [Equation 19] on the contracted image ELray (x, y) using the circular structural element M (u, v), and after expansion, An image FLray (x, y) is obtained.

[Formula 19]
FLray (x, y) = MAX _{u = −R, R; v = −R, R} [M (u, v) × ELray (x + u, y + v)]

  In the above [Equation 19], MAX indicates that it takes the maximum value, and the subscript “u = −R, R; v = −R, R” of MAX indicates that u and v are from −R to R. It shows performing an operation on a range. That is, for each pixel ELray (x, y) of “x = R,..., Xs−R−1; y = R,. R: Conversion is performed to the maximum pixel value of M (u, v) = 1 in the adjacent pixels of v = −R, R ″. Therefore, in the example of FIG. 12, the calculation is performed using 225 pixels as the (2R + 1) × (2R + 1) pixels of the circular structure element, and the maximum value is given as Fray (x, y).

  When the expanded image FLray (x, y) is obtained by executing the processing according to the above [Equation 19], this FLray (x, y) is replaced with ELray (x, y), and the above [Equation 19 ] Can be repeatedly executed. That is, the expansion process may be repeatedly executed. The number of repetitions needs to be set to the same number as the number of repetitions of the contraction process, but normally the contraction process is set to one and is not repeated.

  An expanded image FLray (x, y) obtained by repeatedly executing the contraction process a predetermined number of times becomes a circular opening image from which fine granular noise has been deleted. If this circular opening image is obtained, then the linear component enhancement means 32 performs a subtraction process from the representative linear opening image (step S770). Specifically, the linear component emphasizing unit 32 executes processing according to the following [Equation 20] using the representative linear opening image Lray (x, y) and the circular opening image FLray (x, y). Then, a linear component emphasized image Tray (x, y) is obtained.

[Formula 20]
Ray (x, y) = Lray (x, y) −FLray (x, y) +128

  As shown in the above [Equation 20], the linear component enhanced image Ray (x, y) is represented by each pixel (x = 0) of the representative linear opening image Lray (x, y) and the circular opening image FLray (x, y). ,..., Xs-1; y = 0,..., Ys-1), a difference calculation is performed, and 128 is added. The integer value to which the offset is added is a half value of the gradation value 256 of the image so that it falls within the pixel value range of 0 to 255. If the difference calculation is simply performed, the pixel values are concentrated in the vicinity of 0, and some pixels become negative values. Therefore, an offset is added so that the pixel value falls within the range of 0 to 255.

  In the second method, since processing is performed on eight types of linear opening images according to eight types of directions, extraction of granular components can be suppressed evenly in each direction changed at an equal interval of 22.5 degrees. . Since the number of images to be created is only eight according to the direction, it is possible to reduce the load of the arithmetic processing.

<3.2. Blood vessel enhancement case>
A blood vessel enhancement example will be described with reference to FIGS. FIG. 18 is a diagram illustrating a blood vessel enhancement example in which the linear component enhancement of the first technique is performed in the blood vessel enhancement image conversion unit 30. FIG. 18A is a color fundus image obtained by photographing. FIG. 18B is a grayscale fundus image Gray obtained by converting the color fundus image shown in FIG. 18A by the grayscale conversion unit 31. FIG. 18C is a linear component enhanced image Ray obtained by converting the grayscale fundus image Gray shown in FIG. 18B by the linear component enhancing means 32 (steps S710 and S720). FIG. 18D is a blood vessel emphasized image Uray obtained by converting the linear component emphasized image Ray shown in FIG. In the blood vessel emphasized image Uray shown in FIG. 18 (d), blood vessels are appropriately extracted, but residual particulate components are conspicuous.

  FIG. 19 is a diagram illustrating a blood vessel enhancement example in which the linear component enhancement of the second technique is performed in the blood vessel enhancement image conversion unit 30. FIG. 19A is the same as FIG. 18A and shows a color fundus image obtained by photographing. FIG. 19B is a grayscale fundus image Gray obtained by converting the color fundus image shown in FIG. 19A by the grayscale conversion unit 31, and is the same as FIG. 18B. FIG. 19C is a linear component enhanced image Ray obtained by converting the grayscale fundus image Gray shown in FIG. 19B by the linear component enhancing means 32 (steps S750 to S770). FIG. 19D is a blood vessel emphasized image Uray obtained by converting the linear component emphasized image Ray shown in FIG. In the blood vessel emphasized image Uray shown in FIG. 19D, it can be seen that blood vessels are appropriately extracted, and the remaining granular components are reduced as compared with FIG.

<4. Example of consistent processing>
Examples of matching processing will be described with reference to FIGS. 20 and 21 are diagrams showing examples of matching processing according to the prior art shown as Patent Document 1. FIG. FIG. 20 shows before alignment processing, and FIG. 21 shows after alignment processing. 20 and 21, the left image is taken in 2011, and the right image is taken in 1985. In the example of FIGS. 20 and 21, the color of the blood vessel portion of both images in the overlay display in FIG. 20 is clearly separated into red and green by performing alignment processing, that is, correction of magnification, angle, and position. On the other hand, in FIG. 21, it can be seen that most of the blood vessel portion changes to yellow, which is a mixed color of red and green, and the positional deviation of the blood vessel portion is reduced. However, if the left end portion of the overlay display in FIG. 21 is observed well, it can be seen that the blood vessels separated into red and green partially remain. However, since the overlay image is unclear overall, It is difficult to analyze clearly whether there is a position shift, and it is difficult to analyze quantitatively. This indicates that the matching process cannot be performed with high accuracy because the matching degree cannot be clearly presented visually in the conventional process.

  FIGS. 22 and 23 are diagrams showing an example in which the alignment processing according to the present invention is performed using the same fundus image used in FIGS. 20 and 21. FIG. 22 shows before the alignment process, and FIG. 23 shows after the alignment process. 22 and 23, the left image is taken in 2011, and the right image is taken in 1985, as in FIGS. In the present invention, since the alignment process is performed after the blood vessel emphasis process is performed, the difference between FIG. 22 before the alignment process and FIG. 23 after the alignment process is clear. As shown in FIG. By doing so, it can be seen that the color of the overlapped portion of both images in the overlay display is clearly changed to yellow, which is a mixed color of red and green, and the positional deviation is remarkably reduced. In the left end portion of the overlay display, which is unclear in FIG. 21, the blood vessels separated into red and green can be clearly identified, and the state of the blood vessel running partially changes with the age of about 26 years. It is also possible to quantitatively calculate the proportion of blood vessel regions that are clearly understood and matched among all the blood vessels that are depicted. This improves alignment accuracy by performing matching processing based on the blood vessel candidate region, and quantitatively grasps the degree of inconsistency after the matching processing, and changes in vascular running and generation of new blood vessels over time. It shows that it can be analyzed.

<5. Modified example>
The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above embodiment, eight directions are set as d = 1... 8 in the linear structural element, but the linear structural element shown in FIG. 15 is reduced to 7 directions or less or expanded to 9 directions or more, and is coarser. Alternatively, a linear opening image may be created with fine angle steps.

  In the above embodiment, a full-color image is used as a fundus image obtained by photographing. However, in recent years, the gradation of a commercial digital camera has been expanded to 10 bits or more, and a full-color image such as 1024 gradations for each color is used. It is also possible to obtain a color image having a larger number of gradations.

  In the above-described embodiment, the consistency index is calculated by the consistency index calculation unit 50 using the correction parameter at the time when the blood vessel enhancement image matching unit 40 determines pass, but the calculated consistency is calculated. When the index satisfies a predetermined condition, it may be automatically determined as acceptable.

1 ... CPU (Central Processing Unit)
2 ... RAM (Random Access Memory)
3 ... Storage device 4 ... Key input I / F
5. Data input / output I / F
DESCRIPTION OF SYMBOLS 6 ... Display part 7 ... Image scanner 10 ... Image digitization means 20 ... Fundus image pre-processing means 30 ... Blood vessel emphasis image conversion means 31 ... Gray scale conversion means 32 ... Linear component emphasis means 33 ... Pixel gradation conversion means 40 ... Blood vessel emphasis image alignment means 50 ... Consistency index calculation means 100 ... Fundus image processing apparatus

Claims (10)

A blood vessel-enhanced image conversion means for emphasizing a blood vessel candidate region and creating a blood vessel-enhanced image after converting each of a plurality of fundus images into a grayscale format;
One of the plurality of blood vessel emphasized images is set as a reference blood vessel emphasized image, and each of the blood vessel emphasized images other than the reference blood vessel emphasized image is set as a corrected blood vessel emphasized image. A blood vessel-enhanced image matching means for performing correction for matching the blood vessel-enhanced image with the magnification, angle and position;
Consistency index calculation means for calculating a consistency index between each corrected blood vessel emphasized image after correction by the blood vessel emphasized image matching means and the reference blood vessel emphasized image;
A fundus image processing apparatus comprising: Image digitizing means for digitizing a fundus image recorded on a photographic recording medium among a plurality of fundus images;
A range including the fundus area corresponding to the fundus is extracted from the digitized fundus image, and a scaling process is performed at the same vertical and horizontal magnification so as to obtain a predetermined number of pixels. Fundus image preprocessing means for converting into a fundus image with a uniform number of pixels while deleting or adding pixels;
The fundus image processing apparatus according to claim 1, wherein the blood vessel emphasized image conversion unit performs processing on the fundus image after the number of pixels is unified.   The fundus image pre-processing means recognizes a nipple region whose luminance is a region larger than a predetermined value in the fundus region, and when the nipple region is located above or below the fundus region, the fundus image is Rotating the fundus image by a predetermined angle unit so that the left eye corresponds to the left eye, and the right eye when the fundus image corresponds to the right eye. The fundus image processing apparatus according to claim 2.   The fundus image pre-processing means makes the number of pixels H in the height direction of the rectangular region circumscribing the fundus region equal to a predetermined number of pixels Sy, and also with respect to the number of pixels W in the width direction of the rectangular region. When scaling processing is performed so that W × Sy / H, and the width W ′ of the rectangular area after scaling processing is W ′ <Sx with respect to a predetermined number of pixels Sx, the width of the rectangular area is Background pixels are added to both ends of the rectangular area so as to be Sx, and when W ′> Sx, background pixels are deleted from both ends of the rectangular area so that the width of the rectangular area is Sx. The fundus image processing apparatus according to claim 2, wherein the fundus image processing apparatus is provided. The blood vessel emphasized image converting means includes
A gray scale conversion unit that performs a predetermined calculation on each color component for a fundus image in a color format and converts the pixel value to be smaller as the first color component is larger and converts the pixel value to a gray scale format;
Linear component enhancement means for performing an opening process on the converted fundus image in the gray scale format using a predetermined structural element to create a linear component enhanced image in which the luminance of the blood vessel candidate region is equal to or higher than a certain level; ,
In the linear component emphasized image, the pixel value satisfying the predetermined condition is replaced with a value near the minimum value of the gradation of the converted blood vessel emphasized image, and the pixel value satisfying the predetermined condition is greater than or equal to the pixel value. Pixel gradation conversion means for correcting the gradation of the pixel so that the pixel value of the linear component enhanced image is in the range from the value near the minimum value of the gradation to the maximum value, and creating a blood vessel enhanced image;
The fundus image processing apparatus according to any one of claims 1 to 4, further comprising:   The blood vessel-enhanced image matching means assigns one of the three color components of RGB to the reference blood vessel-enhanced image, and assigns a color component different from the color component assigned to the reference blood-vessel-enhanced image to the corrected blood vessel enhancement 6. A color composite image is created by combining the reference blood vessel emphasized image assigned to an image and artificially colored with each color component and the corrected blood vessel emphasized image. The fundus image processing apparatus according to claim 1.   The consistency index calculating means sets a sum C1 and C2 of pixel values of pixels having a pixel value exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image, and the reference blood vessel emphasized image and the correction blood vessel are corrected. When the sum of the geometric mean values of the pixel values having the pixel value exceeding the threshold value at the same position in both the blood vessel emphasized images is C3, a value proportional to C3 / C1 is obtained if C1 ≦ C2. The fundus image processing apparatus according to any one of claims 1 to 6, wherein a value proportional to C3 / C2 is calculated as a consistency index if C1> C2.   The consistency index calculating means sets C1 ′ and C2 ′ as the number of pixels having pixel values exceeding a predetermined threshold in the reference blood vessel emphasized image and the corrected blood vessel emphasized image, and the reference blood vessel emphasized image and the correction blood vessel correction image. When the number of pixels having a pixel value exceeding the threshold value at the same position in both blood vessel emphasized images is C3 ′, if C1 ′ ≦ C2 ′, a value proportional to C3 ′ / C1 ′ is set as C1 ′> The fundus image processing apparatus according to any one of claims 1 to 7, wherein if C2 ', a value proportional to C3' / C2 'is calculated as a consistency index.   When the number of pixels having a pixel value exceeding a predetermined threshold value is C1a and C2a in the reference blood vessel emphasized image and the corrected blood vessel emphasized image, the consistency index calculation means is the reference blood vessel if C1a ≦ C2a. The correlation coefficient between the C1a pixel group having a pixel value exceeding the threshold in the blood vessel enhanced image and the C1a pixel group in the corrected blood vessel enhanced image is calculated as a consistency index, and if C1a> C2a, For example, a correlation coefficient between C2a pixel groups in the reference blood vessel enhancement image corresponding to C2a pixel groups having a pixel value exceeding the threshold in the corrected blood vessel enhancement image is calculated as a consistency index. The fundus image processing apparatus according to any one of claims 1 to 8, wherein:   A program for causing a computer to function as the fundus image processing apparatus according to any one of claims 1 to 9.