WO2019207800A1

WO2019207800A1 - Ophthalmic image processing device and ophthalmic image processing program

Info

Publication number: WO2019207800A1
Application number: PCT/JP2018/017327
Authority: WO
Inventors: 友洋宮城; 幸弘樋口; 徹哉加納; 壮平宮崎; 涼介柴
Original assignee: 株式会社ニデック
Priority date: 2018-04-27
Filing date: 2018-04-27
Publication date: 2019-10-31
Also published as: JPWO2019207800A1; JP7196908B2

Abstract

An ophthalmic image processing device according to the present invention processes an ophthalmic image of a subject eye. A control unit of the ophthalmic image processing device acquires an ophthalmic image captured by an ophthalmic image imaging unit. The control unit acquires an automatic diagnosis result for each of a plurality of diseases in the subject eye by inputting the ophthalmic image to a mathematical model learned by a machine learning algorithm. On the basis of the plurality of automatic diagnosis results for the ophthalmic image, the control unit generates reference information indicating, in stages, the degree to which at least any of the plurality of diseases is present in the subject eye.

Description

Ophthalmic image processing apparatus and ophthalmic image processing program

The present disclosure relates to an ophthalmic image processing apparatus that processes an ophthalmic image that is an image of an eye to be examined, and an ophthalmic image processing program that is executed in the ophthalmic image processing apparatus.

Conventionally, various diagnoses for an eye to be examined are performed based on ophthalmic images obtained by an ophthalmologic photographing apparatus (for example, an optical coherence tomography (OCT), a fundus camera, a laser scanning optometry apparatus (SLO), or the like). .

JP2015-104581A

One aspect of the technology in this disclosure will be described. The inventor of the present invention makes an attempt to automatically output diagnostic results for each of a plurality of diseases to an ophthalmic image processing apparatus based on an ophthalmic image. Here, since the progress of disease and the like vary depending on the eye to be examined, it is difficult to accurately output all automatic diagnosis results for each disease. Therefore, when a plurality of automatic diagnosis results for each of a plurality of diseases are output, a user (for example, a doctor or the like) determines whether or not to perform a detailed diagnosis of the eye to be examined based on the plurality of automatic diagnosis results. Cannot be done easily.

Explain other aspects. Mathematical model trained by training data set (input training data and training data for output) when automatic diagnosis result of eye to be examined is obtained by inputting ophthalmic image to mathematical model trained by machine learning algorithm Is used. Here, unless an appropriate ophthalmic image is input to the mathematical model, it is difficult to improve the accuracy of the automatic diagnosis, and thus it is difficult to generate useful information.

The present disclosure is to provide an ophthalmic image processing apparatus and an ophthalmic image processing program capable of solving at least one of the plurality of aspects and generating useful information based on an ophthalmic image.

A first aspect of an ophthalmic image processing apparatus provided by an exemplary embodiment of the present disclosure is an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined, and the control unit of the ophthalmic image processing apparatus is an ophthalmologic image capturing unit. Acquiring the ophthalmic image captured by a unit, and inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm to obtain an automatic diagnosis result for each of a plurality of diseases in the eye to be examined, Reference information that indicates in stages the degree to which at least one of the diseases is present in the eye to be examined is generated based on the plurality of automatic diagnosis results for the ophthalmologic image.

A second aspect of an ophthalmic image processing apparatus provided by a typical embodiment of the present disclosure is an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined, and acquires the ophthalmic image captured by an ophthalmic image capturing unit. Image acquisition means, automatic diagnosis result acquisition means for acquiring an automatic diagnosis result for at least one disease in the eye to be examined by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm, and the image When the range of the ophthalmic image acquired by the acquisition unit is wider than the target range of the ophthalmic image input to the mathematical model, the image of the target range from the ophthalmic image acquired by the image acquisition unit is Image extracting means for extracting as the ophthalmic image to be input to the mathematical model.

A first aspect of an ophthalmic image processing program provided by an exemplary embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined. Is executed by the control unit of the ophthalmic image processing apparatus, and the ophthalmic image is input to an image acquisition step of acquiring the ophthalmic image captured by the ophthalmic image capturing unit and a mathematical model trained by a machine learning algorithm. An automatic diagnosis result acquisition step for acquiring an automatic diagnosis result for each of a plurality of diseases in the eye to be examined, and reference information that indicates in steps the degree to which at least one of the plurality of diseases is present in the eye to be examined Generating a reference information based on the plurality of automatic diagnosis results for the ophthalmic image To execute the ophthalmologic image processing apparatus.

A second aspect of the ophthalmic image processing program provided by the exemplary embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined. Is executed by the control unit of the ophthalmic image processing apparatus, and the ophthalmic image is input to an image acquisition step of acquiring the ophthalmic image captured by the ophthalmic image capturing unit and a mathematical model trained by a machine learning algorithm. An automatic diagnostic result acquisition step for acquiring an automatic diagnostic result for at least any disease in the eye to be examined, and an ophthalmic image in which the range of the ophthalmic image acquired in the image acquisition step is input to the mathematical model Before being acquired in the image acquisition step when it is wider than the target range of An image of the target range of ophthalmic images, to execute an image extraction step of extracting as the ophthalmologic image to be input to the mathematical model, to the ophthalmologic image processing apparatus.

According to the ophthalmic image processing apparatus and the ophthalmic image processing program according to the present disclosure, useful information based on the ophthalmic image is generated.

1 is a block diagram showing a schematic configuration of an ophthalmic image processing system 100. FIG. It is a figure which shows the imaging |

photography position

35H, 35V of the front image 30 and the two-dimensional tomographic image of the fundus of the eye to be examined. It is a flowchart of a reference information generation process. It is a figure which shows an example of the automatic diagnosis result in this embodiment. It is a figure which shows an example of the corresponding | compatible reference information setting screen. It is a figure which shows an example of 40 A of patient data display screens when the newest reference information is matched with patient data. It is a figure which shows an example of the display screen 40B of patient data when high-stage reference information is matched with patient data. It is a figure which shows an example of the display screen 50 of a two-dimensional tomographic image. It is a figure which shows an example of the list display screen 60 which displays the whole list of patient data. It is a figure which shows an example of the list display screen 60 which displays the list of the patient data corresponding to the reference information of the selected step. It is a flowchart of a patient data output process.

<Overview>
The control unit of the ophthalmic image processing apparatus exemplified in the present disclosure acquires the ophthalmic image captured by the ophthalmic image capturing unit. The control unit obtains an automatic diagnosis result for each of a plurality of diseases in the eye to be examined by inputting an ophthalmologic image into a mathematical model trained by a machine learning algorithm. The control unit generates reference information, which is information indicating in a stepwise manner the degree to which at least one of a plurality of diseases is present in the eye to be examined, based on a plurality of automatic diagnosis results for the ophthalmologic image.
In this case, even if the user does not grasp each of the plurality of automatic diagnosis results, the determination as to whether or not to make a detailed diagnosis of the eye to be examined is easily made based on the reference information. Therefore, according to the ophthalmic image processing apparatus in the present disclosure, useful information based on the ophthalmic image is appropriately generated.

Note that the number of stages of reference information to be generated is preferably 5 or less, and more preferably 3 or less. By setting the number of stages to 5 stages or less or 3 stages or less, the user can more easily determine whether or not to make a detailed diagnosis. However, the number of stages can be six or more. For example, the control unit may generate the reference information in 101 steps from 0% to 100%.

Mathematical models trained by a machine learning algorithm (hereinafter sometimes simply referred to as “mathematical models”) may be configured to output the presence probability of each disease as an automatic diagnosis result. The control unit may generate reference information based on the existence probability of each disease. In this case, the degree to which at least one of a plurality of diseases exists is more accurately reflected in the reference information.

The ophthalmic image (that is, an ophthalmic image used for automatic diagnosis) input to the mathematical model may be an ophthalmic image acquired by an OCT apparatus. An ophthalmologic image (for example, a two-dimensional tomographic image, a three-dimensional tomographic image, etc.) acquired by the OCT apparatus includes deep information in addition to information on the surface of the tissue. Therefore, the ophthalmologic image acquired by the OCT apparatus is used for automatic diagnosis, thereby further improving the accuracy of automatic diagnosis. However, an ophthalmologic image taken by an apparatus other than the OCT apparatus (for example, at least one of a fundus camera, a laser scanning optometry apparatus (SLO), etc.) may be used for automatic diagnosis.

The control unit may input an instruction from a user who selects at least one of a plurality of stages of reference information. The control unit may extract patient data corresponding to reference information at a stage selected by an input instruction from patient data including ophthalmologic images of a plurality of subjects or eyes. In this case, a plurality of patient data are appropriately managed according to the stage of the generated reference information.

The control unit may display a list of patient data extracted according to the reference information on the display unit. In this case, the user can appropriately manage a plurality of patient data by looking at the list displayed on the display unit.

In addition, it is possible to change the display method of a list of a plurality of patient data. For example, the control unit may rearrange a plurality of patient data according to the reference information (that is, for each stage of the reference information), and display the rearranged list on the display unit. In this case, the user can manage a plurality of patient data according to the degree of presence of the disease.

The control unit may transmit patient data corresponding to reference information at a specific stage among patient data including ophthalmic images to another device via a network. In this case, patient data is appropriately processed according to the stage of the generated reference information. For example, an ophthalmic image processing apparatus according to the present disclosure may be installed in a health examination facility, and a doctor who performs a detailed diagnosis may be using the device in another facility. In this case, the ophthalmologic image processing apparatus transmits patient data corresponding to reference information at a specific stage (for example, a stage where the degree of presence of disease is the highest) to a device used by a doctor via a network. Also good. As a result, patient data that is highly necessary for detailed diagnosis is appropriately transmitted to the device used by the doctor.

Also, the control unit may process patient data by a method different from a method of transmitting patient data to other devices. For example, the control unit may create a report of patient data corresponding to reference information at a specific stage among one or a plurality of patient data. The report may include, for example, at least one of various patient data (patient name, etc.), ophthalmic images, ophthalmic image analysis results, reference information, and the like. The report may be created with data in a specific format (for example, PDF data), or may be created by printing on paper. In this case, a report is appropriately created according to the stage of the generated reference information.

Note that the “specific stage” may be set according to, for example, an instruction (operation instruction or the like) input to the ophthalmic image processing apparatus by the user, or may be set in advance. The control unit may transmit corresponding patient data to another device each time reference information of a specific stage is generated. Further, the control unit may extract patient data corresponding to reference information at a specific stage from a plurality of patient data, and transmit the extracted one or more patient data to another device. Needless to say, the “specific stage” is not limited to one stage and may be a plurality of stages. Further, the control unit may cause the display unit to display details (for example, past history) of patient data corresponding to the reference information at a specific stage.

In addition, when only the reference information of one of the left eye and right eye patient data of the same subject is in a specific stage, the control unit outputs the patient data of the other eye (for example, other data At least one of transmission to a device, report generation, display on a display unit, etc.) may also be performed. For example, when glaucoma develops in one eye, it tends to develop in the other eye. Therefore, when the reference information for one eye is in a specific stage, the user can also output the patient data of the other eye (for example, the eye with the lower degree of disease), so that the user can The other eye can be diagnosed based on the result of the eye diagnosis. Therefore, diagnosis is performed more appropriately.

In addition, when only the reference information of one of the left-eye and right-eye patient data of the same subject is in a specific stage, the control unit stores the eye patient data corresponding to the reference information in the specific stage. Depending on the type of disease, it may be determined whether to output only patient data or to output patient data for both eyes. Further, the control unit may determine whether or not to output both-eye patient data according to an instruction input by the user.

The control unit is different from the eye corresponding to the reference information at a specific stage when only the reference information of one of the patient data of the left eye and the right eye of the same subject is at a specific stage. For the eyes, a notification that prompts the user to pay attention (for example, follow-up observation) may be given. This notification may be performed according to the type of disease, and it may be determined whether to perform notification according to the setting by the user.

Further, the control unit stores data corresponding to reference information at a specific stage (at least one of patient data and data of each ophthalmic image) and data corresponding to reference information other than the specific stage. You may change the storage method. For example, when the data corresponds to reference information other than a specific stage, the control unit may omit the process of saving the data in the storage device, delete the data from the storage device, Data amount reduction processing (for example, conversion processing to a report file with a small data amount, etc.) may be executed. In this case, it is suppressed that the capacity of the storage device is compressed, and data is appropriately managed.

For each subject or eye to be examined, patient data including one or more ophthalmic image data may be stored in the storage means. The control unit may generate reference information for each of the plurality of ophthalmic images when one patient data includes a plurality of ophthalmic images. When a plurality of ophthalmic images are included in one patient data, the control unit generates reference information (hereinafter referred to as “latest reference information”) generated for an ophthalmic image having the latest timing. Any of reference information at the stage where the degree of disease is the highest (hereinafter referred to as “high-stage reference information”) may be associated with each patient data. In this case, appropriate reference information corresponding to the situation is associated with each patient data. For example, when the latest reference information is associated with the patient data, the patient data is appropriately managed based on the latest state of the eye to be examined. When the high-level reference information is associated with the patient data, the patient data is managed in consideration of the degree of the presence of the disease in the past ophthalmic image.

Note that the control unit may set which of the latest reference information and the high-level reference information is associated with the patient data in accordance with a selection instruction input by the user. In this case, the user can associate appropriate reference information with the patient data in accordance with his / her diagnosis policy or the like. Also, which of the latest reference information and the high-level reference information is associated with the patient data may be automatically set according to various conditions.

In addition, when extracting the patient data corresponding to the reference information at the stage selected by the input instruction from the plurality of patient data, the control unit extracts the latest reference information associated with each patient data or the high-level reference information. Patient data may be extracted with reference to reference information. In addition, when the latest reference information or high-level reference information associated with the patient data is in a specific stage, the control unit may transmit the patient data to another device, create a report, or the like. .

The control unit may cause the display unit to display reference information corresponding to each of a plurality of ophthalmologic images included in the patient data when displaying the patient data on the display unit. In this case, the user can appropriately grasp the reference information generated for each ophthalmic image. For example, when a plurality of ophthalmologic images photographed at different times are included in one patient data, the user can appropriately perform follow-up observation of the eye to be examined in consideration of each reference information. it can.

The control unit may input an instruction from the user for selecting whether to display the reference information on the display unit. The control unit may switch between displaying and hiding reference information on the display unit according to the input instruction. For example, the user may not want to show the reference information to the subject. In addition, experienced users may not want to show reference information to other users in order to educate inexperienced users. In addition, the user may want to make a diagnosis or the like without referring to the reference information. Therefore, the user can perform various operations such as diagnosis more appropriately by switching the display and non-display of the reference information at his / her own will.

The control unit may extract ophthalmic images that differ in the degree of presence of the disease indicated by the generated reference information and the presence or absence of the disease determined by actual diagnosis. Further, the control unit may extract an ophthalmologic image in which the automatic diagnosis result obtained by the mathematical model is different from the actually performed diagnosis result. The extracted ophthalmic image may be provided (for example, transmitted) to the manufacturer of the ophthalmic image processing apparatus. In this case, the manufacturer can improve the accuracy of subsequent automatic diagnosis results or reference information by causing the mathematical model to train the received ophthalmic image as training data.

The control unit may edit the reference information generated for the ophthalmologic image in accordance with an instruction input by the user. In this case, the user can appropriately manage the patient data even if the content of the generated reference information may be different from the actual diagnosis result. When the edited reference information is displayed on the display unit, the control unit may indicate that the edited reference information is the edited reference information by highlighting or commenting. In addition, the user who can input the reference information editing instruction may be limited by setting. Further, the control unit may edit the automatic diagnosis result obtained by the mathematical model in accordance with an instruction input by the user.

When the range of the acquired ophthalmic image (that is, the ophthalmic image captured by the ophthalmic image capturing unit) is wider than the target range of the ophthalmic image to be input to the mathematical model, the control unit performs a target from the acquired ophthalmic image. An automatic diagnosis result may be acquired by extracting a range image and inputting the extracted ophthalmic image into a mathematical model. In this case, a highly accurate automatic diagnosis result is appropriately acquired regardless of the range of the ophthalmic image captured by the ophthalmic image capturing unit.

Note that a specific method for extracting the target range from the acquired ophthalmic image can be selected as appropriate. For example, a predetermined range of a two-dimensional tomographic image (B-scan image) captured by the OCT apparatus is the target range, and the range of the two-dimensional tomographic image actually captured by the OCT apparatus may be wider than the target range. . In this case, the control unit may extract a predetermined range from the captured two-dimensional tomographic image. In addition, a predetermined range of the two-dimensional tomographic image captured by the OCT apparatus is a target range, and an image actually captured by the OCT apparatus is captured by scanning each of a plurality of different scanning lines with measurement light. In some cases, it is a map image (three-dimensional OCT data). In this case, the control unit may extract a two-dimensional tomographic image to be automatically diagnosed from the captured map image. Further, the target range is a map image of a predetermined range imaged by the OCT apparatus, and the map image actually imaged by the OCT apparatus may be wider than the predetermined range. In this case, the control unit may extract a target range from a wide range of captured map images. Further, the target range is a predetermined range of the front image of the tissue (for example, the fundus) of the eye to be examined, and the range of the actually captured front image may be wider than the target range. In this case, the control unit may extract the target range from the photographed front image.

Further, the control unit may automatically extract the target range from the acquired ophthalmic image. In this case, the control unit may perform image processing on the acquired ophthalmologic image and extract a target range based on the result of the image processing. For example, the control unit may perform image processing on the acquired ophthalmologic image, detect a reference position (for example, the position of the macula on the fundus), and extract a target range based on the reference position. Further, when the ophthalmologic image is taken by the OCT apparatus, the control unit may specify the position irradiated with the measurement light in the tissue of the eye to be examined, and extract the target range based on the specified position. The control unit may extract the target range based on an instruction input by the user. That is, the user may manually specify the target range.

Note that the control unit extracts at least a target range image from the acquired ophthalmic image, and executes the process of inputting the extracted ophthalmic image into the mathematical model, at least of the plurality of other processes exemplified in the present disclosure. Any one (for example, processing for generating reference information) may be omitted. In addition, when an automatic diagnosis result is acquired by extracting an image of the target range, the automatic diagnosis result may be an automatic diagnosis result for each of a plurality of diseases in the eye to be examined, or any disease in the eye to be examined It may be an automatic diagnosis result for.

<Embodiment>
(System configuration)
Hereinafter, one exemplary embodiment of the present disclosure will be described with reference to the drawings. As an example, in this embodiment, a personal computer (hereinafter referred to as “PC”) 1 acquires ophthalmic image data of an eye to be examined (hereinafter simply referred to as “ophthalmic image”) from the ophthalmic image capturing apparatus 11 and acquires the acquired data. Various processes are performed on the ophthalmologic image. That is, in this embodiment, the PC 1 functions as an ophthalmic image processing apparatus. However, the device that functions as an ophthalmic image processing apparatus is not limited to the PC 1. For example, the ophthalmic image capturing device 11 may function as an ophthalmic image processing device. A portable terminal such as a tablet terminal or a smartphone may function as an ophthalmic image processing apparatus. In addition, the control units of a plurality of devices (for example, the CPU 3 of the PC 1 and the CPU 13 of the ophthalmologic image capturing apparatus 11) may perform various image processing in cooperation.

As shown in FIG. 1, an ophthalmic image processing system 100 exemplified in this embodiment includes a plurality of PCs used at each base (for example, a health check facility, a hospital, etc.). FIG. 1 illustrates a PC 1 used at a base A that is a health check facility and a PC 21 used at a base B that is a hospital.

The PC 1 is used by a user at the site A (for example, a laboratory technician and a doctor). The PC 1 includes a control unit 2 that performs various control processes and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller that controls the memory and a storage device 4 that can store programs, data, and the like. The storage device 4 stores an ophthalmic image processing program for executing ophthalmic image processing described later. In addition, the communication I / F 5 connects the PC 1 to another device (for example, the PC 21) via the network 9 (for example, the Internet).

The PC 1 is connected to the operation unit 7 and the monitor 8. The operation unit 7 is operated by the user in order for the user to input various instructions to the PC 1. For the operation unit 7, for example, at least one of a keyboard, a mouse, a touch panel, and the like can be used. A microphone or the like for inputting various instructions may be used together with the operation unit 7 or instead of the operation unit 7. The monitor 8 is an example of a display unit that can display various images.

The PC 1 can exchange various data (for example, ophthalmic image data) with the ophthalmic image capturing apparatus 11. A method in which the PC 1 exchanges data with the ophthalmic image capturing apparatus 11 can be selected as appropriate. For example, the PC 1 may exchange data with the ophthalmic image capturing apparatus 11 through at least one of wired communication, wireless communication, a removable storage medium (for example, a USB memory), and the like.

As the ophthalmologic image photographing device 11, various devices for photographing an image of the eye to be examined can be used. As an example, the ophthalmologic image capturing apparatus 11 used in the present embodiment is an OCT apparatus capable of capturing a tomographic image or the like of the tissue of the eye to be examined (the fundus in the present embodiment). Therefore, since an automatic diagnosis described later is executed based on an OCT image including information on the deep part of the tissue, the accuracy of the automatic diagnosis is improved. However, an apparatus other than the OCT apparatus (for example, at least one of a fundus camera, a laser scanning optometry apparatus (SLO), or the like) may be used. In addition, an image of a tissue other than the fundus of the eye to be examined (for example, the anterior segment) may be captured by the ophthalmologic image capturing device 11.

The ophthalmic image capturing apparatus 11 includes a control unit 12 that performs various control processes and an ophthalmic image capturing unit 16. The control unit 12 includes a CPU 13 that is a controller that controls the memory and a storage device 14 that can store programs, data, and the like.

The ophthalmologic image capturing unit 16 includes various configurations necessary for capturing an ophthalmic image of the eye to be examined. For example, when an OCT apparatus is used as the ophthalmic image capturing apparatus 11, the ophthalmic image capturing section 16 includes an OCT light source, a scanning section for scanning OCT light, an optical system for irradiating the eye to be examined, OCT light, A light receiving element for receiving light reflected by the tissue of the optometer is included. Further, the ophthalmologic image capturing unit 16 of the present embodiment includes a front observation optical system that captures a front image of the tissue of the eye to be examined (the fundus in the present embodiment). The front image is a two-dimensional image when the tissue is viewed from the direction along the optical axis (front direction) of the OCT measurement light. For the configuration of the front observation optical system, for example, at least one of the configurations such as SLO and fundus camera can be adopted. The ophthalmologic imaging apparatus 11 may acquire the three-dimensional OCT data of the tissue, and may acquire an image when viewing the tissue from the front direction (a so-called “Enface image”) based on the three-dimensional OCT data. In this case, the front observation optical system may be omitted.

PC 21 is used by a user at site B. The PC 21 includes a control unit 22 that performs various control processes and a communication I / F 25. The control unit 22 includes a CPU 23 that is a controller that controls the memory and a storage device 24 that can store programs, data, and the like. The communication I / F 25 connects the PC 21 to another device (for example, PC 1) via the network 9. An operation unit 27 and a monitor 28 are connected to the PC 21.

(Ophthalmological imaging)
With reference to FIG. 2, an example of an ophthalmic image capturing method used in the automatic diagnosis (details will be described later) of the present embodiment will be described. FIG. 2 is a diagram showing the relationship between the front image 30 of the fundus of the eye to be examined and the imaging positions (that is, the scanning positions of the OCT measurement light) 35H and 35V of two two-dimensional tomographic images used in automatic diagnosis. In the present embodiment, the front image 30 is captured by a front observation optical system provided in the ophthalmic image capturing unit 16 of the ophthalmic image capturing apparatus 11. The front image 30 shown in FIG. 2 shows fundus tissues such as the optic disc 31, the macula 32, and the fundus blood vessel 33.

As an example, in the present embodiment, a two-dimensional tomographic image photographed at a photographing position 35H extending in the horizontal direction around the reference position (macular 32 in the present embodiment) and a photographing position 35V extending in the vertical direction around the reference position. The two-dimensional tomographic image photographed in is used for automatic diagnosis. Each of the

imaging positions

35H and 35V is 6 mm in length. However, the reference position, the length of the shooting position, the angle of the shooting position, the number of images used for automatic diagnosis, and the like can be changed as appropriate. In addition, ophthalmic images other than the two-dimensional tomographic image (for example, a front image of the fundus, a three-dimensional tomographic image, etc.) may be used for automatic diagnosis. A plurality of types of ophthalmic images of the same eye to be inspected, taken by each of a plurality of modalities, may be used for automatic diagnosis. When multiple types of ophthalmic images are used for automatic diagnosis, multiple types of automatic diagnosis results may be output by different algorithms for each of the plurality of ophthalmic images, or automatically from a plurality of types of ophthalmic images by one algorithm. A diagnosis result may be output.

The CPU 13 of the ophthalmologic image capturing apparatus 11 according to the present embodiment performs guide display of two-dimensional tomographic

image capturing positions

35H and 35V suitable for automatic diagnosis on the fundus front image 30 before tomographic image capturing. . Specifically, the CPU 13 detects the reference position, and shows a guide display of the optimum photographing

positions

35H and 35V centered on the reference position on the front image 30 displayed on the monitor (not shown). As a result, an ophthalmic image suitable for automatic diagnosis is easily taken. The reference position may be detected by performing image processing or the like on the front image 30, or a position designated by the user may be detected as the reference position.

Further, the ophthalmologic image capturing apparatus 11 may automatically capture a two-dimensional tomographic image suitable for automatic diagnosis. In this case, the CPU 13 detects the reference position (the macular 32 in the present embodiment), and scans the OCT measurement light at the

optimal imaging positions

35H and 35V centered on the reference position, thereby capturing an ophthalmic image. Good.

Note that the ophthalmic image capturing apparatus 11 may capture an ophthalmic image in a wider range than an image range used in an automatic diagnosis described later. In this case, in the present embodiment, before automatic diagnosis is performed, an image of the target range for automatic diagnosis is extracted from a wide range of ophthalmic images.

(Ophthalmological image processing)
Hereinafter, the ophthalmic image processing of this embodiment will be described in detail. In the ophthalmic image processing according to the present embodiment, an automatic diagnosis result for at least one of a plurality of diseases in the eye to be examined (each of the plurality of diseases in the present embodiment) is obtained by using a mathematical model trained by a machine learning algorithm. To be acquired. Further, reference information is generated based on the acquired plurality of automatic diagnosis results. Reference information is information that indicates in a stepwise manner the degree to which at least one of a plurality of diseases is present in the eye to be examined. Furthermore, in this embodiment, various processes such as a process of displaying reference information on the monitor 8 and a process of extracting patient data according to the reference information are executed. Processing described below is executed by the CPU 3 in accordance with an ophthalmic image processing program stored in the storage device 4.

(Reference information generation process)
The reference information generation process will be described with reference to FIGS. 3 and 4. In the reference information generation process, reference information is generated for the ophthalmic image. First, the CPU 3 acquires an ophthalmic image of the eye to be examined (S1). In the present embodiment, the CPU 3 acquires the ophthalmic image captured by the ophthalmic image capturing unit 16 of the ophthalmic image capturing apparatus 11 from the ophthalmic image capturing apparatus 11. However, the ophthalmologic image acquisition method can be changed as appropriate. For example, when the ophthalmic image capturing device 11 executes the reference information generation process, the CPU 13 of the ophthalmic image capturing device 11 may acquire an ophthalmic image stored in the storage device 14.

Next, the CPU 3 determines whether or not the range of the ophthalmic image acquired in S1 is wider than the target range of the ophthalmic image input to a mathematical model (details will be described later) used in automatic diagnosis (S2). If the range of the ophthalmologic image acquired in S1 is the same as the target range for automatic diagnosis (S2: NO), the process proceeds to S4 as it is. When the range of the ophthalmic image acquired in S1 is wider than the target range of automatic diagnosis (S2: YES), the CPU 3 extracts an image of the target range of automatic diagnosis from the ophthalmic image acquired in S1 (S3). ). Although details will be described later, in the present embodiment, a predetermined range of two two-dimensional tomographic images (B-scan images) captured by the OCT apparatus is a target range for automatic diagnosis. For example, when the range of the two-dimensional tomographic image acquired in S1 is wider than the target range, the CPU 3 extracts a predetermined target range from the two-dimensional tomographic image acquired in S1. When the image acquired in S1 is a map image (three-dimensional OCT data) taken by scanning each of a plurality of different lines with OCT measurement light, the CPU 3 is an object of automatic diagnosis. Two two-dimensional tomographic images are extracted from the map image.

In addition, CPU3 of this embodiment extracts the image of a target range automatically in S3. Specifically, the CPU 3 performs image processing on the acquired ophthalmologic image, and extracts a target range image by detecting a reference position (a position of the macular in the present embodiment). Further, the CPU 3 may acquire information on the position where the measurement light is irradiated in the tissue of the eye to be examined, and extract an image of the target range based on the acquired position information. Note that the CPU 3 may extract an image of the target range based on an instruction input by the user.

Next, the CPU 3 acquires an automatic diagnosis result for the ophthalmologic image (S4). In the present embodiment, the CPU 3 acquires an automatic diagnosis result for each of a plurality of diseases in the eye to be examined by inputting an ophthalmologic image into a mathematical model trained by a machine learning algorithm (see FIG. 4).

The method for acquiring the automatic diagnosis result in this embodiment will be described in detail. As a machine learning algorithm, for example, a neural network, random forest, boosting, support vector machine (SVM), etc. are generally known.

Neural network is a technique that mimics the behavior of biological nerve cell networks. Neural networks include, for example, feedforward (forward propagation) neural networks, RBF networks (radial basis functions), spiking neural networks, convolutional neural networks, recursive neural networks (recurrent neural networks, feedback neural networks, etc.), probabilities Neural networks (such as Boltzmann machines and Bayesian networks).

Random forest is a method of generating a large number of decision trees by learning based on randomly sampled training data. When a random forest is used, the branch of a plurality of decision trees that have been learned as classifiers in advance is traced, and the average (or majority vote) of the results obtained from each decision tree is taken.

Boosting is a technique for generating a strong classifier by combining a plurality of weak classifiers. A strong classifier is constructed by sequentially learning simple and weak classifiers.

SVM is a method of configuring a two-class pattern classifier using linear input elements. For example, the SVM learns the parameters of the linear input element based on a criterion (hyperplane separation theorem) for obtaining a margin maximizing hyperplane that maximizes the distance to each data point from the training data.

In this embodiment, a multilayer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer. In each layer, a plurality of nodes (also referred to as units) are arranged. Specifically, in this embodiment, a convolutional neural network (CNN), which is a kind of multilayer neural network, is used.

Mathematical model refers to a data structure for predicting the relationship between input data and output data, for example. The mathematical model is constructed by being trained using a training data set. The training data set is a set of training data for input and training data for output. In the training data for input, an ophthalmologic image of the eye to be inspected in the past (in this embodiment, the OCT measurement light was scanned by scanning the imaging positions extending in the horizontal direction and the vertical direction around the macula. Two two-dimensional tomographic images) are used. In order to improve the accuracy of the automatic diagnosis, it is desirable that the target range of the ophthalmologic image used for the automatic diagnosis matches the image range of the training data for input as much as possible. As the output training data, diagnosis result data such as a disease name and a disease position are used. The mathematical model is trained so that when certain input training data is input, output training data corresponding to the input training data is output. For example, the correlation data (for example, weight) of each input and output is updated by training.

CPU 3 inputs an ophthalmic image to the mathematical model. As a result, as shown in FIG. 4, an automatic diagnosis result for each of a plurality of diseases (in the example of FIG. 4, disease A to disease I) is output. In the present embodiment, the probability that each disease exists and the position of the disease (specifically, the position where it is determined that there is a possibility that the disease exists) are output. Note that if an image of the target range is extracted in S3, the CPU 3 inputs the ophthalmic image extracted in S3 to the mathematical model. As a result, the accuracy of automatic diagnosis is improved.

When the position of the disease is detected by automatic diagnosis, the CPU 3 displays the image of the disease when displaying an image (for example, at least one of the front image of the fundus and the two-dimensional tomographic image in the present embodiment). At least one of position display, highlighted display of diseased part, enlarged display of diseased part, etc. is executed. Therefore, the user can easily confirm the position where it is determined that there is a possibility that the disease exists.

Further, when it is determined that the probability that a specific disease is present is equal to or greater than the threshold, the CPU 3 changes the display form of various images on the monitor 8 according to the type of the disease that is highly likely to exist. For example, in the case of macular diseases such as age-related macular degeneration, central serous chorioretinopathy, retinal detachment, an abnormality may be seen in the thickness of the retina. Therefore, when it is determined that there is a high possibility of macular disease, the CPU 3 determines the thickness map indicating the retina thickness distribution when the fundus is viewed from the front, the retinal thickness analysis chart, and the retina of the normal eye. At least one of the comparison image and the like with the thickness is displayed on the monitor 8. In the case of diabetic retinopathy, abnormalities may be seen in the fundus blood vessels. Therefore, when it is determined that the possibility of diabetic retinopathy is high, the CPU 3 displays the OCT angiography on the monitor 8.

Returning to the explanation of FIG. Next, the CPU 3 generates reference information that indicates in steps the degree to which at least one of the diseases exists based on a plurality of automatic diagnosis results obtained for each of the plurality of diseases (S5). The CPU 3 stores the generated reference information in the storage device 4 in association with the ophthalmologic image targeted for automatic diagnosis (S6).

As an example, in this embodiment, reference information is generated in three stages of “warning”, “caution”, and “normal” in descending order of the degree of presence of at least one of the diseases. By setting the number of stages to 5 or less (more desirably 3 or less), the user can easily determine whether or not to perform a detailed diagnosis. However, the number of stages can be six or more.

Also, a specific method for generating reference information based on a plurality of automatic diagnosis results can be selected as appropriate. As an example, as shown in FIG. 4, the CPU 3 of the present embodiment displays reference information for “warning” when there is even one disease whose existence probability is determined to be equal to or higher than a threshold value α (for example, 60%). Generate. If the highest existence probability is equal to or higher than a threshold value β (for example, 20%) and lower than the threshold value α, the CPU 3 generates reference information for “caution”. If the presence probability of all the diseases is less than the threshold value β, the CPU 3 generates “normal” reference information. However, the CPU 3 may generate the reference information by other methods. For example, the CPU 3 may generate the reference information in consideration of the number of diseases whose existence probability exceeds a predetermined threshold. That is, the CPU 3 may generate reference information indicating that the degree of presence of a disease is higher when the number of diseases whose existence probability exceeds a predetermined threshold is greater than when the number of diseases is small.

Note that the timing for executing the reference information generation process can be selected as appropriate. When the PC 1 according to the present embodiment acquires an ophthalmic image from the ophthalmic image capturing apparatus 11, the PC 1 can automatically execute a reference information generation process for the acquired ophthalmic image. A reference information generation start button 54 (see FIG. 8) is provided on at least a part of the display screen displayed on the monitor 8 by the PC 1. The user operates the reference information generation start 54 button via the operation unit 7 (for example, a mouse) and inputs an instruction to start generation of reference information, thereby starting generation of reference information about a desired ophthalmic image. Can be made. Therefore, the user can generate reference information for past ophthalmic images or the like for which reference information has not yet been generated. Further, when the ophthalmic image capturing apparatus 11 executes the reference information generation process, the CPU 13 may execute the reference information generation process in parallel with the imaging of the ophthalmic image. In this case, the CPU 13 may cause the display unit to display the generated reference information together with the ophthalmologic image being captured.

Also, the time and accuracy of automatic diagnosis processing using a mathematical model are in a trade-off relationship with each other. In other words, high-accuracy automatic diagnosis processing often takes a long time. In the present embodiment, both an algorithm that can be processed at high speed and an algorithm that can execute high-precision automatic diagnosis are employed. When acquiring an ophthalmic image (S1), the CPU 3 of the present embodiment first performs automatic diagnosis using a high-speed algorithm (S4), and generates reference information (S5). If the generated reference information is reference information (for example, “normal”) indicating that the degree of presence of the disease is low, the reference information generation processing for the ophthalmic image is terminated. On the other hand, if the generated reference information is reference information indicating that the presence of a disease is high (for example, “caution” and “warning”), automatic diagnosis is performed again using a high-precision algorithm (S4). Reference information is generated (S5). As a result, reference information is efficiently generated. Note that the CPU 3 may perform automatic diagnosis using a high-precision algorithm in the background while performing automatic diagnosis using a high-speed algorithm. In this case, after the reference information is generated in a short time, finally, highly accurate reference information is generated.

(Setting of reference information corresponding to patient data)
With reference to FIG. 5, the setting of the reference information matched with patient data is demonstrated. In the present embodiment, patient data is created for each subject or eye to be examined and stored in the storage device 4. Details of the patient data will be described later with reference to FIGS. 6 and 7. One patient data includes one or more ophthalmic images. Furthermore, when a plurality of ophthalmic images are included in one patient data, the PC 1 of the present embodiment can generate reference information for each of the plurality of ophthalmic images.

Here, in this embodiment, when reference information is generated for each of a plurality of ophthalmic images included in one patient data, the user can set the type of reference information associated with the patient data. it can. Specifically, as shown in FIG. 5, the user has the highest degree of presence of reference information (hereinafter referred to as “latest reference information”) generated for an ophthalmologic image having the latest shooting timing and the disease. It is possible to set which stage reference information (hereinafter referred to as “high stage reference information”) is associated with patient data. The user can manage the patient data based on the latest state of the eye to be examined by associating the latest reference information with the patient data. On the other hand, by associating high-level reference information with patient data, the user can manage patient data in consideration of the degree of past disease. In the present embodiment, the user opens the corresponding reference information setting screen 38, selects either the latest reference information or the high-level reference information, and operates the “OK” button, whereby the type of reference information associated with the patient data is selected. Can be set. The CPU 3 associates the selected type of reference information with each patient data.

(Display patient data)
An example of a method for displaying patient data will be described with reference to FIGS. FIG. 6 shows an example of a patient data display screen 40A when the latest reference information is associated with patient data. FIG. 7 shows an example of a patient data display screen 40B in the case where high-level reference information is associated with patient data. FIG. 8 shows an example of the two-dimensional

tomographic images

51 and 52 displayed when the thumbnail 47 of June 20, 2017 in FIG. 6 is selected.

First, the various display fields will be described. As shown in FIGS. 6 and 7, the patient data display screen is provided with a patient ID display field 41, a patient name display field 42, a gender display field 43, a left and right eye display field 44, and a corresponding reference information display field 45. It has been. The left and right eye display field 44 is provided with a check box for selecting the left eye and the right eye. The user checks one of the left eye and right eye check boxes to select either the left eye or the right eye among one or more ophthalmic images included in the selected (displayed) patient data. Can be displayed. In the corresponding reference information display field 45, the type of reference information selected on the corresponding reference information setting screen 38 (see FIG. 5) described above is displayed.

As shown in FIGS. 6 and 7, on the patient data display screens 40 </ b> A and 40 </ b> B, each ophthalmic image (in this embodiment, one set (two) of two-dimensional tomographic images that are targets for generating reference information). A thumbnail 47 of the front image corresponding to is displayed. Each thumbnail 47 is provided with a reference information display field 48 and a date display field 49. In the reference information display field 48, reference information generated for each ophthalmic image is displayed. That is, when displaying patient data on the monitor 8, the CPU 3 can display reference information corresponding to each of a plurality of ophthalmologic images included in the patient data. As a result, the follow-up observation of the eye to be examined is facilitated. In the example shown in FIGS. 6 and 7, “warning” indicates warning, “note” indicates caution, and “correct” indicates normal. For an ophthalmologic image for which reference information has not yet been generated, a reference information display field 48 indicates that reference information has not been generated. In the examples shown in FIGS. 6 and 7, “/” indicates that the reference information has not yet been generated. The date display field 49 displays the date when the ophthalmologic image was taken.

As shown in FIG. 6, when the latest reference information is associated with patient data, it indicates which is the latest ophthalmic image that is the basis of the reference information associated with the entire patient data. The display for is performed. As an example, in the example illustrated in FIG. 6, the frame of the thumbnail 47M of the latest ophthalmic image is displayed in a different manner from the frame of the other thumbnails 47, thereby indicating which is the latest ophthalmic image. ing. However, the display method for showing the ophthalmic image on which the reference information is based can be changed as appropriate. For example, the ophthalmologic image that is the basis of the reference information may be displayed by changing the display method of the reference information attached to the thumbnail 47M instead of the frame of the thumbnail 47M.

Also, when the latest reference information is associated with patient data, a plurality of thumbnails 47 are displayed side by side in the order in which the ophthalmic images were taken. Therefore, the user can easily grasp the progress of the eye to be examined by looking at the patient data display screen 40A.

As shown in FIG. 7, when the high-level reference information is associated with the patient data, the ophthalmic image (that is, the presence of the disease is the basis of the reference information associated with the entire patient data). A display for indicating which is the highest ophthalmic image) is performed. As an example, in the example illustrated in FIG. 7, similarly to FIG. 6, the frame of the thumbnail 47 </ b> N of the ophthalmic image with the highest degree of disease presence is displayed in a manner different from the frame of the other thumbnails 47.

Further, when the high-level reference information is associated with the patient data, a plurality of thumbnails 47 are displayed in order of the degree of presence of the disease. Therefore, the user can grasp a plurality of ophthalmologic images according to the degree of presence of the disease.

When any one of the thumbnails 47 displayed on the patient

data display screens

40A and 40B is selected by the user, the CPU 3 displays an ophthalmic image corresponding to the selected thumbnail 47 (in this embodiment, reference information). An ophthalmologic image display screen 50 including a base two-dimensional tomographic image) is displayed on the monitor 8. As shown in FIG. 8, in the ophthalmic image display screen 50 of the present embodiment, two selected two-dimensional

tomographic images

51 and 52 that are the basis for generating reference information together with the selected thumbnail 47, reference information 48, and date 49. Is displayed. Note that the display mode of the ophthalmologic image display screen 50 can be changed. For example, a graph or the like indicating the thickness of a specific layer in the fundus may be mainly displayed as the two-dimensional

tomographic images

51 and 52.

Further, in the present embodiment, when the reference information generated for the ophthalmologic image is different from the actual diagnosis result, the user can input an instruction to edit the reference information via the operation unit 7 or the like. The CPU 3 edits the selected reference information in accordance with an instruction input by the user. As a result, patient data is managed more appropriately. The CPU 3 sets the display mode of the edited reference information to a display mode different from the unedited reference information. Therefore, the user can easily grasp whether or not the reference information has been edited. In addition, the user can limit the users who can edit the reference information to specific users by operating the operation unit 7.

(List of patient data)
An example of a method for displaying a list of patient data will be described with reference to FIGS. 9 and 10. When an instruction to display a list of patient data is input by the user, the CPU 3 causes the monitor 8 to display a patient data list display screen 60 as shown in FIG. The list display screen 60 includes a list display unit 61 and a search condition input unit 62. The list display unit 61 displays a list of various types of information regarding each patient data created for each subject or each eye to be examined. Specifically, in the list display unit 61 of the present embodiment, reference information associated with patient data is displayed in addition to ID, name, age, and gender information included in the patient data. Therefore, the user can manage patient data appropriately based on the reference information. As described above, in this embodiment, the user can set which of the latest reference information and the high-level reference information is associated with the patient data. The CPU 3 causes the list display unit 61 to display the reference information selected by the user among the latest reference information and the high-level reference information in association with each patient data. Needless to say, the list display screen 60 may include other information (for example, photographing date and time of an ophthalmic image).

The search condition input unit 62 is input with various search conditions when the user wants to search patient data desired by the entire list. The search conditions in this embodiment include reference information in addition to the patient's name, age, gender, and the like. When searching for patient data based on the reference information, the user operates the operation unit 7 to select at least one of “warning”, “caution”, “normal”, and “non-generated”, so that a plurality of pieces of reference information are obtained. Enter instructions to select at least one of the stages. The CPU 3 extracts patient data corresponding to the reference information (in the present embodiment, the latest reference information or the high-level reference information) at the stage selected by the input instruction from the patient data. The CPU 3 causes the monitor 8 to display a list of patient data extracted according to the reference information at the selected stage. FIG. 10 shows a list display screen 60 when a search is performed with reference information “warning” from the state shown in FIG. 9. As described above, according to this embodiment, patient data is appropriately managed based on reference information.

In the list display unit 61 shown in FIG. 9, a list of a plurality of patient data is displayed side by side in order of photographing date and time of the ophthalmic image. However, the CPU 3 may display a list of a plurality of patient data side by side according to the reference information (that is, for each stage of the reference information).

(Show / hide reference information)
In the present embodiment, the user can input an instruction to select whether to display reference information on the monitor 8 to the PC 1. The selection instruction may be input by operating the operation unit 7 by the user, for example. The CPU 3 switches between displaying and hiding the reference information illustrated in FIGS. 6 to 10 on the monitor 8 in accordance with an instruction input from the user. For example, when an instruction to hide the reference information is input, the CPU 3 hides the reference information displayed in FIGS. Therefore, the user can perform a task such as diagnosis more appropriately.

(Output of patient data)
With reference to FIG. 11, the patient data output process which CPU3 performs is demonstrated. In the patient data output process, a process for transmitting patient data to another device and a process for outputting a report of patient data are performed. First, the CPU 3 determines whether or not an instruction to select a reference information stage has been input (S11). The selection instruction may be input by operating the operation unit 7 by the user, for example. When the selection instruction is input (S11: YES), the CPU 3 adds the reference information (the latest reference information or the high-level reference information in this embodiment) at the stage selected from the patient data according to the input instruction. Corresponding patient data is extracted (S12).

Note that the CPU 3 specifies the reference information when only one reference information of the patient data of the left eye and the right eye of the same subject is in a specific stage (for example, a stage where the degree of disease is highest). The patient data of the other eye of the same subject may be extracted together with the patient data of the eye corresponding to the reference information at the stage. In this case, the user can diagnose the other eye based on the diagnosis result of one eye. In this case, the CPU 3 may notify the user that attention should be paid to the other eye. Further, the CPU 3 determines whether to extract only the patient data of the eye corresponding to the reference information at a specific stage or to extract the patient data of both eyes according to the type of the disease that is highly likely to exist. Alternatively, it may be determined according to an instruction from the user.

Next, the CPU 3 determines whether or not a patient data transmission instruction has been input (S14). When a transmission instruction is input (S14: YES), the CPU 3 transmits patient data to another device (for example, the PC 21 at the base B shown in FIG. 1) via the network 9 (S15). If the reference information stage is selected in S11, the patient data extracted in S12 (that is, patient data corresponding to the reference information in the selected stage) is transmitted in S15. In addition, the report mentioned later may be transmitted to another device instead of patient data or together with patient data.

Next, the CPU 3 determines whether or not a report output instruction has been input (S17). When an output instruction is input (S17: YES), the CPU 3 outputs a report of patient data (S18). If the reference information stage is selected in S11, a report of the patient data extracted in S12 (that is, patient data corresponding to the reference information of the selected stage) is output in S18. . Note that the report output method may be a method of printing on paper or a method of outputting data in a specific format (for example, PDF data). Needless to say, the information included in the report can be selected as appropriate.

Next, the CPU 3 determines whether or not an end instruction has been input (S19). If an end instruction has not been input (S19: NO), the process returns to the determination in S11. When the end instruction is input (S19: YES), the patient data output process ends.

The patient data transmitted to the other device in S15 may be the entire patient data including one or a plurality of ophthalmic images, or a part of the patient data (for example, the latest ophthalmic image). There may be. The same applies to the output of the report.

In this embodiment, the process of outputting patient data (that is, the process of transmitting patient data and the process of outputting reports) is performed in response to the input of an instruction from the user. However, the timing for outputting patient data can be changed as appropriate. For example, the CPU 3 sequentially generates reference information for each of a plurality of ophthalmic images, and outputs patient data each time reference information at a specific stage (for example, reference information of “warning” and “caution”) is generated. Processing may be performed. For example, the ophthalmologic image capturing apparatus 11 can execute a reference information generation process (see FIG. 3) and a patient data output process (see FIG. 11). In this case, the CPU 16 of the ophthalmic image capturing apparatus 11 may generate reference information for the captured ophthalmic image every time an ophthalmic image of the eye to be examined is captured. The CPU 16 may output patient data each time reference information at a specific stage is generated.

The CPU 3 also includes data corresponding to reference information at a specific stage (at least one of patient data, ophthalmic images included in the patient data, analysis result data, etc.) and data corresponding to reference information at a specific stage. Are stored in the storage device 4 in different ways. For example, the CPU 3 may omit the process of storing at least a part of the data in the storage device 4 when the data corresponds to reference information other than a specific stage (for example, “warning” and “caution”). At least a part of the data may be deleted from the storage device 4. Further, the CPU 3 may execute a process for reducing the amount of data stored in the storage device 4.

(Providing data to manufacturers)
In the present embodiment, the user can input the result of actually diagnosing an ophthalmologic image to the PC 1. The CPU 3 can extract ophthalmic images in which the degree of the disease indicated by the reference information generated for the ophthalmic image is different from the presence or absence of the disease determined by the user by the actual diagnosis. Further, the CPU 3 can extract ophthalmic images in which the automatic diagnosis result obtained by the mathematical model is different from the result of the diagnosis actually performed by the user. The CPU 3 provides (for example, transmits via the network 9) the extracted ophthalmic image to the manufacturer of the ophthalmic image processing apparatus and the ophthalmic image processing program. The manufacturer can improve the accuracy of subsequent automatic diagnosis results or reference information by training a mathematical model using the provided ophthalmic images as training data.

The technology disclosed in the above embodiment is merely an example. Therefore, it is possible to change the technique exemplified in the above embodiment. For example, as described above, the ophthalmologic photographing apparatus 11 can execute the reference information generation process. In this case, the CPU 16 of the ophthalmologic photographing apparatus 11 may change the photographing method of the eye to be examined according to at least one of the automatic diagnosis result and the reference information generation result. For example, the CPU 16 first takes ophthalmic images (two two-dimensional tomographic images in the above embodiment) necessary for automatic diagnosis and generation of reference information. Next, when the automatic diagnosis result and the generation result of the reference information are predetermined results (for example, when the reference information is “warning” or “caution”), the CPU 16 captures an additional ophthalmic image (for example, Map photographing for photographing a three-dimensional tomographic image of the optometric tissue may be executed. In this case, a more appropriate image is taken as necessary according to at least one of the automatic diagnosis result and the reference information generation result.

Note that the process of acquiring an ophthalmic image in S1 of FIG. 3 is an example of an “image acquisition step”. The process of extracting the image of the target range in S2 and S3 in FIG. 3 is an example of “image extraction step”. The process of acquiring the automatic diagnosis result in S4 of FIG. 3 is an example of “automatic diagnosis result acquisition step”. The process of generating reference information in S5 of FIG. 3 is an example of a “reference information generation step”. CPU3 which performs the process which acquires an ophthalmologic image by S1 of FIG. 3 is an example of an "image acquisition means." The CPU 3 that executes the process of extracting the image in the target range in S2 and S3 in FIG. 3 is an example of the “image extracting unit”. CPU3 which performs the process which acquires an automatic diagnostic result by S4 of FIG. 3 is an example of an "automatic diagnostic result acquisition means."

1 PC
3 CPU
7 Operation unit 8 Monitor 9 Network 11 Ophthalmic image capturing device 13 CPU
16 Ophthalmology image capturing unit 21 PC
30 Front image 38 Corresponding reference

information setting screen

40A, 40B Patient data display screen 48 Reference information display column 50 Ophthalmic

image display screen

51, 52 Two-dimensional tomographic image 60 List display screen 62 Search condition input unit 100 Ophthalmic image processing system

Claims

An ophthalmologic image processing apparatus that processes an ophthalmologic image of an eye to be examined,
The controller of the ophthalmic image processing apparatus
Obtaining the ophthalmic image captured by the ophthalmic image capturing unit;
By inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm, an automatic diagnosis result for each of a plurality of diseases in the eye to be examined is obtained,
An ophthalmologic image processing apparatus that generates reference information that indicates in steps the degree to which at least one of the plurality of diseases is present in the eye to be examined based on the plurality of automatic diagnosis results for the ophthalmologic image.
The ophthalmic image processing apparatus according to claim 1,
The controller is
An instruction to select at least one of the plurality of stages of the reference information;
An ophthalmic image processing apparatus, wherein patient data corresponding to reference information at a stage selected by an input instruction is extracted from patient data including the ophthalmic images of a plurality of subjects or eyes.
The ophthalmic image processing apparatus according to claim 2,
The controller is
A list of patient data extracted according to the selected reference information is displayed on a display unit.
An ophthalmic image processing apparatus according to any one of claims 1 to 3,
The controller is
An ophthalmic image processing apparatus which transmits patient data corresponding to the reference information at a specific stage among one or a plurality of patient data including the ophthalmic image to another device via a network.
An ophthalmic image processing apparatus according to any one of claims 1 to 4,
For each subject or eye, patient data including data of one or more ophthalmic images is stored in the storage means,
The controller is
When a plurality of the ophthalmic images are included in one patient data, the reference information can be generated for each of the plurality of ophthalmic images,
When a plurality of the ophthalmologic images are included in one patient data, the reference information generated for the ophthalmologic image with the newest shooting timing and the highest degree of presence of a disease Any one of the reference information is associated with each of the patient data.
An ophthalmologic image processing apparatus according to any one of claims 1 to 5,
For each subject or eye, patient data including data of one or more ophthalmic images is stored in the storage means,
The controller is
When a plurality of the ophthalmic images are included in one patient data, the reference information can be generated for each of the plurality of ophthalmic images,
When the patient data is displayed on a display unit, the reference information corresponding to each of the plurality of ophthalmic images included in the patient data is displayed on the display unit.
An ophthalmic image processing apparatus according to any one of claims 1 to 6,
The control unit
Enter an instruction to select whether to display the reference information on the display unit,
An ophthalmologic image processing apparatus that switches between displaying and hiding reference information on the display unit in accordance with an input instruction.
The ophthalmic image processing apparatus according to claim 1,
The controller is
When the acquired range of the ophthalmic image is wider than the target range of the ophthalmic image input to the mathematical model, the target range is extracted from the acquired ophthalmic image;
The ophthalmic image processing apparatus characterized by acquiring the automatic diagnosis result by inputting the extracted ophthalmic image into the mathematical model.
An ophthalmologic image processing apparatus that processes an ophthalmologic image of an eye to be examined,
Image acquiring means for acquiring the ophthalmic image captured by the ophthalmic image capturing unit;
Automatic diagnosis result acquisition means for acquiring an automatic diagnosis result for at least any disease in the eye to be examined by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm;
When the range of the ophthalmic image acquired by the image acquisition unit is wider than the target range of the ophthalmic image input to the mathematical model, an image of the target range is obtained from the ophthalmic image acquired by the image acquisition unit. Image extracting means for extracting as the ophthalmic image to be input to the mathematical model;
An ophthalmic image processing apparatus comprising:
An ophthalmic image processing program executed by an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined,
The ophthalmic image processing program is executed by the control unit of the ophthalmic image processing apparatus,
An image acquisition step of acquiring the ophthalmic image captured by the ophthalmic image capturing unit;
An automatic diagnosis result acquisition step of acquiring an automatic diagnosis result for each of a plurality of diseases in the eye to be examined by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm;
A reference information generating step for generating, in a step-wise manner, reference information indicating a degree at which at least one of the plurality of diseases is present in the eye to be examined based on the plurality of automatic diagnosis results for the ophthalmic image;
Is executed by the ophthalmic image processing apparatus.
An ophthalmic image processing program executed by an ophthalmic image processing apparatus that processes an ophthalmic image of an eye to be examined,
The ophthalmic image processing program is executed by the control unit of the ophthalmic image processing apparatus,
An image acquisition step of acquiring the ophthalmic image captured by the ophthalmic image capturing unit;
An automatic diagnosis result acquisition step of acquiring an automatic diagnosis result for at least any disease in the eye to be examined by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm;
When the range of the ophthalmic image acquired in the image acquisition step is wider than the target range of the ophthalmic image input to the mathematical model, an image of the target range is obtained from the ophthalmic image acquired in the image acquisition step. Extracting an image as the ophthalmic image to be input to the mathematical model;
Is executed by the ophthalmic image processing apparatus.