WO2018070285A1

WO2018070285A1 - Image processing device and image processing method

Info

Publication number: WO2018070285A1
Application number: PCT/JP2017/035787
Authority: WO
Inventors: 子盛黎; 昌宏荻野
Original assignee: 株式会社日立製作所
Priority date: 2016-10-14
Filing date: 2017-10-02
Publication date: 2018-04-19
Also published as: JP2018061771A

Abstract

The present invention provides a medical image processing device that makes adjustments, automatically and with a high degree of accuracy, to the process of extracting an image feature quantity of a lesion shadow so as to reduce the burden on a user. This image processing device presents a suspected lesion site image detected from image data captured for medical purposes. The image processing device has: an image feature label learning unit (21) for classifying image a feature label with regard to the suspected lesion site image; an image feature quantity extraction unit (23) for extracting an image feature quantity with regard to the suspected lesion site image using a learning parameter for the image feature label obtained through learning; a display unit (11) for displaying the suspected lesion site image; a user input unit (10); and an image feature label learning update unit (29) for updating the learning parameter in accordance with an input from the user input unit (10).

Description

Image processing apparatus and image processing method

The present invention relates to an image processing apparatus, and more particularly to an image processing technique for processing medical images.

In diagnosis using a medical imaging apparatus typified by an X-ray CT (X-ray Computed Tomography) apparatus and an MRI (Magnetic Resonance Imaging) apparatus, the captured three-dimensional medical image is re-created as a continuous two-dimensional section. In general, the image is interpreted by observing the two-dimensional cross-sectional image.

With the advancement of these imaging devices, the three-dimensional resolution of the generated three-dimensional medical image is also improved, and the data size tends to increase. In particular, the two-dimensional cross-section generation interval described above can be made finer, and more detailed observation of the lesion appearing on the medical image is possible. The number of two-dimensional sections is also increasing. In particular, in the CT apparatus, it has become possible to capture a high-quality three-dimensional medical image with a low dose, and the number of CT image capturing opportunities tends to increase.

For these reasons, development of computer-aided diagnosis technology called CAD (Computer Aided Detection) is being promoted to reduce the burden on doctors and technicians when reading vast 3D medical images and mainly to prevent oversight of lesions. ing. This CAD aims to automatically or semi-automatically apply image processing technology to detect shadows, measure sizes, identify normal / abnormal shadows, and distinguish between types of abnormal shadows using a computer.

Here, we will describe CAD that aims to present shadows with high suspicion of lesions based on image features. Since the purpose of this CAD is to prevent a doctor from overlooking it, it is often desirable to present all shadows with a high suspicion of lesions. However, on the other hand, if there are too many shadows to present, there is a problem that the burden on the doctor who examines each suspicion increases. Therefore, there is a need for a method for presenting suspected shadows in a form desired by a doctor and reducing the burden on the doctor.

In order to solve these problems, for example, in Non-Patent Document 1, CAD data is improved by continuously collecting and re-learning diagnostic data of facilities using CAD systems developed by machine learning. A method for achieving this has been proposed.

In the automatic detection of a suspicious lesion by CAD, a formula that defines the suspicion of a lesion is generally set using a feature amount obtained from an image, and a shadow having a high suspicion of the lesion is presented. However, since the image feature quantity of the lesion shadow is unique, the feature quantity extraction method set based on the CAD development data set cannot be applied to the image quality and findings of the actual operation. The estimation performance of the suspicion of the lesion is not obtained. In addition, as an adjustment of the detection accuracy of the shape in accordance with the wishes of the doctor who is the user, the contribution rate of which feature amount is high in the method of adjusting the threshold of the suspicion of the shadow to be presented and the method of calculating the suspicion However, in any case, the image feature extraction processing cannot be adjusted.

An object of the present invention is to provide an image processing apparatus and an image processing method that enable adjustment of image feature amount extraction processing of a lesion shadow and reduce a burden on a user when interpreting a medical image. It is in.

In order to achieve the above object, according to the present invention, an image processing apparatus that presents a suspected lesion area image detected from image data, and that performs learning for classifying image feature labels related to the suspected lesion area image A feature label learning unit, an image feature amount extraction unit that extracts an image feature amount of a suspicious lesion region image using a learning parameter of an image feature label obtained by learning of the image feature label learning unit, and a suspicious lesion region image are displayed An image processing apparatus having a display unit, a user input unit, and an image feature label learning update unit that updates a learning parameter in response to an input from the user input unit.

In order to achieve the above object, according to the present invention, there is provided a processing method of an image processing apparatus that includes a display unit and a user input unit, and presents a suspected lesion area image detected from image data. Learns to classify the image feature labels related to the suspicious lesion region image, extracts the image feature amount of the suspicious lesion region image using the learning parameters of the image feature label obtained by learning, and extracts the suspicious lesion region image. Provided is an image processing method configured to display on a display unit and update a learning parameter in accordance with an input from a user input unit.

According to the present invention, it becomes possible to adjust the image feature extraction processing of the lesion shadow along the shape desired by the user.

1 is a block diagram illustrating an example of the overall configuration of an image processing apparatus according to Embodiment 1. FIG. FIG. 3 is a flowchart illustrating a flow of machine learning processing in the image processing apparatus according to the first embodiment. FIG. 3 is a flowchart illustrating a flow of a suspicious area malignancy estimation process in the image processing apparatus according to the first embodiment. FIG. 3 is a flowchart illustrating a flow of image feature amount extraction processing in the image processing apparatus according to the first embodiment. The flowchart figure of the flow of the process which adjusts a lesion suspicious area malignancy estimation process based on Example 1. FIG. FIG. 3 is a schematic diagram illustrating an example of a plurality of image feature labels according to the first embodiment. FIG. 3 is a diagram illustrating an example of a configuration of an image label learning device for classifying image feature labels according to the first embodiment. FIG. 6 is a diagram illustrating an example of image feature amount extraction processing of a suspected lesion area image according to the first embodiment. Explanatory drawing which shows an example of the display part which displays image data and a lesion suspicious area image based on Example 1, and the user interface which selects an image feature label and malignancy right / wrong information. FIG. 6 is a block diagram illustrating an example of the overall configuration of an image processing apparatus according to a second embodiment. FIG. 10 is a flowchart illustrating a flow of image feature amount extraction processing in the image processing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an image label learning device for classifying image feature labels according to the second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

In the present embodiment, in order to calculate the malignancy level for a suspected lesion area image detected from a medical image in a form desired by the user with high accuracy, an image feature amount extraction process related to a suspected lesion area image according to an input from the user 1 is an example of an image processing apparatus capable of adjusting the image quality. That is, an image processing apparatus that presents a suspected lesion area image detected from image data, an image feature label learning unit that performs learning for classifying image feature labels related to a suspected lesion area image, and an image feature label learning unit An image feature amount extraction unit that extracts an image feature amount of a suspicious lesion region image using a learning parameter of an image feature label obtained by learning, a display unit that displays a suspicious lesion region image, a user input unit, and a user It is an Example of the image processing apparatus of a structure provided with the image feature label learning update part which updates the learning parameter of an image feature label according to the input from an input part.

A processing method of an image processing apparatus that includes a display unit and a user input unit and presents a suspected lesion area image detected from image data, wherein the image processing apparatus classifies image feature labels related to a suspected lesion area image. For learning, using the learning parameters of the image feature label obtained by learning, extracting the image feature amount of the suspicious lesion region image, displaying the suspicious lesion region image on the display unit, and inputting it from the user input unit Accordingly, the embodiment is an embodiment of an image processing method configured to update learning parameters.

In this embodiment, a reconstructed three-dimensional medical image obtained by a CT medical image photographing apparatus will be described as an example. However, the structure of this embodiment is an image process based on data obtained by another medical image photographing apparatus. The apparatus can also be applied. For example, even if the data is obtained by an MRI imaging apparatus or the like, it is applicable to obtain a three-dimensional image that can be expressed as a stack of a plurality of two-dimensional cross sections, and that are supposed to show lesion characteristics in the pixel distribution. Can do.

In addition, the suspected lesion area in the present embodiment refers to a point and an area with a high suspected lesion, which are determined based on the medical knowledge of the interpreting doctor, the medical evidence (evidence) for the disease diagnosis, and the like. Here, it is assumed that the target lesion is highly likely to be judged from the difference in luminance from the surrounding area, that is, the region with low suspicion of the lesion, or the difference in luminance value distribution when it appears on the medical image. For example, in the case of a pulmonary nodule, it is generally known that the CT value appears on the CT image as a region including many pixels higher than the surrounding air region. Other objects that contain many high-luminance pixels on the chest CT image include blood vessels and bones, but depending on the distribution shape of the high-luminance values, they are distinguished from those, and the lesions, ie pulmonary nodules in this example, are suspected It is said that the height can be distinguished.

FIG. 1 is a diagram illustrating an example of a system configuration including an image processing apparatus according to the first embodiment. As shown in FIG. 1, the image processing apparatus 100 includes a user input unit 10, an image feature label learning unit 21, an image feature label learning parameter storage unit 22, an image feature amount extraction unit 23, and a lesion suspected area malignancy learning. Unit 24, suspected lesion malignancy estimation parameter storage unit 25, suspected lesion malignancy estimation unit 26, image feature label learning update unit 28, suspected lesion malignancy learning update unit 29, and display unit 11 It is composed of The image processing apparatus 100 is configured by a normal computer, the display unit 11 is a display thereof, the storage unit is configured by a memory thereof, and each functional block such as the image feature amount extraction unit 23 is a central processing unit (CPU). ) Is executed by program execution. Further, the medical image DB 20, the diagnostic image, and the suspected lesion area image 27 are realized by the external storage device or the like. In FIG. 1, the image feature quantity extraction unit 23 is shown as three blocks, but these are functional blocks that execute the same process of extracting the feature quantity from each input target image. The middle block of the three blocks corresponds to step S203 in FIG. 2, which will be described later, the right block corresponds to step S302 in FIG. 3, and the left block corresponds to step S502 in FIG.

Next, the operation processing of the image processing apparatus 100 shown in FIG. 1 will be described using the flowcharts shown in FIGS. FIG. 2 is a flowchart showing an operation process of extracting an image feature amount by learning an image feature label and learning of a suspected lesion region malignancy estimation using the extracted image feature amount for a suspected lesion region image. . These operation processes are executed by the image feature label learning unit 21, the image feature amount extraction unit 23, and the suspected lesion area malignancy learning unit 24.

First, in step S201 of FIG. 2, the image feature label learning unit 21 receives information on a suspected lesion region image and the corresponding image feature label from the medical image DB 20. Here, the image feature label refers to the type of image feature related to the suspected lesion area, such as the size of the area of the shadow area, the brightness, the presence / absence of contact with the surrounding existing structure, the occurrence site, and the shape. FIG. 6 shows image feature labels 61 to 66 as an example of such image feature labels of the suspicious lesion area.

Next, in step S202, the image feature label learning unit 21 performs machine learning for classifying the image feature labels 70, and generates learning parameters thereof. Here, the CNN (Convolutional Neural Network) method, which is a known deep learning method, can be used. FIG. 7 shows an example of the configuration of a learning device (network) using the CNN method. In the learning process using the CNN method, an input learning image is identified by repeating a convolution layer 71 that performs a number of image filtering processes and a pooling layer 72 that samples from the output of the convolution layer. It is possible to automatically generate an image feature amount that optimally expresses the feature of the image, that is, as accurately as possible.

In FIG. 7, only two layers of the convolution layer 71 and the pooling layer 72 are shown for convenience, but in an actual case, it is possible to improve performance by setting a larger number of layers. The last layer of the CNN network configuration shown in FIG. 7 is an identification layer (Classification layerer) 73. Here, the probability (score) that an input image belongs to a preset type (class) is calculated and the result (Result) ) 74, that is, a process of classifying (identifying) the input image. In this embodiment, six types of image feature labels 61 to 66 shown in FIG. 6 are set so as to identify large area area, small area area, sternal contact type, luminance intensity, luminance intensity, and tubular. It is also possible to change settings such as the type of image feature label and the number of types depending on the feature of the target image in the suspected lesion area.

In this embodiment, a CNN network is set for each class of the image feature label 70. Input learning images for learning each CNN network are positive sample data that is an image belonging to the image feature label class and negative sample data that is an image belonging to another image feature label class. That is, the image feature label learning unit 21 sets a CNN network for each class of image feature labels. When the network learning is completed, the image feature label learning parameter storage unit 22 acquires the parameters of each convolution layer 71, the pooling layer 72, and the identification layer 73 from the image feature label learning unit 21, and uses them as image feature label learning parameters. save. The image feature label learning parameter is a parameter related to the configuration of a convolution layer or a pooling layer in a known CNN method, for example, the total number of convolution layers or pooling layers (2 in the case of FIG. 7) or a convolution operation. It is a parameter such as the coefficient and size of the convolution filter of time.

In this embodiment, the purpose of identifying the type of the image feature label 70 is to extract the image feature amount of the highly unique lesion suspicious area image in a form that can be automatically adjusted with high accuracy. FIG. 8 shows an example of image feature amount extraction processing of a suspected lesion area image by the image feature amount extraction unit 23. In step S203, the image feature quantity extraction unit 23 receives the image feature label learning parameters from the image feature label learning parameter storage unit 22, and samples them from the convolution layer 81 shown in FIG. 8 and the output of the convolution layer. An image feature amount of the input suspicious region image 80 is extracted using a CNN network constituted by repetition of a pooling layer 82 and a classification layer 83. In this embodiment, an output vector of any pooling layer 82 of the CNN network or an identification score vector obtained by concatenating results (Result) 84 that are identification scores of networks of each class may be used as the image feature amount. . A vector concatenated with the output vector of the pooling layer 72 and the identification score vector may be used. As described above, the image feature amount extraction unit 23 can identify the type of the image feature label related to the suspected lesion region image using the learning parameter.

Next, in steps S204 to S205, the suspected lesion malignancy learning unit 24 generates an estimated parameter of the suspected lesion malignancy using machine learning in order to calculate the malignancy of the suspected lesion region. In step S 204, the suspected lesion area malignancy learning unit 24 receives the image feature amount of the suspected lesion area image obtained by the processing of FIG. 8 from the image feature amount extraction unit 23. Further, the suspected lesion area malignancy learning unit 24 acquires malignancy information corresponding to the suspected lesion area image from the medical image DB 20. In step S 205, the suspicious lesion area malignancy learning unit 24 creates a learning device that estimates the suspicious area malignancy using the image feature amount of the suspicious area image and the corresponding malignancy information. Here, a well-known SVM (Support Vector Machine) method and a Ranking SVM method that can rank malignancy can be used. When the learning process is completed, the suspected lesion malignancy estimation parameter storage unit 25 receives and stores the suspected lesion malignancy estimation parameter from the suspected lesion malignancy learning unit 24. The lesion suspicious area malignancy estimation parameter is a parameter in a known machine learning method such as the SVM method. For example, in the feature amount space, the parameter is a boundary line or a boundary surface for classification, for example, a straight line. When dividing, the coefficients a and b of a straight line (y = a * x + b) are obtained.

Next, a process for estimating the malignancy of an input suspicious area image as a diagnosis target at the time of diagnosis will be described using the flowchart shown in FIG. The operation processing of this flowchart is executed by the image feature amount extraction unit 23 and the suspected lesion malignancy estimation unit 26. In step S 301, the image feature quantity extraction unit 23 receives a diagnostic image and a lesion suspected area image 27. In step S302, the image feature amount extraction unit 23 receives the image feature label learning parameter from the image feature label learning parameter storage unit 22, and extracts the image feature amount of the input suspicious region image using the CNN network. This is the same process as the process of FIG. 8 described in step S203.

In step S303, the suspected lesion malignancy estimation unit 26 receives the suspected lesion malignancy estimation parameter stored in the suspected lesion malignancy estimation parameter storage unit 25. Further, the suspected lesion area malignancy estimation unit 26 receives the image feature amount of the suspected lesion area image previously extracted from the image feature amount extraction unit 23. The suspected lesion area malignancy estimation unit 26 calculates the malignancy of the suspected lesion area in the suspected lesion area image using the suspected lesion area malignancy estimation parameter. In step S304, the display unit 11 receives the diagnostic image, the suspected lesion area image 27, and the corresponding suspected lesion area malignancy, and displays them as the diagnosis result of the image processing apparatus. When there are a plurality of suspected lesion area images, they can be ranked and displayed based on the malignancy estimation result. A detailed display method will be described later using the display screen example shown in FIG.

Next, a process for updating the image feature amount extraction process using the image feature label relating to the lesion area image input and displayed by the doctor who is the user will be described using the flowchart shown in FIG. The operation processing of this flowchart is executed by the image feature label learning update unit 28 and the image feature amount extraction unit 23. First, in step S401, the image feature label learning update unit 28 receives a displayed suspicious lesion area image. In step S 402, the image feature label learning update unit 28 receives from the user input unit 10 an image feature label corresponding to the suspected lesion region image input by the user.

In step S403, the image feature label learning update unit 28 determines whether to update the image feature amount extraction process. For example, when the image feature label learning update unit 28 acquires a predetermined number of suspected lesion area images, the image feature amount extraction processing may be updated. Further, the image feature amount extraction process may be updated when a predetermined accumulation period has elapsed. Further, the image feature amount extraction process may be updated in accordance with a user instruction. That is, when the image feature label learning update unit 28 acquires a predetermined number of suspected lesion area images, when the acquired suspected lesion area image has passed a predetermined accumulation period, or when a user instruction is input from the user input unit 10 If it has been done, the image feature quantity extraction unit 23 updates the image feature quantity extraction process.

When updating the image feature amount extraction process (Yes), in step S404, the image feature label learning update unit 28 receives the image feature label learning parameter from the image feature label learning parameter storage unit 22. In step S405, the image feature label learning update unit 28 updates the learning parameters of the CNN network related to the image feature label. The image feature label learning parameter storage unit 22 receives and stores the updated image feature label learning parameter. This makes it possible to automatically adjust the image feature amount extraction process more appropriately for a new lesion-suspicious area image. The image feature amount extraction unit 23 can extract the image feature amount of the suspicious lesion region image again using the image feature label learning parameter updated by the image feature label learning update unit 28.

Next, a process for updating the suspected lesion malignancy estimation parameter using the malignancy correctness information regarding the suspected lesion image input and displayed by the user will be described with reference to the flowchart shown in FIG. The operation processing of this flowchart is executed by the image feature amount extraction unit 23 and the lesion suspicious area malignancy learning update unit 29. In other words, the image processing apparatus according to the present embodiment includes the suspected lesion malignancy learning update unit 29 that updates the suspected lesion malignancy estimation parameter in accordance with the input from the user input unit 10 for each suspected lesion image. ing.

In step S501, the suspected lesion malignancy learning update unit 29 receives from the user input unit 10 correctness / incorrectness information of malignancy corresponding to the displayed suspected lesion region image input by the user. In step S502, the image feature amount extraction unit 23 receives the displayed suspicious lesion region image, further receives the image feature label learning parameter from the image feature label learning parameter storage unit 22, and uses the CNN network to input the lesion. Extract the image feature amount of the suspicious area image. This is the same processing as step S203. The lesion suspicious area malignancy learning update unit 29 receives the extracted image feature amount.

In step S503, the suspicious lesion malignancy learning update unit 29 determines whether to update the suspected lesion malignancy. Here, similarly to the update of the image feature amount extraction process, for example, when the suspected lesion area malignancy learning update unit 29 acquires a predetermined number of image feature amounts, the suspected lesion area malignancy may be updated. Further, the suspected lesion area malignancy may be updated after a predetermined accumulation period. Further, the suspicious lesion area malignancy may be updated in accordance with a user instruction. When updating the suspected lesion malignancy (Yes), in step S504, the suspected lesion malignancy learning update unit 29 receives the suspected lesion malignancy estimation parameter storage unit 25 from the suspected lesion malignancy estimation parameter storage unit 25.

Next, in step S505, the suspected lesion malignancy learning update unit 29 performs learning again in order to calculate a new lesion suspected region malignancy estimation parameter. Here, a known online learning method may be used. The suspected lesion malignancy estimation parameter storage unit 25 receives and stores the updated suspected lesion malignancy estimation parameter. In step S506, the suspected lesion malignancy estimation unit 26 receives the updated lesion suspected region malignancy estimation parameter, updates the malignancy estimation result regarding the displayed suspected lesion region image, and displays the result. Output to. That is, in the image processing apparatus of the present embodiment, the suspected lesion area malignancy estimation unit 26 uses the suspected lesion area malignancy learning update unit 29 to update the suspected lesion area image. The malignancy of the suspicious area is estimated and displayed again.

Here, the operation of the display unit 11 will be described with reference to FIG. FIG. 9 shows an example of a user interface 91 that displays diagnostic images, diagnosis results using the image processing apparatus 100, image feature label presentation for user input, malignancy correctness information, and the like. . The display unit 11 receives the diagnostic image and the suspected lesion area image 27. Further, the display unit 11 receives image feature labels corresponding to the respective suspected lesion area images from the image feature amount extraction unit 23. Further, the display unit 11 receives from the suspected lesion malignancy estimation unit 26 the malignancy estimation result and the estimated score corresponding to each suspected lesion image. The user interface 91 displays a diagnostic image, a suspected lesion area image area 92, an image feature label presentation selection area 94, a malignancy degree correct / incorrect information selection area 95, and the like.

The diagnostic image and suspected lesion area image area 92 of the user interface 91 display a diagnostic image and a suspected lesion area image. At that time, it is possible to rank and display the suspected lesion region images using the malignancy estimation result. That is, the display unit 11 displays the image data, the suspected lesion area image, and the identification result of the image feature label corresponding to the suspected lesion area image, ranks the estimated score of the suspected lesion area malignancy, It is possible to rearrange the suspected lesion region images in descending order of malignancy. In addition, a location corresponding to the selected suspicious lesion image 93 can be displayed on the diagnostic image using a mark or the like.

Further, the image feature label presentation selection area 94 of the user interface 91 displays an image example of a predetermined image feature label, presents an image feature label corresponding to the selected suspicious lesion region image, and selects it for the image feature user. Or can be modified. The user interface 91 can display the image of the image feature label and the identification result of the image feature label corresponding to the suspected lesion area image, and allow the user to select the correct image feature label corresponding to the suspected lesion area image. Therefore, a check box is arranged below a predetermined image feature label displayed in the image feature label presentation selection area 94. Still further, in the malignancy correctness / incorrectness information selection area 95 of the user interface 91, the correctness / incorrectness information TP, FP, TN, FN of the malignancy corresponding to the selected suspicious lesion region image is displayed, and the user can select any one of four types. Can be selected. That is, the user interface 91 can allow the user to select correct / incorrect information corresponding to the suspected lesion area image.

Furthermore, the user interface 91 can allow the user to add a new image feature label in addition to the predetermined image feature label displayed in the image feature label presentation selection area 94. In other words, the user interface 91 can cause the user to newly add a suspected lesion area image in addition to a suspected lesion area image, and select an image feature label and correct / incorrect information corresponding to the image. That is, the image feature label learning unit 21 of the image processing apparatus according to the present embodiment adds the type of the image feature label according to the image feature label newly added from the user input unit 10 regarding the suspected lesion area image, Learning parameters can be updated. Further, the user can add a new suspected lesion area image by using the diagnostic image and the diagnostic image displayed in the suspected lesion area image area 92, and can select malignancy correctness / incorrectness information corresponding thereto. .

In the first embodiment described above, the image processing apparatus 100 does not include a medical image capturing apparatus. However, the image processing apparatus 100 may include a medical image capturing apparatus, and the image processing apparatus 100 is one of the medical image capturing apparatuses. It may function as a part. According to the present embodiment, image processing that enables adjustment of the extraction process of the image feature amount of the lesion shadow and can reduce the burden on the interpreter who is the user when interpreting a large amount of three-dimensional medical images. An apparatus and a medical image photographing apparatus can be provided.

In the second embodiment, in addition to a predetermined image feature label, a user defines and adds a new image feature label, and further adds a suspected lesion area image corresponding to the newly added image feature label to the image DB. It is an Example of an image processing apparatus provided with DB update part. FIG. 10 shows an example of a system configuration including the image processing apparatus according to the second embodiment.

As illustrated in FIG. 10, the image processing apparatus 100 according to the present exemplary embodiment includes a user input unit 10, an image feature label learning unit 21, an image feature label learning parameter storage unit 22, an image feature amount extraction unit 23, and a suspected lesion. Region malignancy learning unit 24, suspected lesion malignancy estimation parameter storage unit 25, suspected lesion malignancy estimation unit 26, suspected lesion malignancy learning update unit 29, medical image DB update unit 30, and display Part 11.

Next, operation processing of the image processing apparatus 100 shown in FIG. 9 will be described using the flowchart shown in FIG. In step S601, the medical image DB update unit 30 receives the displayed suspected lesion area image. In step S 602, the medical image DB update unit 30 receives an image feature label corresponding to the suspected lesion region image added by the user from the user input unit 10. In step S 603, the suspected lesion malignancy learning update unit 29 receives from the user input unit 10 correctness / wrongness information of malignancy corresponding to the displayed lesion suspected region image input by the user. In step S 604, the medical image DB update unit 30 updates the medical image DB 20 using the displayed diagnostic image, suspected lesion area image, and a newly added image feature label corresponding thereto.

In step S605, it is determined whether to update the image feature amount extraction process. For example, the image feature amount extraction processing may be updated when the medical image DB update unit 30 acquires a predetermined number of images. Further, the image feature amount extraction process may be updated when a predetermined accumulation period has elapsed. Further, the image feature amount extraction process may be updated in accordance with a user instruction. When updating the image feature amount extraction processing (Yes), in step S606, the image feature label learning unit 21 performs machine learning for classifying the image feature label including the added image feature label, and sets the learning parameter. Generate. This is the same as the processing in step S202.

Note that the learning of the image feature label in step S606 may use a network configuration using the CNN method shown in FIG. Further, the network configuration shown in FIG. 12 may be used. That is, as shown in the network configuration of FIG. 12, for each class of image feature label 120, one multi-class image feature using one convolution layer 121, pooling layer 122, identification layer 123, and result 124. A CNN network for identifying the label may be set.

According to the configuration of the present embodiment, in addition to a predetermined image feature label, a user defines and adds a new image feature label to new clinical data, and further, a lesion suspected region image corresponding to the newly added image feature label Can be added to the image DB.

In addition, this invention is not limited to the above-mentioned Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

In addition, each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a memory, a hard disk, a recording device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

According to the present invention described above, it is possible to adjust the extraction process of the image feature amount of the lesion shadow, and to reduce the burden on the interpreter who is the user when interpreting a large amount of three-dimensional medical images. The above description includes various inventions in addition to the present invention described in the claims. An example is given below.

<Example 1>
An image processing apparatus that presents a suspected lesion image detected from image data,
An image feature amount extraction unit that extracts an image feature amount of the suspected lesion region image using a learning parameter of the image feature label obtained by learning for classifying image feature labels related to the suspected lesion region image;
Using the image feature amount extracted by the image feature amount extraction unit, using the parameter for estimating the malignancy level of the suspected lesion area obtained by learning for estimating the malignancy level of the suspected lesion area, the malignancy of the suspected lesion area A suspected lesion malignancy estimation unit for calculating
An image processing apparatus comprising:

<Example 2>
An image processing apparatus described in Example 1,
A display unit for displaying a suspected lesion image;
A user input section;
An image feature label learning update unit that updates the learning parameter in response to an input from the user input unit;
An image processing apparatus.

<Example 3>
An image processing apparatus described in Example 2,
When the image feature label learning update unit has acquired a predetermined number of suspected lesion area images, or when the acquired suspected lesion area image has passed a predetermined accumulation period, or when the user's instruction is input, The image feature amount extraction unit updates the image feature amount extraction process.
An image processing apparatus.

<Example 4>
An image processing apparatus described in Example 2,
The image feature label is a type of image feature related to the suspected lesion area.
An image processing apparatus.

<Example 5>
An image processing apparatus described in Example 4,
The image feature label is the size of the shadow area of the suspected lesion area, the brightness, the presence or absence of contact with the surrounding existing structure, the occurrence site, and the shape.
An image processing apparatus.

<Example 6>
An image processing method of an image processing apparatus for presenting a suspected lesion area image detected from image data,
The image processing device
Learning to classify image feature labels related to the suspected lesion image,
Using the learning parameter of the image feature label obtained by the learning, extract the image feature amount of the suspected lesion image,
Using the extracted image feature amount, learn the suspicious lesion malignancy for estimating the malignancy of the suspicious region,
Using the lesion suspicious area malignancy estimation parameter obtained by learning the lesion suspicious area malignancy, calculating the malignancy of the suspected lesion area,
An image processing method.

<Example 7>
An image processing method described in Example 6,
The image feature label is the size of the area of the shadow region, which is the type of the image feature related to the suspected lesion region, the brightness, the presence or absence of contact with the surrounding existing structure, the occurrence site, or the shape.
An image processing method.

<Example 8>
An image processing method described in Example 6,
The image processing apparatus includes a display unit and a user input unit,
Displaying the suspected lesion area image and the malignancy of the suspected lesion area on the display unit;
Updating the learning parameter in response to an input from the user input unit;
An image processing method.

<Example 9>
An image processing method described in Example 8,
When the image processing apparatus acquires a predetermined number of suspected lesion area images, when the acquired suspected lesion area image has passed a predetermined accumulation period, or when an instruction from the user is input, the image feature Update quantity extraction process,
An image processing method.

10 User Input Unit 11 Display Unit 20 Medical Image DB
21 Image feature label learning unit 22 Image feature label learning parameter storage unit 23 Image feature amount extraction unit 24 Suspicious region malignancy learning unit 25 Suspicious region malignancy estimation parameter storage unit 26 Suspicious region malignancy estimation unit 27 Diagnostic image And suspected lesion image 28 image feature label learning update unit 29 suspected lesion region malignancy learning update unit 30 medical image DB update units 61-66, 70, 120 image feature labels 71, 81, 121

convolutional layers

72, 82, 122 pooling

Layers

73, 83, 123 Discrimination layers 74, 84, 124 Result 80 Suspected lesion image 91 Interface 92 Diagnosis image and suspected lesion image area 93 Selected suspected lesion image 94 Image feature label presentation selection area 95 Grade accuracy Information selection area 100 Image processing apparatus

Claims

An image processing apparatus that presents a suspected lesion image detected from image data,
An image feature label learning unit that performs learning to classify image feature labels related to the suspected lesion image;
An image feature amount extraction unit that extracts an image feature amount of the suspected lesion region image using a learning parameter of the image feature label obtained by learning of the image feature label learning unit;
A display unit for displaying the suspected lesion area image;
A user input section;
An image processing apparatus comprising: an image feature label learning update unit that updates the learning parameter in response to an input from the user input unit.
The image processing apparatus according to claim 1,
The image feature amount extraction unit includes:
Using the learning parameters updated by the image feature label learning update unit, the image feature amount of the suspected lesion area image is extracted again.
An image processing apparatus.
The image processing apparatus according to claim 1,
Using the image feature amount extracted by the image feature amount extraction unit, a lesion suspected region malignancy learning unit for estimating the malignancy of a suspected lesion region,
Using the lesion suspected area malignancy estimation parameter obtained in the suspected lesion area malignancy learning unit, the lesion suspected area malignancy estimation unit for calculating the malignancy of the suspected lesion area,
A suspected lesion area malignancy learning update unit that updates the suspected lesion area malignancy estimation parameter in response to an input from the user input unit for each of the suspected lesion area images;
An image processing apparatus.
The image processing apparatus according to claim 3,
The suspected lesion area malignancy estimation unit is:
Using the lesion suspicious area malignancy estimation parameter updated from the lesion suspicious area malignancy learning update unit, reestimating the malignancy of the lesion suspicious area related to the lesion suspicious area image,
An image processing apparatus.
The image processing apparatus according to claim 1,
The image feature amount extraction unit identifies the type of the image feature label related to the lesion suspicious area image using the learning parameter.
An image processing apparatus.
The image processing apparatus according to claim 1,
The image feature label learning unit
According to the image feature label newly added from the user input unit related to the suspected lesion region image, the type of the image feature label is added, and the learning parameter is updated.
An image processing apparatus.
The image processing apparatus according to claim 1,
The display unit
Displaying the image data, the suspected lesion area image, and the identification result of the image feature label corresponding to the suspected lesion area image, ranking the estimated score of the suspected lesion area malignancy, Rearrange the suspected lesion image in descending order of
An image processing apparatus.
The image processing apparatus according to claim 7,
The display unit
Displaying the image of the image feature label and the identification result of the image feature label corresponding to the suspected lesion region image, and allowing the user to select the correct image feature label corresponding to the suspected lesion region image;
An image processing apparatus.
The image processing apparatus according to claim 7,
The display unit
Allowing the user to select correct / incorrect information corresponding to the suspected lesion image,
An image processing apparatus.
The image processing apparatus according to claim 7,
The display unit
In addition to the suspected lesion area image, the user is allowed to add a new suspected lesion area image and select the corresponding image feature label and correct / incorrect information.
An image processing apparatus.
The image processing apparatus according to claim 1,
The image feature label learning unit
For each image feature label class, set up a CNN (Convolutional Neural Network) network,
Among the learning data for learning each of the CNN networks, positive sample data is an image belonging to the respective image feature label class, and negative sample data is an image belonging to the other image feature label class. is there,
An image processing apparatus.
An image processing method of an image processing apparatus that includes a display unit and a user input unit and presents a suspected lesion area image detected from image data,
The image processing apparatus includes:
Learning to classify image feature labels related to the suspected lesion image,
Using the learning parameter of the image feature label obtained by the learning, extract the image feature amount of the suspected lesion image,
Displaying the suspected lesion image on the display unit;
Updating the learning parameter in response to an input from the user input unit;
An image processing method.
The image processing method according to claim 12,
The image processing apparatus includes:
Using the updated learning parameters, the image feature amount of the lesion suspicious area image is extracted again.
An image processing method.
The image processing method according to claim 12,
The image processing apparatus includes:
Identifying the type of image feature label for the suspected lesion image using the learning parameter;
An image processing method.
The image processing method according to claim 12,
The image processing apparatus includes:
According to the image feature label newly added from the user input unit related to the suspected lesion region image, the type of the image feature label is added, and the learning parameter is updated.
An image processing method.