JP4874701B2 - Similar image retrieval apparatus and method, and program - Google Patents

Description

  The present invention relates to a similar image search apparatus and method for searching for an image similar to an image to be processed, and a program therefor.

  Conventionally, image diagnosis for diagnosing a subject by interpreting an X-ray image or CT image of the subject has been performed in various fields. In image diagnosis, it may be difficult to make an appropriate diagnosis if the image reader is inexperienced or there is an experience, but the diagnosis target is an unusual case. In order to solve such a problem, for example, it is considered effective to make a diagnosis while referring to a case of an image similar to an image to be diagnosed in past cases.

Therefore, a large number of images that have been diagnosed in the past and diagnostic reports that include the diagnosis results and findings at that time are stored so that diagnosis can be performed while referring to past cases of images similar to the images to be diagnosed. A method of searching for an image similar to an image to be diagnosed and a diagnosis report thereof from a case database and displaying the search result has been proposed by the present applicant (Patent Documents 1 and 2).
Japanese Patent Laid-Open No. 2005-065728 JP 2004-005364 A

  By the way, recently, a plurality of images including the same part of the same subject with different imaging devices (imaging modalities) used for imaging, for example, a CT image and an MRI image, can be simultaneously read, and the symptom of the subject can be detected. For example, in Japanese Patent Application Laid-Open No. 2003-000575, an imaging modality composed of a mammogram (breast radiation image) and an ultrasound image (echo) can be narrowed down and diagnosis reliability can be improved. Discloses a method for determining whether a tumor is benign or malignant using two different images.

  However, the method of searching for an image related to a past case similar to the image to be diagnosed and displaying the search result is a method of searching for an image of a single imaging modality, and the imaging modality is different. There was no idea of searching for multiple images. In other words, the above search method can obtain the benefit of improving the reliability of diagnosis by looking at cases similar to the image to be diagnosed, but can interpret images with different imaging modalities from this benefit. Therefore, there is a problem that it is impossible to simultaneously obtain the benefit of improving the reliability of diagnosis.

  In view of the above circumstances, the present invention improves the reliability of diagnosis by interpreting images having different imaging modalities and the benefit of being able to see similar images related to past cases similar to images to be diagnosed It is an object of the present invention to provide a similar image retrieval apparatus and method capable of simultaneously obtaining the benefit of being able to perform the same and a program therefor.

  The similar image search device of the present invention stores a large number of image sets including an image obtained by photographing a subject with the first photographing device and an image obtained by photographing the subject with the second photographing device. An image database and a first image obtained by photographing the same kind of subject as the subject with the first photographing device and a subject identical to the first image are obtained by photographing with the second photographing device. Image input means for receiving the input of the second image as a search target, first feature quantity acquisition means for calculating and acquiring feature quantities for the first and second images, and storing them in the image database Second feature amount acquisition means for acquiring a feature amount for an image obtained by photographing with the first and second photographing devices of the image set for each image set, and the first feature The feature amount acquired by the amount acquisition means Comparing the feature quantity acquired by the second feature quantity acquisition means, the overall similarity between the first and second images and the images of the image set stored in the image database Similarity calculation means for calculating each image set and image sets included in the upper predetermined number in descending order of the overall similarity from among a large number of image sets stored in the database. It is characterized by comprising search means for searching as a set, and output means for outputting search result information representing a search result by the search means.

  In the similar image search device of the present invention, the similarity calculating means is a photographing device used for photographing between an image set composed of the first and second images and an image set stored in the image database. For each combination of images having the same image quality, a similarity calculation unit for each photographing device that calculates the similarity between the images by comparing the feature amounts of both images constituting the combination with each other, and calculating for each combination And a total similarity calculating unit that calculates the total similarity using a plurality of similarities including the similarities.

  In the similar image search device of the present invention, the similarity calculation unit includes the feature amount acquired by the first feature amount acquisition unit and the feature amount acquired by the second feature amount acquisition unit. Each may be expressed by one vector, and the overall similarity may be calculated based on the difference between the vectors.

  The similar image search device of the present invention further comprises a diagnostic database storing diagnostic data related to the image set stored in the image database, and the search means is related to the searched similar image set. The diagnostic data may be searched from the diagnostic database, and the output means may output the searched diagnostic data as the search result information together with the related similar image set.

  The diagnosis data may be any one or more of disease name, treatment policy, findings, patient name, doctor name in charge, imaging date and time, follow-up information, diagnosis information, and definitive diagnosis information.

  The similar image search device of the present invention further includes a region-of-interest setting unit that sets a region of interest on the first image and / or the second image, and the first feature amount acquiring unit is the set item. The feature amount of the image of the region of interest may be calculated.

  The similar image search device of the present invention further includes alignment means for aligning a subject between the first and second images and obtaining a corresponding positional relationship between the images, and the region of interest setting means includes A region-of-interest specifying means for accepting specification of a region of interest on the first image, and a region on the second image corresponding to the region of interest specified on the first image on the second image The region of interest includes a corresponding region-of-interest determining unit that determines the region based on the corresponding positional relationship between the images, and the first feature amount acquiring unit includes the interest on the first and second images. The feature amount of the image in the area may be calculated.

  The similar image retrieval method of the present invention is obtained by photographing a first image obtained by photographing a subject with the first photographing device and a subject identical to the first image with the second photographing device. An image input step of accepting the input of the second image as a search target, a first feature amount acquisition step of calculating and acquiring a feature amount for the first and second images, and the same type as the subject For each image set in the image database storing a large number of image sets including an image obtained by photographing the subject with the first photographing device and an image obtained by photographing the subject with the second photographing device, A second feature amount acquisition step for acquiring a feature amount for an image obtained by photographing with the first and second imaging devices in the image set, and a feature acquired by the first feature amount acquisition step. Amount and the second feature amount Comparing the feature amounts acquired in the step, the overall similarity between the first and second images and the images of the image set stored in the image database is determined for each image set. A similarity calculating step for calculating, and a search step for searching, as a similar image set, image sets included in the upper predetermined number in descending order of the overall similarity from among a large number of image sets stored in the database. And an output step for outputting search result information representing a search result by the search step.

  The program of the present invention stores a large number of image sets including an image obtained by photographing a subject with a first photographing device and an image obtained by photographing the subject with a second photographing device. An image database and a first image obtained by photographing the same kind of subject as the subject with the first photographing device and a subject identical to the first image are obtained by photographing with the second photographing device. Image input means for receiving the input of the second image as a search target, first feature quantity acquisition means for calculating and acquiring feature quantities for the first and second images, and storing them in the image database Second feature amount acquisition means for acquiring a feature amount for an image obtained by photographing with the first and second photographing devices of the image set for each image set, and the first feature Acquired by quantity acquisition means Comparing between the first and second images and the images of the image set stored in the image database by comparing the feature amount with the feature amount acquired by the second feature amount acquisition unit A similarity calculation means for calculating a global similarity for each image set, and images included in the upper predetermined number in descending order of the overall similarity from a large number of image sets stored in the database The computer is caused to function as a similar image search device by functioning as search means for searching the set as a similar image set, and output means for outputting search result information representing the search result by the search means. It is.

  The image database and the diagnostic database do not need to be a single database, but may be a plurality of databases. For example, it may be composed of a plurality of databases existing in a single facility, or may be a plurality of databases scattered in a plurality of different facilities that can be connected via a network. That is, it includes a form in which images and diagnostic data in a database existing in each facility are shared and used.

  Further, the image sets stored in the database do not need to be stored as a set in the same place. If they are stored in association with each other based on the subject or diagnosis, they are stored in different places. May be.

  The search result information output by the output means means a searched image set itself, a data file name for specifying the searched image set, and the like.

  Here, the feature amount is, for example, the shape, size, shading pattern characteristics, position, direction, number of shadows appearing in the image, shading pattern characteristics of the entire image, structure (lung, heart, etc.) In addition to index values representing the shape and size of the image, index values used to determine the degree of malignancy of the detected abnormal shadow (for example, in the case of a mass shadow, circularity, edge information of the contour, region) Density histogram information, etc .; see Japanese Patent Application Laid-Open No. 9-167238), and the like, which can determine similarity based on image data. When a region of interest is set in the image, an index value indicating the shape, size, shading pattern characteristic, etc. of the shadow appearing in the image of the region of interest is also included in the feature amount.

  According to the similar image search apparatus and method of the present invention, input of a plurality of images having different shooting modalities representing the same subject is received, and feature amounts are calculated for the plurality of images, while being stored in the database. For an image set including a plurality of images with different shooting modalities representing the same subject of the same type as the subject, a feature amount is set for an image having the same shooting modality as the plurality of images among images constituting the image set. Obtaining and comparing the feature quantities related to the plurality of images with the feature quantities related to the image set stored in the database to calculate the overall similarity, and in the descending order of the overall similarity, the top predetermined Since a search is made for a number of image sets from the database and the search results are output, the diagnosis results are selected from a large number of image sets related to past cases. An image set similar to a plurality of images with different imaging modalities as elephants can be searched and output, and similar images related to past cases similar to images to be diagnosed can be viewed, and imaging By reading images having different modalities, it is possible to simultaneously obtain the benefit of improving the reliability of diagnosis.

  In addition, since images similar to the image to be diagnosed are searched not for each image but for each image set, a plurality of images having similar image characteristics but low relevance between images are similar. An image that is meaningful in terms of diagnosis is retrieved without being retrieved as an image.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of a similar image search system according to an embodiment of the present invention. This similar image search system captures images when a set of a plurality of medical images having different imaging modalities obtained by imaging the same part of the same patient as the same subject with different types of imaging devices is input as a search target. A similar image set having the same modality as that of this image set and similar features such as an image and diagnostic data related to the similar image set are searched from the database, and the search result is output. Here, the image data represents an image, and the following description will be given without particularly distinguishing the image from the image data.

  The similar image retrieval system according to the present embodiment stores a case database 10 that stores a large number of case image sets including a plurality of images having different imaging modalities representing the same subject and diagnostic data related to the case image sets. And an image input unit (image input means) 20 that accepts input of a plurality of images S1 and S2 having different shooting modalities representing the same subject as search targets, and the images S1 and S2 are aligned with each other. An image registration unit (positioning unit) 30 that acquires the corresponding positional relationship B12 of the two, and a region of interest designating unit (region of interest designating unit) 40 that designates a region of interest (ROI) on one of the images S1 and S2. The region on the other image corresponding to the region of interest on the one image is set as the region of interest on the other image based on the corresponding positional relationship B12. A corresponding region-of-interest determination unit (corresponding region-of-interest determination means) 50, and a first feature amount acquisition unit (first unit) that calculates and acquires feature amounts of the images S1 and S2 and the image of the region of interest. Feature amount acquisition means) 60 and a second feature amount acquisition unit (second feature amount acquisition means) for acquiring feature amounts for images having the same imaging modality as the images S1 and S2 among the images constituting the case image set. ) 70 and the feature amount related to the image set SS composed of the images S1 and S2 and the feature amount related to the case image set are compared, and a similarity calculation unit that calculates the overall similarity LT between the two image sets ( (Similarity calculation means) 80, a search unit (search means) 90 for searching case image sets and their diagnostic data included in the upper predetermined number from the case database 10 in descending order of the total similarity LT, and a search unit 9 And an output unit (output means) 100 for outputting the search result information indicating the search results, and a display unit 110 for displaying the search result information output by the output unit 100 on the monitor screen by.

  The case database 10 includes an image database 11 that stores a large number of case image sets including a plurality of images having different imaging modalities that represent the same subject, and a diagnostic database 12 that stores diagnostic data related to each case image set. ing.

  The image database 11 includes a CT image, an MRI image, a PET image, a simple X-ray image, and the like obtained by imaging the same subject using a CT imaging device, an MRI imaging device, a PET imaging device, an X-ray imaging device, or the like. A large number of case image sets, which are a series of images and have been diagnosed in the past, are stored for each subject and for each diagnosis. Note that processing result information obtained by processing performed at the time of past diagnosis is attached to the images constituting the case image set. This processing result information includes various feature amounts calculated for the image, information on the position of the region of interest set on the image, feature amounts calculated for the image of the region of interest, and the like. .

  The diagnosis database 12 stores diagnosis data related to each case image set stored in the image database 11 in a state associated with each case image set. This diagnosis data includes various data relating to diagnosis such as disease name, diagnosis information, definitive diagnosis information, treatment policy, findings, doctor name in charge, photographing date and time, follow-up information, and the like. When the diagnostic database 12 is constructed, diagnostic data, particularly diagnostic information and definitive diagnostic information may be manually registered and stored in the diagnostic database, or PACS and electronic medical record data may be automatically stored. You may make it memorize | store in the form collected.

  The image input unit 20 receives an input of a plurality of images S1 and S2 having different imaging modalities representing the same subject, which are images to be searched. Here, as an example, a CT image S1 representing the same part of the same patient. And the input of the MRI image S2.

  As shown in FIG. 2, the image aligning unit 30 is a CT standard density distribution model image M1 (first standard density distribution model image) representing a standard density distribution of an image obtained by imaging with a CT imaging apparatus. ) And a standard density distribution model image M2 for MRI (second standard density) representing a standard density distribution of an image obtained by photographing with an MRI photographing apparatus. Corresponding positions of corresponding anatomical structures between the second standard concentration distribution model image storage unit 32 storing the distribution model image), the CT standard concentration distribution model image M1, and the MRI standard concentration distribution model image M2. Corresponding anatomical structures are aligned between the CT image correspondence position storage unit 33 that stores the BM 12, and the CT image S1 and the CT standard density distribution model image M1, and a first corresponding position BS1 is obtained. Take The second position for acquiring the second corresponding position BS2 by aligning the corresponding anatomical structure between the first position matching unit 34 and the MRI image S2 and the standard density distribution model image M2 for MRI. The matching position B12 and the corresponding position B12 of the anatomical structure between the CT image S1 and the MRI image S2 are the same as the corresponding position BM12 between the CT standard density distribution model image M1 and the MRI standard density distribution model image M2. The corresponding position acquisition unit 36 is obtained from the corresponding position BS1 and the second corresponding position BS2.

  The CT image S1, the MRI image S2, the CT standard density distribution model image M1, and the MRI standard density distribution model image M2 are volume data composed of a large number of voxel data, and hold density value information for each voxel data. Anatomical structures such as bones and organs are represented by shades of voxel data.

  The standard density distribution model image is an anatomical structure based on a large number of CT images and MRI images taken in the past, in which a normal person has a standard body shape and tissues such as internal organs in a standard size and position. Is a model image representing a standard density value distribution. Since the density value that appears in each image varies depending on the modality that was taken, the CT standard density distribution model image M1 represented by the standard density value that appears on the image when taken with the CT imaging apparatus, and the MRI imaging apparatus Then, an MRI standard density distribution model image M2 represented by standard density values appearing on the image is prepared.

  These standard density distribution model images can acquire the average position and density value of each organ from a number of images taken in the past. In addition, a model image generated so that each voxel data included in the standard concentration distribution model image belongs to which anatomical structure (bone, viscera, fat, etc.) belongs is preferable. For example, if coordinate values (x, y, z) are given to the CT standardity distribution model image M1 (CT_Model), the density value and the tissue name can be understood as shown in the following equations (1) and (2). 1 standard density distribution model image storage unit 31.

Density value CT_Model_intensity (x, y, z) = 100 (1)
Organization name CT_Model_tissue (x, y, z) = Liver (2)
Similarly, when coordinate values (x, y, z) are given to the MRI standard density distribution model image M2 (MRI_Model), the density value and the tissue name can be understood as shown in the following equations (3) and (4). The data is stored in the second standard density distribution model image storage unit 32.

Concentration value MRI_Model_intensity (x ', y', z ') = 80 (3)
Organization name MRI_Model_tissue (x ', y', z ') = liver (4)
In the inter-model image corresponding position storage unit 33, the corresponding positions BM12 of the anatomical structures of the CT standard density distribution model image M1 and the MRI standard density distribution model image M2 are stored in correspondence with each voxel data. To do. That is, a relationship represented by the following formula (5) is obtained in advance.

CT_Model (x, y, z) = MRI_Model (x ', y', z ') (5)
This corresponding position may be obtained manually.

  Here, a registration method by the image registration unit 30 will be described.

  First, the first alignment unit 34 aligns the CT image S1 and the CT standard density distribution model image M1 to obtain a first corresponding position BS1. Images taken with the same imaging modality are more accurate than the alignment of images taken with different imaging modalities because the density distribution of anatomical structures such as organs and tissues shown on the image are similar. Good alignment is possible.

  For example, the body surface is recognized based on the pixel value of the CT image S1, and the shape of the body surface of the CT standard density distribution model image M1 stored in the first standard degree distribution model image storage unit 31 substantially matches. As described above, alignment is performed using a technique such as warping to obtain the first corresponding position BS1.

  In this way, the first corresponding position BS1 as in the following equation (6) is obtained between the CT image S1 (CT_Data) and the CT standard density distribution model image M1.

CT_Model (x1, y1, z1) = CT_Data (x, y, z) (6)
Similarly, the second alignment unit 35 aligns the MRI image S2 (MRI_Data) and the MRI standard density distribution model image M2 stored in the second standard density distribution model image storage unit 32. Then, a second corresponding position BS2 as shown in the following equation (7) is obtained.

MRI_Model (x2, y2, z2) = MRI_Data (x ', y', z ') (7)
The corresponding position acquisition unit 36 obtains the coordinate values (x ′, y ′,...) Of the MRI image S2 corresponding to the coordinate values (x, y, z) of the CT image S1 from the above equations (5), (6), (7). z ′) can be obtained. By obtaining this for all voxel data, the CT image S1 and the MRI image S2 are aligned, and a corresponding positional relationship B12 between these images is obtained.

  The region-of-interest designating unit 40 designates a region of interest on the CT image S1 or MRI image S2 to be searched, and the operator views the image displayed on the monitor screen of the display unit 110 while viewing the image. For example, a portion suspected of being a lesion is designated using a position indicating means (not shown) such as a mouse.

  When the region of interest is specified on one of the CT image S1 or the MRI image by the region of interest specifying unit 40, the corresponding region of interest determining unit 50 selects the region of interest on the other image corresponding to the region of interest on the one image. The region is determined as a region of interest on the other image based on the corresponding positional relationship B12 between the CT image S1 and the MRI image S2 acquired by the image alignment unit 30, and for example, on the CT image S1. When the region of interest R1 is designated, the region on the MRI image S2 representing the same subject portion as the subject portion corresponding to the region of interest R1 is automatically determined as the region of interest R2.

  In general, when an operator finds a suspected lesion in an image to be diagnosed, the operator designates that portion as a region of interest and calculates the similarity using the feature amount of the image of the region of interest. In the case where there are a plurality of images to be diagnosed, it is conceivable that the same subject portion on the plurality of images is set as a region of interest. In such a case, as described above, if the same subject portion on a plurality of images is set as a region of interest by only one designation, the burden on the operator is reduced.

  The first feature amount acquisition unit 60 calculates and acquires a feature amount C1 for the CT image S1 and the MRI image S2. As the feature amount, for example, the shape of the shadow appearing in the image, Examples include size, shading pattern characteristics, position, direction, number, shading pattern characteristics of the entire image, and index values representing the shape and size of structures (lung, heart, etc.). When the region of interest R1 is set on the CT image S1, or when the region of interest R2 is set on the MRI image S2, the shape of the shadow appearing in the image of the region of interest, An index value representing the size, the density pattern characteristic, etc. is also calculated as a feature amount.

  The second feature amount acquisition unit 70 has images having the same imaging modality as the CT image S1 and the MRI image S2 among the images constituting the case image set, that is, the imaging modalities are “CT imaging apparatus” and “MRI imaging apparatus”. Is obtained in the processing result information attached to the images constituting the case image set, and is set on the various feature amounts already calculated for the image and on the image. When the calculated feature amount is included in the image of the region of interest, the feature amount is read and acquired. When such a feature amount is not included, a new feature amount is calculated. Is what you get.

  The similarity calculation unit 80 includes the feature amount acquired by the first feature amount acquisition unit 60, that is, the feature amount C1 related to the image set SS including the CT image S1 and the MRI image S2, and the second feature amount acquisition unit. The feature amount acquired by 70, that is, the feature amount C2 related to the case image set is compared to calculate the overall similarity LT between the two image sets. From the CT image S1 and the MRI image S2, For each combination of images having the same imaging modality between the image set SS and the case image set, the feature amounts in both images constituting the combination are compared with each other, and the similarities L1 and L2 between the images are compared. Using the similarity calculation unit for each modality (photographing device similarity calculation means) 81 and a plurality of similarities L1 and L2 calculated for each combination. And a general similarity calculation section (general similarity calculation means) 82 for calculating a.

  More specifically, the modality-specific similarity calculation unit 81 sets both of the images constituting the combination with respect to the combination of images having the imaging modality “CT imaging apparatus” between the image set SS and the case image set. The feature quantities acquired for the images are compared with each other to calculate a similarity L1 for this combination. For the combination of images whose imaging modality is “MRI imaging apparatus”, both images constituting this combination are used. Is obtained by comparing the feature quantities acquired for the combination, and calculating the similarity L1 for this combination. Here, various known methods can be considered as the similarity calculation method. For example, each feature amount group acquired for two images is represented by a point on a multidimensional feature amount space. A method is conceivable in which the similarity is calculated by a predetermined formula in which the similarity is higher as the Euclidean distance is smaller, and the similarity is smaller as the distance is larger.

  Further, the total similarity calculation unit 82 adds, for example, the similarity L1 and L2 calculated by the modality similarity calculation unit 81 using a predetermined weighting factor or takes a product, thereby obtaining the image set SS. And the total similarity LT between the case image set and the case image set is calculated.

  The search unit 90 sets the case image sets included in the upper predetermined number in the descending order of the overall similarity LT among the many case image sets stored in the image database 11 as the similar image set LS. And the diagnostic data related to the set of similar images is searched from the diagnostic database 12. Here, it is assumed that the search unit 90 searches for the case image sets included in the top four in descending order of the overall similarity LT as the similar image set LS.

  Next, the operation of the similar image search system of the present embodiment configured as described above will be described. FIG. 3 is a diagram showing a processing flow in the similar image search system according to the present embodiment.

  First, the image input unit 20 receives an input of an MRI image S2 of the same subject as the CT image S1 of a certain subject to be diagnosed, and the display unit 110 displays the CT image S1 and the MRI image S2 on the monitor screen. (Step ST1).

  FIG. 4 shows an MRI image S2 of the chest of the same subject as the CT image S1 of the chest of a subject to be diagnosed. A circular shadow Q suspected of being a tumor shadow appears in the left lung portion of the CT image S1 and the MRI image S2.

  Then, the image alignment unit 30 aligns the subject between the CT image S1 and the MRI image S2 by the above-described processing, and acquires the corresponding positional relationship B12 between the CT image S1 and the MRI image S2 (step S12). ST2).

  Here, when the operator designates a region suspected of being a lesion for one of the CT image S1 or the MRI image S2 displayed on the monitor screen of the display unit 110, the region of interest designating unit 40 The designated region is set as the region of interest on the one image (step ST3), and the corresponding region-of-interest determining unit 50 corresponds to the region designated on the one image based on the correspondence positional relationship B12. The region on the other image is determined as the region of interest on the other image (step ST4).

  In FIG. 5, the operator designates a region surrounding a shadow Q suspected of being a lesion on one of the CT image S1 or MRI image S2 as a region of interest, and a corresponding region on the other image is also a region of interest. The designated state is shown, and the region of interest R1 is set on the CT image S1, and the region of interest R2 is set on the MRI image S2.

  Here, when the operator clicks the “similar image search button” provided on the screen, the first feature quantity acquisition unit 60 performs the feature quantity, the image of the region of interest R1 and the interest of the CT image S1 and the MRI image S2. The feature amount C1 including the feature amount for the image in the region R2 is calculated and acquired (step ST5).

  When the feature quantity relating to the image set SS including the CT image S1 and the MRI image S2 is obtained by the first feature quantity acquisition unit 60, the second feature quantity acquisition unit 70 is stored in the image database 11. For each case image set, images having the same imaging modality as those of the CT image S1 and the MRI image S2 among images constituting the case image set, that is, images having the imaging modality of “CT imaging apparatus” and “MRI imaging apparatus”. In the processing result information identified and attached to these images, various feature quantities already calculated for the image, feature quantities calculated for the image of the region of interest set on the image, etc. Is included, the feature values are read and acquired.If these feature values are not included or if there are not enough feature values to be acquired, It is newly calculated amount, to obtain the feature quantity C2 of case image set (step ST6).

  When the feature amount C1 related to the image set SS including the CT image S1 and the MRI image S2 and the feature amount C2 related to each case image set stored in the image database 11 are acquired, the similarity calculation unit 81 by modality In addition, between the image set SS and each case image set, for the combination of images whose imaging modality is “CT imaging apparatus”, the feature amounts acquired for both images constituting this combination are compared. Then, the similarity L1 for this combination is calculated, and for the combination of images whose imaging modality is “MRI imaging apparatus”, the feature quantities acquired for both images constituting this combination are compared, A similarity L1 for this combination is calculated (step ST7). Then, the total similarity calculation unit 82 adds the similarities L1 and L2 calculated by the modality-specific similarity calculation unit 81 for each case image set using a predetermined weighting coefficient, thereby obtaining the image set SS. An overall similarity LT between the case image set is calculated (step ST8).

  When the overall similarity LT for each case image set is calculated, the search unit 90 selects the top four in descending order of the overall similarity LT from among a large number of case image sets stored in the image database 11. Is searched from the image database 11 as a similar image set LS, and diagnostic data related to the similar image set LS is searched from the diagnostic database 12 (step ST9). Then, the output unit 100 outputs search result information representing the search result, for example, a data file name representing the similar image set LS and related diagnostic data, image data of the similar image set LS, and diagnostic data itself. (Step ST10), the display unit 110 displays the data file name representing the image data of the similar image set LS and the related diagnostic data on the monitor screen, or the image or diagnostic data represented by the image data is displayed. The diagnostic information to be displayed is displayed on the monitor screen.

  FIG. 6 shows an example of the monitor screen of the display unit 110 after the similar image search is performed. On the monitor screen, a CT image S1 and an MRI image S2 in the upper left part, thumbnail images of the four similar image sets LS searched in the upper right part, and one similar image set selected from the similar image sets LS in the lower left part The enlarged image of LS1, and diagnostic data (diagnostic report) RPM1 related to the image selected from the similar image set which is an enlarged image are displayed in the lower right part.

  The operator can update and display an enlarged image of the image by clicking the desired thumbnail image, and can click on any of the enlarged similar image sets to display the image. The diagnostic report can be updated and displayed.

  As described above, according to the similar image search system of the present embodiment, input of a plurality of images S1 and S2 having different shooting modalities representing the same subject is received, and feature amounts are calculated for the plurality of images S1 and S2. On the other hand, with respect to a case image set stored in the image database 11 and including a plurality of images having different imaging modalities representing the same subject of the same type as the subject, the plurality of images among the images constituting the case image set A feature amount is acquired for images having the same imaging modality, and the feature amount related to the plurality of images S1 and S2 is compared with the feature amount related to the case image set stored in the database, The similarity LT is calculated, and the upper predetermined number of case image sets are searched from the image database 11 in descending order of the overall similarity TL, and the search result Since output is performed, a case image set similar to a plurality of images S1 and S2 having different imaging modalities to be diagnosed is searched for and output from a large number of case image sets relating to past cases. The benefit of being able to see similar images related to past cases that can be used as a reference during diagnosis and the benefit of improving the reliability of diagnosis by interpreting images with different imaging modalities be able to.

  In addition, according to the present embodiment, images similar to the images S1 and S2 to be diagnosed are searched not for each image, but for each image set. A plurality of images that are less relevant to each other are not retrieved as similar images, and images that are meaningful in diagnosis are retrieved.

  Further, according to the present embodiment, not only a similar image set similar to the plurality of images S1 and S2 to be diagnosed but also diagnostic data related to the similar image set is output. This makes it possible to provide effective diagnosis support information when an operator of this system is at a loss in diagnosis at the time of image interpretation.

  In this embodiment, in order to simplify the description, the imaging modality of the image to be searched is described as two types, “CT imaging apparatus (simple)” and “MRI imaging apparatus”. However, the imaging modality is “ Similar images in the case of three or more types such as “CT imaging apparatus (simple)”, “CT imaging apparatus (with contrast medium)”, “MRI imaging apparatus”, “RI imaging apparatus”, “PET imaging apparatus”, etc. Search can be performed. In this case, images of all shooting modalities may be searched as search targets, or images of two or more types of shooting modalities may be selected as search targets. For the similar image set LS obtained by the search, only the image of the imaging modality used for the search may be displayed on the screen, or all the images constituting the similar image set LS are displayed on the screen. You may make it do.

  In the present embodiment, it is assumed that the first feature quantity acquisition unit 60 and the second feature quantity acquisition unit 70 acquire a predetermined type of feature quantity. It is also possible to provide a selection means for selecting several types of the feature quantity types, and to acquire only the feature quantity of the type selected by the selection means.

  In this embodiment, when setting a region of interest on the CT image S1 and MRI image S2 that are diagnosis targets, the region of interest on one image is set manually, and the region of interest on the other image is automatically set. Of course, all of them may be set manually, or the region of interest may be set only on one of the images. In addition, the region of interest may be set as the region of interest as it is by using the image diagnosis support apparatus that automatically extracts the lesion.

  In the present embodiment, the similarity calculation unit 80 calculates the similarity by comparing the feature quantities acquired for each image constituting the combination for each combination of images having the same shooting modality. Further, the calculation is performed in two steps of calculating the overall similarity LT from the similarity calculated for each combination. For example, the image set acquired by the first feature amount acquisition unit 60 is used. The feature amount related to SS, C1, and the feature amount C2 related to the case image set acquired by the second feature amount acquisition unit 70 are each expressed by one vector, and based on the magnitude of the difference between the vectors, You may make it perform the calculation in a lump that calculates said comprehensive similarity LT.

Configuration diagram of similar image retrieval system according to this embodiment Configuration diagram of image alignment unit 30 The figure which shows the processing flow in the similar image search system by this embodiment. Diagram showing images with different shooting modalities that represent the same subject that is to be diagnosed and searched The figure which shows a mode that the region of interest was each set on two images used as search object Figure showing a display example of search results

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Case database 11 Image database 12 Diagnostic database 20 Image input part 30 Image registration part 31 1st standard density distribution model image memory | storage part 32 2nd standard density distribution model image memory part 33 Model image corresponding | compatible position memory part 34 1st 1 alignment unit 35 second alignment unit 36 corresponding position acquisition unit 40 region of interest designation unit 50 corresponding region of interest determination unit 60 first feature amount acquisition unit 70 second feature amount acquisition unit 80 similarity calculation unit 81 Modality similarity calculation unit 82 Total similarity calculation unit 90 Search unit 100 Output unit 110 Display unit

Claims (7)

An image database for storing a number of image sets including a medical image obtained by photographing a subject with the first photographing device and a medical image obtained by photographing the subject with the second photographing device;
Obtained by photographing the first medical image obtained by photographing the same kind of subject as the subject with the first photographing device and the same subject as the first medical image with the second photographing device. Image input means for accepting input of a second medical image as a search target;
First feature amount acquisition means for calculating and acquiring each feature amount of the first and second medical images;
For each image set stored in the image database, a second feature amount acquisition that acquires each feature amount of a medical image obtained by imaging with the first and second imaging devices in the image set. Means,
The feature quantity acquired by the first feature quantity acquisition unit and the feature quantity acquired by the second feature quantity acquisition unit are compared, and the first and second medical images and the image database are compared. Similarity calculating means for calculating a total similarity between the stored medical images and the medical images for each image set;
Search means for searching, as a similar image set, an image set included in the upper predetermined number in descending order of the overall similarity from a large number of image sets stored in the database;
Output means for outputting search result information representing a search result by the search means ,
The similarity calculation means includes:
For each combination of medical images in which the imaging device used for imaging is the same between the image set composed of the first and second medical images and the image set stored in the image database, A similarity calculation unit for each photographing apparatus that calculates the similarity between the two medical images by comparing the feature amounts of the two medical images that constitute each other;
A similar image search apparatus comprising: a total similarity calculating unit that calculates the total similarity using a plurality of similarities calculated for each combination . A diagnostic database storing diagnostic data associated with the image set stored in the image database;
The search means searches the diagnostic database for the diagnostic data related to the searched similar image set;
It said output means, the diagnostic data that is the search, the relevant the similar image set with the search results, characterized in that output as information claim 1 Symbol placement of the similar image retrieval apparatus. The diagnostic data, disease name, treatment policy, findings, patient name, doctor name, photographing date, observation information, diagnostic information, claims, characterized in that any one or more of definitive diagnosis information 2 The similar image search device described. A region of interest setting means for setting a region of interest on the first medical image and / or the second medical image;
The first feature amount acquisition unit, similar image retrieval apparatus 3 according claim 1, characterized in that calculates a feature quantity of the image of the set region of interest. Alignment means for aligning a subject between the first and second medical images and obtaining a corresponding positional relationship between the medical images;
The region of interest setting means is
A region-of-interest designating unit that accepts designation of a region of interest on the first medical image;
Based on the corresponding positional relationship between the medical images, the region on the second medical image corresponding to the region of interest designated on the first medical image is set as the region of interest on the second medical image. A corresponding region of interest determination means to determine,
5. The similar image search apparatus according to claim 4 , wherein the first feature quantity acquisition unit calculates a feature quantity of an image of a region of interest on the first and second medical images. Input of a first medical image obtained by photographing a subject with the first photographing device and a second medical image obtained by photographing the same subject as the first medical image with the second photographing device An image input step for accepting as a search target,
A first feature amount acquisition step of calculating and acquiring each feature amount of the first and second medical images;
A large number of image sets including a medical image obtained by photographing the same kind of subject as the subject with the first photographing device and a medical image obtained by photographing the same subject with the second photographing device are stored. A second feature amount acquisition step of acquiring, for each image set in the image database, each feature amount of a medical image obtained by photographing with the first and second photographing devices of the image set;
The feature quantity acquired by the first feature quantity acquisition step and the feature quantity acquired by the second feature quantity acquisition step are compared, and the first and second medical images and the image database are compared. A similarity calculation step for calculating, for each image set, a total similarity between the stored medical images and the medical images;
A search step of searching, as a similar image set, an image set included in the upper predetermined number in descending order of the overall similarity from a large number of image sets stored in the database;
Possess an output step of outputting the search result information indicating a search result by the search step,
The similarity calculation step includes:
For each combination of medical images in which the imaging device used for imaging is the same between the image set composed of the first and second medical images and the image set stored in the image database, A similarity calculation step for each imaging device that calculates the similarity between the two medical images by comparing the feature amounts in the two medical images that constitute each other, and
A similar image search method comprising: an overall similarity calculation step of calculating the overall similarity using a plurality of similarities including similarities calculated for each combination . Computer
An image database for storing a number of image sets including a medical image obtained by photographing a subject with the first photographing device and a medical image obtained by photographing the subject with the second photographing device;
Obtained by photographing the first medical image obtained by photographing the same kind of subject as the subject with the first photographing device and the same subject as the first medical image with the second photographing device. Image input means for accepting input of a second medical image as a search target;
First feature amount acquisition means for calculating and acquiring each feature amount of the first and second medical images;
For each image set stored in the image database, a second feature amount acquisition that acquires each feature amount of a medical image obtained by imaging with the first and second imaging devices in the image set. Means,
The feature quantity acquired by the first feature quantity acquisition unit and the feature quantity acquired by the second feature quantity acquisition unit are compared, and the first and second medical images and the image database are compared. Similarity calculating means for calculating a total similarity between the stored medical images and the medical images for each image set;
Search means for searching, as a similar image set, an image set included in the upper predetermined number in descending order of the overall similarity from a large number of image sets stored in the database;
A program for functioning as output means for outputting search result information representing a search result by the search means ,
The similarity calculation means includes:
For each combination of medical images in which the imaging device used for imaging is the same between the image set composed of the first and second medical images and the image set stored in the image database, A similarity calculation unit for each photographing apparatus that calculates the similarity between the two medical images by comparing the feature amounts of the two medical images that constitute each other;
A program comprising: a total similarity calculation unit that calculates the total similarity using a plurality of similarities calculated for each combination.

Images