JP2018106560A - Image processing apparatus, image processing method, image processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus that specifies a small area to be prioritized in performing processing for obtaining a conversion for deformation position alignment, and is capable of performing alignment of at least a partial area earlier compared to other areas.SOLUTION: An image processing apparatus acquires information on a first conversion for aligning a first image and a second image, determines an order of acquiring a second conversion for each of a plurality of partial regions constituting the first image, based on information on the accuracy of alignment in a first converted image obtained by converting the first image through the first conversion, and acquiring information relating to the second conversion for aligning to the second image, for each of the plurality of partial regions, according to the determined order.SELECTED DRAWING: Figure 3

Description

  The present disclosure relates to an image processing apparatus, an image processing method, an image processing system, and a program.

  When a doctor conducts medical care using medical images, a plurality of medical images obtained by photographing the same region at a plurality of time points may be observed in order to find a lesion site and follow up. In that case, in order to compare the region where the pathological condition has changed, there is a case where deformation position alignment is performed between two three-dimensional images at different time points when the image is taken.

  Non-Patent Document 1 divides the entire image into a plurality of small areas, searches for the vicinity of the small area of the other image corresponding to the small area of one image, and sets one image in the area with high similarity. It is disclosed that a shift amount for displacing the image is obtained, and a deformation for aligning one image with the other image is obtained based on the shift amount in each small region.

Kano A, Doi K, MacMahon H et al: Digital image subtraction of temporally sequential check for images of interference change. Med Phys 21 (3): 453-461, 1994

  It is considered that the accuracy of the alignment is improved by repeating the process for obtaining the transformation for the alignment between the two images, but the order of the areas to be aligned is not considered.

  An image processing apparatus according to an embodiment of the present invention includes a first acquisition unit configured to acquire information related to a first conversion for aligning a first image and a second image, and the first conversion. Based on the information regarding the accuracy of the alignment in the first converted image obtained by converting the first image, the second conversion is performed on each of the plurality of partial areas constituting the first image. A determination unit that determines an acquisition order; a second unit that acquires, for each of the plurality of partial areas, information related to a second transformation for aligning with the second image according to the determined order; Acquisition means.

  By specifying a small region in which priority is given to processing for obtaining transformation for deformation alignment, alignment of at least some of the regions can be performed earlier than other regions.

It is a figure which shows an example of a structure of the system containing the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the hardware constitutions of the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the function structure contained in the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the process performed by the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the process performed by the image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the process performed by the image processing apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
In the field of diagnosis and examination using medical images, it is possible to compare medical images of subjects taken at different times and observe changes over time. At that time, since the state of the subject imaged at each time is different, a process of aligning a plurality of images (deformation alignment) may be performed by deforming the image for the purpose of canceling this. . Furthermore, in order to facilitate the observation of the temporal change, a temporal difference image that is a difference between the medical images at different times subjected to deformation positioning may be acquired. In acquiring a time-difference image, it is preferable to emphasize only changes in a region related to a disease, but changes in regions other than a region related to a disease are reflected due to a difference in imaging state at each time. There is a case.

  Therefore, for the purpose of comparing medical images of subjects taken at different times and assisting a doctor in observing a region related to a disease, it is preferable that the deformation alignment is performed with high accuracy.

  On the other hand, a medical image to be observed by a doctor, such as a registered medical image or a time-difference image, can be generated by repeating a process of aligning a deformed position by processing a large amount of medical image data. The time required to do this will increase.

  In the first embodiment, the time required for the doctor to start observation can be reduced by accurately performing the positioning with priority given to the area that is presumed to be the area that the doctor wants to observe preferentially. The purpose is to do. The region that the doctor wants to observe preferentially is a region that has changed over time due to, for example, a disease progressing or healing. The image processing apparatus according to the first embodiment performs processing for deformation positioning by giving priority to an area where a difference is generated between two medical images.

[System configuration including image processing apparatus]
FIG. 1 is a diagram illustrating an example of the configuration of a system including an image processing apparatus according to the first embodiment. The client computer 100, the electronic medical record management apparatus 110, the examination order management apparatus 120, the imaging apparatus 130, the image management apparatus 140, and the report management apparatus 150 are connected via the network 160.

  The client computer 100 is a computer used by a user such as a doctor. The doctor uses the client computer 100 to communicate with the electronic medical record management apparatus 110, the examination order management apparatus 120, the image management apparatus 140, and the report management apparatus 150 via the network 160. The user can browse the electronic medical record stored in the electronic medical record management server 111 using the client computer 100, and can store the created electronic medical record in the server 111. The user can create an inspection order using the client computer 100 and transmit it to the imaging apparatus 130. The inspection order is stored in the inspection order management server 121. A user can manage processing performed in the imaging apparatus 130 using the client computer 100 and input an instruction for the processing. A user can acquire and browse a medical image from the image management server 141 using the client computer 100. The user can describe the result of observing the medical image using the client computer 100 in the report, and can save the report in the report management server 151.

  The electronic medical chart management apparatus 110 creates an electronic medical chart and provides a service for managing the created electronic medical chart. The electronic medical record management apparatus 110 is connected to the electronic medical record management server 111. The electronic medical record management server 111 is a server for storing the electronic medical record. The electronic medical record management apparatus 110 may be a HIS (Hospital Information System).

  The inspection order management apparatus 120 provides a service for creating an inspection order for the imaging apparatus 130 and the like, transmitting the inspection order to an appropriate imaging apparatus, and managing the inspection order. Further, the inspection order management apparatus 120 may provide a service that performs management related to inspection work by assigning devices, inspection rooms, inspection technicians, and the like used in the inspection. The inspection order management apparatus 120 is connected to the inspection order management server 121. The inspection order management server 121 is a server for storing inspection orders. The inspection order management apparatus 120 is, for example, a RIS (Radiology Information System).

  The photographing device 130 is a device for photographing a medical image. The imaging apparatus 130 may be, for example, an MRI (magnetic resonance imaging) apparatus, an X-ray CT (computed tomography) apparatus, an ultrasonic imaging apparatus, a photoacoustic tomography (PAT) apparatus, or a PET (positron emission CT) apparatus. It is at least one of a single photon emission computed tomography (OCT) device and an OCT (optical coherence tomography) device. The imaging apparatus 130 receives an examination order from the examination order management apparatus 120, obtains data based on the examination order, performs image processing as appropriate, and obtains a medical image. The imaging apparatus 130 transmits information regarding the progress of the inspection to the inspection order management apparatus 120. The imaging apparatus 130 transmits the medical image to the image management server 141 and stores it.

  In the first embodiment, the imaging apparatus 130 includes an image processing unit 300 that performs image processing for acquiring a medical image used for diagnosis by a doctor. The image processing unit 300 performs image processing for generating a difference image for comparing the medical image acquired by the imaging apparatus 130 with the past medical image of the same subject acquired from the image management server 141. From this point of view, the photographing apparatus 130 is an example of an image processing apparatus. Details of the configuration of the image processing unit 300 and the processing performed by the image processing unit 300 will be described later.

  The image management apparatus 140 provides a service for storing and managing medical images. The image management apparatus 140 is connected to the image management server 141. For example, the image management apparatus 140 stores the medical image acquired by the imaging apparatus 130 and the accompanying information related to the medical image in the image management server 141. The incidental information includes, for example, information related to the date of imaging of the medical image, the imaging device, imaging conditions, and the subject. The image management server 141 is, for example, a PACS (Picture Archiving and Communications Systems).

  The report management apparatus 150 provides a service for creating and managing an image diagnosis report describing a result of observation of a medical image by a doctor. The report management apparatus 150 is connected to the report management server 151. The report management server 151 stores the diagnostic imaging report.

  Information transmitted and received in the above system is based on DICOM (Digital Imaging and Communications in Medicine) standard. DICOM is a standard that defines the format of medical images and the communication protocol between devices that handle these images. Data to be exchanged based on DICOM is called an information object (IOD: Information Object Definitions). Hereinafter, the information object may be referred to as an IOD or an object. Examples of IODs include medical images, patient information, examination information, structured reports, and the like, and various data related to examinations and treatments using medical images can be targeted.

  A medical image that is handled based on DICOM, that is, a medical image that is an IOD is composed of supplementary information and image data. An IOD is composed of a collection of data elements called DICOM data elements. A tag for identifying the data element is added to each DICOM data element. For example, a tag indicating image data is added to the DICOM data element with respect to image data that is pixel data.

  In the first embodiment, a case where the photographing apparatus 130 includes the image processing unit 300 and the photographing apparatus 130 functions as an image processing apparatus will be described as an example. However, the present invention is not limited to this. For example, the image processing apparatus having the image processing unit 300 may be configured as an independent apparatus, or may be configured as an image processing server using a cloud.

[Hardware configuration of image processing apparatus]
FIG. 2 is a diagram illustrating an example of a hardware configuration of the image processing apparatus according to the first embodiment. In the first embodiment, the hardware configuration of a console (computer) that constitutes the photographing apparatus 130 is shown. The client computer 100, the electronic medical record management apparatus 110, the examination order management apparatus 120, the image management apparatus 140, and the report management apparatus 150 have the same hardware configuration. Further, the electronic medical record management server 111, the examination order management server 121, the image management server 141, and the report management server 151 may have the same hardware configuration, and may not have an unnecessary configuration as a server device.

  The image processing apparatus includes a CPU 200, ROM 201, RAM 202, storage device 203, USB 204, communication circuit 205, graphics board 206, and HDMI (registered trademark) 207. These are communicably connected by BUS. The BUS is used to transmit and receive data between connected hardware and to transmit commands from the CPU 200 to other hardware.

  A CPU (Central Processing Unit) 200 is a control circuit that integrally controls the image processing apparatus and each unit connected thereto. The CPU 200 performs control by executing a program stored in the ROM 201. Further, the CPU 200 executes a display driver which is software for controlling the display unit 209 and performs display control on the display unit 209. Further, the CPU 200 performs input / output control for the operation unit 208.

  A ROM (Read Only Memory) 201 stores a program and data in which a control procedure by the CPU 200 is stored. The ROM 201 stores a boot program for the image processing apparatus and various initial data. Also, various programs for realizing the processing of the image processing apparatus are stored.

  A RAM (Random Access Memory) 202 provides a working storage area when the CPU 200 performs control by an instruction program. The RAM 202 has a stack and a work area. The RAM 202 stores a program for executing processing in the image processing apparatus and each unit connected thereto, and various parameters used in the image processing. The RAM 202 stores a control program executed by the CPU 200 and temporarily stores various data when the CPU 200 executes various controls.

  The storage device 203 is an auxiliary storage device that stores various data. The storage device 203 is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

  A USB (Universal Serial Bus) 204 is a connection unit that connects to the operation unit 208.

  The communication circuit 205 is a circuit for performing communication with each unit constituting the system and various external devices connected to the network 160. The communication circuit 205 stores output information in a transfer packet, for example, and outputs the information to an external device via the network 160 by a communication technique such as TCP / IP. The image processing apparatus may have a plurality of communication circuits in accordance with a desired communication form.

  The graphics board 206 includes a GPU (Graphics Processing Unit) and a video memory. For example, the GPU performs image processing according to the first embodiment, generates data regarding information to be displayed on the display unit 209, and outputs the data to the display unit 209.

  An HDMI (registered trademark) (High Definition Multimedia Interface) 207 is a connection unit connected to the display unit 209.

  The CPU 200 and the GPU are examples of processors. The ROM 201, the RAM 202, and the storage device 203 are examples of memories. The image processing apparatus may have a plurality of processors. In the first embodiment, the function of each unit of the image processing apparatus is realized by the processor of the image processing apparatus executing a program stored in the memory.

  In addition, the image processing apparatus may include a CPU, GPU, or ASIC (Application Specific Integrated Circuit) that performs a specific process. The image processing apparatus may include a field-programmable gate array (FPGA) in which specific processing or all processing is programmed.

[Functional configuration of image processing unit]
FIG. 3 is a diagram illustrating an example of a functional configuration of the image processing unit 300 included in the image processing apparatus according to the embodiment of the present invention. For example, the image processing apparatus that is the photographing apparatus 130 may have various functions related to image processing, but only the functional configuration related to the deformation alignment process according to the first embodiment will be described below for the sake of simplicity. .

  In the following, the first image (past image) taken at the first time point and the second image (current image) taken at the second time point newer than the first time point are used as inputs. A case where the deformation position of the image and the current image is aligned will be described as an example. Both the past image and the current image are medical images obtained by photographing the same part of the subject.

  Further, the deformation position alignment is performed using the current image as a reference image (standard image for alignment) and the past image as a floating image (an image deformed by alignment). The alignment is a process for aligning the positions of corresponding features between two images. Alignment obtains a conversion function that maps a position on one image to a position on the other image, and applies the conversion function to one image, thereby converting one image into the other image. This is done by the approaching process. For example, by moving each pixel constituting the past image using the conversion function, a deformed aligned image is acquired. If a pixel with no pixel value is generated in the transformed image, the pixel with other pixel values set by a pixel value interpolation method such as linear interpolation is applied to the pixel with no pixel value set. The pixel value can be set based on the value and position.

  Note that the two images obtained by performing deformation alignment and obtaining the difference image are not limited to images acquired with the same modality over time, but may be images acquired with different modalities at the same time, for example. Further, in the deformation alignment, the current image may be a floating image and the past sub-image may be a reference image.

  The image processing unit 300 includes a correction processing unit 310, a rough alignment processing unit 320, and a detailed alignment processing unit 330.

  The correction processing unit 310 performs a process of correcting the images for the two input images S1 (past image) and S2 (current image). For example, the two input images S1 and S2 may have different image sizes and image resolutions depending on the acquired photographing apparatus. Further, for example, the two input images S1 and S2 may have different density value profiles due to a difference in imaging conditions such as X-ray irradiation dose. Accordingly, the correction processing unit 310 performs a correction process for unifying the image size and the image resolution in the input images S1 and S2, for example. Further, the correction processing unit 310 performs a correction process for bringing the density value profile of the input image S1 close to the density value profile of the input image S2, for example. The input image S1 corrected by the correction processing unit 310 is defined as S1 ′. Further, the input image S2 corrected by the correction processing unit 310 is defined as S2 ′.

  The approximate alignment processing unit 320 performs alignment on S1 ′ and S2 ′ acquired by the correction processing unit 310. Specifically, the processing unit 320 performs rigid body transformation such as affine transformation such as image rotation, parallel movement, and enlargement / reduction on S1 ′, and performs alignment so that the degree of coincidence between the images with S2 ′ increases. I do. The processing unit 320 acquires a deformation field that increases the degree of coincidence between the input images of S1 ′ and S2 ′. And the process part 320 deform | transforms S1 'based on the acquired deformation field, and acquires S1' '. The processing unit 320 may use an ICP (Iterative Closest Point) algorithm, which is a registration algorithm using image geometric features such as lines and surfaces.

  The detailed alignment processing unit 330 performs alignment processing on S1 ″ acquired by the approximate alignment processing unit 320 and S2 ′ acquired by the correction processing unit 310. Here, the detailed alignment processing unit 330 performs alignment processing with less alignment error than the alignment processing performed by the approximate alignment processing unit 320. Hereinafter, the alignment performed by the approximate alignment processing unit 320 may be referred to as approximate alignment, and the more accurate alignment performed by the detailed alignment processing unit 330 may be referred to as detailed alignment. The detailed alignment processing unit 330 repeatedly performs detailed alignment processing to acquire a deformation field that increases the degree of coincidence between the images of S1 ″ and S2 ′. And the process part 330 deform | transforms S1 '' based on the acquired deformation field, and acquires O1. The image O1 is a past image that is aligned and corrected with respect to the current image. Further, the processing unit 330 acquires the image S2 ′ as the image O2 that is the corrected current image. The processing unit 330 performs detailed alignment using an FFD (Free-Form Deformation) method that performs non-linear conversion using, for example, a B-splines function. Also, the processing unit 330 is an example of non-rigid alignment, LDDMM (Large Deformation Metric Metal Mapping, Miller et al. 1993, Proceedings of the National Academic Sciences in the United States. Et al., 1995, Geometric methods in Applied Imaging, San Diego, CA; Granander and Miller, 1996, Statistical computing and graphics newsletter 7, 3-8). It may be.

  In the process of acquiring the image O1 and the image O2, the processing unit 330 divides into, for example, a plurality of partial regions that configure S1 ′ based on the information regarding the accuracy of the detailed alignment performed previously. Hereinafter, each of the plurality of partial areas constituting S1 ′ and S2 ′ is referred to as a small area. Based on the information regarding the accuracy, the processing unit 330 determines the order in which the detailed alignment is repeated for each of the plurality of partial regions (that is, each of the small regions). Then, the processing unit 330 repeatedly performs the detailed alignment process based on the determined order.

  Further, the processing unit 330 performs a process of acquiring a difference between the image O1 and the image O2, and acquires a difference image O3. Details of processing performed by the processing unit 330 will be described later.

[A series of processing by the image processing device]
[Detail alignment]
FIG. 4 is a flowchart illustrating an example of a series of processes performed by the detailed alignment processing unit 330. In the following processing, unless otherwise specified, the main body that realizes each processing is the CPU 200 or the GPU.

In step S401, the processing unit 330 acquires an image to be subjected to detailed alignment. Here, the processing unit 330 reads out from the memory two image data of S1 _{HIGH ″} that is an image acquired by the approximate alignment processing unit 320 and S2 _HIGH ′ processed by the correction processing unit 310. S1 _{HIGH ″} is an image processed so as to increase the degree of coincidence with the current image S2 by the processing by the correction processing unit 310 and the processing by the approximate alignment processing unit 320 on the past image S1 as described above. is there. Further, S2 _HIGH ′ is an image obtained by performing processing by the correction processing unit 310 on the current image S2 as described above.

In step S402, the processing unit 330 performs resolution conversion processing on the image acquired in step S401. The processing unit 330 lowers the resolution of the image acquired in step S401. This is because a low-resolution image can be roughly aligned. Here, the processing unit 330 converts the resolutions of S1 _{HIGH ″} and S2 _HIGH ′ read in step S401 into half the resolution, and sets them as S1 _{LOW ″} and S2 _LOW ′. In the following, in order to simplify the description, the resolution conversion in the first embodiment is performed at two levels of resolution (low resolution LOW: half of the original resolution, high resolution HIGH: original resolution). However, the number of resolution stages and the resolution pitch are not limited to the above.

In step S403, the processing unit 330 performs detailed alignment processing (first detailed alignment processing) on the two images S1 _{LOW ″} and S2 _LOW ′ acquired in step S402. The processor 330 in the first detailed alignment processing, by using a method such as FFD and _LDDMM, acquires the deformation field _{D LOW_n} such higher degree of coincidence between the images of the _{S1 LOW''} and _{S2 LOW} '. Then, the processing unit 330 acquires S1 _{LOW_n ″} by converting S1 _{LOW ″} based on the acquired deformation field D _{LOW_n} . n is a counter value and increases every time the process of step S403 is executed.

In step S404, the processing unit 330 acquires an evaluation value related to the degree of coincidence between the two images of S1 _{LOW_n ″} and S2 _LOW ′ acquired in step S403. Based on the evaluation value, the processing unit 330 determines whether or not to repeat the process in step S403. The evaluation value is, for example, an SSD (Sum of Squared Difference). The SSD is the sum of squares of differences between pixel values at the same coordinate position in two images. As another example of the evaluation value, a root mean square error RMSE (Root Mean Squared Error) or a mean absolute error MAE (Mean Absolute Error) may be used. The smaller these evaluation values are, the higher the degree of coincidence between images is, and the more similar images are indicated. For example, when using SSD as the evaluation value, the processing unit 330 determines that the iterative process of step S403 has converged when the evaluation value is equal to or less than a predetermined threshold value, and proceeds to step S405. If the evaluation value is greater than the predetermined threshold, the processing unit 330 determines that the iterative process in step S403 has not converged, and continues the process in step S403. By repeatedly performing the process of step S403 until convergence, S1 _{LOW_n ″} can be set to a degree of coincidence equal to or higher than a predetermined reference with respect to S2 _LOW ′.

In step S <b> 405, the processing unit 330 divides S <b> 1 _{HIGH ″} and S <b> 2 _HIGH ′ acquired in step S <b> 401 into a predetermined number M of small regions. These images are divided based on a prescribed number of divisions and a prescribed number of voxels. Further, the processing unit 330 may be divided by a method specified by the user or a method corresponding to each imaged part (for example, chest, abdomen, head, etc.). Hereinafter, a case where M = 9 and S1 _{HIGH ″} and S2 _HIGH ′ are divided into nine equal small regions will be described as an example. S1 _{HIGH ″} is an example of the first image, and S2 _HIGH ″ is an example of the second image. In this respect, the processing unit 330 is an example of an image acquisition unit.

In step S406, the processing unit 330 based on the deformation field _{D LOW_n} acquired in step S403, an image of one small region obtained by dividing in step S405 _{S1 HIGH_m''} and the _{S2 HIGH_m} ', small A detailed alignment process (second detailed alignment process) is performed for each region. S1 _{HIGH_m ″} is an example of one of a plurality of partial regions constituting the first image S1 _{HIGH ″} . The processing unit 330 acquires a deformation field D _{HIGH_m_n} that enhances the degree of coincidence between the small region images S1 _{HIGH_m ″} and S2 _{HIGH_m} ′. m is a value indicating any one of the small areas divided in step S405, and the processing unit 330 can specify the small area based on m. n is a counter value, and increases every time a process of step S505 described later is executed. Details of the processing in step S406 will be described later with reference to FIG.

In step S407, the processing unit 330 acquires the small region image S1 _{HIGH_m ″} converted based on the deformation field D _{HIGH_m_n} acquired in step S406. The processing unit 330 may perform processing for converting the small region image S1 _{HIGH_m ″} in step S407, or may acquire S1 _{HIGH_m ″} converted in step S505 in FIG. 5 described later. Accordingly, the processing unit 330 acquires S1 _{HIGH_m_n ″,} which is an image of a small region aligned with S2 _{HIGH_m} ′.

In step S _<b> 408, the processing unit 330 acquires a difference image O _<b> 3 _m that indicates the difference between the two images of S _<b> 1 _{HIGH_m_n ″} acquired in step S _<b> 407 and S _<b> 2 _{HIGH_m} ′. Since the small region images S1 _{HIGH_m_n ″} and S2 _{HIGH_m} ′ are aligned in step S407, the difference image O3 _m is compared with S1 (past image) and S2 (current image), and the changed portion is emphasized. Will be.

In step S409, the processing unit 330 causes the display unit 209 to display the difference image O3 _m acquired in step S408. The processing unit 330 may cause the display unit 209 to display the difference image O3 _m , the aligned image S1 _{HIGH_m_n ″,} and the reference image S2 _{HIGH_m} ′ in a comparable manner.

  In step S410, the processing unit 330 determines whether or not the second detailed alignment process has been performed on each of the small areas divided in step S405 by the processes in steps S406 and S407. When the second detailed alignment process is performed on all the small areas, the process illustrated in FIG. 4 is terminated. If there is a small area that has not been subjected to the second detailed alignment process, the process returns to step S406 and the process is repeated.

  In the example illustrated in FIG. 4, the example in which the result of the detailed positioning process for a certain small region is displayed on the display unit 209 in step S409 has been described, but the present invention is not limited thereto. For example, the processing unit 330 may cause the display unit 209 to display the result of the small area on which the second detailed alignment process has been performed through the repetition process of steps S406 to S409. The processing unit 330 may cause the display unit 209 to sequentially display the difference images corresponding to the small regions in the order in which the second detailed alignment processing is performed by the repetition processing of step S406 to step S409. The processing unit 330 may display the result on the display unit 209 after it is determined in step S410 that the second detailed alignment processing has been performed on all the small regions. Moreover, although the case where the result of the second detailed alignment process is displayed on the display unit 209 in step S408 has been described as an example, the present invention is not limited to this. For example, you may display on the display part (not shown) connected to the client computer 100 instead of the display part 209 of an image processing apparatus. In step S <b> 408, the processing unit 330 may transmit the image of the small area on which the second detailed alignment process has been performed to the image management server 141 for storage.

[Second detailed alignment processing]
FIG. 5 is a flowchart showing an example of detailed processing in step S406 shown in FIG.

In step S501, the processing unit 330 _converts the resolution of the deformation field D _{LOW_n} acquired in steps S403 and S404 into a high resolution, and acquires the deformation field D _{HIGH — 0} . The deformation field D _{HIGH — 0} is an example of information regarding the first conversion. In this respect, the processing unit 330 is an example of a first acquisition unit.

In step S502, the processing unit 330 _converts the high-resolution small region image S1 _{HIGH_m ″} obtained in step S405 based on the deformation field D _{HIGH_0} acquired in step S501, and acquires S1 _{HIGH_m_0 ″} . S1 _{HIGH_m — 0 ″} is an example of the first converted image. As a result, the result of the low-resolution alignment acquired in steps S402 to S404 can be reflected on the high-resolution small area image.

In step S _<b> 503, the processing unit 330 acquires evaluation values of the respective small regions in the two images of S _<b> 1 _{HIGH_m — 0} ″ and S _<b> 2 _{HIGH_m} ′. The evaluation value is an evaluation value related to the degree of coincidence of two images for each small region, as described in step S404. The processing of the processing unit 330 is the same as that in step S404, and the detailed description is omitted here by using the above description.

  In step S504, the processing unit 330 acquires the order of the small regions for which detailed alignment is performed based on the evaluation value for each small region acquired in step S503. In the first embodiment, a small area having a high evaluation value, that is, a low degree of coincidence between images is preferentially processed. If the degree of coincidence between images is low, there is a possibility that there is a large change between S1 (past image) and S2 (current image), and it is presumed that there is a high possibility that the region will be a notable area in the observation by the doctor. Because. Note that the order of the small areas for which the detailed alignment is performed is not limited to this. For example, the order instructed by the user may be used, or the small area having the lowest degree of matching is processed first, and then the neighborhood of the small area is processed. It may be the order to do. From this point of view, the processing unit 330 is an example of a determination unit that determines the order of partial areas (small areas) for which detailed alignment is performed for each of a plurality of partial areas.

In step S505, the processing unit 330 performs detailed alignment processing (second detailed alignment processing) of the small region images S1 _{HIGH_m_n−1 ″} and S2 _{HIGH_m} ′ in accordance with the order acquired in step S504. Similarly to step S403, the processing unit 330 uses a method such as FFD or _LDDMM to acquire a deformation field D _{HIGH_m_n} that increases the degree of coincidence between images of S1 _{HIGH_m_n−1 ″} and S2 _{HIGH_m} ′. D _{HIGH — m — n} is an example of information regarding the second conversion. In this respect, the processing unit 330 is an example of a second acquisition unit. Then, the processing unit 330 acquires the _{S1 HIGH_m_n''} by converting the _{S1 HIGH_m_n''} based on deformation field obtained. From this point of view, the processing unit 330 is an example of an alignment unit that aligns a partial region including the first image with the second image.

  That is, in the second detailed alignment, iterative processing of the detailed alignment (first detailed alignment) between the past image and the current image is given priority, and a small region with a low image matching degree in the first detailed alignment is prioritized. This is done for each small area. The processing unit 330 is an image obtained as a result of performing the first detailed alignment, and the iterative process for the intermediate image in the detailed alignment repetitive process is performed for each small region in accordance with the priority based on the first detailed alignment result. Do. As a result, a highly accurate alignment image is acquired with priority given to a small region having a low image matching degree as a result of the first detailed alignment. The processing performed by the processing unit 330 in step S505 is the same as the processing in step S403, although the target image is different.

In step S506, the processing unit 330 acquires an evaluation value related to the degree of coincidence between the two images of S1 _{HIGH_m_n ″} and S2 _{HIGH_m} ′ acquired in step S505. Similarly to step S404, when the evaluation value such as SSD is equal to or less than a predetermined threshold value, it is determined that the iterative process of step S505 has converged, and the process proceeds to step S507. If the evaluation value is greater than the predetermined threshold, the processing unit 330 determines that the step S505 repetition process has not converged, and continues the process of step S505. The processing performed by the processing unit 330 in step S506 is the same as the processing in step S404, although the target image is different.

  With the configuration of the first embodiment as described above, an image to be aligned is divided into small areas, and an area where the degree of coincidence between images is low, that is, an area that is presumed to be observed by a doctor preferentially. Priority can be given to repeated positioning. Thereby, it is possible to perform highly accurate alignment by giving priority to an area presumed to be observed by a doctor. A doctor can acquire a region to be preferentially observed for an image on which high-accuracy alignment has been performed in a short time.

[Second Embodiment]
In the second embodiment, an example in which an image obtained by the second detailed alignment for each small region is synthesized and presented to the user in addition to the first embodiment will be described. The detailed description of the second embodiment will be omitted by using the above description for the configuration and processing common to the first embodiment.

  The configuration of the image processing apparatus according to the second embodiment is the same as that of the first embodiment.

  FIG. 6 is a flowchart illustrating an example of processing performed by the image processing apparatus according to the second embodiment. The processes common to the flowchart shown in FIG. 4 are denoted by the same reference numerals as those in FIG. In the second embodiment, step S601 is performed instead of step S405, and step S602 is added.

In step S <b> 601, the processing unit 330 divides S <b> 1 _{HIGH ″} and S <b> 2 _HIGH ′ acquired in step S <b> 401 into a predetermined number M of small regions. These images are divided based on a prescribed number of divisions and a prescribed number of voxels. Further, the processing unit 330 may be divided by a method specified by the user or a method corresponding to each imaged part (for example, chest, abdomen, head, etc.).

  In the alignment of the divided small areas, there is a possibility that the alignment accuracy of the end of the small area image is lowered. In the second embodiment, the processing unit 330 divides each small region with a size including a predetermined margin. Specifically, a predetermined pixel width is added as a margin in each of the vertical direction (X direction), the horizontal direction (Y direction), and the depth direction (Z direction). In the processing after step S406 in the second embodiment, an image in which a pixel width corresponding to a predetermined margin is added to an end of the target small region image in the first embodiment is targeted.

In step S602, the processing unit 330 synthesizes each small region image S1 _{HIGH_m_n ″} aligned up to step S407. As in the first embodiment, m is a value indicating one of the small areas divided in step S601, and the processing unit 330 can specify the small area based on m. n is a counter value and increases every time the process of step S505 is executed. Details of the processing in step S406 will be described later with reference to FIG. Here, a description will be given of a process of combining the small regions S1 _{HIGH_1_n1 ''} , S1 _{HIGH_2_n2 ''} , and S1 _{HIGH_3_n3} '' when the second detailed alignment has been completed. . The processing unit 330 acquires empty image data having the same image size as S1 ′ or S2 ′. Then, the processing unit 330 based on the coordinates of _{S2 HIGH_m} 'when divided from S2' into small areas, the small areas _{S1 HIGH_1_n1'', S1 HIGH_2_n2'',} Paste _{S1 HIGH_3_n3''} each on an empty image data To obtain a composite image.

In the portion corresponding to the margin set in step S601, a portion that overlaps with an adjacent small region occurs at the time of composition. For example, in a pixel overlapping with an adjacent small region, the processing unit 330 sets an average value of the pixel value of the adjacent small region and the pixel value of the margin as the pixel value of the composite image. In another example, the processing unit 330 may process the pixel value of the margin and the pixel value of the adjacent small region using an average value filter or an intermediate value filter, and smooth the boundary between the adjacent small regions. . In another example, the processing unit 330 may blend the pixel value of the margin and the pixels of the adjacent small area overlapping the margin at a predetermined ratio. In that case, the processing unit 330 may increase the ratio of the pixels in the adjacent small regions to the ratio of the pixels in the margin in consideration that it is estimated that the margin pixels have lower alignment accuracy. . Note that since S2 _{HIGH_m} ′ is a reference image for alignment, it is not necessary to perform image synthesis in step S601.

  In the composite image, the processing unit 330 may set a region corresponding to a small region for which the second detailed alignment has not been completed yet to a specific pixel value, or may delete a pixel. The processing unit 330 generates the composite image so that the region where the second detailed alignment is completed and the region where the second detailed alignment is not completed can be distinguished.

In step S408, the processing unit 330 acquires a difference image between the combined image combined in step S407 and the corrected current image S2 _HIGH ′ acquired in step S401. The processing unit 330 may set the region of the difference image corresponding to the small region for which the second detailed alignment has not been completed in step S602 to the pixel value of S2 _HIGH ′, or delete the pixel Alternatively, a specific pixel value may be set.

  With the configuration of the second embodiment as described above, an image to be aligned is divided into small areas, and an area where the degree of coincidence between images is low, that is, an area that is presumed to be observed by a doctor preferentially. Priority can be given to repeated positioning. Thereby, it is possible to perform highly accurate alignment by giving priority to an area presumed to be observed by a doctor. A doctor can acquire a region to be preferentially observed for an image on which high-accuracy alignment has been performed in a short time. Furthermore, while the doctor observes the registered image, a composite image in which the newly aligned image is combined can be generated. Accordingly, the doctor can observe other regions together while observing the region having high priority.

[Modification]
In steps S405 and S601, the detailed alignment processing unit 330 divides each image used for alignment into small areas so that the user can specify the division number M via the user interface displayed on the display unit 209. It may be. The division number M may be changed according to the size of the image.

  In the process of determining the order in which the processing unit 330 performs the second detailed alignment process on the small area in step S406, the user can specify the order via the user interface displayed on the display unit 209. Good. In another example, the user may be able to input information regarding a site that the user wants to observe preferentially via the user interface or information regarding a site that seems to be a lesion. In this case, the processing unit 330 may perform the second detailed alignment process by giving priority to the small area corresponding to the input part. In another example, the processing unit 330 may acquire information related to a past diagnosis result of the subject from the report management server 151 and may acquire information related to a region that the doctor has focused on in the diagnosis result. . In this case, the processing unit 330 may perform the second detailed alignment process by giving priority to a small area corresponding to the area that the doctor has focused on in the past diagnosis results. The processing unit 330 may determine the order in which the second detailed alignment process is performed based on the analysis result of the images included in the past diagnosis results. When the difference image is included in the information related to the past diagnosis result and the information related to the order of the small regions subjected to the second detailed alignment process is included, the processing unit 330 performs the processing in the past diagnosis result. The second detailed alignment process may be performed in the same order as the order.

  In step S <b> 406, the processing unit 330 may perform the second detailed alignment process on the small areas in the order of the lower evaluation value. It is considered that the second detailed alignment can be completed more quickly in a small region having a high similarity between images. For example, for the purpose of comparing medical images with different modalities acquired at the same time, it can be presented to the user preferentially from the small area where the detailed alignment has been completed, and the user workflow can be improved. .

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The image processing apparatus in each of the above-described embodiments may be realized as a single apparatus, or may be configured to execute the above-described processing by combining a plurality of apparatuses so that they can communicate with each other. include. The above-described processing may be executed by a common server device or server group. The plurality of apparatuses constituting the image processing apparatus and the image processing system need only be able to communicate at a predetermined communication rate, and do not need to exist in the same facility or in the same country.

  In the embodiment of the present invention, a software program that realizes the functions of the above-described embodiments is supplied to a system or apparatus, and the computer of the system or apparatus reads and executes the code of the supplied program. Includes form.

  Therefore, in order to realize the processing according to the embodiment by a computer, the program code itself installed in the computer is also one embodiment of the present invention. Further, based on instructions included in a program read by the computer, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing. .

  Embodiments appropriately combining the above-described embodiments are also included in the embodiments of the present invention.

Claims (14)

First acquisition means for acquiring information relating to a first transformation for aligning the first image and the second image;
Based on the information on the accuracy of the alignment in the first converted image obtained by converting the first image by the first conversion, for each of a plurality of partial regions constituting the first image Determining means for determining an order of obtaining the second transformation;
Second acquisition means for acquiring, in accordance with the determined order, information relating to a second transformation for aligning with the second image for each of the plurality of partial regions;
An image processing apparatus comprising:   The determination unit performs the processing so as to perform the process of obtaining the second transformation by giving priority to a partial region having a low alignment accuracy among the plurality of partial regions constituting the first image. The image processing apparatus according to claim 1, wherein the determination is performed.   The image processing apparatus according to claim 1, wherein the second acquisition unit acquires the second conversion by repeating a process for acquiring the first conversion.   The first image and the second image are acquired by aligning an image acquired at the first time point with an image acquired at a second time point that is newer than the first time point. The image processing apparatus according to claim 1, further comprising a third acquisition unit.   The image processing according to claim 4, wherein the alignment corresponding to the first conversion and the second conversion is alignment with higher accuracy than alignment by the third acquisition unit. apparatus.   The image processing apparatus according to claim 1, further comprising a combining unit that combines the partial regions aligned by the second acquisition unit to acquire a combined image. .   7. The image processing apparatus according to claim 1, further comprising a difference unit configured to obtain a difference image by subtracting the partial region aligned by the second acquisition unit from the second image. 8. The image processing apparatus according to item 1.   The difference means for acquiring a difference image by subtracting the synthesized image obtained by synthesizing the partial regions aligned by the second acquisition means and the second image, The image processing apparatus according to claim 1.   The first acquisition means converts the resolution of the first image and the second image to be low, and aligns the first image and the second image of the low-converted resolution. The image processing apparatus according to claim 1, wherein the first conversion is acquired.   The second acquisition unit converts the first image and the second image to a resolution higher than the resolution converted by the first acquisition unit, and the first of the resolution converted higher. The image processing apparatus according to claim 9, wherein the second conversion for aligning the partial region of the image and the second image having the resolution that has been highly converted is acquired. Image acquisition means for acquiring a first image and a second image;
Determining means for determining an order for obtaining a transformation for aligning the first image with the second image for each of a plurality of partial regions constituting the first image;
Alignment means for aligning a partial area included in the first image with the second image based on the determined order;
An image processing apparatus comprising: First acquisition means for acquiring information relating to a first transformation for aligning the first image and the second image;
Based on the information on the accuracy of the alignment in the first converted image obtained by converting the first image by the first conversion, for each of a plurality of partial regions constituting the first image Determining means for determining an order of obtaining the second transformation;
Second acquisition means for acquiring, in accordance with the determined order, information relating to a second transformation for aligning with the second image for each of the plurality of partial regions;
An image processing system comprising: A first acquisition step of acquiring information relating to a first transformation for aligning the first image and the second image;
Based on the information on the accuracy of the alignment in the first converted image obtained by converting the first image by the first conversion, for each of a plurality of partial regions constituting the first image A determining step of determining an order of obtaining the second transformation;
A second acquisition step of acquiring, with respect to each of the plurality of partial areas, information related to a second transformation for alignment with the second image according to the determined order;
An image processing method comprising:   A program for causing a computer to execute the image processing method according to claim 13.

Images