CN111583204B - Organ localization method based on two-dimensional serial magnetic resonance images based on network model - Google Patents

Abstract

The invention discloses a network model-based organ positioning method for a two-dimensional sequence magnetic resonance image, which comprises the following steps: data set preparation, preprocessing and expansion processing; constructing an improved organ preliminary positioning network model based on the fast R-CNN; optimizing and adjusting the organ primary positioning network model by using the verification set to obtain an organ positioning network model; performing organ preliminary positioning on the verification set image to obtain a plurality of target detection frames and corresponding credibility scores aiming at each two-dimensional slice image, and reserving target candidate frames with credibility greater than a threshold value in each image; constructing a spatial curve fitting model based on sequence correlation processing; preprocessing a two-dimensional sequence magnetic resonance image of an organ to be positioned, inputting the preprocessed two-dimensional sequence magnetic resonance image into an organ positioning network model, and obtaining a target candidate frame with the credibility being more than a threshold value in each image according to an organ initial positioning result; the organ is finally located based on the sequence correlation process. The pair of organs is accurately positioned and has good generalization performance.

Description

Organ localization method based on two-dimensional serial magnetic resonance images based on network model

technical field

The invention relates to a method for locating organs in magnetic resonance images, in particular to a method for locating organs in two-dimensional sequence magnetic resonance images based on a network model.

Background technique

Magnetic resonance (MR) imaging has become the main imaging modality for prostate auxiliary diagnosis due to its superior spatial resolution and tissue contrast. Compared with transrectal ultrasonography (TRUS), it facilitates targeted biopsy and therapy that is important for localization of prostate tumor lesions, volume assessment, and prostate cancer staging. However, the current inspection of prostate MR images is performed by radiologists based on visual inspection of each slice image, so it is a rather time-consuming, tedious and subjective task.

Organ localization is important for many medical image processing tasks such as image registration, organ segmentation, and lesion detection. Effective assessment of the initial position of organs can greatly improve the performance of subsequent processing. For example, for organ segmentation, the initial positioning of the organ can focus the segmentation task on the region of interest, which can improve the segmentation speed, reduce memory storage and reduce the risk of false positive segmentation.

At present, in the segmentation and detection of various organs/tissues in medical images, some semi-automatic or fully automatic methods for computer-aided diagnosis have been proposed. However, due to differences in scanners and scanning protocols, image brightness, imaging artifacts, and heterogeneity of signal intensity around the rectal coil, and differences in the size and shape of the glands themselves, as well as the differences between the glands and surrounding tissue structures, Prostate segmentation and detection still face great challenges due to inherent differences such as contrast, lack of strong borders, etc.

In recent years, region-based two-stage object detection algorithms led by Faster R-CNN have been widely used in the field of medical image processing due to their excellent detection accuracy and better detection efficiency. However, since the network was originally designed for object detection in natural images and there are still certain limitations in the detection of small objects, it cannot uniquely and accurately locate organs in magnetic resonance images.

SUMMARY OF THE INVENTION

In order to overcome the deficiencies of the prior art, the present invention provides an improved two-dimensional sequence magnetic resonance image organ localization method based on Faster R-CNN, which makes full use of the similarities and differences between natural image features and medical image features to optimize organ localization. Accuracy and detection success rate.

For this reason, the present invention adopts the following technical solutions:

A method for organ localization of two-dimensional sequence magnetic resonance images based on a network model, comprising the following steps:

S1, prepare the dataset:

Collect multiple 2D sequential magnetic resonance images of an organ, label the organ regions in these images with rectangular target boxes, and then divide them into training sets and validation sets;

S2, data set preprocessing: perform pixel intensity maximum and minimum normalization processing, center cropping processing, and image size normalization processing on each of the two-dimensional sequence MR images in sequence;

S3, performing data expansion processing on the training set image obtained in step S2 and the rectangular target frame labeling of the corresponding organ;

S4, build an improved organ localization network model based on faster R-CNN, including the following steps:

1) Build the target detection network architecture based on the improved Faster R-CNN, use the ResNet-50 with spatial attention mechanism to replace the VGG16 architecture in the classic Faster R-CNN, and use the ImageNet large natural dataset to train ResNet-50 , get the initial training weight parameters of the network;

2) Use the training set image expanded in step S3 and the corresponding rectangular target frame annotation as the input of the network in step 1), and use the multi-task loss function composed of classification loss and regression loss to iteratively adjust the parameters of the entire network architecture to complete the network. Train and generate a network model for preliminary positioning of organs;

S5, the verification set image preprocessed in step S2 and the corresponding rectangular target frame annotation are input into the preliminary organ localization network model generated by S4, and the organ localization model is optimized and adjusted according to the output result to obtain the organ localization network model;

S6, using the network model constructed in S5 to preliminarily locate the organs on the validation set images preprocessed in step S2, to obtain several target detection frames and corresponding reliability scores for each of the two-dimensional slice images, and retain each image The target candidate frame in the image whose reliability is greater than a certain threshold;

S7, construct a spatial curve fitting model based on sequence correlation processing:

1) For each two-dimensional slice image with the preliminary detection result in S6, arrange the axial slice images belonging to the same sequence in the order before and after;

2) For each sequence sorted above, extract the key points of the target frame in the slice image with a single reliable target candidate frame, and use these key points to perform spatial curve fitting in the sequence direction to determine the best curve fitting. The fitting method is least squares quadratic polynomial fitting;

S8, the two-dimensional sequence magnetic resonance image of a certain organ to be located is preprocessed according to the method of step S2 and then input into the organ location network model obtained in step S5 to obtain a preliminary organ location result;

S9, obtain several target detection frames and corresponding reliability scores of each two-dimensional slice image from the preliminary positioning result of the organ output in step S8, and retain target candidate frames whose reliability is greater than a certain threshold in each image;

S10, perform final positioning of the organ based on sequence association processing:

1) For each two-dimensional slice image with preliminary detection results in S9, arrange it in the order of the sequence;

2) For the above sorted sequence images, extract the key points of the target frame in the slice image with a single reliable target candidate frame, and perform the sequence direction on these key points according to the least squares quadratic polynomial fitting method obtained in S7. The spatial curve fitting of ;

3) Use the fitted spatial curve and the minimum spatial Euclidean distance to screen and determine the target frame closest to the fitting position in the two-dimensional image with multiple prediction frames as the final positioning frame of the organ, and update the spatial curve fitting after each screening. combined parameters;

4) Fitting the two-dimensional images of the deletion detection at the beginning and end of each sequence using the final updated space curve to complete the final positioning of the organ.

Wherein, when the pixel intensity is normalized in step S2, the normalized pixel value of the pixel value x _i of the pixel i is: norm _i =( _xi -Pmin)/(Pmax-Pmin)×255, norm _i ∈ [0,255], where Pmax and Pmin refer to the maximum and minimum pixel values of the slice image where pixel i is located, respectively.

In step S2, when the center cropping process is performed, if the size of the original image is W×H, the size after cropping is αW×αH, where α is a scale coefficient, preferably, the α=2/3.

Preferably, in step S2, the image size is normalized to 600×600.

In step S3, data expansion processing is performed by horizontal flipping, small-amplitude horizontal translation or vertical translation, small-angle random rotation and elastic deformation. Preferably, the random rotation angle of the small angle is -5°˜5°.

In step S6, the threshold value is 0.80-0.90.

Inspired by the excellent performance of the Faster R-CNN model in natural image target detection, the present invention is based on the model, and makes full use of the prior information of the center position of the organ in the image and the evolution law between the sequence images. The model is further improved and perfected to achieve accurate positioning of prostate organs in magnetic resonance images, laying a good foundation for subsequent medical image tasks such as organ registration and segmentation. The improved two-dimensional sequence magnetic resonance organ positioning method based on Faster R-CNN of the present invention makes full use of the similarities and differences between natural image features and medical image features to optimize the positioning accuracy and detection success rate of organs.

Experiments show that the method can achieve more accurate localization and good generalization performance without too much complex image preprocessing work. The recall rate of this method for organ area reaches 96.91%, and the success rate of organ localization is as high as 99.39%.

Description of drawings

Fig. 1 is the flow chart of the method of the present invention;

Figure 2 is an example of axial magnetic resonance images of the prostate from different medical centers used in the present invention;

Fig. 3 is the specific structure of some network blocks in the present invention: (a) convolution attention block; (b) identification block; (c) spatial attention module;

Figure 4 is a partial preliminary detection result based on the improved Faster R-CNN: (a) an example of a slice image with multiple target detection results; (b) an example of a slice image lacking target detection results;

Figure 5 is a schematic diagram of spatial curve fitting and updating based on sequence correlation, wherein: (1) the projection of the key points of the discarded target frame on the X-Z and X-Y planes; (2) the drawing and adjustment of the spatial fitting curve; ( 3) The projection of the spatial fitting curve on the X-Z plane; (4) The projection of the spatial fitting curve on the X-Y plane

6 is a partial detection result diagram of the method of the present invention on the verification set: wherein the red mark is the true outline of the prostate organ, and the yellow rectangle is the organ location result of the algorithm;

7 is a graph of the axial slice test results of the method of the present invention in a practical case: (a) the 5th slice; (b) the 11th slice; (c) the 15th slice; (d) the 18th slice slice.

FIG. 8 is a flow chart of the model training link and the testing link in the present invention.

Detailed ways

Since the initial positioning of organs is very important for medical image processing such as organ segmentation, image registration, and lesion detection, effective evaluation of the initial position of organs can help remove most of the interfering information in the image background, thereby improving subsequent processing. Based on the two-stage detection algorithm Faster R-CNN, the invention introduces a spatial attention module that pays attention to the positional features of organs and a correlation process that improves the detection effect between sequences, and shows excellent performance on the test data set with high heterogeneity. The detection effect has laid a good foundation for the subsequent processing of medical images.

The method of the present invention will be described in detail below with reference to specific embodiments.

Embodiment 1 A method for organ localization of two-dimensional sequence magnetic resonance images based on network model

In this embodiment, the method for locating prostate organs is described in detail by taking a prostate magnetic resonance data set with a small scale and high heterogeneity as an example. As shown in Figure 8, the method includes the following steps:

S1, dataset preparation:

The prostate magnetic resonance images used in this example are from the Prostate Segmentation Challenge PROMISE12 dataset published at the 2012 MICCAI conference. The dataset contains 80 cases of T2-weighted magnetic resonance images from four medical centers, 50 of which have expert manual segmentation masks, each center using different acquisition equipment and acquisition protocols to acquire images . Sample images from the four centers are shown in Figure 2. Among them, 1.5T and 3T represent the magnetic field strength, and the presence or absence of ERC represents whether a rectal coil was used during image acquisition.

In this embodiment, the above-mentioned 50 cases of prostate MR images with expert manual segmentation masks are divided into training set and validation set in a case-by-case basis according to a ratio of 4:1.

S2, dataset image preprocessing:

(1) Preprocessing of training set and validation set images:

The following four basic preprocessing operations are performed on the training set and validation set images obtained above: (a) Since the inter-layer resolution of the original 3D image in the axial direction is higher than the intra-layer resolution, the present invention will The 3D images are unfolded in an axial sequence to form a 2D sequence MR image. (b) Secondly, due to the large difference in the intensity of the images from different acquisition centers, the intensity maximum and minimum values are normalized for each 2D slice image. For example, the normalized pixel value of the pixel value x _i of the pixel i is: norm _i =( _xi -Pmin)/(Pmax-Pmin)×255, norm _i ∈[0,255], where Pmax and Pmin refer to the pixel i respectively The pixel maximum and minimum of the slice image where it is located. (c) In addition, considering the prior information that the organ is usually located at the center of the image, in order to enhance the detection performance for small objects, the image is centrally cropped (if the size of the original image is W×H, then the cropped size is αW× αH, α are proportional coefficients, in this embodiment, α=2/3). (d) Finally, the image size is uniformly normalized, which is normalized to 600×600 in this embodiment.

(2) Generation of target box labels for training set and validation set

To train and validate deep learning networks with labeled data, segmentation masks (ie, pixel-level annotations) corresponding to training and validation images are used to generate image-level annotations based on each training and validation image, ie, rectangular objects Box callout. In this method, for the segmentation mask images corresponding to the training and verification images, after the center cropping and size normalization operations of (c)-(d) in the above (1), the circumscribed rectangle of the true outline of the organ is used as the position of the organ label, and use the training set and validation set images and the corresponding label images with position labels as the input of the subsequent training and validation network.

For datasets without segmentation masks, the regions of the organs in these images can be labeled with rectangular target boxes first.

S3, data augmentation processing:

Since deep learning model training often requires large-scale data sets to better learn the generalization features of image data, and medical images are often difficult to obtain due to issues such as patient privacy, the present invention aims at training set images and corresponding segmentation masks. The code image carries out data expansion processing in four ways: horizontal flip, small-amplitude horizontal or vertical translation, small-angle random rotation (-5°～5°) and elastic deformation. The number of expanded images is 16 times the number of original images.

S4, construct an improved preliminary organ localization network model based on faster R-CNN:

(1) Network structure construction:

The present invention adopts the same network structure as Faster R-CNN, the difference is that the ResNet-50 architecture with spatial attention mechanism is used to replace the VGG16 in the original Faster R-CNN to realize the feature extraction and processing of the input image. Classification and recognition of objects in images, as shown in Figure 1. Among them, the specific structure of the convolutional attention module and the embedding method of the spatial attention module are shown in Figure 3(a); the specific structures of the identification block and the spatial attention module are shown in Figures 3(b) and (c) Show. The addition of the spatial attention module is to make better use of the central location information of organs in the image and enhance the sensitivity of the network to location feature learning. Taking F∈R ^C×H×W as the intermediate feature map input, the calculation formula of the attention feature map after adding the spatial attention module is:

Attention(F)=sigmoid( _f7 ^×7 [Favg,Fmax]) _Favg (F)= _{Global_avgpool} (F),

Fmax(F)= _{Global_maxpool} (F)

Among them, F _avg , F _max ∈ R ^1×H×W , f ^7×7 represents the convolution operation with the 7×7 convolution kernel.

(2) Network training strategy and loss function settings

1) Training strategy

Due to the limitation of medical data sets, the present invention uses the idea of transfer learning to pre-train the network on the large-scale natural image set of ImageNet, and uses the pre-trained weight parameters as the initial weight of the method, and uses the The network is further trained on the annotated prostate dataset. In the training process, Adam optimizer with parameters β1=0.9, β2=0.999, epsilon=10 ^-8 is selected, and a fixed learning rate of 2×10 ^-5 is adopted.

2) Loss function settings

The Faster R-CNN network includes two tasks: regression and classification. The regressor is used to perform regression prediction on the candidate frame, and the classifier is used to classify the objects in the candidate frame. Therefore, the network training process uses a multi-task loss function, and its calculation formula is as follows:

Among them, L _cls and L _reg represent classification loss and regression loss, respectively, as follows:

为anchor[i]的真实标签，即满足

i) Classification loss: a binary cross-entropy loss function is used, where p _i is the probability that anchor[i] is predicted to be the target,

is the true label of anchor[i], that is, it satisfies

其中，

t_i＝{t_x,t_y,t_w,t_h}是一个向量，表示预测候选框anchor[i]的参数化坐标；

对应的真实目标框的坐标向量。ii) Regression loss:

in,

t _i ={t _x , t _y , t _w , t _h } is a vector representing the parameterized coordinates of the prediction candidate frame anchor[i];

The coordinate vector of the corresponding real target box.

(3) Network training

1) Feature extraction

The training set image obtained in S3 above is input into the feature extraction network. With the continuous reduction of the resolution of the convolutional layer, the network can better learn the global features (ie generalization features) of the image, and the addition of the residual structure further optimizes the features. extraction process. As shown in Figure 1, the image undergoes four downsampling operations with stride 2 in stages 1 to 4, so the final convolutional feature map size is about 1/16 of the original input image size (take the input image Taking the size of 600×600 as an example, the size of the output convolutional feature map is 38×38).

2) Prediction of target candidate frame

(a) Based on the convolution feature map obtained in step 1), the RPN network first performs a 3×3 convolution operation on it, and then takes each pixel in the convolution map as the center to form several different sizes and different The scaled anchors are mapped to the convolutional feature maps to obtain several target candidate box proposals. In this embodiment, considering the size of the organ itself and its proportion in the image, anchors with sizes {64, 128, 256, 512} and aspect ratios {0.5, 1, 2} are selected. 4×3=12 anchors are formed at each pixel.

(b) Based on the convolution feature map obtained in 1) and the target candidate frame proposals of different sizes and sizes obtained in (a), perform ROI pooling operation to obtain target candidate frame feature maps of the same size;

(c) Use stage 5 shown in Figure 1 to perform further convolution operations on the feature map of the target candidate frame obtained in (b), and then use the fully connected layer to obtain the category probability prediction and position of each target candidate frame respectively. offset prediction;

(d) Perform Non-Maximum Suppression (NMS) with a selected threshold to remove the set of object candidate boxes with high overlap in the same class.

(e) Use the label image corresponding to the training image to evaluate the loss function for the classification and position prediction of the target candidate frame screened in (d), so as to use the gradient descent backpropagation method to iterate the parameters of the deep convolutional neural network. Update, take the network parameters obtained after iterating to the maximum set times as the optimal network parameters, complete the training, and generate the initial positioning network model of the organ.

S5. Apply the network model constructed above to the preliminary positioning of the validation set:

The preprocessed verification set image (not including the target frame annotation) described in S2 is input into the network model constructed above, and the output obtains several target detection frames and corresponding target detection frames for each two-dimensional slice image in the verification set image. reliability score. Retain the target candidate boxes whose reliability is greater than the threshold (the threshold is 0.80 in this method) in each image.

S6, final localization of the organ based on sequence association processing:

In the preliminary detection result of step S4, the phenomenon of multiple target detection frames appearing in individual slice images as shown in Fig. 4(a) and the phenomenon of lack of target detection frames in (b) appear because this method ignores characteristics of the medical image itself. For the detection of organ localization, the position of the organ does not change greatly between the two-dimensional sequence MR images of the same individual, and the evolution of the surface contour of the organ has a certain law, so it can be described by a mathematical model. This method introduces Sequence correlation processing to improve the above problems. Specifically divided into the following steps:

(1) Two-dimensional image serialization:

For each two-dimensional slice image with preliminary detection results in S4, the axial slice images belonging to the same sequence are arranged in the order of the front and back, so as to prepare for further sequence correlation processing.

(2) The target frame is finally determined:

Taking the detection of prostate organs as an example, each image contains prostate organs and the organs are unique. Therefore, using the uniqueness of the target and the slight variability of the position of the organs in the sequence image, first, for each sequence in the validation set obtained in (1), nine of the target frames in the slice images with a single reliable target candidate frame belonging to the same sequence are analyzed. Three key points (including the center point of the rectangular frame, the four vertices and the midpoints of the four sides) are fitted to the three-dimensional space curve, and the fitting method adopts the least squares quadratic polynomial fitting.

The two problems in Figure 4(a) and (b) above are dealt with as follows:

1) For the phenomenon that there are two or more target frames in individual two-dimensional images in each sequence (as shown in Figure 4(a)): use the fitted spatial curve and the minimum spatial Euclidean distance to screen and determine if there are multiple predictions The target box closest to the fitting position in the 2D image of the box is the position of the final organ, and the spatial curve fitting parameters are updated after each screening. Figure 5 is a schematic diagram describing this process, in which the mathematical formula for target box screening is based on:

分别表示第j个关键点处y方向和z方向的拟合预测值，

分别表示网络预测出的某个目标框的第j个关键点处的y方向和z方向的坐标值。Among them, b ^* represents the organ prediction frame finally screened out of the slice image,

represent the fitted predicted values in the y-direction and z-direction at the jth key point, respectively,

Represent the coordinate values of the y-direction and z-direction at the jth key point of a target frame predicted by the network, respectively.

II) For the prostate base and bottom organs at the beginning and end of the sequence in each sequence, there may be no target frame due to the small target (as shown in Figure 4(b)):

Due to the limitation of the target detection network for small target detection, this method uses the final updated spatial curve of each sequence to fit the two-dimensional image of the missing detection in the sequence to obtain the target location of the organ. Thus, the final localization of the organs in the validation set images is achieved.

Validation set positioning effect evaluation

Combined with the positioning results of the validation set images obtained in the above S6 and the real annotations of the organs in the validation set images in S2, the detection effects of the above methods are evaluated from both qualitative and quantitative perspectives:

From a qualitative perspective, (a)-(d) in Figure 6 are the detection results obtained on the validation set. Among them, the red line represents the real outline of the organ, and the yellow rectangle represents the detection result of the prostate organ by the method of the present invention. Judging from the test results, this method can achieve more accurate and unique organ localization, and at the same time, it can well avoid organs such as the bladder and rectum that are close to organs in shape and size.

Evaluation from a quantitative point of view, through multiple experiments, it is shown that the recall rate of this method for the real area of the organ reaches 96.91%, and the area recall rate of 50% is used as the standard for the localization success rate. Up to 99.39%.

From the perspective of detection efficiency, the method takes only 0.3s (Intel Core i7 3GHz processor) for organ localization detection in a two-dimensional slice image, which can well meet the actual needs in medical tasks.

Preferably, the preprocessed verification set is input into the network model formed in step S4, and the training network is further optimized and adjusted to obtain an optimized and adjusted network model, and then step S5 is performed.

Although the location of the prostate organ has been described as an example, it can be understood that this method is also applicable to the detection and location of tissues and organs such as kidney and pancreas.

Embodiment 2

The method for locating the prostate organ to be located using the above method comprises the following steps:

(1) Image preprocessing:

For the 3D magnetic resonance prostate organ image of a certain case, firstly, the 2D serialization process is performed. Secondly, the maximum and minimum intensity normalization processing is performed to normalize the pixel value of the two-dimensional slice image to [0, 255]. Finally, the image is appropriately cropped according to the position and proportion of the organ in the image, and the image size is normalized to 600×600.

(2) Based on Faster R-CNN to improve the organ localization network model to perform preliminary localization of organs:

1) Feature extraction: Input the two-dimensional slice image obtained in (1) into the convolutional neural network constructed by S4 to obtain a convolutional feature map about the image;

2) Generate target candidate frame suggestions: Input the convolutional feature map obtained in 1) into the RPN network to generate target candidate frame suggestions;

3) ROI pooling (i.e. region of interest pooling): Take 1) the output convolution feature map and 2) the output target candidate frame proposal as input, map the target candidate frame proposal to the corresponding position of the feature map, and map the mapping. After the feature map is pooled, the target region feature map of uniform size is obtained;

4) Classification prediction: classify according to the feature map of the target area obtained in 3), distinguish whether the object contained in the corresponding target candidate frame is an organ or a background, and obtain a confidence score corresponding to the category;

5) Bounding box regression: According to the feature map of the target area obtained in 3), the bounding box regression is performed to refine the bounding box, and finally the precise position of the detection box is obtained;

6) According to the target boxes belonging to the organs and their positions obtained in 4) and 5), perform non-maximum suppression (NMS) with a threshold of N _t (N _t is a fixed value) to remove those with a high degree of overlap. A set of target frames, and retain target candidate frames whose reliability is greater than a certain fixed threshold in each 2D slice image (the threshold is recommended to be between 0.80 and 0.90). Thus, the positions of several target detection frames in each two-dimensional slice image and the corresponding reliability scores are obtained.

(3) Final positioning of organs based on sequence association processing:

1) Arrange the above-mentioned slice images in the order of the original sequence;

2) Perform the least squares quartic polynomial space curve fitting on the nine key points of the target frame (including the center point of the rectangular frame, the four vertices and the midpoint of the four sides) in the sequence image with a single reliable target candidate frame. Secondly, use the fitted space curve and the minimum space Euclidean distance to screen and determine the target frame closest to the fitting position in the two-dimensional image with multiple prediction frames as the final positioning frame of the organ, and update the space curve fitting after each screening. matching parameters.

3) For the organ detection at the beginning and end of the sequence, due to the small target, there may be no target frame. The final updated space curve is used to fit the two-dimensional image of the missing detection to obtain the target location of the organ, thus realizing the target location of the organ. Organ localization detection of the entire 2D sequence of images.

In this embodiment, the effect diagram of prostate organ localization in the axial slice image is shown in Figure 7, where (a)-(d) respectively show the 5th, 11th, 15th, and 18th slices of the axial serial slices. The test effect chart. By comparing the positioning effect of the prostate organs in the four images, it can be seen that this method has a very good effect on the positioning of prostate organs in the overall sequence slice, especially the organ positioning at the beginning and end of the sequence, which is also rarely involved in other algorithms or Incomparable; in addition, from the longitudinal point of view, this method can not only achieve more accurate prostate localization, but also can well exclude the bladder, rectum and other regions with similar shape and size to the prostate organs, which shows that this method is useful for the feature learning of target organs. The effect is better.

To sum up, the advantage of the method for organ localization and identification of two-dimensional sequence magnetic resonance images according to the present invention is that a general organ localization method is constructed by using a magnetic resonance data set with a small scale and high heterogeneity, which is very convenient. The subjectivity of traditional slice-based visual inspection is well avoided. In addition, the present invention realizes the preliminary detection and positioning of organs by improving the existing target detection algorithm Faster R-CNN used in natural image processing; The evolution law of position and shape size between layers) greatly improves the organ localization effect at the beginning and end of the sequence. This method has better recognition speed and accuracy, and can well meet the real-time requirements in medical image processing.

Claims (8)

1. an organ localization method based on a two-dimensional sequence magnetic resonance image of a network model, comprising the following steps: S1, prepare the dataset: Collect multiple 2D sequential magnetic resonance images of an organ, label the organ regions in these images with rectangular target boxes, and then divide them into training sets and validation sets; S2, data set preprocessing: perform pixel intensity maximum and minimum normalization processing, center cropping processing, and image size normalization processing on each of the two-dimensional sequence MR images in sequence; S3, performing data expansion processing on the training set image obtained in step S2 and the rectangular target frame labeling of the corresponding organ; S4, construct an improved preliminary organ localization network model based on Faster R-CNN, including the following steps: 1) Build an improved target detection network architecture based on Faster R-CNN, use ResNet-50 with a spatial attention mechanism to replace the VGG16 architecture in the classic Faster R-CNN, and use the ImageNet large natural dataset to train ResNet-50, Get the initial training weight parameters of the network; 2) Use the training set image expanded in step S3 and the corresponding rectangular target frame annotation as the input of the network in step 1), and use the multi-task loss function composed of classification loss and regression loss to iteratively adjust the parameters of the entire network architecture to complete the network. Train and generate a network model for preliminary positioning of organs; S5, the verification set image preprocessed in step S2 and the corresponding rectangular target frame annotation are input into the preliminary organ localization network model generated by S4, and the organ localization model is optimized and adjusted according to the output result to obtain the organ localization network model; S6, using the network model constructed in S5 to preliminarily locate the organs on the validation set images preprocessed in step S2, to obtain several target detection frames and corresponding reliability scores for each of the two-dimensional slice images, and retain each image The target candidate frame in the image whose reliability is greater than a certain threshold; S7, construct a spatial curve fitting model based on sequence correlation processing: 1) For each two-dimensional slice image with the preliminary detection result in S6, arrange the axial slice images belonging to the same sequence in the order before and after; 2) For each sequence sorted above, extract the key points of the target frame in the slice image with a single reliable target candidate frame, and use these key points to perform spatial curve fitting in the sequence direction to determine the best curve fitting. The fitting method is least squares quadratic polynomial fitting; S8, the two-dimensional sequence magnetic resonance image of a certain organ to be located is preprocessed according to the method of step S2 and then input into the organ location network model obtained in step S5 to obtain the preliminary location result of the organ; S9, obtain several target detection frames and corresponding reliability scores of each two-dimensional slice image from the preliminary positioning result of the organ output in step S8, and retain target candidate frames whose reliability is greater than a certain threshold in each image; S10, perform final positioning of the organ based on sequence association processing: 1) For each two-dimensional slice image with preliminary detection results in S9, arrange it in the order of the sequence; 2) For the above sorted sequence images, extract the key points of the target frame in the slice image with a single reliable target candidate frame, and perform the sequence direction on these key points according to the least squares quadratic polynomial fitting method obtained in S7. The spatial curve fitting of ; 3) Use the fitted spatial curve and the minimum spatial Euclidean distance to screen and determine the target frame closest to the fitting position in the two-dimensional image with multiple prediction frames as the final positioning frame of the organ, and update the spatial curve fitting after each screening. combined parameters; 4) Fitting the two-dimensional images of the deletion detection at the beginning and end of each sequence using the final updated spatial curve to complete the final positioning of the organ. 2. organ localization method according to claim 1, is characterized in that: in step S2, when pixel intensity is normalized, the pixel value after the normalization of pixel value x _i of pixel i is: norm _i =(x _i -Pmin)/(Pmax-Pmin)×255, norm _i ∈[0,255], Among them, Pmax and Pmin respectively refer to the maximum and minimum pixel values of the slice image where the pixel i is located. 3 . The organ localization method according to claim 1 , wherein in step S2 , when the center cropping process is performed, if the size of the original image is W×H, the size after cropping is αW×αH, and α is a proportional coefficient. 4 . 4 . The organ localization method according to claim 3 , wherein: the α=2/3. 5 . 5 . The organ localization method according to claim 1 , wherein in step S2 , the image size is normalized to 600×600. 6 . 6 . The organ positioning method according to claim 1 , wherein in step S3 , data expansion processing is performed by horizontal flipping, small-amplitude horizontal translation or vertical translation, small-angle random rotation and elastic deformation. 7 . 7 . The method for organ positioning according to claim 6 , wherein the random rotation angle of the small angle is -5° to 5°. 8 . 8 . The organ localization method according to claim 1 , wherein the threshold value in step S6 is 0.80-0.90. 9 .

Images