CN111383213B - Mammary gland image retrieval method for multi-view discrimination metric learning - Google Patents

Abstract

A breast image retrieval method for multi-view discrimination metric learning uses multi-view metric to learn distance metrics capable of fully representing different views of breast images, and utilizes Fisher discrimination to measure similarity between breast image pairs, so that similar medical images are tightly mapped in a metric space, and dissimilar breast images are separated from each other as much as possible. Experimental results of the real breast image dataset show that the MVML-FDA model helps to form good distance measures from the CC and MLO views of the breast image to distinguish benign nodules, malignant nodules, benign calcifications and malignant calcifications of the breast.

Description

Mammary gland image retrieval method for multi-view discrimination metric learning

Technical Field

The invention relates to the field of medical images, in particular to a breast image retrieval method for multi-view discrimination measurement learning.

Background

Breast cancer is one of the most common cancers, especially female, according to world cancer reports by the world health organization. At present, the incidence rate of breast cancer in China is 14.7%, 170 thousands of breast cancer patients are newly increased annually, and about 50 thousands of breast cancer patients die. But screening of early breast cancer can effectively reduce the death rate of breast cancer and remarkably improve the survival rate of patients. In contrast to other diagnostic methods, medical imaging is the gold standard for diagnosing breast cancer, e.g., mammography, nuclear magnetic resonance, and CT scanning are all common imaging methods for breast cancer screening. Currently, early breast cancer screening relies solely on the experience of the physician, and decision accuracy depends largely on the physician's knowledge, experience, and the quality and quantity of information available. In addition, the manual processing of images is expensive and time consuming, especially for inexperienced physicians, and diagnosing breast cancer by analyzing medical images is laborious and difficult. Thus, assistance techniques that provide knowledge support to physicians during their decision making are significant.

With the popularization of internet technology, the medical image database is larger and larger in scale, and more past cases can be provided for doctors. Meanwhile, with the rapid development of artificial intelligence technology, machine learning classification methods are applied to diagnosis, treatment and prognosis of breast cancer. Watson software developed by IBM corporation, for example, is used for breast cancer treatment.

Essentially, the breast cancer decision process itself relies on historical empirical knowledge. The content-based image retrieval method and the medical case reasoning case-basedreasoning (CBR) have technical advantages in medical diagnosis over traditional machine learning classification techniques. Medical image retrieval is a core technology of a CBR system, and can retrieve medical images similar to images to be diagnosed from a medical database. Medical images are very different from natural images, and firstly, the resolution of the medical images is higher but the vast majority of the medical images are gray images; secondly, important information of medical images is mostly concentrated in a small area; third, semantic content may vary greatly between visually similar medical images. So the effect of the conventional natural image method directly applied to medical images is often not ideal. In addition, the acquisition of the medical image is directly related to the medical equipment and the measurement position. For example, the image capture position commonly adopted by the breast image during the conventional physical examination is the oblique position inside and outside the double-sided breast, and the head and tail positions or the side position are added when the pathological changes are found. In clinical practice, the height and the pathological change position of a patient are required to be taken into consideration at the same time, so that the obtained breast images are often different in the part and tissue with the focus of shooting, and the breast images with the same target and different shooting angles are effectively considered, so that the accuracy of image retrieval can be improved.

Disclosure of Invention

In order to solve the problem, a Multi-view discriminant metric learning (Multi-View Metric Learning with FisherDiscriminantAnalysis, MVML-FDA) medical image retrieval method is provided for assisting in diagnosis of breast diseases. The MVML-FDA learns a robust metric space between multiple perspectives based on a Fisher discriminant model, such that similar breast images are tightly mapped in the metric space, and dissimilar breast images are separated from each other as much as possible. The comprehensive information of the breast image is characterized from different visual angles, and potential descriptive characteristics are extracted, so that the medical image retrieval capability is improved.

A breast image retrieval method for multi-view discrimination metric learning comprises the following steps:

step 1, obtaining a data set of the multi-view medical image according to the multi-view medical image obtained from different shooting positions;

step 2, constructing a multi-view discrimination measurement learning MVML-FDA model through a data set, measuring the similarity between breast image pairs by Fisher discrimination, tightly mapping similar medical images in a measurement space, separating dissimilar breast images from each other as far as possible, and solving the MVML-FDA optimization problem;

step 3, solving the MVML-FDA optimization problem to obtain an optimal solution, namely an MVML-FDA model;

and 4, inputting medical images of a plurality of visual angles, weight parameters, maximum iteration times, maximum neighbor numbers and convergence threshold values by using the obtained MVML-FDA model, outputting an optimal medical image projection matrix, and completing medical image retrieval.

Further, in step 1, the multi-view medical image dataset obtained according to the different film capturing positions is { X } ¹ ,X ² ,...,X ^T (wherein X is ^t Representing a data matrix at a t-th viewing angle, X ^t Is the ith vector of (2)Representing sample x _i A feature vector at a t-th view angle; thus, the medical image pair at the t-th view can be written as +.>Wherein->Representing any medical image pair,/->Representing the corresponding image pair label, +.>Instruction image->And->Are similar in terms of both semantics and vision, l _m,i = -1 illustrates image->And->Are semantically or visually dissimilar.

Further, in step 2, through mahalanobis metric learning, multi-view discriminant metric learning (Multi-View Metric Learning with Fisher Discriminant Analysis, MVML-FDA) aims to learn a mapping space, so that the distance between similar samples is far smaller than the distance between different samples, i.e. there is a more separable inter-class distance and a more compact intra-class distance in the new space;

MVML-FDA constructs multi-view learning intra-class k adjacency graph based on sample spatial distributionAnd inter-class k adjacency graph->In-class k adjacency matrix for view t>The elements of (a) may be written in the form:

inter-class k adjacency matrix H for the t-th view _b ^t The elements of (a) may be written in the form:

based on Fisher discrimination, firstly defining a class correlation matrix and an inter-class correlation matrix applicable to multi-view learning; class-dependent correlation matrix S at view angle t ^t _w Expressed as:

where N is the number of samples,and->Respectively picture->Is a diagonal matrix of (c) and a laplace matrix,

class-dependent correlation matrix at view angle tExpressed as:

wherein the method comprises the steps ofAnd->Respectively picture->Diagonal matrix and laplace matrix, +.>

In the MVML-FDA method, the class-dependent matrix for each view angleDescribing the compactness within the class in this view, while the inter-class correlation matrix for each view +.>Describing the separability of data between classes in the view; in order to fully exploit the complementary information represented by the different views, the MVML-FDA defines a series of non-negative parameters Δ= [ Δ ] ₁ ,Δ ₂ ,...,Δ _T ]The weights for each view are expressed so that the MVML-FDA defined optimization problem can be expressed as:

redundancy in the low-dimensional representation of the data should be avoided as much as possible when learning the distance measure, for which purpose the orthogonality of the projection directions, i.e. W, is taken into account in the MVML-FDA method ^T W＝I；

Further, as can be seen from equation (7), the view weight Δ _t The larger the value, the greater the contribution of view t in learning projection matrix W; when the formula (7) reaches the optimal condition, the sum of the weights of all visual angles is 1; the weight delta in the formula (7) is calculated _t Rewritten asWherein the weight parameter r acts like fuzzy membership and the value of r satisfies r>1, a step of; thus, the MVML-FDA optimization problem is further expressed as:

further, in the step 3, the MVML-FDA optimization problem is solved, and as the formula (8) is a nonlinear non-convex problem, an iterative method is used to obtain a locally optimal solution of the parameters through alternate calculation;

first, the matrix W is fixed, lagrange multiplier lambda is introduced, and the formula (8) is written

The extreme value corresponding to the formula (9) is required to beAt the same time consider->The method comprises the following steps:

simplifying to obtain delta _t Is solved by (a) analysis:

next, fix Δ _t The value update matrix W, equation (8) is written in the form:

the optimization problem (12) is converted into the following problem by following the Ky Fan theory

By characteristic decompositionSolving to obtain an optimal solution W of W ^* I.e. calculate +.>The feature vector corresponding to the maximum K feature values is obtained ^* And then, calculating the Markov distance between the medical image pair, and obtaining the MVML-FDA model based on the analysis.

Further, in step 4, the specific searching process is as follows:

inputting medical images of T visual angles, a weight parameter r, a maximum iteration number T, a maximum neighbor number k and a convergence threshold epsilon;

step 1: an initial value Δ= [1/T, ], 1/T is set]Calculating W using formula (13) ⁰ ；For i＝1,...,T；

Step 2: calculation of delta using (11) _t ；

Step 3: calculation of W using (13) ⁱ ；

Step 4: if |W ⁱ -W ^i-1 The Step is turned to Step 5 if the I is less than epsilon;

step 5: outputting an optimal projection matrix W ^* ：W ^* ＝W ⁱ 。

The beneficial effects achieved by the invention are as follows: experimental results of the real breast image dataset show that the MVML-FDA model helps to form good distance measures from the CC and MLO views of the breast image to distinguish benign nodules, malignant nodules, benign calcifications and malignant calcifications of the breast.

Drawings

Fig. 1 is a schematic diagram of a MVML-FDA multi-view learning model in an embodiment of the present invention.

FIG. 2 is a schematic representation of benign nodules, malignant nodules, benign calcifications and malignant calcifications in CC views and MLO views used in experiments in accordance with embodiments of the present invention.

Fig. 3 is a detailed information table of feature extraction in an embodiment of the present invention.

FIG. 4 is a table of comparison of the performance of 4 comparison algorithms based on 4 feature extractions on CAR, EER and AUC indicators in an embodiment of the present invention.

Fig. 5 shows AUC values based on 4 feature extractions for different parameters r of the MVML-FDA method in an embodiment of the present invention.

Fig. 6 shows AUC values based on 4 feature extractions for different parameters k of the MVML-FDA method in an embodiment of the present invention.

FIG. 7 shows the results of MVML-FDA and 4 comparison algorithms on CAR, EER and AUC metrics for various feature extraction schemes in an example of the present invention.

Fig. 8 is a graph comparing ROC curves obtained by the 4 th feature extraction method for 5 algorithms in the embodiment of the present invention.

Detailed Description

The technical scheme of the invention is further described in detail below with reference to the attached drawings.

In the field of medical image retrieval, similarity measurement refers to calculating a distance between a medical image and a feature vector by using a given medical image pair, and is commonly used for judging the type of the medical image. For medical images, the similarity measure includes semantic relevance and visual similarity. The semantic relevance metric depends on the class labels of the medical images, e.g. the labels of both breast images are healthy, then it is stated that both breast images have a semantic relevance. Visual similarity measures describe feature similarity, typically from a visual perspective, to describe the similarity of medical images. The definition of the similarity measure generally considers the use of a distance measure, and common distance measure methods include euclidean distance, included angle cosine distance, mahalanobis distance, and Minkowski distance. Conventional metric learning is based on labeled samples of a medical image dataset, and label information is presented in the form of paired constraints on the samples. Assume that a feature-extracted medical image dataset is represented asx _i An i-th sample representing an input space, two images x, taking the mahalanobis distance as an example _i And x _j The distance measure between can be written as:

wherein the semi-positive matrix a can be decomposed into a=w ^T W, matrix W ^d×m (m.ltoreq.d) is referred to as a metric matrix. Thus (2)Formula (1) may be represented as:

thus, computing the mahalanobis metric essentially learns a mapping space such that similar images output positive values near zero and dissimilar images output larger values.

In the actual medical image processing process, multi-view medical image data are often encountered, which are from different view angles cut into to measure and diagnose the same object. And multi-view learning is based on a consistency principle and a complementation principle, and the relevance and the difference among a plurality of views are utilized in the learning process, so that more efficient and robust performance than single-view learning can be achieved. The multi-view method commonly used in the field of medical image retrieval at present is more studied to solve the problem of feature fusion by utilizing a multi-view technology. A multi-view feature fusion method for medical image retrieval based on query is provided at present, and the method iteratively learns the optimal subspace by fusing a plurality of features obtained from a user example. In addition, a multi-view local linear embedding medical image retrieval method is provided, which reserves local geometric structures in each feature space according to local linear embedding criteria and assigns different weights to different feature spaces. Finally, a global coordinate alignment and alternate optimization technique is used to learn a smooth low-dimensional embedding from different characteristics. Das et al propose four different feature extraction techniques-binarization-based, transform-based, texture-based and shape-based techniques for content-based multi-view image retrieval.

A breast image retrieval method for multi-view discrimination metric learning comprises the following steps:

the multi-view medical image data set obtained according to different shooting positions is { X } ¹ ,X ² ,...,X ^T (wherein X is ^t Representing a data matrix at a t-th viewing angle, X ^t Is the ith vector of (2)Representing sample x _i Feature vectors at the first view angle. Thus, the medical image pair at the t-th view can be written as +.>Wherein->Representing any medical image pair,/->Representing the corresponding image pair label, +.>Instruction image->And->Are similar in terms of both semantics and vision, l _m,i = -1 illustrates image->And->Are semantically or visually dissimilar. Through mahalanobis metric learning, the MVML-FDA aims at learning a mapping space such that the distance between similar samples is much smaller than the distance between different samples, i.e. there is a more separable inter-class distance and a more compact intra-class distance in the new space, and a schematic diagram of the MVML-FDA multi-view learning model is given in fig. 1.

MVML-FDA constructs multi-view learning intra-class k adjacency graph based on sample spatial distributionAnd inter-class k adjacency graph->In-class k adjacency matrix for view t>The elements of (a) may be written in the form:

inter-class k adjacency matrix for the t-th viewThe elements of (a) may be written in the form:

based on Fisher discrimination, a intra-class correlation matrix and an inter-class correlation matrix suitable for multi-view learning are defined first. Class-dependent correlation matrix at view angle tExpressed as:

where N is the number of samples,and->Respectively picture->Is a diagonal matrix of (c) and a laplace matrix,

class-dependent correlation matrix at view angle tExpressed as:

wherein the method comprises the steps ofAnd->Respectively picture->Diagonal matrix and laplace matrix, +.>

In the MVML-FDA method, the class-dependent matrix for each view angleDescribing the compactness within the class in this view, while the inter-class correlation matrix for each view +.>The separability of data between classes in this view is described. In order to fully exploit the complementary information represented by the different views, the MVML-FDA defines a series of non-negative parameters Δ= [ Δ ] ₁ ,Δ ₂ ,...,Δ _T ]The weights for each view are expressed so that the MVML-FDA defined optimization problem can be expressed as:

in learning distance measurement, the number should be avoided as much as possibleRedundancy in terms of low-dimensional representation, for which reason orthogonalization of projection directions, i.e. W, is considered in the MVML-FDA method ^T W=i. Further, as can be seen from equation (7), the view weight Δ _t The larger the value, the greater the contribution of view t in learning the projection matrix W. The sum of the viewing angle weights required to be satisfied when equation (7) reaches the optimum is 1. The method shows that the medical image features under different visual angles can be fully utilized, the method is different from single visual angle learning, only the best visual angle is selected, and the proposed multi-visual angle discrimination measurement learning model can mine and balance different characterization information of the features under the multi-visual angle, so that the medical image retrieval capability is improved. But in extreme cases delta may occur _t Approaching the case of 1, at this time, the tth view plays an absolute dominant role in equation (7), while the roles of the other views are impaired. To avoid this, the weights Δ in equation (7) are inspired by fuzzy clustering _t Rewritten asWherein the weight parameter r acts like fuzzy membership and the value of r satisfies r>1. Thus, the MVML-FDA optimization problem can be further expressed as:

next, a solution method of MVML-FDA is described. Equation (8) is a nonlinear non-convex problem, and the locally optimal solution of the parameters is obtained through alternate calculation by using an iterative method. First, the matrix W is fixed, the Lagrangian multiplier lambda is introduced, and the formula (8) can be written as

The extreme value corresponding to the formula (9) is required to beAt the same time consider->The method can obtain the following steps:

by simplification, delta can be obtained _t Is solved by (a) analysis:

next, fix Δ _t The value updates the matrix W. Formula (8) may be written in the following form:

the optimization problem (12) obeys the Ky Fan theory and can be converted into the following problem

By characteristic decompositionSolving to obtain an optimal solution W of W ^* I.e. calculate +.>And the feature vectors corresponding to the maximum K feature values. Obtaining W ^* The mahalanobis distance between the medical image pair can then be obtained from equation (2). Based on the above analysis, a MVML-FDA model can be obtained, the algorithm of which is described below.

Inputting medical images of T visual angles, a weight parameter r, a maximum iteration number T, a maximum neighbor number k and a convergence threshold epsilon;

step 1: an initial value Δ= [1/T, ], 1/T is set]Calculating W using formula (13) ⁰ ；For i＝1,...,T；

Step 2: use type(11) Calculating delta _t ；

Step 3: calculation of W using (13) ⁱ ；

Step 4: if |W ⁱ -W ^i-1 The Step is turned to Step 5 if the I is less than epsilon;

step 5: outputting an optimal projection matrix W ^* ：W ^* ＝W ⁱ 。

Next, the search method of the invention is experimentally verified, firstly, the data set and experimental setting used in the experiment are introduced, and secondly, the feature selection method used in the experiment is introduced. The MVML-FDA performance was then evaluated on a real data set and finally compared to a comparison algorithm.

The dataset of the experiment was from the Breast Cancer Digital Repository (BCDR) dataset. The dataset consisted of female patients with portugal teeth, ranging in age from 28 years to 82 years. BCDR consisted of 1010 patients, including 3703 MLO and CC mammograms and 1044 clinically confirmed lesions. Both the MLO and CC images are gray-scale digitized images, the images are 720 x 1167 pixels, and the gray scale is 256. In addition to patient age and breast density, the dataset includes a selected set of binary attributes for indicating that the physician observes abnormal information such as nodules, calcification, and matrix deformation. Thus, the clinical data for each instance of the BCDRF01 dataset includes a total of 8 attributes per instance: the 6 binary attributes associated with observed abnormalities, the ordinal attribute of breast density, and the digital attribute containing the age of the patient at study. In the experiment, 400 pairs of MLO and CC mammograms in the BCDR database, namely 800 breast image cases, are selected as experimental data. MLO and CC serve as two different perspectives in multi-perspective learning, respectively. Random 75% data of each view angle is selected from 800 images to be used as a training image set for training a model, and the rest 25% data are used as a query image set, so that breast images of the same patient cannot be simultaneously present in the training image set and the query image set. The whole training and testing process is carried out 5 times, and the average calculation result of 5 times is taken as a final result. Fig. 2 presents schematic representations of benign nodules, malignant nodules, benign calcifications and malignant calcifications in CC view and MLO view used in the experiments. Wherein fig. 2 (a) and fig. 2 (b) are from the same benign nodule patient; FIGS. 2 (c) and 2 (d) are from the same malignant nodule patient; fig. 2 (e) and fig. 2 (f) are from the same benign calcified patient; fig. 2 (g) and fig. 2 (h) are from the same malignant calcified patient.

In the experiment for evaluating the MVML-FDA algorithm, the MVML-FDA conversion method under 3 different visual angles is designed: MVML-FDA-CC indicates that MVML-FDA only considers the breast image under CC single view; MVML-FDA-MLO means that MVML-FDA only considers the breast image under the single view angle of MLO; MVML-FDA- (MLO+CC) means that the breast images at CC and MLO views are mixed at a single view angle. In experiments comparing the performance of MVML-FDA algorithms, 4 algorithms were compared, including: adaptive multiview learning machine (AMVL), multi-view CVM (MvCVM), global and Local Structural Risk Minimization (GLSRM) and Multi-view learning with Universum (MVU). In order to ensure fairness of the comparison experiment, the optimal parameters are selected in a grid search mode. Specifically, the number of neighbors of the graph in AMVL is set to {5,10,15,20}; the number of prescriptions r is set to {2,4,., 10}. MvCVM uses Gaussian kernelsWherein the kernel width parameter sigma of each view angle is used for training the average l of the view angles corresponding to the sample ₂ Norm square s is referenced and is in grid { s/64, s/32,., 64 _s Searching and selecting; each view penalty factor is in grid {1,10 ¹ ,...,10 ⁶ Search and select in }. Left and right weights and offsets {0.1,0.2, & gt, 1}, regularization parameters {10 } in GLSRM ^-3 ，10 ^-2 ,...,10 ³ Learning rate 0.99, relaxation amount of visual angle 10 ^-6 . MVU the learning rate of each view is set to 0.99, regularization parameter {10 } ^-2 ，10 ^-1 ,...,10 ² Viewing angle relaxation amount 10 ^-6 . The evaluation criteria are used herein as classification accuracy (Classification Accuracy Rate, CAR), equal Error Rate (EER) and area under ROC curve (area underthe Roc curve, AUC).

Features of the breast image are extracted from 4 aspects of intensity description (Intensity statistics), texture description, multi-scale texture description and image gradient, and detailed information of feature extraction is shown in a table of fig. 3. In addition to the features extracted in fig. 3, 8 features per clinical data were used in the experiment.

The validity of the proposed multi-view method MVML-FDA was verified using the structured single-view methods MVML-FDA-CC, MVML-FDA-MLO and the mixed single-view method MVML-FDA- (MLO+CC). Figure 4 shows a comparison of performance of 4 comparison algorithms based on 4 feature extractions on CAR, EER and AUC indicators. The following results were obtained from the results of fig. 4:

1) The CAR, EER and AUC performances of the MVML-FDA-CC and MVML-FDA-MLO methods under 4 feature extraction methods are general, and the main reason is that the 2 methods are single-view methods, and although the methods are very similar to the MVML-FDA methods in structure, the methods have no coordination ability of different view angles, and the methods are limited to the samples per se under a single-view frame, but cannot obtain optimal spatial projection due to the difference between the samples.

2) MVML-FDA- (MLO+CC) combines the MLO view and the CC view into one dataset process where the MLO view and the CC view are converted into a subset of datasets, respectively. Because of the large difference between the breast images obtained from a portion of the MLO view and the CC view, the CAR, EER and AUC performance of MVML-FDA- (MLO+CC) in the 4 feature extraction methods are significantly lower than that of the MVML-FDA-CC and MVML-FDA-MLO methods.

3) The MVML-FDA has the multi-view collaborative learning capability, and can fully utilize the information of the multiple view spaces to learn the optimal value of the space projection. Thus, MVML-FDA has significant performance advantages over the other 3 methods.

From formula (5), 2 parameters are involved in the MVML-FDA method: a weight index r and a maximum number k of neighbors. The effect of the parameters r and k on MVML-FDA performance is discussed below. Figures 5-6 show the mean values of CAR, EER and AUC for the MVML-FDA method at different parameters r and k values for the 4 feature extraction methods. From the results in the figures, we conclude that:

1) The weight index r is a parameter similar to the fuzzy index in fuzzy clustering and is a parameter of the degree of 'fuzzification'; the weight index r can also be regarded as a smoothing parameter, which controls the importance of different views in the construction of the MVML-FDA method. From the results in fig. 5, it is seen that the weight index r is less sensitive in breast images, and the average of CAR, EER and AUC at different r values is not much different. Therefore, to expedite training of the MVML-FDA method, the value of r can be manually fixed.

2) The maximum number k of neighbors has a significant impact on the performance of the proposed MVML-FDA approach. As can be seen from equations (3) - (4), the parameter k represents the correlation matrix in the classAnd inter-class correlation matrix->Is a neighbor number in (a). When the k value of the neighbor number is too small, only an image sample very similar to the input sample acts on the MVML-FDA method, and in this case, the MVML-FDA method is prone to over-fitting, and the final performance error of the MVML-FDA increases. When the k value of the neighbor number is too large, more image samples are needed to construct the intra-class correlation matrix and the inter-class correlation matrix. In this case, samples further from the input image samples will also contribute to the model building, and the final classification result is susceptible to negative effects. Therefore, the k-value in the MVML-FDA method needs to be selected by using a cross-validation method in consideration of the distribution and the intrinsic structure of the data set, which is appropriate.

The MVML-FDA method was compared to the performance of the 4 multi-view methods GLSRM, MVU, mvCVM and AMVL on CAR, EER and AUC indicators. FIG. 7 shows the results of MVML-FDA and 4 comparison algorithms on CAR, EER and AUC metrics for various feature extraction modes. From the experimental results of fig. 7, the following conclusions can be drawn:

1) The results of the 5 multi-view approach on the CAR, EER and AUC indices are all superior to the single view approach shown in table 2. The multi-view method is described to improve the accuracy of medical image retrieval by using complementary information between different views.

2) The MVML-FDA method presented herein resulted in a CAR, EER and AUC that were all superior to the other 4 multi-view methods. Because the MVML-FDA method considers view weighting on the basis of collaborative learning, the medical image retrieval capability is improved. AMVL is an unsupervised method and cannot effectively utilize semantic information of medical images. Both GLSRM and MVU methods use euclidean distance, and the effect of measuring euclidean distance for high dimensional data tends to be poor. The MvCVM method is established in the theory that the classification interval between multi-view data is maximized, the information between the same-category data can not be fully utilized, meanwhile, the approximation accuracy threshold of the MvCVM method can also influence the performance of an algorithm, and the performances of the CAR, the EER and the AUC on the breast medical image are inferior to those of the MVML-FDA method.

3) In addition, when using the feature extraction method based on image gradient (i.e. feature method 4), all methods achieved higher CAR, EER and AUC performance than other feature methods, and also demonstrated that the feature extraction method based on image gradient is more suitable for extracting feature vectors of breast medical images.

To better compare the performance ratio of MVML-FDA to the comparison algorithm, fig. 8 compares ROC curves obtained under the 4 th feature extraction method for the 5 algorithms. From fig. 8, it can be seen that MVML-FDA is superior to another 4 multi-view learning methods. The experimental results achieved consistent results on CAR, EER, AUC and ROC curve 4 performance indicators, indicating that it is appropriate to use these 4 indicators to evaluate the results of a search for breast medical images.

The above description is merely of preferred embodiments of the present invention, and the scope of the present invention is not limited to the above embodiments, but all equivalent modifications or variations according to the present disclosure will be within the scope of the claims.

Claims (4)

1. A breast image retrieval method for multi-view discrimination metric learning is characterized in that: the method comprises the following steps:

step 1, obtaining a data set of the multi-view medical image according to the multi-view medical image obtained from different shooting positions;

step 2, constructing a multi-view discrimination measurement learning MVML-FDA model through a data set, measuring the similarity between breast image pairs by Fisher discrimination, mapping similar medical images in a measurement space, separating dissimilar breast images from each other, and raising an MVML-FDA optimization problem;

in step 2, MVML-FDA constructs a multi-view learning intra-class k adjacency graph based on the spatial distribution of samplesAnd inter-class k adjacency graph->In-class k adjacency matrix for view t>The elements of (a) are written in the following form:

inter-class k adjacency matrix for the t-th viewThe elements of (a) are written in the following form:

based on Fisher discrimination, firstly defining a class correlation matrix and an inter-class correlation matrix applicable to multi-view learning; class-dependent correlation matrix at view angle tExpressed as:

where N is the number of samples,and->Respectively is (picture)>Is a diagonal matrix of (c) and a laplace matrix,

class-dependent correlation matrix at view angle tExpressed as:

wherein the method comprises the steps ofAnd->Respectively is (picture)>Diagonal matrix and laplace matrix, +.>

In the MVML-FDA method, the class-dependent matrix for each view angleDescribing the compactness within the class in this view, while the inter-class correlation matrix for each view +.>Describing the separability of data between classes in the view; in order to fully exploit the complementary information represented by the different views, the MVML-FDA defines a series of non-negative parameters Δ= [ Δ ] ₁ ,Δ ₂ ,...,Δ _T ]The weights for each view are expressed, so that the MVML-FDA defined optimization problem is expressed as:

the orthogonality of the projection directions, i.e. W, is taken into account in the MVML-FDA method ^T W＝I；

Further, as can be seen from equation (7), the view weight Δ _t The larger the value, the greater the contribution of view t in learning projection matrix W; when the formula (7) reaches the optimal condition, the sum of the weights of all visual angles is 1; the weight delta in the formula (7) is calculated _t Rewritten asWherein the weight parameter r acts like fuzzy membership and the value of r satisfies r>1, a step of; thus, the MVML-FDA optimization problem is further expressed as:

step 3, solving the MVML-FDA optimization problem to obtain an optimal solution, namely an MVML-FDA model;

and 4, inputting medical images of a plurality of visual angles, weight parameters, maximum iteration times, maximum neighbor numbers and convergence threshold values by using the obtained MVML-FDA model, outputting an optimal medical image projection matrix, and completing medical image retrieval.

2. The breast image retrieval method for multi-view discrimination metric learning according to claim 1, wherein: in step 1, the multi-view medical image dataset obtained according to different film capturing positions is { X } ¹ ,X ² ,...,X ^T (wherein X is ^t Representing a data matrix at a t-th viewing angle, X ^t Is the ith vector of (2)Representing sample x _i A feature vector at a t-th view angle; thus, the medical image pair at the t-th view is written +.>Wherein->Representing any medical image pair,/->Representing the corresponding image pair label, +.>Instruction image->And->Semantically and visually similar, < >>Instruction image->And->Are semantically or visually dissimilar.

3. The breast image retrieval method for multi-view discrimination metric learning according to claim 1, wherein: in the step 3, solving the MVML-FDA optimization problem, wherein the formula (8) is a nonlinear non-convex problem, and an iteration method is used for obtaining a local optimal solution of parameters through alternate calculation;

first, the matrix W is fixed, lagrange multiplier lambda is introduced, and the formula (8) is written

The extreme value corresponding to the formula (9) is required to beAt the same time consider->The method comprises the following steps:

simplifying to obtain delta _t Is solved by (a) analysis:

next, fix Δ _t The value update matrix W, equation (8) is written in the form:

the optimization problem (12) is converted into the following problem by following the Ky Fan theory

By characteristic decompositionSolving to obtain an optimal solution W of W ^* I.e. calculate +.>The feature vector corresponding to the maximum K feature values is obtained ^* And then, calculating the Markov distance between the medical image pair, and obtaining the MVML-FDA model based on the analysis.

4. A breast image retrieval method for multi-view discriminant metric learning according to claim 3, wherein: in step 4, the specific searching process is as follows:

inputting medical images of T visual angles, a weight parameter r, a maximum iteration number T, a maximum neighbor number k and a convergence threshold epsilon;

step 1: an initial value Δ= [1/T, ], 1/T is set]Calculating W using formula (13) ⁰ ；

For i＝1,...,T；

Step 2: calculation of delta using (11) _t ；

Step 3: calculation of W using (13) ⁱ ；

Step 4: if |W ⁱ -W ^i-1 The Step is turned to Step 5 if the I is less than epsilon;

step 5: outputting an optimal projection matrix W ^* ：W ^* ＝W ⁱ 。