CN107545276B - Multi-view learning method combining low-rank representation and sparse regression - Google Patents

Abstract

The invention discloses a multi-view learning method combining low-rank representation and sparse regression. The method includes the following steps: extracting low-level features and high-level attribute features respectively on a SUN data set with an image memory score label; The three parts of low-rank representation, combined sparse regression model and multi-view consistency loss are placed under the same framework to form a whole, and a multi-view model combining low-rank and sparse regression is constructed. The problem of memorability is to obtain the relationship between the underlying image features, image attribute features and image memory under the optimal parameters; combine the low-level features and high-level attribute features of the image, and use the relationship results obtained under the optimal parameters to predict the database test. Set image memory, and use relevant evaluation criteria to verify the prediction results. The invention combines the multi-view learning framework of low-rank representation and sparse regression to accurately predict the memorability of image regions.

Description

A Multi-View Learning Approach to Joint Low-Rank Representation and Sparse Regression

technical field

The invention relates to the field of low-rank representation and sparse regression, in particular to a multi-view learning method combining low-rank representation and sparse regression.

Background technique

Humans have the ability to remember thousands of images, but not all images are stored in the brain in the same way. Some representative images are remembered at a glance, while other images are easily lost from memory. Image memory is used to measure how well images are remembered or forgotten after a specific period of time. Previous research work has shown that memory for pictures is related to the inherent property of images that memory for pictures is consistent across time intervals and across observers. In this context, as with many other high-level image attributes (such as popularity, interest, mood, and aesthetics), several research works have begun to explore the potential correlation between image content representation and image memory.

Analyzing image memorability can be applied in several fields such as user interface design, video summarization, scene understanding, and advertising design. For example, image collections or videos can be summarized using memorability as a guiding criterion by selecting meaningful images. By enhancing consumers' memory of the target brand, unforgettable advertisements can be designed to help merchants expand their influence.

Recently, low-rank representation (LRR) has been successfully applied in multimedia and computer vision domains. To better handle the feature representation problem, LRR is used to reveal the underlying low-dimensional subspace structure embedded in the data by decomposing the original data matrix into a low-rank representation matrix while eliminating irrelevant details. Traditional methods are often insufficient for outlier handling. To address this issue, some recent studies also focus on sparse regression learning.

However, one of the main shortcomings of these works is that feature representation and memory prediction are performed in two separate stages. That is, when determining the pattern of feature combinations for image memorability prediction, the final performance of the regression step is mainly determined by the processed features. While reference [1] proposes a feature encoding algorithm for joint low-rank and sparse regression to deal with outliers. Likewise, Reference [2] develops a joint graph embedding and sparse regression framework for dimensionality reduction. But they are all designed for visual classification problems, not image memory prediction tasks.

SUMMARY OF THE INVENTION

The present invention provides a multi-view learning method that combines low-rank representation and sparse regression. The present invention combines a multi-view learning framework of low-rank representation and sparse regression to accurately predict the memorability of image regions, as described below:

Extract low-level features and high-level attribute features respectively for the SUN dataset with image memory score labels;

The three parts of low-rank representation, combined sparse regression model and multi-view consistency loss are placed under the same framework to form a whole, and a multi-view model combining low-rank and sparse regression is constructed;

The multi-vision adaptive regression algorithm is used to solve the problem of automatically predicting the memorability of images, and the relationship between image underlying features, image attribute features and image memory is obtained under the optimal parameters;

Combine the low-level features and high-level attribute features of the image, use the relationship results obtained under the optimal parameters to predict the image memory of the database test set, and use the relevant evaluation criteria to verify the prediction results;

The problem of automatically predicting the memorability of the image by using the multi-vision adaptive regression algorithm is: converting the equivalent problem through the slack variable Q:

s.t.X=XA+E, Q=Aw

Among them, A is the mapping matrix of the low-rank representation; w is the linear dependence between the low-rank feature representation and the output memory score; E is the sparse error constraint part; α is the balance parameter between the prediction error part and the regularization part ; β is the parameter to control sparsity; λ > 0 is the balance parameter; X is the input feature; y is the memorability score vector; * is the nuclear norm representation; φ is the constraint term of graph regularization; operator;

Two slack variables Y ₁ and Y ₂ are introduced to obtain the augmented Lagrangian function:

Among them, <,> represents the inner product operation of the matrix, Y ₁ and Y ₂ represent the Lagrangian operator matrix, and μ > 0 is the positive penalty parameter. The above methods are combined as:

in

Introduce variable t, define A _t , E _t , Q _t , w _t , Y _{1, t} , Y _{2, t} and μ as the result of the t-th iteration of the variables, and the result of the t+1-th iteration is as follows:

The iterative result of A:

in,

Fixing w, A, Q to get the optimization result of E is as follows:

By fixing E, A, Q, the result of optimizing w is as follows:

The above problem is a ridge regression problem, and the optimal solution is

Finally, fix E, w, A, optimize Q, get:

Y ₁ and Y ₂ are updated with the following scheme:

Y _1,t+1 =Y _1,t + μ _t (X-XA _t+1 -E _t+1 )

Y _2,t+1 =Y _2,t + μ _t (Q _t+1 -A _t+1 w _t+1 )

为求偏导的符号；in,

is the symbol for partial derivative;

The multi-view model of the joint low-rank and sparse regression is specifically:

in:

G(φ _H ,φ _l )=tr[(X _l A _l w _l ) ^T X _H A _H w _H ]

为用作高级特征预测误差的损失函数；

为用作低级特征预测误差的损失函数；

为用于解决过拟合问题的图形正则化表示；X_H为高级属性特征；A_H为高级属性特征低秩表示的映射矩阵；E_H为是高级属性特征稀疏误差约束部分；w_H为高级属性特征的低秩表示和输出记忆度分数之间的线性依赖关系；A_l为低级特征低秩表示的映射矩阵；E_l为低级属性特征稀疏误差约束部分；X_l为低级属性特征；w_l为低级属性特征的低秩表示和输出记忆度分数之间的线性依赖关系。

is the loss function used as the prediction error for advanced features;

is the loss function used as the prediction error of low-level features;

is the graphic regularization representation used to solve the overfitting problem; X _H is the high-level attribute feature; A _H is the mapping matrix of the low-rank representation of the high-level attribute feature; E _H is the sparse error constraint part of the high-level attribute feature; w _H is the high-level attribute feature Linear dependence between low-rank representation of attribute features and output memory score; A _l is the mapping matrix of low-rank representation of low-level features; E _l is the sparse error constraint part of low-level attribute features; X _l is low-level attribute features; w _l is the linear dependence between the low-rank representation of low-level attribute features and the output memory score.

The method also includes acquiring an image memorability dataset.

The low-level features include: scale-invariant feature transformation features, search tree features, directional gradient histogram features, and structural similarity features.

The advanced attribute features include: 327-dimensional scene category attribute features and 106-dimensional object attribute features.

The beneficial effects of the technical scheme provided by the present invention are:

1. Combine low-rank representation and sparse regression for image memorability prediction, in which low-rank constraints are used to reveal the inherent structure of the embedded original data, and sparse constraints are used to remove outliers and redundant information. When low-rank representation and sparse regression work together When executed, the low-rank representation shared by all features can capture the internal structure of features, thereby improving the accuracy of prediction;

2. The present invention is based on the multi-vision adaptive regression (MAR) algorithm to solve the optimization problem of the objective function with fast convergence.

Description of drawings

Figure 1 is a flowchart of a multi-view learning method combining low-rank representation and sparse regression;

Figure 2 is an example of a database image marked with an image memory score;

Fig. 3 is the algorithm convergence diagram;

Figure 4 is a comparison chart of the results of this method and other methods.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention are further described in detail below.

Example 1

Studies have shown that image attribute features are very high-level semantic features compared to their original underlying features, and the visual features of images are studied and image memory is predicted. A multi-view learning method combining low-rank representation and sparse regression, see Figure 1. The method includes the following steps:

101: Obtain an image memory data set;

Among them, the image memorability dataset ^[1] contains 2,222 images from the SUN dataset ^[11] . The memory score of the image is obtained by Amazon Mechanical Turk's Visual Memory Game, and the image memorability is a continuous value from 0 to 1. The higher the value, the harder the image is to remember. Sample images with various memory scores are shown in Figure 2.

102: Extract low-level features and high-level attribute features respectively for the SUN dataset with image memory score labels;

Among them, the extracted low-level features include: SIFT (Scale-invariant feature transform, SIFT), Gist (Search Tree, Generalized Search Trees), HOG (Histogram of Oriented Gradient) and SSIM (Structure Similarity, structural similarityindex)) feature, the four low-level features together constitute the low-level feature library. The embodiment of the present invention simultaneously uses two types of advanced attribute features, including: a 327-dimensional scene category attribute feature and a 106-dimensional object attribute feature.

The scene category attribute covers 327 scene categories, and the object attribute feature is marked by 106 object categories, and the specific dimension is set according to actual application needs, which is not limited in this embodiment of the present invention. .

103: The low-rank representation, the sparse regression model and the multi-view consistency loss are placed under the same framework to form a whole, and the Mv-JLRSR (multi-view joint low-rank and sparse regression) model is constructed;

104: Use the multi-vision adaptive regression (MAR) algorithm to solve the problem of automatically predicting the memorability of images, and obtain the relationship between the underlying image features, image attribute features and image memory under optimal parameters;

105: Combine the low-level features and high-level attribute features of the image, use the relationship results obtained under the optimal parameters to predict the image memory degree of the database test set, and use the relevant evaluation criteria to verify the prediction results.

To sum up, in the embodiment of the present invention, the low-rank constraints are used to reveal the internal structure of the original data through the above steps 101 to 105, and the outliers and redundant information of the features are removed by using the sparse constraints. When the low-level representation and the sparse regression are executed jointly , the lowest-level representation shared by all features can not only capture the global structure of all modalities, but also represent the requirements of regression; since the formulated objective function is not smooth and difficult to solve, the Multi-View Adaptive Regression (MAR) algorithm is used to solve automatic Predict the memorability problem of images to solve optimization problems with fast convergence.

Example 2

The scheme in Embodiment 1 is further introduced below in conjunction with specific calculation formulas, and is described in detail below:

201: The image memorability dataset contains 2,222 images from the SUN dataset;

The data set is known to those skilled in the art, and details are not described in this embodiment of the present invention.

202: Perform feature extraction on the pictures of the SUN dataset with image memory score labels, the extracted SIFT, Gist, HOG and SSIM features form a low-level feature library, and use two types of high-level attribute features, including 327-dimensional scene categories properties, 106-dimensional object properties

D_i代表此类特征的维数，N代表数据库中所含图像个数(2222)。这些特征构成特征库B＝{B₁,...,B_M}。Among them, the above data set includes 2222 pictures in various environments, and each picture is marked with the image memory score. Figure 2 shows a sample of the pictures marked with the memory score in the database. Features are expressed as

D _i represents the dimension of such features, and N represents the number of images contained in the database (2222). These features constitute a feature library B={B ₁ , . . . , B _M }.

203: Establish an Mv-JLRSR model, combine low-rank representation and sparse regression on the basis of the extracted low-level features and high-level attribute features, establish a more robust feature representation, and establish an accurate regression model.

The general framework defined by the Mv-JLRSR model is as follows:

where F(A,w) is the loss function used as prediction error; L(A,E) represents the feature encoder based on low-rank representation; G(A) is the graph regularization representation used to solve the overfitting problem ; A is the mapping matrix of the low-rank representation; w is the linear dependence between the low-rank feature representation and the output memory score; E is the sparse error constraint part.

The image memorability dataset ^[1] contains 2,222 images from the SUN dataset ^[11] , and the memory scores of the images are obtained through the Visual Memory Game of Amazon Mechanical Turk; combined with the regression training of adaptive transfer learning, the linear regression method is adopted Train the extracted feature library. The score prediction of image memory degree is divided into two aspects. On the one hand, the feature representation is directly used to predict the image memory degree, and the mapping matrix w _i from each type of image underlying features to image memory degree is obtained. On the other hand, the image high-level attribute features (in It also plays a very important role in predicting the image memory score. Combined with low-rank learning, the relationship between each type of image attribute and image memory is obtained; according to the initial image feature vector set X∈R ^N×D , the goal of the Mv-JLRSR model is It combines low-rank representation and sparse regression based on the extracted visual cues to enhance robust feature representation and accurate regression models.

A detailed introduction to each part:

Due to low-rank constraints, noise or redundant information can be removed to help reveal the essential structure of the data. Therefore, these extracted low-level and high-level features can be integrated into feature learning to deal with these problems. LRR assumes that the original feature matrix contains the potential lowest-rank structural component shared by all samples and its unique error matrix:

where A∈R ^D×D is the low-rank projection matrix of N samples, E∈R ^N×D is the only sparse error part constrained with the _l1 norm to handle random errors, λ>0 is the balance parameter, X is the input feature; D is the feature dimension after low-rank constraint; rank is the low-rank representation of the feature.

Since the above equation is difficult to optimize, the nuclear norm ||A|| _* ( _* indicates that the nuclear norm refers to the sum of the singular values of the matrix) is used to approximate the rank of A, so the formula of L(A, E) can be defined as follows:

来解决预测问题。在最小二乘误差部分加入岭正则化^[6]后，获得具有岭回归的典型最小二乘问题。In the framework proposed by the embodiments of the present invention, the problem of image memory prediction is regarded as a standard regression problem. The lasso ^[5] regression method is proposed to minimize the least squares error by establishing a linear relationship v between the input feature matrix X and the memorability score vector y

to solve the prediction problem. After adding ridge regularization ^[6] to the least squares error part, a typical least squares problem with ridge regression is obtained.

where α is a balance parameter between the prediction error part and the regularization part.

From the perspective of matrix decomposition, the transformation vector v can be decomposed into a product of two components, i.e. applying a low-rank projection matrix A to capture the low-rank structure shared between samples, and applying a coefficient vector w to transform the transformed Samples are associated with their memory scores. Introduce Q=Aw and define the loss function F(A,w) as:

Based on the idea of multi-learning, graph regularization is adopted to maintain the consistency of the geometric structure to solve this problem. The core idea of graph regularization is that the samples are close in feature representation, so the memory scores are also close, and vice versa. Geometric consistency between features and memory scores is achieved by minimizing the graph regularizer G(A):

Among them, L=BS is the Laplacian operator, B is the angle matrix, B _ii =∑ _j S _ij , S is the weight matrix calculated by the Gaussian similarity function, and its calculation is obtained by the Gaussian similarity function:

where y _i and y _j are the popularity scores of the i-th sample and the j-th sample, N _K denotes that y _i is the K neighbors of yi _i , and σ is a radius parameter, which is simply set as all video pairs The median value of the Euclidean distance on .

Often multiple features are extracted to represent images from different views. This is because these multiple representations can provide compatible and complementary information. For image memory prediction tasks, the natural choice is to integrate these multiple representations to represent images for better performance, rather than relying on a single feature. The embodiments of the present invention extract high-level attribute features and low-level visual features.

Therefore, the Mv-JLRSR model is defined as:

in:

A∈R ^D×D is a low-rank projection matrix of N samples to capture the underlying low-rank structure shared between samples, and E∈R ^N×D utilizes the L ₁ norm to account for random errors.

为用作高级特征预测误差的损失函数；

为用作低级特征预测误差的损失函数；

为用于解决过拟合问题的图形正则化表示；X_H为高级属性特征；A_H为高级属性特征低秩表示的映射矩阵；E_H为是高级属性特征稀疏误差约束部分；β为控制稀疏的参数；w_H为高级属性特征的低秩表示和输出记忆度分数之间的线性依赖关系；A_l为低级特征低秩表示的映射矩阵；E_l为低级属性特征稀疏误差约束部分；X_l为低级属性特征；E_l为低级属性特征稀疏误差约束部分；w_l为低级属性特征的低秩表示和输出记忆度分数之间的线性依赖关系；y为训练样本的标签。in,

is the loss function used as the prediction error for advanced features;

is the loss function used as the prediction error of low-level features;

is the graphic regularization representation used to solve the overfitting problem; X _H is the high-level attribute feature; A _H is the mapping matrix of the low-rank representation of the high-level attribute feature; E _H is the sparse error constraint part of the high-level attribute feature; β is the control sparsity parameter; w _H is the linear dependence between the low-rank representation of high-level attribute features and the output memory score; A _l is the mapping matrix of low-level feature low-rank representation; E _l is the sparse error constraint part of low-level attribute features; X _l is the low-level attribute feature; E _l is the sparse error constraint part of the low-level attribute feature; w _l is the linear dependence between the low-rank representation of the low-level attribute feature and the output memory score; y is the label of the training sample.

β||A_lw_l||+λ||X_lA_lw_l-y||_F是定义的误差函数，通过在输入特征矩阵(X_H和X_l)和可记忆度分数向量y之间建立线性向量v来解决预测问题。

β||A _l w _l ||+λ||X _l A _l w _l -y|| _F is the error function defined by summing the input feature matrix (X _H and X _l ) and the memorability score vector y A linear vector v is established between to solve the prediction problem.

是为了保证特征相近的样本记忆力分数也是接近的。

It is to ensure that the memory scores of samples with similar characteristics are also close.

Initialize α, β, λ and φ in F _H (φ _H ) and _FL (φ _L ) in the Mv-JLRSR model; fix A, E, w and Q respectively and derive them, and repeat the derivation process until the error reaches the set minimum value.

The following is a detailed introduction to the solution process, and the multi-vision adaptive regression (MAR) algorithm ^[7] is used to solve the problem of automatically predicting the memorability of the image, and then solve the optimization problem;

First, a slack variable Q is introduced to transform the above equivalent problem:

Then, two slack variables Y ₁ and Y ₂ are introduced to obtain the augmented Lagrangian function:

Where <,> represents the inner product operation of the matrix, Y ₁ and Y ₂ represent the Lagrangian operator matrix, μ>0 is the positive penalty parameter, * is the nuclear norm representation; φ is the constraint term for graph regularization; The methods are combined into:

in

This method adopts the alternate iteration method to solve. Each subproblem is treated separately by approximating the quadratic term h(A,Q,E,w,Y ₁ , Y ₂ , μ) as a second-order Taylor expansion. To better understand this process, a variable t is introduced, and A _t , E _t , Q _t , w _t , Y _{1, t} , Y _{2, t} and μ are defined as the results of the t-th iteration of the variables, Therefore, the result of the t+1th iteration is as follows:

The iterative result of A:

in,

Then fix w, A, Q to get the optimization result of E as follows:

By fixing E, A, Q, the result of optimizing w is as follows:

The above problem is actually the well-known ridge regression problem, and its optimal solution is

Finally, fix E, w, A, and optimize Q, you can get:

Furthermore, the Lagrangian multipliers Y ₁ and Y ₂ can be updated by the following scheme:

Y _1,t+1 =Y _1,t + μ _t (X-XA _t+1 -E _t+1 )

Y _2,t+1 =Y _2,t + μ _t (Q _t+1 -A _t+1 w _t+1 )

为求偏导的符号。in,

Symbols for partial derivatives.

The relationship between the predicted score and the actual score is studied under the selected evaluation criteria, and the algorithm performance results are obtained.

In the embodiment of the present invention, the database is randomly divided into 10 groups, and the above steps are performed for each group to obtain 10 groups of correlation coefficients, and the average value is taken to evaluate the performance of the algorithm. The evaluation criteria selected by this method include Ranking Correlation and R-value, which are also described in detail in Embodiment 3.

Example 3

Below in conjunction with concrete experimental data, Fig. 3 to Fig. 4 carry out feasibility verification to the scheme in embodiment 1 and 2, see the following description for details:

The Image Memorability dataset contains 2,222 images from the SUN dataset. The memory score of the image is obtained by Amazon Mechanical Turk's Visual Memory Game, and the image memorability is a continuous value from 0 to 1. The higher the value, the harder the image is to remember, and sample images with various memory scores are shown in Figure 2.

This method adopts two evaluation methods:

Ranking Correlation (RC): Obtain the ranking relationship between the actual memory ranking and the predicted memory score, and use the ranking-related Spearman correlation coefficient standard to measure the correlation coefficient between the two rankings. Its value range is [-1, 1], and the higher the value, the closer the two sorts are:

Among them, N is the number of images in the test set, the element r _1i in r ₁ is the position where the ith image is sorted in the real result, and the element r _2i in r ₂ is the position where the ith image is sorted in the predicted result.

R-value: Evaluate the correlation between predicted scores and actual scores for easy comparison of regression models. The value range of R-value is [-1,1], 1 means positive correlation, -1 means negative correlation:

是所有图像真实记忆度分数的均值；v_i是图像预测记忆度分数向量，

是所有图像预测记忆度分数的均值。Among them, N is the number of images in the test set, s _i is the image real memory score vector,

is the mean of the real memory score of all images; v _i is the image prediction memory score vector,

is the mean of the predicted memory scores for all images.

In the experiment, this method is compared with the following four methods:

LR (Liner Regression): Use the linear prediction function to train the relationship between the underlying features and the memory score;

SVR (Support Vector Regression): Support Vector Regression, string together the underlying features, combined with RBF kernel function to learn nonlinear functions to predict image memory;

MRR ^[9] (Multiple Rank Regression): Use multi-order left projection vector and right projection vector to establish a regression model;

MLHR ^[10] (Multi-Level via Hierarchical Regression): Multi-Level Regression Based Multimedia Information Analysis.

Figure 3 verifies the convergence of the algorithm; Figure 4 shows the performance comparison results of this method and other methods, and it can be seen that this method is better than other methods. Contrastive methods only explore the relationship between underlying features and memory prediction. This method combines the underlying features and image attribute features to predict the image memory degree under the same framework, and obtains a relatively stable model. The experimental results verify the feasibility and superiority of this method.

references

[1] Zhang Z, Li F, Zhao M, et al. Joint low-rank and sparse principal feature coding for enhanced robust representation and visual classification [J]. IEEE Transactions on Image Processing, 2016, 25(6): 2429-2443 .

[2]Shi X,Guo Z,Lai Z,et al.A framework of joint graph embedding and sparse regression for dimensionality reduction[J].IEEE Transactions on ImageProcessing,2015,24(4):1341-1355.

[3] P.Isola, J.Xiao, A.Torralba, and A.Oliva, "What makes an imagememorable?" in Proc.Int.Conf.Comput.Vis.Pattern Recognit., 2011, pp.145–152.

[4] P.Isola, D.Parikh, A.Torralba, and A.Oliva, "Understanding the intrinsic memorability of images," in Proc.Adv.Conf.Neural Inf.Process.Syst., 2011, pp.2429–2437 .

[5]Tibshirani R.Regression shrinkage and selection via the lasso[J].Journal of the Royal Statistical Society.Series B(Methodological),1996:267-288.

[6] Hoerl A E, Kennard R W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 1970, 12(1):55-67.

[7] Purcell S, Neale B, Todd-Brown K, et al. PLINK: a tool set for whole-genome association and population-based linkage analyses. The American Journal of Human Genetics, 2007, 81(3):559-575.

[8] Q. You, H. Jin, and J. Luo, “Visual sentiment analysis by attending on local image regions,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017.

[9] Hou C, Nie F, Yi D, et al. Efficient image classification via multiplerank regression. IEEE Transactions on Image Processing, 2013, 22(1): 340-352.

[10] Sundt B.A multi-level hierarchical credibility regression model[J]. Scandinavian Actuarial Journal, 1980, 1980(1): 25-32.

[11] J. Xiao, J. Hays, K. Ehinger, A. Oliva, A. Torralba et al., “Sun database: Large-scale scene recognition from abbey to zoo,” in Proc.Int.Conf.Comput. Vis. Pattern Recognit., 2010, pp. 3485–3492.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (4)

1. A multi-view learning method combining low-rank representation and sparse regression, characterized in that the method comprises the following steps: Extract low-level features and high-level attribute features respectively for the SUN dataset with image memory score labels; The three parts of low-rank representation, combined sparse regression model and multi-view consistency loss are placed under the same framework to form a whole, and a multi-view model combining low-rank and sparse regression is constructed; The multi-vision adaptive regression algorithm is used to solve the problem of automatically predicting the memorability of images, and the relationship between image underlying features, image attribute features and image memory is obtained under the optimal parameters; Combine the low-level features and high-level attribute features of the image, use the relationship results obtained under the optimal parameters to predict the image memory of the database test set, and use the relevant evaluation criteria to verify the prediction results; The problem of automatically predicting the memorability of the image by using the multi-vision adaptive regression algorithm is: converting the equivalent problem through the slack variable Q:

s.t.X=XA+E, Q=Aw Among them, A is the mapping matrix of the low-rank representation; w is the linear dependence between the low-rank feature representation and the output memory score; E is the sparse error constraint part; α is the balance parameter between the prediction error part and the regularization part ; β is the parameter to control sparsity; λ > 0 is the balance parameter; X is the input feature; y is the memorability score vector; * is the nuclear norm representation; φ is the constraint term of graph regularization; operator; Two slack variables Y ₁ and Y ₂ are introduced to obtain the augmented Lagrangian function:

Among them, <,> represents the inner product operation of the matrix, Y ₁ and Y ₂ represent the Lagrangian operator matrix, and μ > 0 is the positive penalty parameter. The above methods are combined as:

in

Introduce variable t, define A _t , E _t , Q _t , w _t , Y _{1, t} , Y _{2, t} and μ as the result of the t-th iteration of the variables, and the result of the t+1-th iteration is as follows: The iterative result of A:

in,

Fixing w, A, Q to get the optimization result of E is as follows:

By fixing E, A, Q, the result of optimizing w is as follows:

The above problem is a ridge regression problem, and the optimal solution is

Finally, fix E, w, A, optimize Q, get:

Y ₁ and Y ₂ are updated with the following scheme: Y _1,t+1 =Y _1,t + μ _t (X-XA _t+1 -E _t+1 ) Y _2,t+1 =Y _2,t + μ _t (Q _t+1 -A _t+1 w _t+1 ) in,

is the symbol for partial derivative; The multi-view model of the joint low-rank and sparse regression is specifically:

in:

G(φ _H ,φ _l )=tr[(X _l A _l w _l ) ^T X _H A _H w _H ]

is the loss function used as the prediction error for advanced features;

is the loss function used as the prediction error of low-level features;

is the graphic regularization representation used to solve the overfitting problem; X _H is the high-level attribute feature; A _H is the mapping matrix of the low-rank representation of the high-level attribute feature; E _H is the sparse error constraint part of the high-level attribute feature; w _H is the high-level attribute feature Linear dependence between low-rank representation of attribute features and output memory score; A _l is the mapping matrix of low-rank representation of low-level features; E _l is the sparse error constraint part of low-level attribute features; X _l is low-level attribute features; w _l is the linear dependence between the low-rank representation of low-level attribute features and the output memory score. 2 . The multi-view learning method combining low-rank representation and sparse regression according to claim 1 , wherein the method further comprises: acquiring an image memorability data set. 3 . 3. A multi-view learning method combining low-rank representation and sparse regression according to claim 1, wherein the low-level features comprise: scale-invariant feature transformation features, search tree features, and directional gradient histogram features , and structural similarity features. 4 . The multi-view learning method combining low-rank representation and sparse regression according to claim 1 , wherein the high-level attribute features include: 327-dimensional scene category attribute features and 106-dimensional object attribute features. 5 .

Images