CN103729462B - A kind of pedestrian retrieval method blocked based on rarefaction representation process - Google Patents

Abstract

The invention discloses a kind of pedestrian retrieval method processing based on rarefaction representation and blocking, the present invention obtains the distance metric between inquiry pedestrian and pedestrian to be measured initially with block-based similarity coupling, then the method using rarefaction representation obtains the coverage extent between inquiry pedestrian and pedestrian to be measured, last comprehensive distance tolerance and coverage extent calculate pedestrian between similarity, improve the accuracy with a group traveling together's coupling under multi-cam.

Description

Pedestrian retrieval method based on sparse representation processing shielding

Technical Field

The invention relates to the technical field of surveillance video retrieval, in particular to a pedestrian retrieval method based on sparse representation processing occlusion.

Background

Surveillance video pedestrian retrieval is a technique to match specific pedestrian objects under multiple cameras with no overlap of illumination areas. In the actual video investigation, an investigator mainly locks, inspects and tracks a suspected target quickly according to a moving picture and a track of a same pedestrian object. The traditional video detection mode of manual browsing needs to consume a large amount of manpower and time, and the time for solving a case is easy to be delayed. The pedestrian re-identification technology is convenient for a video inspector to quickly and accurately find moving pictures and tracks of suspected targets, and has important significance for improving the case rate of a public security department and maintaining the life and property safety of people.

The existing pedestrian retrieval (also called pedestrian re-identification) method can be divided into two categories:

the first category mainly constructs robust visual features and then performs similarity measurement using standard distance functions (e.g., euclidean distance, etc.). For example, a pedestrian re-identification method based on multiple local feature matching of symmetric segmentation, firstly, a body is segmented horizontally and vertically by using color feature clues; secondly, extracting various color and texture characteristics of each region, and weighting the visual characteristics based on a horizontal central axis; finally, the characteristics are comprehensively used for representing and matching the object;

the second type has no strict requirement on feature construction, more accurate distance measurement is carried out mainly by learning a proper scale, difference vectors of samples of the same type and difference vectors of different samples are respectively expressed into different Gaussian distributions, then the distance between the samples is measured by using the ratio of probability, and finally the ratio of the Gaussian distributions is converted into a form of Mahalanobis distance, so that a proper Mahalanobis distance function is learned.

The method is to sort the to-be-detected set according to the distances of the appearance features of the pedestrian object to be inquired and all the to-be-detected pedestrian objects, and does not consider the self-shielding and other pedestrian or object shielding conditions caused by the visual angle transformation under different cameras. However, in an actual video monitoring environment, the same pedestrian often has a shielding situation under multiple cameras, so that the appearance features have significant differences, and the retrieval result is inaccurate.

Disclosure of Invention

Aiming at the defects of the existing method, the invention provides a pedestrian retrieval method based on sparse representation processing shielding, and the accuracy of matching of the same pedestrian under multiple cameras is improved.

In order to achieve the purpose, the invention adopts the following technical scheme: a pedestrian retrieval method for processing occlusion based on sparse representation is characterized by comprising the following steps:

step 1: dividing an inquiry pedestrian image P and a pedestrian image Q to be detected into m rows and n columns of small image blocks, and representing the P and the Q as a set of image blocks, namely P = { P = { P =_ij|i=1,…,m;j=1,…,n}，Q={Q_ijI =1, …, m, j =1, …, n }, wherein m is greater than or equal to 1, and n is greater than or equal to 1;

step 2: extracting the characteristics of each image block of the P and the Q, and expressing the P and the Q by the characteristics based on the image blocks;

and step 3: carrying out similarity matching based on the image blocks on the P and the Q to obtain a distance measurement result of the P and the Q based on the similarity matching of the image blocks;

and 4, step 4: calculating sparse representation of the P image blocks relative to the Q, and obtaining the shielding degree of the P relative to the Q; calculating sparse representation of the image block of Q relative to the P, and obtaining the shielding degree of Q relative to the P;

and 5: and calculating the similarity between P and Q according to the distance measurement result of matching P and Q based on the similarity of the image blocks in the step 3 and the shielding degree between P and Q in the step 4.

Preferably, the features of the image block described in step 2 are grayscale, color, and SIFT features.

Preferably, the obtaining of the result of the distance metric based on similarity matching of P and Q in step 3 includes the following steps:

step 3.1: determining a search area, determining a search area for each image block in the P image block set, and setting the P image block as P_abThen P is_abThe search area consists of a plurality of image blocks in the image block set of Q, and the image block P_abThe search area of (a) is represented as:

step 3.2: calculating P_abTo the search areaThe Euclidean distance of each image block, the block with the minimum distance is the distance P_abThe most similar block, which is denoted as:then the distance measurement result of P and Q based on similarity matching of image blocks is: $d(P,Q)=\underset{m,n}{\mathrm{\Σ}}d(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*}).$

preferably, said computing a sparse representation of said P image block relative to said Q in step 4 is implemented by: let the image block of P be P_abThen P is_abSparse representation with respect to said Q:

$\underset{{\mathrm{\β}}_{\mathrm{Pab}}}{\min }{\left|\right|V_{p_{\mathrm{ab}}}-D_{P_{\mathrm{ab}}}^Q{\mathrm{\β}}_{P_{\mathrm{ab}}}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|{\mathrm{\β}}_{P_{\mathrm{ab}}}\left|\right|}_1$

the presentation error is:whereinIs P_abIs determined by the feature vector of (a),is P_abThe sparse coefficient vector of (a) is,is P_abA dictionary of, andλ is a parameter of the sparse representation model;

the step 4 of calculating the sparse representation of the image block of Q relative to P specifically implements the following process: let the image block of Q be Q_abThen Q is_abSparse representation with respect to said P:

$\underset{{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}}{\min }{\left|\right|V_{Q_{\mathrm{ab}}^{*}}-D_{Q_{\mathrm{ab}}^{*}}^P{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}\left|\right|}_1$

the presentation error is:whereinIs composed ofIs determined by the feature vector of (a),is composed ofThe sparse coefficient vector of (a) is,is composed ofA dictionary of, andλ is a parameter of the sparse representation model.

Preferably, the similarity between P and Q is calculated by the following formula:

$\mathrm{sim}(P,Q)=\underset{m,n}{\mathrm{\Σ}}\mathrm{\ρ}(P_{\mathrm{ab}},Q)\·\mathrm{sim}(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})\·\mathrm{\ρ}(Q_{\mathrm{ab}}^{*},P)$

wherein, $\mathrm{sim}(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})=\exp \left(\frac{-d{(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})}^2}{2{\mathrm{\σ}}^2}\right),\mathrm{\ρ}(P_{\mathrm{ab}},Q)=\exp (-{\left(E_{P_{\mathrm{ab}}}^Q\right)}^2),$ sigma is the bandwidth of the gaussian function,based on image blocks P for said P and Q_abAndthe greater sim (P, Q), the more similar said P and said Q.

The invention has the following advantages and positive effects:

(1) compared with the prior art, the method introduces the idea of processing the shielding based on sparse representation, calculates the similarity between the inquired pedestrian and the pedestrian to be detected through the distance and the shielding degree based on the similarity matching of the blocks, and can obtain a more accurate pedestrian retrieval result;

(2) the method not only considers the mismatching caused by the posture change, but also considers the influence of the shielding on similarity calculation, and is more robust to the shielding caused by the visual angle change.

Drawings

FIG. 1: is a method flow diagram of an embodiment of the invention.

FIG. 2: is a technical scheme schematic diagram of the embodiment of the invention.

Detailed Description

The technical solution of the present invention is further explained by the following embodiments with reference to the accompanying drawings.

Referring to fig. 2, the embodiment provided by the present invention uses MATLAB7 as a simulation experiment platform to perform tests on a commonly used pedestrian retrieval dataset VIPeR. The VIPeR data set has 632 pedestrian image pairs under two cameras, and obvious differences of visual angles, illumination and the like exist between the cameras.

Referring to fig. 1, the technical scheme adopted by the invention is as follows: a pedestrian retrieval method based on sparse representation processing occlusion comprises the following steps:

step 1: dividing the inquiry pedestrian image P and the pedestrian image Q to be detected into m rows and n columns of small image blocks, and representing P and Q as a set of image blocks, namely P = { P = { (P)_ij|i=1,…,m;j=1,…,n}，Q={Q_ijI =1, …, m, j =1, …, n }, where m > 1, n > 1, the image size is 128 × 48 pixels, the image is partitioned by a window of 10 × 10 pixels with a step size of 4, each image is divided into 30 × 10 image blocks.

Step 2: extracting the characteristics of each image block of P and Q, wherein the characteristics of the image blocks are the gray scale, color and SIFT characteristics, and the P and Q are expressed by the characteristics based on the image blocks; the gray scale feature is 100 dimensions, the color feature is represented as 672 dimensions by using LAB and SIFT feature, and the feature of all blocks in each image is added to represent the appearance feature of a pedestrian object in one image.

And step 3: similarity matching based on the image blocks is carried out on the P and the Q, and a distance measurement result of the P and the Q based on the similarity matching of the image blocks is obtained; the specific implementation comprises the following substeps:

step 3.1: determining a search area, determining a search area for each image block in the P image block set, and setting the P image block as P_abThen P is_abThe search area of (A) is composed of a plurality of image blocks in the image block set of (Q), the image block (P)_abThe search area of (a) is represented as: $S_{P_{\mathrm{ab}}}^Q=\left\{Q_{\mathrm{ij}}\right|i=a-1,...,a...,a+1;j=1,...,n\},$ take l =1 as an example;

step 3.2: calculating P_abTo the search areaThe Euclidean distance of each image block, the block with the minimum distance is the distance P_abThe most similar block, which is denoted as:the distance metric result of P and Q matching based on similarity of image blocks is: $d(P,Q)=\underset{m,n}{\mathrm{\Σ}}d(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*}).$

and 4, step 4: calculating sparse representation of an image block of P relative to Q, and obtaining the shielding degree of P relative to Q; calculating sparse representation of the image block of Q relative to P, and obtaining the shielding degree of Q relative to P;

sparse representation of image blocks in which P is computed relative to Q, which is concreteThe current process is as follows: let P's image block be P_abThen P is_abSparse representation with respect to Q:

$\underset{{\mathrm{\β}}_{\mathrm{Pab}}}{\min }{\left|\right|V_{p_{\mathrm{ab}}}-D_{P_{\mathrm{ab}}}^Q{\mathrm{\β}}_{P_{\mathrm{ab}}}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|{\mathrm{\β}}_{P_{\mathrm{ab}}}\left|\right|}_1$

the presentation error is:whereinIs P_abIs determined by the feature vector of (a),is P_abThe sparse coefficient vector of (a) is,is P_abA dictionary of, andλ is a parameter of the sparse representation model;

the sparse representation of the image block of Q relative to P is calculated, and the specific implementation process is as follows: let the image block of Q be Q_abThen Q is_abSparse representation with respect to P:

$\underset{{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}}{\min }{\left|\right|V_{Q_{\mathrm{ab}}^{*}}-D_{Q_{\mathrm{ab}}^{*}}^P{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}\left|\right|}_2^2+\mathrm{\λ}{\left|\right|{\mathrm{\β}}_{Q_{\mathrm{ab}}^{*}}\left|\right|}_1$

the presentation error is:whereinIs composed ofIs determined by the feature vector of (a),is composed ofThe sparse coefficient vector of (a) is,is composed ofA dictionary of, andλ is a parameter of the sparse representation model.

And 5: calculating the similarity between P and Q according to the distance measurement result of the similarity matching of the image blocks in the step 3P and the step Q and the shielding degree between the step 4P and the step Q; calculating the similarity between P and Q, and obtaining by adopting the following formula:

$\mathrm{sim}(P,Q)=\underset{m,n}{\mathrm{\Σ}}\mathrm{\ρ}(P_{\mathrm{ab}},Q)\·\mathrm{sim}(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})\·\mathrm{\ρ}(Q_{\mathrm{ab}}^{*},P)$

wherein, $\mathrm{sim}(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})=\exp \left(\frac{-d{(P_{\mathrm{ab}},Q_{\mathrm{ab}}^{*})}^2}{2{\mathrm{\σ}}^2}\right),\mathrm{\ρ}(P_{\mathrm{ab}},Q)=\exp (-{\left(E_{P_{\mathrm{ab}}}^Q\right)}^2),$ sigma is the bandwidth of the gaussian function,based on image blocks P for P and Q_abAndthe greater sim (P, Q), the more similar P and Q.

The pedestrian retrieval evaluation index uses a CMC value, and the CMC value is the probability that the correct pedestrian object exists in the previous r results returned in N times of queries. When the former r results are returned, the higher the CMC value, the better the pedestrian retrieval performance is indicated. The test procedure of this example was repeated 10 times and the average CMC value was calculated. The method for identifying pedestrians based on characteristics by the ELF is compared with the method for processing the shielded result by sparse representation. The specific results are shown in Table 1, which are CMC values when the previous 1, 10, 25 and 50 results are returned on the VIPeR;

table 1: CMC values at return of the first 1, 10, 25, 50 results on VIPeR

Method of producing a composite material	Rank@1	10	25	50
					ELF	12	31	41	58
SDALF	20	39	49	66
					KISS	20	49	62	78
Sparse representation handling occlusion	31	50	65	79

From comparison of CMC values in Table 1, it can be found that the performance of the pedestrian re-identification method based on sparse representation processing occlusion provided by the invention is obviously superior to that of a comparison algorithm.

According to the method, firstly, the distance measurement between the query pedestrian and the pedestrian to be detected is obtained through block-based similarity matching, then the shielding degree between the query pedestrian and the pedestrian to be detected is obtained through a sparse representation method, and finally the similarity between pedestrian pairs is calculated through the distance measurement and the shielding degree, so that the matching accuracy of the same pedestrian under multiple cameras is improved.

The present invention is not limited to the above embodiments, and any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (4)

1. A pedestrian retrieval method for processing occlusion based on sparse representation is characterized by comprising the following steps:

step 1: dividing the inquiry pedestrian image P and the pedestrian image Q to be detected into m rows and n columns of small image blocks, and expressing the P and the Q as a set of image blocks, namely P ═ { P ═ P }_ij|i＝1,…,m；j＝1,…,n}，Q＝{Q_ij1, | i ═ …, m; j is 1, …, n, wherein m is more than or equal to 1, n is more than or equal to 1;

step 2: extracting the characteristics of each image block of the P and the Q, and expressing the P and the Q by the characteristics based on the image blocks;

and step 3: carrying out similarity matching based on the image blocks on the P and the Q to obtain a distance measurement result of the P and the Q based on the similarity matching of the image blocks; the specific implementation comprises the following substeps:

step 3.1: determining a search area, determining a search area for each image block in the P image block set, and setting the P image block as P_abThen P is_abThe search area consists of a plurality of image blocks in the image block set of Q, and the image block P_abThe search area of (a) is represented as:

step 3.2: calculating P_abTo the search areaThe Euclidean distance of each image block, the block with the minimum distance is the distance P_abThe most similar block, which is denoted as:then the distance measurement result of P and Q based on similarity matching of image blocks is:

and 4, step 4: calculating sparse representation of the P image blocks relative to the Q, and obtaining the shielding degree of the P relative to the Q; calculating sparse representation of the image block of Q relative to the P, and obtaining the shielding degree of Q relative to the P;

and 5: and calculating the similarity between P and Q according to the distance measurement result of matching P and Q based on the similarity of the image blocks in the step 3 and the shielding degree between P and Q in the step 4.

2. The pedestrian retrieval method based on sparse representation processing occlusion of claim 1, wherein: the characteristics of the image block in the step 2 are gray scale, color and SIFT characteristics.

3. The pedestrian retrieval method based on sparse representation processing occlusion of claim 1, wherein: the step 4 of calculating the sparse representation of the image block of P relative to Q specifically implements the following process: let the image block of P be P_abThen P is_abSparse representation with respect to said Q:

$\underset{{\mathrm{\β}}_{Pab}}{min}\left|\right|V_{p_{ab}}-D_{P_{ab}}^Q{\mathrm{\β}}_{P_{ab}}|{|}_2^2+\mathrm{\λ}|\left|{\mathrm{\β}}_{P_{ab}}\right|{|}_1$

the presentation error is:whereinIs P_abIs determined by the feature vector of (a),is P_abThe sparse coefficient vector of (a) is,is P_abA dictionary of, andλ is a parameter of the sparse representation model;

the step 4 of calculating the sparse representation of the image block of Q relative to P specifically implements the following process: let the image block of Q be Q_abThen Q is_abSparse representation with respect to said P:

$\underset{{\mathrm{\β}}_{Q_{ab}^{*}}}{min}\left|\right|V_{Q_{ab}^{*}}-D_{Q_{ab}^{*}}^P{\mathrm{\β}}_{Q_{ab}^{*}}|{|}_2^2+\mathrm{\λ}|\left|{\mathrm{\β}}_{Q_{ab}^{*}}\right|{|}_1$

the presentation error is:whereinIs composed ofIs determined by the feature vector of (a),is composed ofThe sparse coefficient vector of (a) is,is composed ofA dictionary of, andλ is a parameter of the sparse representation model.

4. The pedestrian retrieval method based on sparse representation processing occlusion of claim 3, wherein: the similarity between P and Q is calculated by adopting the following formula:

$sim(P,Q)=\underset{m,n}{\Σ}\mathrm{\ρ}(P_{ab},Q)\·sim(P_{ab},Q_{ab}^{*})\·\mathrm{\ρ}(Q_{ab}^{*},P)$

wherein,sigma is the bandwidth of the gaussian function,based on image blocks P for said P and Q_abAnd Q^* _abThe greater sim (P, Q), the more similar said P and said Q.