CN101533512A - Method for automatically extracting interesting image regions based on human visual attention system - Google Patents

Abstract

The invention discloses a method for automatically extracting interesting image regions based on a human visual attention system, which mainly solves the problem that the prior method for extracting interesting regions cannot extract a plurality of interesting regions and edge information. The method comprises the following steps: calculating the local brightness contrast, global brightness contrast and edge of an input image; fusing characteristic graphs corresponding to the three characteristics through global nonlinear normalization merging algorithm so as to generate a contrast graph; calculating the position characteristics of the input image so as to establish a weighting graph; establishing a saliency graph corresponding to the input image through the contrast graph and the weighting graph; and segmenting out the interesting regions of the input image according to the saliency graph. The method can effectively extract a plurality of interesting regions in the input image, and is used in the technical field of image analysis and image compression.

Description

Method for automatically extracting interesting image regions based on human visual attention system

Technical field

The present invention relates to extract the method for interesting image regions, particularly extract the method for interesting image regions, be used for graphical analysis and Image Compression field by simulating human vision attention system.

Technical background

Along with the fast development of computer network communication technology, the internet provides information service, and especially image information develops rapidly.

The data of image information amount is huge, and how effectively processing image information becomes the research focus of image information service.For the human visual system, the information that image provided not is of equal importance everywhere.Some zone of image provides people to understand the required main contents of image, is referred to as area-of-interest; And the other zone only provides some less important background content.Searching is published picture as area-of-interest, and graphical analysis, compression of images etc. is all had important meaning.

W.Osberger and A.J.Maeder are at article " Automatic identification ofperceptually important regions in the image ", in Proc.Int ' l Conf.Pattern Recognition, 1998, among the pp.17-20, a kind of method of extracting area-of-interest based on human visual attention system is proposed.This method is come the analysis image characteristic factor with the basis that is divided into of image, determines area-of-interest then.Therefore partitioning algorithm is depended in the success or failure of this method.People such as Itti Laurent are at article " A model of saliency-based visual attention for rapid scene analysis ", IEEE Trans.Pattern Analysis and Machine Intelligence, vol.20, no.11, pp1254-1259, in 1998, a kind of method of extracting area-of-interest by these three characteristic factors of color, direction and brightness of analyzing each pixel is proposed.This method is easy to analyze and realize, and can accurately orient the approximate location of area-of-interest.Because the feature that adopts the gaussian pyramid model to come analysis image in this method, this is a kind of algorithm of down-sampling, and therefore unavoidable meeting loses some specifying informations of image in the operation, as marginal information etc.C.M.Priviter and L.W.Stark are at article " Algorithms for definingvisual region-of-interesting:Comparison with eye fixations ", IEEETrans.Pattern Anal.Machine Intell.vol.22, no.9, pp.970-982, in 2000, proposed based on the single feature of image, the segmentation algorithm directly extracts area-of-interest from image.This method is simple to operate, is easy to realize.For simple image, can effectively extract its area-of-interest; But for the image with complex background, this method effect is poor.SooYeong Kwak etc. are at article " Automatic salient-object extraction using the contrast map andsalient point ", in Advances in Multimedia Information Processing-PCM2004.vol.3332 of LNCS, pp.138-145, Springer Berlin. and K.B.Chul etc. are at article " Automatic object-of-interest segmentation fromnature images ", in PROC.Int ' l Conf Pattern Recognition, 2006, among the pp.45-48, the application focus window of proposition detects the method for well-marked target.These methods therefore also can lose detailed information such as edge in operating process, and when a plurality of target occurring simultaneously in the image, application focus window method detect a plurality of well-marked targets and can lose efficacy owing to all be to adopt the model of Itti to set up remarkable figure.

Summary of the invention

The objective of the invention is to overcome the defective and the deficiency that exist in the above-mentioned prior art, a kind of method for automatically extracting interesting image regions based on human visual attention system simple to operate is provided, have the area-of-interest of the image of complex background with effective extraction, and a plurality of target objects that occur in the image.

For achieving the above object, simulating human vision attention of the present invention system, influence human visual system's rudimentary factor and senior factor in the analysis image, the contrast figure that these three the rudimentary factors in the local luminance contrast of the main analyzing influence vision system of the present invention, overall luminance contrast and edge are set up, and the weight map of this senior factor foundation of position, set up significantly figure in conjunction with contrast figure and weight map, performing step is as follows:

(1) these three low-level features factors of local luminance contrast, overall luminance contrast and edge of each pixel of difference calculating input image;

(2) use these three low-level features factors of local luminance contrast, overall luminance contrast and edge that overall non-linear normalizing merge algorithm merges input picture, generate contrast figure;

(3), obtain the weight map of input picture correspondence according to the weight of each pixel of input picture this pixel of the position calculation in image in image;

(4), generate the remarkable figure of input picture by described contrast figure and weight map;

(5) area-of-interest of input picture is determined in the indication of the remarkable figure of basis.

Described calculating local luminance contrast is the luminance difference that calculates in each pixel subrange peripheral with it, and calculation procedure is as follows:

(2a) adopt Gaussian function that input picture is done smoothing processing:

$I\left({\mathrm{\σ}}_i\right)=I\⊗G\left({\mathrm{\σ}}_i\right)$

Wherein, I is the former figure of input picture, and G (σ) is a Gaussian function, I (σ _j) be the image after the smoothing processing;

(2b) with Gaussian difference function calculation pixel with it absolute brightness difference in peripheral little field:

DoG(x，y，σ ₁，σ ₂)＝|I(x，y，σ ₁)-I(x，y，σ ₂)|

Wherein: σ _iValue decision central pixel point and the degree of association of peripheral little field interior pixel point, establish σ here ₁=1, I (x, y, σ ₁) be through first variance σ ₁The pixel of image after the Gaussian function smoothing processing, I (x, y, σ ₂) be through second variance σ ₂The pixel of image after the Gaussian function smoothing processing, DoG (x, y, σ ₁, σ ₂) be the local luminance contrast value;

(2c) get two different σ ₂Value obtains two width of cloth luminance difference figure DoG1 and DoG2 respectively, by overall non-linear normalizing fusion method this two width of cloth image is merged, and obtains local luminance contrast figure.

Described according to the weight of each pixel of input picture this pixel of the position calculation in image in image, utilize following formula to calculate:

$W_{\mathrm{center}}(x,y)=\left\{\begin{array}{cc}1& \mathrm{if}(x,y)\∈\mathrm{center}\\ \frac{1+{\cos }^2\frac{\mathrm{\πr}}{2R}}{2}& \mathrm{else}\end{array}\right.$

Wherein ' center ' is picture centre

The zone, r is that (x, y) to the distance of input picture central area, R is the distance of input picture edge to its center to point.

The present invention has following advantage:

1) the present invention according to influencing a plurality of factor analysis input pictures that the human vision system is paid close attention in the image, extracts the human eye area-of-interest, so can accurately extract the input picture area-of-interest owing to adopt the method for simulating human vision system.

2) each pixel that the present invention is based on input picture is done computing, so design process is simple, is easy to realize.

3) the present invention is owing to these three features of local luminance contrast, overall luminance contrast and edge of analyzing each pixel of input picture, so can extract the marginal information of object.

4) the present invention is owing to adopt according to the remarkable indication of figure and seek area-of-interest, thereby can extract a plurality of area-of-interests simultaneously.

Embodiment

With reference to Fig. 1, specific implementation step of the present invention is as follows:

Step 1, these three low-level features factors of local luminance contrast, overall luminance contrast and edge of each pixel of calculating input image respectively.

(1a) calculate the local luminance contrast

Gaussian difference (Differential of Gaussian) function can be represented central pixel point and its difference between the pixel of field on every side effectively, so adopt the Gaussian difference function to calculate the interior luminance contrast of regional area.

With reference to Fig. 2, being calculated as follows of local luminance contrast among the present invention:

At first, adopt Gaussian function that input picture is done smoothing processing:

$I\left({\mathrm{\σ}}_i\right)=I\⊗G\left({\mathrm{\σ}}_i\right)$

Wherein, I is the former figure of input picture, and G (σ) is a Gaussian function, I (σ _i) be the image after the smoothing processing, Be the convolution symbol;

Secondly, employing Gaussian difference function calculation pixel is absolute brightness difference in peripheral little field with it, works as σ _iValue is from little change when big, the smoothed processing of image fuzzy more, and each central pixel point has also comprised the information of more surrounding pixel point simultaneously, when getting different σ _iDuring value, the Gaussian difference function is:

DoG(x，y；σ ₁，σ ₂)＝|I(x，y；σ ₁)-I(x，y；σ ₂)| (A)

I (x, y, σ wherein ₁) be through first variance σ ₁The pixel of image after the pairing Gaussian convolution nuclear G1 smoothing processing, I (x, y, σ ₂) be through second variance σ ₂The pixel of image after the Gaussian function smoothing processing, we establish σ here ₁=1, and σ ₁≠ σ ₂, DoG (x, y, σ ₁, σ ₂) be that the Gaussian difference function is in (x, the value of y) locating;

Then, when utilizing in the Gaussian difference function detected image target object, its effect and σ ₂The value and the size of target object relevant, for the little target object of size, get its corresponding little σ ₂Value, it is better that local contrast detects effect; And, get its big σ for the big target object of size ₂Value, it is better that local contrast detects effect; Under the situation, get its minimum without any target object priori in about image

Value and maximum thereof

Value obtains two Gaussian convolution nuclear G21 and G22, obtains two width of cloth luminance difference figure DoG1 and DoG2 according to formula (A),

At last, merge this two width of cloth luminance difference figure DoG1 and DoG2, obtain local contrast with the non-linear normalizing act of union.

(1b) calculate overall luminance contrast

When certain regional brightness is protruded in view picture figure among the width of cloth figure, certainly will attract observer's notice, overall luminance contrast is represented the contrast in brightness of each pixel and full figure, computing formula is as follows:

$G_{\mathrm{contrast}}(x,y)=\frac{|L_m(x,y)-L_M|}{L_m(x,y)+L_M}$

L wherein _m(x is that (x y) is the average gray value in 7 * 7 fields at center, L with some y) _MIt is the average gray value of full figure.

(1c) edge calculation zone

Fringe region is the highstrung zone of human visual system in the image, adopts canny operator extraction edge of image feature usually, and threshold value is made as 0.5.

Step 2 is set up the contrast figure of input picture.

The contrast figure that sets up input picture is undertaken by overall non-linear normalizing merge algorithm, this algorithm can overallly promote those only the characteristic pattern of minority peak value, overall situation compacting those have a lot of places all to have the characteristic pattern of similar big small leak in entire image, and its concrete steps are as follows:

(2a) will normalize to same dynamic range (0-1) by step (1a), (1b) and the local luminance contrast that (1c) calculates figure, overall luminance contrast figure and outline map;

(2b) seek the global maximum M of this three width of cloth figure respectively, and calculate all local peaked mean value M;

(2c) respectively to the corresponding amplification (M-M) of this three width of cloth figure overall situation ²Doubly, obtain figure after three width of cloth normalization;

(2d) three width of cloth figure additions after the normalization are obtained luminance contrast figure.

Step 3 is set up the weight map of input picture.

The human visual system pays close attention to the central area of input picture more, so the central area pixel weights of input picture are big, and the pixel weights of its fringe region are little, and concrete computing formula is as follows:

$W_{\mathrm{center}}(x,y)=\left\{\begin{array}{cc}1& \mathrm{if}(x,y)\∈\mathrm{center}\\ \frac{1+{\cos }^2\frac{\mathrm{\πr}}{2R}}{2}& \mathrm{else}\end{array}\right.$

Wherein ' center ' is picture centre

The zone, r is that (x, y) to the distance of input picture central area, R is the distance of edge to the input picture center to point.

Step 4 is set up the remarkable figure of input picture.

According to the contrast figure and the weight map of above-mentioned steps 2 and step 3 gained input picture, the remarkable figure that sets up the image correspondence is:

SM(x，y)＝CM(x，y)×W _center(x，y)

Wherein, CM is the contrast figure of input picture, W _CenterWeight map for input picture.

The brightness value of each point is represented the susceptibility of its correspondence image pixel among this remarkable figure, and significantly the brightness value of certain point is big more among the figure, represents that this conspicuousness in former figure is high more, attracts the vision system more attention.

Step 5, the area-of-interest of extraction input picture.

(5a) according to the characteristics of the high place attraction vision system more concern of remarkable figure brightness, setting threshold extracts the high zone of brightness value among the remarkable figure:

$\mathrm{ROI}(x,y)=\left\{\begin{array}{cc}1& \mathrm{SM}(x,y)\≥T\\ 0& \mathrm{else}\end{array}\right.---\left(B\right)$

Wherein, ROI is the figure after the input picture binaryzation, and 1 represents the effective coverage, and 0 represents inactive area; T is a segmentation threshold, here we establish the T value among the SM figure peaked half.

(5b) obtain a width of cloth binary map according to formula (B), this binary map is made morphology handle, removing the interfered cell territory that some cause because of noise, its value is that 1 zone is area-of-interest in the binary map after treatment.

Claims (3)

1. the method for automatically extracting interesting image regions based on human visual attention system comprises the steps:

(1) these three low-level features factors of local luminance contrast, overall luminance contrast and edge of each pixel of difference calculating input image;

(2) use these three low-level features factors of local luminance contrast, overall luminance contrast and edge that overall non-linear normalizing merge algorithm merges input picture, generate contrast figure;

(3), obtain the weight map of input picture correspondence according to the weight of each pixel of input picture this pixel of the position calculation in image in image;

(4), generate the remarkable figure of input picture by described contrast figure and weight map;

(5) area-of-interest of input picture is determined in the indication of the remarkable figure of basis.

2. the method for claim 1, the described calculating local luminance of its step (1) contrast is the luminance difference that calculates in each pixel subrange peripheral with it, calculation procedure is as follows:

(2a) adopt Gaussian function that input picture is done smoothing processing:

$I\left({\mathrm{\σ}}_i\right)=I\⊗G\left({\mathrm{\σ}}_i\right)$

Wherein, I is the former figure of input picture, G (σ _i) be Gaussian function, I (σ _i) be the image after the smoothing processing;

(2b) with Gaussian difference function calculation pixel with it absolute brightness difference in peripheral little field:

DoG(x，y，σ ₁，σ ₂)＝|I(x，y，σ ₁)-I(x，y，σ ₂)|

Wherein: σ _iValue decision central pixel point and the degree of association of peripheral little field interior pixel point, establish σ here ₁=1,

I (x, y, σ ₁) be through first variance σ ₁The pixel of image after the Gaussian function smoothing processing,

I (x, y, σ ₂) be through second variance σ ₂The pixel of image after the Gaussian function smoothing processing, DoG (x, y, σ ₁, σ ₂) be the local luminance contrast value;

(2c) get two different σ ₂Value obtains two width of cloth luminance difference figure DoG1 and DoG2 respectively, by overall non-linear normalizing fusion method this two width of cloth image is merged, and obtains local luminance contrast figure.

3. method according to claim 1, wherein step (3) is described according to the weight of each pixel of input picture this pixel of the position calculation in image in image, utilizes following formula to calculate:

$W_{\mathrm{center}}(x,y)=\left\{\begin{array}{cc}1& \mathrm{if}(x,y)\∈\mathrm{center}\\ \frac{1+{\cos }^2\frac{\mathrm{\πr}}{2R}}{2}& \mathrm{else}\end{array}\right.$

Wherein ' center ' is picture centre

The zone, r is that (x, y) to the distance of input picture central area, R is the distance of input picture edge to its center to point.

Images