CN101388115A - Depth image autoegistration method combined with texture information - Google Patents

Abstract

A depth image automatic registration process combining with texture information is mainly used to reconstruct the three-dimensional model of various actual objects, which comprises the steps: firstly extracting a texture image from scanning data or generating a texture image according a depth image, secondly extracting interest pixels in the texture image based on the characteristics of SIFT, and finding out candidate sets therein matched with the interest pixels through the crosscheck method, thirdly finding out correct matched pixel pairs in the candidate sets according to geometry information constraint, fourthly finding out matched apex pairs corresponding to the matched pixel pairs in a three-dimensional space, and calculating the rigid replacement matrix between two depth images, fifthly optimizing the result through using the improved ICP algorism, sixthly dividing the input sequences of several depth images into a plurality of bar type subsequences based on the registrations of two depth images, finally combining the subsequences through the strategy of forward searching, and constructing an integral three-dimensional model. The invention can fast register large-scale three-dimensional scanning data to generate a three-dimensional model, and has strong anti-noise capability and excellent universality.

Description

A kind of depth image autoegistration method of combined with texture information

Technical field

The present invention relates to a kind of method of rebuilding the object dimensional model according to 3 d scan data automatically, a kind of depth image autoegistration method of combined with texture information particularly, belong to computer virtual reality and computer graphics techniques field, be mainly used in the reconstruction of various real-world object three dimensional lifelike models, particularly be applied to the reconstructing three-dimensional model of historical relic in the digital museum.

Background technology

Rebuild the three-dimensional model of object in the real world, by more and more widely be applied in aspects such as virtual emulation, computer animation, reverse engineering, computer-aided design (CAD) and digital museum.Along with the continuous development of 3-D scanning equipment, the data that collect based on 3-D scanning equipment are rebuild the model of real-world object, become a kind of popular three-dimensional modeling method gradually.The spatial digitizer of main flow not only can obtain the depth information on model surface summit, and can also obtain their colouring information, the depth image (range image) and the texture image (texture image) of resolution such as promptly can collect simultaneously, depth image be also referred to as cloud data (point cloud data, PCD).The geometric configuration of real-world object is often complicated, and the visual angle of spatial digitizer is limited, therefore, in order to obtain the whole geological information of three-dimensional object surface, need be in a plurality of different viewpoints scannings, the scan-data that will collect each time is stitched together again, reverts to a complete three-dimensional model, and this process is exactly the registration of scan-data.

At present, can be divided into two processes for the scan-data registration: two width of cloth view registrations (pair-wise registration) and several view registrations (multi-view registration).For two amplitude deepness images in the zone that overlaps, the purpose of two width of cloth view registrations is optimal motion matrixes of finding out between this two amplitude deepness image, under the situation of error minimum, wherein an amplitude deepness image is spliced on another width of cloth.The registration of two amplitude deepness images generally was divided into for two steps: rough registration (coarse registration) and meticulous registration (accurate registration).At first, calculate the estimated value of rigid body permutation matrix between two amplitude deepness images, then, optimize the result of back by minimizing a predefined error function in the meticulous registration stage in the rough registration stage.Similarly, the purpose of several data registrations is the given multi-amplitude deepness images of splicing, and it mainly comprises two aspects: find out the syntople between these depth images, the overlapping region is promptly arranged between which depth image and minimize global error.

In two kinds of thick registrations commonly used, the method of the rough registration of one class is a kind of local feature description of definition symbol, calculate the eigenwert of all points then, find out three pairs or above character pair point between two amplitude deepness images by comparing and mating these eigenwerts again, use Horn (referring to Horn at last, B.Closed-form solution of absolute orientation using unit quaternions.Journal of the Optical Society of America 4 (1987), 629-642.) method, just can calculate the rigid body permutation matrix between two width of cloth data.Common local feature description's symbol has: image rotating (spin images) is (referring to Huber, D.F.Automaticthree-dimensional modeling from reality.PhD thesis, Carnegie Mellon University, 2002.Chair-Martial Hebert.), based on the descriptor of body surface derivative characteristic (referring to Gal R, Cohen-Or D.Salientgeometric features for partial shape matching and similarity.ACM Trans.Graph.2006,25,130-150.), based on the descriptor of integration body (referring to Gelfand, N., Mitra, N.J., Guibas, L.J., andPottmann, H.Robust global registration.In Symposium on Geometry Processing (2005), pp.197-206.) with based on the descriptor of metric space (referring to Li, X., and Guskov, I.Multi-scale features forapproximate alignment of point-based surfaces.In SGP ' 05:Proceedings of the third Eurographicssymposium on Geometry processing (Aire-la-Ville, Switzerland, Switzerland, 2005), EurographicsAssociation, p.217.); The method of another kind of rough registration is based on as random sampling consistance (RANdom SAmpleConsensus, RANSAC) random algorithm of algorithm.For example Chen is (referring to Chen, C.-S., Hung, Y.-P., and Cheng, J.-B.Ransac-based darces:A new approach to fast auto-matic registration of partially overlappingrange images.IEEE Trans.Pattern Anal.Mach.Intell.21,11 (1999), 1229-1234.) method that proposes, by on two amplitude deepness images, choosing some reference point in advance and introducing space constraints and improved the RANSAC algorithm, between registration speed and result's robustness, find an equilibrium point.

Although the character representation of existing local geometric features remains unchanged, even under the certain noise condition, also has robustness under rigid body translation.But, since low dimensional feature and high dimensional feature registration speed and stable aspect intrinsic contradictions and depth image in have noise and a large amount of geometrical defects, for the registration of large-scale data, use local geometric features to be difficult to obtain better effects merely.In addition, for any local feature description symbol, the eigenwert on each summit is all calculated according to its neighborhood inner vertex, therefore, for the different summit of body surface, if the geometric properties of their neighborhoods is similar, will obtain close eigenwert at these places, summit so, this makes the unique point of seeking coupling between two amplitude deepness images become the thing of a difficulty.The constraint condition that adds other is carried out beta pruning or feature is done polymerization all is the effective ways that address this problem, but they have increased the time complexity of Feature Points Matching algorithm greatly.

Consider that the research work at the two dimensional image registration has obtained achievement preferably, researchers begin to attempt the problem that the combined with texture image solves two amplitude deepness image registrations.Wherein Roth is (referring to Roth G.Registering two overlapping rangeimages.In:3-D Digital Imaging and Modeling, Proceedings.Second Intemational Conference on, 1999,191-200.) by the correct corresponding relation of triangle searching that the match interest summit is formed, the efficient of this method and accuracy are not high.Bendels is (referring to Bendels GH, Degener P, Wahl R, Kortgen M, Klein R.Image-basedregistration of 3d-range data using feature surface elements.In VAST, 2004,115-124.) behind the corresponding vertex of finding out the interest pixel, earlier feature extraction is done on these summits and neighborhood thereof, utilize the constraint of geological information to find out matching relationship correct between them again, the method of finally finishing registration .Bendels the scan-data quality better and between data angle change hour, can obtain registration results preferably.Seo is (referring to Seo JK, Sharp GC, Lee SW.Range data registrationusing photometric features.In:Computer Vision and Pattern Recognition, CVPR 2005,1140-145.) utilizing optical correction techniques to handle three-dimensional rotation and the projection distortion problem that exists between texture image on the Bendels method basis, but can only correcting plane or the texture in the zone of almost plane, so be difficult to surface geometry information complex objects.Liu Xiaoli (referring to Liu Xiaoli, Peng Xiang, Li Ameng, Gao Pengdong. the degree of depth picture coupling of combined with texture information..Computer-aided design (CAD) and graphics journal, 2007,19 (3), 340-345.) use the character pixel on the Sobel operator extraction texture image, find out the characteristic of correspondence pixel by normalized related coefficient then, at last by Hausdorff apart from check these character pixels right validity. this method relies on the information of texture image merely and finds out matched pixel, and needs man-machine interaction to select the overlapping region, only is applicable to that simple shooting angle changes the registration of little texture image.

In the meticulous registration of two amplitude deepness images, researchers have proposed many accurate method for registering, wherein that tool milestone significance is iterative closest point (Iterative Closest Point, ICP) algorithm is (referring to Besl, P.J., and McKay, N.D.Amethod for registration of 3-d shapes.IEEE Trans.Pattern Anal.Mach.Intell.14,2 (1992), 239-256. and Chen, Y., and Medioni, G.Object modelling by registration of multiple range images.ImageVision Comput.10,3 (1992), 145-155.).The ICP algorithm is an original state with the initial position of two amplitude deepness images, then by an iterative process, constantly reduce all corresponding point between two amplitude deepness images distance and, up to reaching certain threshold value, thereby calculate the kinematic matrix between two amplitude deepness images.ICP algorithm and its mutation generally need an original state preferably, therefore need the process of a rough registration to calculate the original state of the estimated value of rigid motion between two amplitude deepness images as accurate registration.

When the registration multi-amplitude deepness image, (V E), is called illustraton of model (model graph) can to set up between each amplitude deepness image of a whole model of expression the figure G of topological relation usually.Wherein, each vertex v of figure _iRepresent an amplitude deepness image, each bar limit e _{I, j}The expression vertex v _iAnd v _jBetween two amplitude deepness images of representative the overlapping region is arranged, and the kinematic matrix between them also is assigned to this edge.When the set of the limit of structural map G, in the worst case, need judge whether they are overlapping to any two amplitude deepness images, algorithm time complexity at this moment is O (n ²).Pingi (Pingi P, Fasano A, Cignoni P, Montani C, Scopigno R.Exploiting the scanning sequence for automatic registration of large sets of range maps.ComputerGraphics Forum 2005,24 (3), 517-526.) simplify this exhaustive algorithm by pre-defined a kind of scanning strategy.This about the map generalization tree representation of whole model make up the least member collection of complete models by these depth images, in case therefore G is fabricated out, just can generate the complete model of tree reconstruction by seeking of G.

But scanning and registration strategies towards band that Pingi proposes are too idealized, are difficult in the reality guarantee all to have the overlapping region between data adjacent arbitrarily in the range image sequence.

Summary of the invention

Technology of the present invention is dealt with problems: overcome the deficiencies in the prior art, a kind of automatic deepness image registration method of combined with texture information is provided, scan-data that can the fast processing big data quantity generates three-dimensional model, and noise resisting ability is strong simultaneously.

Technical solution of the present invention: a kind of automatic deepness image registration method of combined with texture information, be divided into the registration of two amplitude deepness images and the registration of multi-amplitude deepness image, wherein:

The step of registration of two amplitude deepness images is as follows:

If P and Q are scan-datas subject to registration, the texture image of P and depth image are respectively I _pAnd R _p, the texture image of Q and depth image are respectively I _qAnd R _q,

The first step is extracted texture image I from P and Q _pAnd I _qOr according to depth image R _pAnd R _qGenerate texture image I _pAnd I _q

Second step is based on SIFT feature extraction texture image I _pAnd I _qIn the interest pixel, and therefrom find out the right Candidate Set C of matched pixel by the method for crosscheck;

The 3rd step, use adaptive RANSAC algorithm, it is right to find out matched pixel correct among the Candidate Set C according to geological information constraint;

In the 4th step, it is right to the coupling summit of correspondence to find out in three dimensions with matched pixel, and according to these summit corresponding relations, calculates the rigid body permutation matrix between two amplitude deepness images;

The 5th step, use the result of improved ICP algorithm optimization in the 4th step, finish the accurate registration of two amplitude deepness images;

The step of registration of several scan-datas is as follows:

If between the expression depth image illustraton of model of topological relation be G (V, E), S=s ₁, s ₂... s _n) be the list entries of n amplitude deepness image subject to registration;

In the 6th step,, the list entries S of multi-amplitude deepness image is divided into the banded subsequence S=of k (W based on two amplitude deepness image registrations ₁, W ₂..., W _k), W wherein _i∈ S (i=1...k) is the subsequence of some amplitude deepness images, W _iAny two adjacent depth images among the ∈ S (i=1...k) all have the overlapping region;

The 7th step, adopt the strategy of search forward, merge subsequence, according to the complete three-dimensional model of subsequence structure that has merged.

In the described first step, according to depth image R _pAnd R _qGenerate texture image I _pAnd I _qMethod be:

It is level and smooth that depth image is Laplacian, obtains its basic mode type, calculates the distance of each summit corresponding vertex on the basic mode type in the depth image again, and with the brightness value of each distance value as each summit respective pixel on texture image.

In described second step, based on SIFT feature extraction texture image I _pAnd I _qIn the interest pixel, and be by the method that the method for crosscheck is therefrom found out the right Candidate Set C of matched pixel:

(1) use the SIFT algorithm to find out I respectively _pAnd I _qIn the interest set of pixels;

(2) according to the Euclidean distance between the SIFT proper vector of two pixels, at I _pAnd I _qThe interest set of pixels between seek corresponding pixel, its method is:

(2.1) deletion does not have the interest pixel of corresponding vertex in depth image;

(2.2) for the interest pixel i in the texture image, if i is then deleted in the summit of i correspondence in depth image on the border of depth image grid;

(2.3) for I _pIn each crucial pixel i ^p, at I _qIn find out and make

Two minimum crucial pixels

With

, establish $d(i^p,i_1^q)\≤d(i^p,i_2^q)$ , definition

With Between diversity factor be ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)=\frac{d(i^p,i_2^q)-d(i^p,i_1^q)}{d(i^p,i_1^q)},$ And pre-defined diversity factor threshold value δ _DiffIf, ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}},$ Then With

Between notable difference is arranged,

Can be used as i ^pRespective pixel, otherwise i ^pAt I _qIn multiple correspondence is arranged, should give up i ^p, when ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}}$ The time, again with same method at I _pIn find out pixel

Respective pixel

If

And i ^pBe same pixel, then i ^pWith

Be I _pAnd I _qBetween a pair of candidate's respective pixel, will

Add Candidate Set C, otherwise still think i ^pAt I _qIn do not have respective pixel.

In described the 3rd step, use adaptive RANSAC algorithm, right method is to find out matched pixel correct among the Candidate Set C according to geological information constraint:

(1) to all elements (i among the Candidate Set C ^p, i ^q) according to d (i ^p, i ^q) sort with ascending order, and the result is put into a formation L;

(2) whether l element is correct coupling before among the check L, and the method for inspection is:

(2.1) from preceding l the element of L 3 elements of picked at random as a sample;

(2.2) calculate the rigid body permutation matrix according to 3 pairs of matched pixel geological information of corresponding vertex in depth image;

(2.3), judge whether all the other L-3 element is correct coupling under this rigid body displacement according to the geological information of corresponding vertex; The element of correct coupling is interior point, and remaining is an exterior point, and point is formed the unanimity collection of this sampling in all, and the number according to interior point upgrades the sampling number upper limit then;

(2.4) repeat above step (2.1)-(2.3), reach the sampling number upper limit, find out consistent sampling of concentrating element (to count out promptly at most) at most then, and the unanimity collection that will sample specifically is as the element set of correct coupling up to cycle index.

In described the 6th step, based on two amplitude deepness image registrations, the step that the list entries of multi-amplitude deepness image is divided into the subsequence of several strips shape is:

(1) with S _iAdd first subsequence W ₁In;

(2) all adjacent depth image s among the registration sequence S successively _i, s _I+1(if i=1..n-1 is s for ∈ S _iWith s _I+1The overlapping region is arranged, then with s _I+1Add s _iThe subsequence at place, otherwise with s _I+1Add a new subsequence, and in figure G, add relevant limit.

In described the 7th step, adopt the strategy of search forward, merge these subsequences, and be according to the step of the complete three-dimensional model of the subsequence structure that has merged:

(1) to subsequence W _iEvery amplitude deepness image among the ∈ S (i=2...k) is retrieved subsequence W successively _jWhether there are depth image and W among the ∈ S (j=i-1...1) _iIn depth image the overlapping region is arranged, if there is such depth image, so just can be with W _iAnd W _jMerge, be about to W _jIn each element be inserted into W _iHeader element before, simultaneously in figure G, add relevant limit;

(2) continuous circulation step (1) is until only remaining a subsequence, perhaps at residue subsequence W _iIn do not have the overlapping region between any two;

(3) rebuild whole model according to figure G.

The present invention's beneficial effect compared with prior art is:

(1) the present invention extracts texture image I from P and Q _pAnd I _qOr according to depth image R _pAnd R _qGenerate texture image I _pAnd I _q, compare with traditional method based on the local geometric features coupling, when texture image was known, the inventive method was not calculated the geometrical characteristic on each summit, had greatly improved speed, and there are the data of how much noises in the energy good treatment.

(2) from P and Q, extract texture image I _pAnd I _qOr according to depth image R _pAnd R _qGenerate texture image I _pAnd I _q, compare with existing method for registering based on texture information, when texture image was unknown, the inventive method can generate texture image according to the low-dimensional geometric properties on summit in the three dimensions, obviously has more versatility.

(3) compare with traditional method, the present invention is based on SIFT feature extraction texture image I based on the local geometric features coupling _pAnd I _qIn the interest pixel, and therefrom find out the right Candidate Set C of matched pixel by the method for crosscheck, can utilize method for registering images to find out corresponding relation between the summit, when handling the scan-data of complex model, have the obvious speed advantage.

(3) compare with the method for registering of existing combined with texture image, the inventive method adopts based on SIFT feature extraction texture image I _pAnd I _qIn the interest pixel, and therefrom find out the right Candidate Set C of matched pixel by the method for crosscheck, promptly introduced process, and selected correct matched pixel in conjunction with depth image to the crosscheck of interest pixel, improved the accuracy of registration results.

(4) the present invention is divided into the list entries S of multi-amplitude deepness image the subsequence of several strips shape, adopt the strategy of search forward, merge subsequence, according to the complete three-dimensional model of subsequence structure that has merged, multi-amplitude deepness image for input, this method based on divide-and-conquer strategy is generation model figure fast, avoided all will doing any two amplitude deepness images in the classic method problem of registration, improve the speed of several view registrations greatly, therefore greatly reduced the time of whole several registration process.

Description of drawings

Fig. 1 is two amplitude deepness image registration process process flow diagrams of the present invention;

Fig. 2 is the autoregistration process of two amplitude deepness images of the embodiment of the invention, wherein: (a) be two depth images subject to registration, (b) be two width of cloth texture images that from scan-data, extract, wherein white point is represented the respective pixel between two width of cloth images, (c) stain in is the summit of the respective pixel correspondence in three dimensions between texture image, (d) is final registration results;

Fig. 3 is the registration process of the multi-amplitude deepness image of the embodiment of the invention, 13 amplitude deepness images for input, earlier all two adjacent width of cloth data are done registration, whole scanning sequence can be divided into 5 ribbon subsequences like this, merge this 5 subsequences then, finally generate a complete model.

Fig. 4,5,6,7 is respectively the registration results of the different model scan-datas that adopt the inventive method, wherein Fig. 4 is the registration results of a model of big foot, Fig. 5 is the registration results of No. two models of big foot, and Fig. 6 is the registration results of Buddha model, and Fig. 7 is the registration results of terra cotta warriors and horses model.

Embodiment

As shown in Figure 1, the present invention includes the registration process of registration He several scan-datas of two width of cloth scan-datas, specific as follows:

1. from scan-data, extract or generate texture image.

If P and Q are scan-datas subject to registration, the texture image of P and depth image are respectively I _pAnd R _p, the texture image of Q and depth image are respectively I _qAnd R _qIf comprised texture image among P and the Q, then directly extracted them and convert gray level image to as I _pAnd I _q,, then at first generate texture image according to depth image if do not comprise texture image among P and the Q or the texture image that comprises can't registration the time.

A given amplitude deepness image R, then

Make that Γ is a kind of geometric properties descriptor, $\&ForAll;v\&Element;R^3,$ (eigenwert of v) representing the v place can be done linear transformation to the eigenwert on all summits to Γ, makes their codomain be [0,1], defines the texture image that is generated by Γ so $I_{\mathrm{\&Gamma;}}(x,y)=\left\{\begin{array}{c}\mathrm{\&Gamma;}\left(R(x,y)\right),R(x,y)\&NotEqual;\mathrm{\&phi;}\\ \mathrm{\&phi;},R(x,y)=\mathrm{\&phi;}\end{array}\right.,$ Can generate a width of cloth 2 d texture image according to three-dimensional geometric properties Γ like this.Any in theory geometric properties descriptor all can be used to generate texture image, but considers the efficient of algorithm and result's accuracy, wishes that then Γ has good discrimination, and is easy to calculate.

The basic thought that the present invention generates texture image is: establishing the depth image that comprises in the scan-data is $R^0=\left\{v_i^0\right|v_i^0\&Element;R^3,i=1...n\},$ And $R^k=\left\{v_i^k\right|v_i^k\&Element;R^3,i=1...n\}$ Be to R ⁰Make the basic mode type that obtains after level and smooth for k time, wherein $v_i^k\&Element;R^k$ Be $v_i^0\&Element;R^0$ Corresponding point after level and smooth, definition so ${\mathrm{\&Gamma;}}_{\mathrm{\&delta;}}\left(v_i^0\right)=\mathrm{sign}\left(v_i^0\right)\&CenterDot;\left|\right|v_i^0-v_i^k\left|\right|,$ I=1...n, wherein $\mathrm{sign}\left(v_i^0\right)=\left\{\begin{array}{c}1,(v_i^0-v_i^k)\&CenterDot;\mathrm{norm}\left(v_i^0\right)\&GreaterEqual;0\\ -1,(v_i^0-v_i^k)\&CenterDot;\mathrm{norm}\left(v_i^0\right)<0\end{array}\right.,$ Be

The normal vector at place.In the specific implementation, the present invention uses the Laplacian based on grid smoothly to obtain the basic mode type. because depth image R ⁰In contained the two-dimensional structure information of a cloud, so gridding R ⁰Be simply also fast.In smoothing process, topological relation remains unchanged between the summit, establishes M ^k=(V ^k, E) be R ^kCorresponding triangle gridding, wherein V ^kRepresent the set on summit and limit respectively with E,

Be M ^kThe border. $\&ForAll;v_i^k=(x_i^k,y_i^k,z_i^k)\&Element;M^k,$ Then $v_i^{k+1}=\left\{\begin{array}{c}\frac{1}{d_i}\underset{j\&Element;N\left(i\right)}{\mathrm{\&Sigma;}}{v}_j^k,v_i^k\&NotElement;{\&PartialD;M}^k\\ v_i^k,v_i^k\&Element;{\&PartialD;M}^k\end{array}\right.,$ N (i)={ j| (i, j) ∈ E}, d wherein _i=| N (i) |, in experiment, generally get k=64.

2. find out the Candidate Set of respective pixel between texture image.

(1) consider that the SIFT feature not only has unchangeability to the convergent-divergent and the rotation of image, simultaneously illumination variation, noise, block with less conditions such as viewpoint variation under robustness is preferably also arranged, the present invention uses the SIFT algorithm to find out I respectively _pAnd I _qIn the interest set of pixels;

(2) at I _pAnd I _qThe interest set of pixels between seek corresponding pixel, for detected each the interest pixel i of SIFT algorithm, SIFT proper vector with sift (i) expression i, d (i then, j)=and ‖ sift (i), sift (j) ‖ can represent the similarity degree between two interest pixel i and the j, more little then these two pixels of this distance are similar more, otherwise big more then their difference of distance are big more.Owing to may there be the complicacy of angle variation and texture image self between depth image; the simple Euclidean distance of SIFT proper vector of using is sought respective pixel as evaluation criterion; introduce a large amount of wrong couplings through regular meeting; simultaneously because the interest pixel of using the SIFT algorithm to find out is too much; the present invention adopts following steps to find out the Candidate Set C of respective pixel between two width of cloth texture images, and its detailed process is:

(2.1) for guaranteeing that the pixel in the result set all has corresponding summit in depth image, deletion does not have the interest pixel of corresponding vertex in depth image;

(2.2) for the interest pixel i in the texture image, if i is then deleted in the summit of i correspondence in depth image on the border of depth image grid;

(2.3) find out I by the method for crosscheck _pAnd I _qBetween the Candidate Set of respective pixel.For I _pIn each crucial pixel i ^p, at I _qIn find out and make

Two minimum crucial pixels

With Might as well establish $d(i^p,i_1^q)\&le;d(i^p,i_2^q)$ , definition

With

Between diversity factor be ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)=\frac{d(i^p,i_2^q)-d(i^p,i_1^q)}{d(i^p,i_1^q)},$ And pre-defined diversity factor threshold value δ _DiffIf, ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}},$ Then think

With

Between notable difference is arranged,

Can be used as i ^pRespective pixel, otherwise think i ^pAt I _qIn multiple correspondence is arranged, should give up i ^p, when ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}}$ The time, again with same method at I _pIn find out pixel

Respective pixel

If

And i ^pBe same pixel, then think i ^pWith

Be I _pAnd I _qBetween a pair of candidate's respective pixel, will Add Candidate Set C, otherwise, still think i ^pAt I _qIn do not have respective pixel.In experiment, generally get δ _Diff=0.2.

3. choose the set of correct matched pixel according to depth image

The coupling that still comprises mistake among the simple respective pixel Candidate Set C that uses the SIFT proper vector to find out, the pixel that method that need to introduce other is found out correct coupling among the C is right. under the known situation of the three-dimensional information of pixel corresponding vertex, can judge the correctness of coupling fully with three-dimensional constraint condition.The present invention uses a kind of adaptive RANSAC algorithm in conjunction with the three-dimensional information on summit, and the pixel of finding out correct coupling from Candidate Set C is right, and its method is:

(1) to all elements (i among the Candidate Set C ^p, i ^q) according to d (i ^p, i ^q) sort with ascending order, and the result is put into a formation L;

(2) need in the three dimensions not three couple of conllinear coupling summit owing to recover minimum of a rigid body permutation matrix, thus the present invention only check among the L before l element whether be correct coupling, generally get l=50 in the experiment.

(2.1) from preceding l the element of L 3 elements of picked at random as a sample;

(2.2) calculate the rigid body permutation matrix according to these 3 pairs of matched pixel geological information of corresponding vertex in depth image;

(2.3), judge whether all the other l-3 element is correct coupling under this rigid body displacement according to the geological information of corresponding vertex.The element of correct coupling is an interior point (inlier), and remaining is exterior point (outlier), and point is formed the unanimity collection (consensus set) of this sampling in all, and the number according to interior point upgrades the sampling number upper limit then;

(2.4) repeat above three steps, reach the sampling number upper limit up to cycle index, the unanimity collection of maximum sampling of counting out in finally getting as a result of collects the element set as correct coupling.

4. it is right to summit corresponding in depth image to find out correct matched pixel, and uses the estimated value that calculates rigid body permutation matrix between two amplitude deepness images based on the method for hypercomplex number.

Between depth image and texture image, there is mapping F _p: R _p→ I _pAnd F _q: R _q→ I _qBecause algorithm of the present invention can guarantee result set C in the 2nd, 3 steps _rIn pixel in depth image, finding corresponding vertex, promptly $\&ForAll;(i_k^p,i_k^q)\&Element;C_r,$ $\&Exists;v_k^p\&Element;R_p,$ $v_k^q\&Element;R_q$ Make $F_p\left(v_k^p\right)=i_k^p,F_q\left(v_k^q\right)=i_k^q,$ So in case find out C _r, can be according to F _pAnd F _qFind out R _pAnd R _qBetween corresponding vertex set, find the solution the rigid body permutation matrix with the method for Horn again.

5. meticulous registration

In case calculate the estimated value of rigid body permutation matrix between two amplitude deepness images, just can use ICP algorithm or its mutation to be original state with this result, finish iteration optimization.In meticulous registration process, the present invention has improved classical ICP algorithm: at first, do not consider to be positioned on the depth image border and close on the summit on border in iterative process, can obtain better accuracy like this; Secondly, the distance of corresponding point and dynamically adjusts distance threshold during according to each iteration, like this in each iterative process, have only the interior corresponding point of threshold range be used to calculate new transformation matrix and the distance that minimizes corresponding point with.

In meticulous registration process, if the distance of corresponding point and not restraining, perhaps converge on a value that surpasses pre-defined threshold value, then thinking can't this two amplitude deepness image of registration, and promptly the result of rough registration is incorrect or do not have an overlapping region between this two amplitude deepness image.In the process of multi-amplitude deepness image registration, this criterion is used to differentiate two amplitude deepness images and whether has the overlapping region.

The step of registration of several scan-datas is as follows:

When registration n amplitude deepness image, the illustraton of model of establishing topological relation between the expression depth image is that (V E), wherein, schemes each vertex v of G to G _i∈ V all represents an amplitude deepness image, | V|=n, each bar limit e of figure G _{I, j}The expression vertex v _iAnd v _jBetween two amplitude deepness images of representative the overlapping region is arranged, and the weights of this edge are from v _iTo v _j, the rigid body permutation matrix, easily know, if e _{I, j}There is then e _{J, i}Also exist, and its weights are e _{I, j}Inverse matrix. the generation tree representation of figure G makes up the least member collection of complete models by these depth images, therefore in case make up the G that publishes picture, just can set by the generation of searching G and rebuild complete model.

6. based on two amplitude deepness image registrations, the list entries of multi-amplitude deepness image is divided into the subsequence of several strips shape.

If S=is (s ₁, s ₂..., s _n) be the list entries of n amplitude deepness image subject to registration, when these data of registration, the present invention need set up illustraton of model G (V equally, E), although can do a registration to judge whether they have the overlapping region to any two amplitude deepness images among the S, if have then to calculate relevant rigid body permutation matrix, all limits among the G thereby structure is published picture.But when handling large-scale data, this is a task very consuming time.Under the scanning strategy of Pingi, $\&ForAll;s_i\&Element;S(1<i<n),$ s _iWith s _I-1And s _I+1Between all have the overlapping region, so just can be successively to two adjacent amplitude deepness image s _i, s _I+1∈ S (i=1...n-1) does registration, the G thereby structure is published picture, and assurance figure G is communicated with.Yet this scanning of Pingi and the strategy of registration are too desirable, are difficult to guarantee two adjacent width of cloth data s among the S in practice _i, s _I+1(i=1...n-1) all there is the overlapping region between.The present invention supposes that the user can be according to a definite sequence when scanning, promptly finish scanning in banded mode, but per two scan stripes interbands are not necessarily continuous, be not necessarily to have the overlapping region between first amplitude deepness image of last amplitude deepness image of last band and next band, under this assumption, sequence S can be divided into k ribbon subsequence: S=(W ₁, W ₂.., W _k), $\&ForAll;W_i\&Element;S$ All satisfy the hypothesis of Pingi, i.e. W _iIn any two adjacent depth images the overlapping region is all arranged.Concrete division methods is:

(1) with s _iAdd first subsequence W ₁In;

(2) all adjacent depth image s among the registration sequence S successively _i, s _I+1∈ S (i=1...n-1) is if s _iWith s _I+1The overlapping region is arranged, then with s _I+1Add s _iThe subsequence at place, otherwise with s _I+1Add a new subsequence, and in figure G, add relevant limit.

7. adopt a kind of strategy of search forward to merge these subsequences, and construct complete three-dimensional model.

(1) to subsequence W _iEvery amplitude deepness image among the ∈ S (i=2...k) is retrieved subsequence W successively _jWhether there are depth image and W among the ∈ S (j=i-1...1) _iIn depth image the overlapping region is arranged, if there is such depth image, so just can be with W _iAnd W _jMerge, be about to W _jIn each element be inserted into W _iHeader element before, simultaneously in figure G, add relevant limit;

(2) up to only remaining a subsequence, perhaps there is not the overlapping region in constantly cyclic process (1) between any two in the residue subsequence, the former shows that figure G is communicated with, and one that promptly can find out it generates tree and constructs complete model; And under one situation of back, a part of the equal representative model of each remaining subsequence, but be to use autoregistration algorithm of the present invention these parts can't be stitched together again.

(3) rebuild whole model according to the generation tree of figure G

Obviously, the strategy of Pingi proposition can be regarded a special case of the present invention as.With traditional O (n that needs any two width of cloth scan-datas of registration ²) several method for registering of time complexity compare, and adopt method of the present invention, the scan-data in the same subsequence can not done registration again, has therefore reduced the time of whole several registration process greatly.

As shown in Figure 2, (a) be two depth images subject to registration, (b) be two width of cloth texture images that from scan-data, extract, wherein white point is represented the respective pixel between two width of cloth images, (c) stain in is the summit of the respective pixel correspondence in three dimensions between texture image, (d) is final registration results; Fig. 2 has illustrated the process of two amplitude deepness image autoregistrations with example.

As shown in Figure 3,, earlier all two adjacent width of cloth data are done registration, whole scanning sequence can be divided into 5 ribbon subsequences like this, merge this 5 subsequences then, finally generate a complete model for 13 amplitude deepness images of input.

Be respectively the registration results of the different model scan-datas that adopt the inventive method as Fig. 4,5,6,7, Fig. 4 is the registration results of a model of big foot, Fig. 5 is the registration results of No. two models of big foot, and Fig. 6 is the registration results of Buddha model, and Fig. 7 is the registration results of terra cotta warriors and horses model.Wherein, comprised texture image in the scan-data of model of big foot and No. two models of big foot, can directly come registration based on known texture image, and do not comprise texture image in the scan-data of Buddha model and terra cotta warriors and horses model, need to generate texture image according to depth image earlier, come registration based on the texture image that generates again.From registration results, for comprising texture image and not comprising the scan-data of texture image, the present invention all can well registration they.

Claims (6)

1, a kind of depth image autoegistration method of combined with texture information is characterized in that: be divided into the registration of two amplitude deepness images and the registration of multi-amplitude deepness image, wherein:

The step of registration of two amplitude deepness images is as follows:

If P and Q are scan-datas subject to registration, the texture image of P and depth image are respectively I _pAnd R _p, the texture image of Q and depth image are respectively I _qAnd R _q,

The first step is extracted texture image I from P and Q _pAnd I _qOr according to depth image R _pAnd R _qGenerate texture image I _pAnd I _q

Second step is based on SIFT feature extraction texture image I _pAnd I _qIn the interest pixel, and therefrom find out the right Candidate Set C of matched pixel by the method for crosscheck;

The 3rd step, use adaptive RANSAC algorithm, it is right to find out matched pixel correct among the Candidate Set C according to geological information constraint;

In the 4th step, it is right to the coupling summit of correspondence to find out in three dimensions with matched pixel, and according to these summit corresponding relations, calculates the rigid body permutation matrix between two amplitude deepness images;

The 5th step, use the result of improved ICP algorithm optimization in the 4th step, finish the accurate registration of two amplitude deepness images;

The step of registration of several scan-datas is as follows:

If between the expression depth image illustraton of model of topological relation be G (V, E), S=(s ₁, s ₂..., s _n) be the list entries of n amplitude deepness image subject to registration;

In the 6th step,, the list entries S of multi-amplitude deepness image is divided into the banded subsequence S=of k (W based on two amplitude deepness image registrations ₁, W ₂..., W _k), W wherein _i∈ S (i=1...k) is the subsequence of some amplitude deepness images, W _iAny two adjacent depth images among the ∈ S (i=1...k) all have the overlapping region;

The 7th step, adopt the strategy of search forward, merge subsequence, according to the complete three-dimensional model of subsequence structure that has merged.

2, the automatic deepness image registration method of combined with texture information according to claim 1 is characterized in that: in the described first step, according to depth image R _pAnd R _qGenerate texture image I _pAnd I _qMethod be:

It is level and smooth that depth image is Laplacian, obtains its basic mode type, calculates the distance of each summit corresponding vertex on the basic mode type in the depth image again, and with the brightness value of each distance value as each summit respective pixel on texture image.

3, the automatic deepness image registration method of combined with texture information according to claim 1 is characterized in that: in described second step, based on SIFT feature extraction texture image I _pAnd I _qIn the interest pixel, and be by the method that the method for crosscheck is therefrom found out the right Candidate Set C of matched pixel:

(1) use the SIFT algorithm to find out I respectively _pAnd I _qIn the interest set of pixels;

(2) according to the Euclidean distance between the SIFT proper vector of two pixels, at I _pAnd I _qThe interest set of pixels between seek corresponding pixel, its method is:

(2.1) deletion does not have the interest pixel of corresponding vertex in depth image;

(2.2) for the interest pixel i in the texture image, if i is then deleted in the summit of i correspondence in depth image on the border of depth image grid;

(2.3) for I _pIn each crucial pixel i ^p, at I _qIn find out and make

Two minimum crucial pixels

With

, establish $d(i^p,i_1^q)\&le;d(i^p,i_2^q),$ Definition

With

Between diversity factor be ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)=\frac{d(i^p,i_2^q)-d(i^p,i_1^q)}{d(i^p,i_1^q)},$ And pre-defined diversity factor threshold value δ _DiffIf, ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}}$ , then

With

Between notable difference is arranged,

Can be used as i ^pRespective pixel, otherwise i ^pAt I _qIn multiple correspondence is arranged, should give up i ^p, when ${\mathrm{diff}}_{i^p}(i_1^q,i_2^q)<{\mathrm{\&delta;}}_{\mathrm{diff}}$ The time, again with same method at I _pIn find out pixel

Respective pixel

If,

And i ^pBe same pixel, then i ^pWith Be I _pAnd I _qBetween a pair of candidate's respective pixel, will

Add Candidate Set C, otherwise still think i ^pAt I _qIn do not have respective pixel.

4, the automatic deepness image registration method of combined with texture information according to claim 1, it is characterized in that: in described the 3rd step, use adaptive RANSAC algorithm, right method is to find out matched pixel correct among the Candidate Set C according to geological information constraint:

(1) to all elements (i among the Candidate Set C ^p, i ^q) according to d (i ^p, i ^q) sort with ascending order, and the result is put into a formation L;

(2) whether l element is correct coupling before among the check L, and the method for inspection is:

(2.1) from preceding l the element of L 3 elements of picked at random as a sample;

(2.2) calculate the rigid body permutation matrix according to 3 pairs of matched pixel geological information of corresponding vertex in depth image;

(2.3), judge whether all the other L-3 element is correct coupling under this rigid body displacement according to the geological information of corresponding vertex; The element of correct coupling is interior point, and remaining is an exterior point, and point is formed the unanimity collection of this sampling in all, and the number according to interior point upgrades the sampling number upper limit then;

(2.4) repeat above step (2.1)-(2.3), reach the sampling number upper limit up to cycle index, it is maximum to find out the consistent element of concentrating then, maximum sampling of counting out in promptly, and the unanimity collection that will sample specifically is as the element set of correct coupling.

5, the automatic deepness image registration method based on colouring information according to claim 1 is characterized in that: in described the 6th step, based on two amplitude deepness image registrations, the step that the list entries of multi-amplitude deepness image is divided into the subsequence of several strips shape is:

(1) with s ₁Add among first subsequence W1;

(2) all adjacent depth image s among the registration sequence S successively _i, s _I+1∈ S (i=1..n-1) is if s _iWith s _I+1The overlapping region is arranged, then with s _I+1Add s _iThe subsequence at place, otherwise with s _I+1Add a new subsequence, and in figure G, add relevant limit.

6, the automatic deepness image registration method based on colouring information according to claim 1, it is characterized in that: in described the 7th step, adopt the strategy of search forward, merge these subsequences, and be according to the step of the complete three-dimensional model of the subsequence structure that has merged:

(1) to subsequence W _iEvery amplitude deepness image among the ∈ S (i=2...k) is retrieved subsequence W successively _jWhether there are depth image and W among the ∈ S (j=i-1...1) _iIn depth image the overlapping region is arranged, if there is such depth image, so just can be with W _iAnd W _jMerge, be about to W _jIn each element be inserted into W _iHeader element before, simultaneously in figure G, add relevant limit;

(2) continuous circulation step (1) is until only remaining a subsequence, perhaps at residue subsequence W _iIn do not have the overlapping region between any two;

(3) rebuild whole model according to figure G.

Images