CN104077804A - Method for constructing three-dimensional human face model based on multi-frame video image - Google Patents
Method for constructing three-dimensional human face model based on multi-frame video image Download PDFInfo
- Publication number
- CN104077804A CN104077804A CN201410253326.5A CN201410253326A CN104077804A CN 104077804 A CN104077804 A CN 104077804A CN 201410253326 A CN201410253326 A CN 201410253326A CN 104077804 A CN104077804 A CN 104077804A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- video
- face
- model
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for constructing a three-dimensional human face model based on a multi-frame video image. The method includes the steps that three-dimensional reconstruction is conducted on a two-dimensional monitoring picture photographed by a video camera with the fixed irradiation position and angle, and therefore a three-dimensional space model and three-dimensional space information of the monitoring picture of the video camera are obtained; a multi-frame continuous sequence video including target movement, shapes, texture and color information is extracted from the input video image; human face feature detection, three-dimensional space positioning and human face feature synchronous tracking and identifying are conducted on the multi-frame continuous sequence video, and three-dimensional human face feature points of the multi-frame continuous sequence video are obtained; superposition calculating is conducted on the three-dimensional human face feature points of the multi-frame continuous sequence video according to the three-dimensional space model of the monitoring picture of the video camera, a three-dimensional human face network is formed, and three-dimensional human face model data are generated. The method has the advantages of being easy and convenient to operate, good in real-time performance and high in accuracy. The method can be widely applied to the field of video image processing.
Description
Technical field
The present invention relates to field of video image processing, especially a kind of method based on multi-frame video picture construction three-dimensional face model.
Background technology
At present, the identity identifying technology based on biological characteristic (as fingerprint, palmmprint and footmark etc.) has been widely used in safety-security area, and the various service application based on Identification of Images also spread to different industries and field gradually.Traditional Identification of Images method, taking two-dimension human face recognition methods as main, comprises Fisher face recognition methods and Eigenface recognition methods etc.But the discrimination of two-dimension human face method of identification is low, there is certain error, cannot meet the active demand of service application.And Identification of Images method based on three-dimensional face model is compared with two-dimension human face method of identification, there is abundanter information, and support to realize by Space Rotating the comparison of multi-angle, recognition accuracy is higher, the existing trend that replaces two-dimension human face method of identification.
The structure of three-dimensional face model is core and the key of the Identification of Images method based on three-dimensional face model.At present, the method for structure three-dimensional face model mainly contains two kinds: a kind of is by the three-dimensional camera of multi-angle, a fixed face to be taken, and is then spliced into a three-dimensional model; Another kind is that the mode scanning by surface profile builds three-dimensional model.Although these two kinds of methods to a certain extent reconstruct three-dimensional face model, it operates more complicated, convenient not.
The structure of three-dimensional face model comprises the processes such as feature extraction, master pattern variation, positioning feature point and texture.Current feature extraction, master pattern changes and texture process is carried out mainly for Static Human Face image, be difficult to the information such as face parameter and attribute (as the situation of facial expression distortion be just difficult to describe) of reflection with movement locus, the method rediscover face to the full extent that cannot adopt similarity measurement or comparison, real-time is lower and data error is larger.
In sum, need in the industry a kind of convenient, real-time and high three-dimensional face model construction method of degree of accuracy at present badly.
Summary of the invention
In order to solve the problems of the technologies described above, the object of the invention is: provide a kind of convenient, in real time and degree of accuracy high and applied range, based on the method for multi-frame video picture construction three-dimensional face model.
The technical solution adopted for the present invention to solve the technical problems is: a kind of method based on multi-frame video picture construction three-dimensional face model, comprising:
A. the two-dimentional monitored picture of the video camera of fixing irradiation position and angle being taken carries out three-dimensional reconstruction, thereby obtains three-dimensional space model and the three-dimensional spatial information of camera supervised picture;
B. from the video image of input, extract the multiframe continuous sequence video that comprises target travel, shape, texture and colouring information;
C. multiframe continuous sequence video is carried out to face characteristic detection, three-dimensional fix, the synchronous recognition and tracking of face characteristic, thereby obtain the three-dimensional face unique point of multiframe continuous sequence video;
D. according to the three-dimensional space model of camera supervised picture, the three-dimensional face unique point of multiframe continuous sequence video is carried out to superposition calculation, thereby form three-dimensional face grid generating three-dimensional faceform data.
Further, described steps A, it comprises:
A1. set up homography solution according to the intrinsic parameter matrix of video camera, described homography solution has reflected the homography relation of ground level in plane and video camera photographic images practically;
A2. known according to the height of video camera, given two length and perpendicular to the reference line of ground level, the visual angle initial point of video camera is calculated;
A3. rebuild video camera visual angle three-dimensional model according to the visual angle initial point of homography solution, video camera and given Visualization Model, thereby obtain three-dimensional space model and the three-dimensional spatial information of camera supervised picture.
Further, described step C, it comprises:
C1. from multiframe continuous sequence video, choose single frame of video as current video frame;
C2. current video frame is carried out to face characteristic detection and face characteristic location, thereby obtain the human face characteristic point of current video frame;
C3. the human face characteristic point of current video frame is carried out to three-dimensional fix, and detect the spatial information of the contained human face characteristic point of current video frame, movement locus and the temporal information of face characteristic;
C4. according to the result detecting, current video frame is carried out to face characteristic and synchronously follow the tracks of and identification automatically, thus the each locus coordinate of the human face characteristic point of definite current video frame in moving process;
C5. continue to choose next single frame of video as current video frame from multiframe continuous sequence video, then return to step C2, thereby generate the three-dimensional system of coordinate matrix of face characteristic according to multiframe continuous sequence video in the continuous motion state under in the same time not.
Further, described step D, it is specially:
Many groups three-dimensional face unique point by multiframe continuous sequence video is carried out three-dimensional face key frame superposition calculation, the list of generating three-dimensional faceform data directory, thus set up structurized three-dimensional face model data list and to three-dimensional face model data directory list carry out stores processor.
Further, described step D, it comprises:
D1. by the human face characteristic point of multiframe continuous sequence video in the three-dimensional space model for the camera supervised picture of people, thereby obtain the three-dimensional face unique point volume coordinate of multiframe continuous sequence video;
D2. according to three-dimensional face unique point volume coordinate, adopt 3-D view stitching algorithm to generate texture image, and the texture image generating is shone upon, thereby obtain real three-dimensional face model data.
Further, described step D2, it comprises:
D21. from multiframe continuous sequence video, rebuild the sparse set of face markers point according to three-dimensional face unique point volume coordinate, and utilize thin plate spline TPS to try one by one to join to sparse set;
D22. the result of trying to join according to TPS is carried out nonlinear transformation to general faceform, thereby obtains the three-dimensional face model of coupling;
D23. adopt 3-D view stitching algorithm to obtain the face textural characteristics information of multiframe continuous sequence video, and the face textural characteristics information obtaining is mapped in the three-dimensional face model of coupling, thereby obtain real three-dimensional face model data.
Further, after described step D, be also provided with step e, described step e, it is specially:
Adopt SFM algorithm to retain the special characteristic of Generic face model, by with Generic face model comparison, revise the three-dimensional face model data that generate and the error of Generic face model data; Then adopt triangle close classification to build final three-dimensional face model by the depth information of point.
Further, adopt triangle close classification to build this step of final three-dimensional face model by the depth information of point in described step e, it comprises:
E21. from three-dimensional face grid, filter out the triangle that needs segmentation according to default threshold value, and the triangle filtering out is carried out to mark;
E22. the triangle of mark is combined into n gridblock according to neighbouring relations, then independent this n gridblock, be designated as B
b 1 , b 2 , b 3 ..., b n , the part simultaneously three-dimensional face grid not being labeled is designated as
r i ;
E23. will
b i the weight on four summits of middle gridblock is adjusted into respectively 0,1/2,1/2 and 0, thereby right
b i carry out grid interpolation subdividing;
E24. segment not doing
r i carry out interpolation subdividing at boundary, thereby make to be positioned at midpoint in borderline insertion point;
E25. will
bwith
rsynthesize, and whether the grid model of judgement after synthetic meet its all leg-of-mutton length of sides and be all less than default threshold value, if so, using the grid model after synthesizing as final three-dimensional face model, otherwise, return to step e 21.
The invention has the beneficial effects as follows: the image information of taking by the video camera of single fixing irradiation position and angle is set up three-dimensional face model, simple to operate, very convenient; By continuous sequence video being carried out to face characteristic detection, three-dimensional fix, the synchronous recognition and tracking of face characteristic, the key frame that extraction comprises face, the change in displacement of face position is carried out to dynamic tracing, set up three-dimensional relationship, determine the spatial relation of everyone face characteristic point, solved prior art and cannot carry out to information such as face characteristic parameter and attribute such as the movement locus of dynamic human face image the problem of synchronous recognition and tracking, real-time better and degree of accuracy higher.Further, adopt SFM algorithm to retain the special characteristic of Generic face model, and adopt triangle close classification to carry out smoothly, further having improved faceform's degree of accuracy and the sense of reality to facial image.
Brief description of the drawings
Below in conjunction with drawings and Examples, the invention will be further described.
Fig. 1 is the flow chart of steps of a kind of method based on multi-frame video picture construction three-dimensional face model of the present invention;
Fig. 2 is the process flow diagram of steps A of the present invention;
Fig. 3 is the process flow diagram of step C of the present invention;
Fig. 4 is the process flow diagram of step D of the present invention;
Fig. 5 is the process flow diagram of step D2 of the present invention;
Fig. 6 is the process flow diagram of step e triangle close classification of the present invention;
Fig. 7 is that embodiment mono-is according to the schematic diagram of the intrinsic Reconstruction three-dimensional space model of video camera;
Fig. 8 is the schematic diagram of in embodiment mono-, multiple image being set up three-dimensional portrait flow process.
Embodiment
With reference to Fig. 1, a kind of method based on multi-frame video picture construction three-dimensional face model, comprising:
A. the two-dimentional monitored picture of the video camera of fixing irradiation position and angle being taken carries out three-dimensional reconstruction, thereby obtains three-dimensional space model and the three-dimensional spatial information of camera supervised picture;
B. from the video image of input, extract the multiframe continuous sequence video that comprises target travel, shape, texture and colouring information;
C. multiframe continuous sequence video is carried out to face characteristic detection, three-dimensional fix, the synchronous recognition and tracking of face characteristic, thereby obtain the three-dimensional face unique point of multiframe continuous sequence video;
D. according to the three-dimensional space model of camera supervised picture, the three-dimensional face unique point of multiframe continuous sequence video is carried out to superposition calculation, thereby form three-dimensional face grid generating three-dimensional faceform data.
With reference to Fig. 2, be further used as preferred embodiment, described steps A, it comprises:
A1. set up homography solution according to the intrinsic parameter matrix of video camera, described homography solution has reflected the homography relation of ground level in plane and video camera photographic images practically;
A2. known according to the height of video camera, given two length and perpendicular to the reference line of ground level, the visual angle initial point of video camera is calculated;
A3. rebuild video camera visual angle three-dimensional model according to the visual angle initial point of homography solution, video camera and given Visualization Model, thereby obtain three-dimensional space model and the three-dimensional spatial information of camera supervised picture.
With reference to Fig. 3, be further used as preferred embodiment, described step C, it comprises:
C1. from multiframe continuous sequence video, choose single frame of video as current video frame;
C2. current video frame is carried out to face characteristic detection and face characteristic location, thereby obtain the human face characteristic point of current video frame;
C3. the human face characteristic point of current video frame is carried out to three-dimensional fix, and detect the spatial information of the contained human face characteristic point of current video frame, movement locus and the temporal information of face characteristic;
C4. according to the result detecting, current video frame is carried out to face characteristic and synchronously follow the tracks of and identification automatically, thus the each locus coordinate of the human face characteristic point of definite current video frame in moving process;
C5. continue to choose next single frame of video as current video frame from multiframe continuous sequence video, then return to step C2, thereby generate the three-dimensional system of coordinate matrix of face characteristic according to multiframe continuous sequence video in the continuous motion state under in the same time not.
Be further used as preferred embodiment, described step D, it is specially:
Many groups three-dimensional face unique point by multiframe continuous sequence video is carried out three-dimensional face key frame superposition calculation, the list of generating three-dimensional faceform data directory, thus set up structurized three-dimensional face model data list and to three-dimensional face model data directory list carry out stores processor.
With reference to Fig. 4, be further used as preferred embodiment, described step D, it comprises:
D1. by the human face characteristic point of multiframe continuous sequence video in the three-dimensional space model for the camera supervised picture of people, thereby obtain the three-dimensional face unique point volume coordinate of multiframe continuous sequence video;
D2. according to three-dimensional face unique point volume coordinate, adopt 3-D view stitching algorithm to generate texture image, and the texture image generating is shone upon, thereby obtain real three-dimensional face model data.
With reference to Fig. 5, be further used as preferred embodiment, described step D2, it comprises:
D21. from multiframe continuous sequence video, rebuild the sparse set of face markers point according to three-dimensional face unique point volume coordinate, and utilize thin plate spline TPS to try one by one to join to sparse set;
D22. the result of trying to join according to TPS is carried out nonlinear transformation to general faceform, thereby obtains the three-dimensional face model of coupling;
D23. adopt 3-D view stitching algorithm to obtain the face textural characteristics information of multiframe continuous sequence video, and the face textural characteristics information obtaining is mapped in the three-dimensional face model of coupling, thereby obtain real three-dimensional face model data.
Be further used as preferred embodiment, after described step D, be also provided with step e, described step e, it is specially:
Adopt SFM algorithm to retain the special characteristic of Generic face model, by with Generic face model comparison, revise the three-dimensional face model data that generate and the error of Generic face model data; Then adopt triangle close classification to build final three-dimensional face model by the depth information of point.
Generic face model, refers to as known standard faces model in the industry.
With reference to Fig. 6, be further used as preferred embodiment, in described step e, adopt triangle close classification to build this step of final three-dimensional face model by the depth information of point, it comprises:
E21. from three-dimensional face grid, filter out the triangle that needs segmentation according to default threshold value, and the triangle filtering out is carried out to mark;
E22. the triangle of mark is combined into n gridblock according to neighbouring relations, then independent this n gridblock, be designated as B
b 1 , b 2 , b 3 ..., b n , the part simultaneously three-dimensional face grid not being labeled is designated as
r i ;
E23. will
b i the weight on four summits of middle gridblock is adjusted into respectively 0,1/2,1/2 and 0, thereby right
b i carry out grid interpolation subdividing;
E24. segment not doing
r i carry out interpolation subdividing at boundary, thereby make to be positioned at midpoint in borderline insertion point;
E25. will
bwith
rsynthesize, and whether the grid model of judgement after synthetic meet its all leg-of-mutton length of sides and be all less than default threshold value, if so, using the grid model after synthesizing as final three-dimensional face model, otherwise, return to step e 21.
Below in conjunction with specific embodiment, the present invention is described in further detail.
Embodiment mono-
The process that the present embodiment carries out faceform to the present invention by video acquisition high-speed downloads device is described in detail.
The process that video acquisition high-speed downloads device carries out faceform is:
(1) 3 D scene rebuilding
In video acquisition high-speed downloads device, the intrinsic parameter of video camera is known, and the picture that video camera is taken carries out 3 D scene rebuilding, and reconstruction procedures is: first, between the ground level in plane and image practically, set up homography solution (homography) H; The rear reality of video camera and the setting height(from bottom) h of ground level of utilizing, default known length and the line perpendicular to ground level, calibrate video camera.Embodiment is as follows:
(1) according to the pin-hole model of video camera, definition matrix
mfor:
, hence one can see that, and the homography relation of the ground level in plane and video camera photographic images can be expressed as practically
(1).
Wherein, the intrinsic parameter matrix that A is video camera.
r 1,
r 2,
r 3for rotation matrix
rthree column vectors,
tfor translation parameters.If the corresponding point between the ground level in plane and image are more than 4 groups practically, can pass through formula (1), can make H more be expanded.
(2) definition video camera optic center point, the visual angle initial point of video camera be (
x c,
y c,
h), order
, can draw according to the spatial relationship of video camera
:
(3) given perpendicular to the reference line of plane practically
l*, and the projection on ground level in shooting photographic images
l, can learn the straight line through impact point according to the spatial relationship of video camera
h t lin plane practically and through point (
x c,
y c, 0).
Therefore, according to step (1)-(3), given camera height
h, can calculate perpendicular to reference line plane and that length is known practically with two
x c,
y cwith
k.
(4) then rebuild video camera visual angle three-dimensional model according to default Visualization Model.
To as shown in Figure 7, (
x c,
y c,
h) be made as the central point of user coordinate system, and this Visualization Model is projected to plane practically.According to space geometry projection relation, any point in user coordinate system (
x w,
y w,
z w) projection in plane practically (
x w ＇,
y w ＇, 0) can calculate by through type (2), formula (2) is:
…（2）
(5) finally utilize homography solution H, can be by this Visualization Model the projection mapping in plane practically in image ground level, thereby the reconstruction of setting up mapping relations and complete 3 D monitoring scene.After reconstruction finishes, can carry out depth information calibration to any point on the three-dimensional portrait of setting up.
(2) obtain three-dimensional face feature space data
After 3 D scene rebuilding completes, the picture in this video camera is calculated, utilize face characteristic detection method to a frame wherein
f1picture carries out face characteristic location, and in the three-dimensional space model that the characteristic point sequence substitution of collecting is rebuild, draw each unique point three-dimensional space data [
x f1 ,
y f1 ,
z f1 ].Start subsequently to read the next frame in video
f2, same method obtain the second frame face characteristic three-dimensional space data [
x f2 ,
y f2 ,
z f2 ], until obtain
fnthe three-dimensional face feature space data of frame [
x fn ,
y fn ,
z fn ].
(3) three-dimensional splicing and mapping
After the three-dimensional face feature space data that obtain, also need to be modeled to the camera imaging figure of multiple angles, this process can be converted into the three-dimensional splicing problem of 3-D view.Specific practice is: the sparse set of rebuilding face markers point from video, utilize thin plate spline TPS(Thin Plate Spline) these sparse set are tried to join one by one, on the basis of then trying to join at TPS, general faceform is carried out to nonlinear transformation and obtain the three-dimensional face model mating, last again by the face texture information in video to shining upon in the three-dimensional face model of this coupling, thereby obtain real three-dimensional face model.
For example, a pair of 3-D view I1 that can splice and I2 are positioned at the scope of one group of given N image.First I1 and I2 splice a kind of new 3-D view I11 obtaining, and then I11 image splices and obtains image I 12 with I3 image, then follow I12 image and I3 image and splice and obtain image I 13.Repeat said process, until can not splice again, thereby a complete three-dimensional face model obtained.
(4) SFM algorithm
In order to ensure the precision of model, the present invention has also adopted SFM(Structure From Motion) algorithm retains the special characteristic of Generic face model, by with Generic face model comparison, revise the error between two width faces.Concrete steps are:
First determine with from
f1obtain [
x f1 ,
y f1 ,
z f1 ] be benchmark trace data.Estimate subsequently motion and the structural change of human face characteristic point.Next motion estimated values is carried out to refinement, finally by estimated value and next frame
f2face coordinate figure compare, judge in its interval of whether calculating in estimation, if not, abandon, continue the extraction of next frame data, the face result that circulation draws thus can ensure its grown form.
(5) triangle close classification
Adopt triangle close classification to set up three-dimensional portrait by the depth information of point, idiographic flow is as follows:
Step 1, screening needs the triangle of segmentation: establishing i triangle maximal side is k, and when this triangle of K>m tense marker, m is default threshold value, and the present invention is taken as 0.15.The all triangles of traversal in grid model, and the triangle that need to divide of mark.
Step 2, the triangle of composite marking: the triangle of mark is combined into n piece according to neighbouring relations, and independent this n piece, be designated as B
b 1 , b 2 , b 3 ..., b n , the part simultaneously three-dimensional face grid not being labeled is designated as
r i .
Step 3, segments independent gridblock: right
b i do grid subdivision, need to adjust the weight on four summits, change respectively 0,1/2,1/2,0 into, can be all mid point in borderline insertion point so always, thereby boundary shape is consistent.
Step 4, adjusts
r i border:
b i on border, insert new point in midpoint, segmented not doing
r i do same adjustment at boundary, thereby synthetic grid is coincide in splicing boundary.
Step 5, synthetic R and B.
Through above-mentioned steps 1-5, the segmentation of R grid completes, and R, and B is also consistent in borderline division points, finally both is combined to the once segmentation just having realized whole original mesh.Repeat above step until all leg-of-mutton length of sides are all less than threshold value, finally reach model accuracy requirement.
Fig. 8 is the embodiment schematic diagram of setting up three-dimensional portrait model by said method.
Compared with prior art, the image information that the present invention takes by the video camera of single fixing irradiation position and angle is set up three-dimensional face model, simple to operate, very convenient; By continuous sequence video being carried out to face characteristic detection, three-dimensional fix, the synchronous recognition and tracking of face characteristic, the key frame that extraction comprises face, the change in displacement of face position is carried out to dynamic tracing, set up three-dimensional relationship, determine the spatial relation of everyone face characteristic point, solved prior art and cannot carry out to information such as face characteristic parameter and attribute such as the movement locus of dynamic human face image the problem of synchronous recognition and tracking, real-time better and degree of accuracy higher; Adopt SFM algorithm to retain the special characteristic of Generic face model, and adopt triangle close classification to carry out smoothly, further having improved faceform's degree of accuracy and the sense of reality to facial image.
More than that better enforcement of the present invention is illustrated, but the invention is not limited to described embodiment, those of ordinary skill in the art also can make all equivalent variations or replacement under the prerequisite without prejudice to spirit of the present invention, and the distortion that these are equal to or replacement are all included in the application's claim limited range.
Claims (8)
1. the method based on multi-frame video picture construction three-dimensional face model, is characterized in that: comprising:
A. the two-dimentional monitored picture of the video camera of fixing irradiation position and angle being taken carries out three-dimensional reconstruction, thereby obtains three-dimensional space model and the three-dimensional spatial information of camera supervised picture;
B. from the video image of input, extract the multiframe continuous sequence video that comprises target travel, shape, texture and colouring information;
C. multiframe continuous sequence video is carried out to face characteristic detection, three-dimensional fix, the synchronous recognition and tracking of face characteristic, thereby obtain the three-dimensional face unique point of multiframe continuous sequence video;
D. according to the three-dimensional space model of camera supervised picture, the three-dimensional face unique point of multiframe continuous sequence video is carried out to superposition calculation, thereby form three-dimensional face grid generating three-dimensional faceform data.
2. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 1, is characterized in that: described steps A, and it comprises:
A1. set up homography solution according to the intrinsic parameter matrix of video camera, described homography solution has reflected the homography relation of ground level in plane and video camera photographic images practically;
A2. known according to the height of video camera, given two length and perpendicular to the reference line of ground level, the visual angle initial point of video camera is calculated;
A3. rebuild video camera visual angle three-dimensional model according to the visual angle initial point of homography solution, video camera and given Visualization Model, thereby obtain three-dimensional space model and the three-dimensional spatial information of camera supervised picture.
3. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 1, is characterized in that: described step C, and it comprises:
C1. from multiframe continuous sequence video, choose single frame of video as current video frame;
C2. current video frame is carried out to face characteristic detection and face characteristic location, thereby obtain the human face characteristic point of current video frame;
C3. the human face characteristic point of current video frame is carried out to three-dimensional fix, and detect the spatial information of the contained human face characteristic point of current video frame, movement locus and the temporal information of face characteristic;
C4. according to the result detecting, current video frame is carried out to face characteristic and synchronously follow the tracks of and identification automatically, thus the each locus coordinate of the human face characteristic point of definite current video frame in moving process;
C5. continue to choose next single frame of video as current video frame from multiframe continuous sequence video, then return to step C2, thereby generate the three-dimensional system of coordinate matrix of face characteristic according to multiframe continuous sequence video in the continuous motion state under in the same time not.
4. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 1, is characterized in that: described step D, and it is specially:
Many groups three-dimensional face unique point by multiframe continuous sequence video is carried out three-dimensional face key frame superposition calculation, the list of generating three-dimensional faceform data directory, thus set up structurized three-dimensional face model data list and to three-dimensional face model data directory list carry out stores processor.
5. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 1, is characterized in that: described step D, and it comprises:
D1. by the human face characteristic point of multiframe continuous sequence video in the three-dimensional space model for the camera supervised picture of people, thereby obtain the three-dimensional face unique point volume coordinate of multiframe continuous sequence video;
D2. according to three-dimensional face unique point volume coordinate, adopt 3-D view stitching algorithm to generate texture image, and the texture image generating is shone upon, thereby obtain real three-dimensional face model data.
6. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 5, is characterized in that: described step D2, and it comprises:
D21. from multiframe continuous sequence video, rebuild the sparse set of face markers point according to three-dimensional face unique point volume coordinate, and utilize thin plate spline TPS to try one by one to join to sparse set;
D22. the result of trying to join according to TPS is carried out nonlinear transformation to general faceform, thereby obtains the three-dimensional face model of coupling;
D23. adopt 3-D view stitching algorithm to obtain the face textural characteristics information of multiframe continuous sequence video, and the face textural characteristics information obtaining is mapped in the three-dimensional face model of coupling, thereby obtain real three-dimensional face model data.
7. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 1, is characterized in that: after described step D, be also provided with step e, and described step e, it is specially:
Adopt SFM algorithm to retain the special characteristic of Generic face model, by with Generic face model comparison, revise the three-dimensional face model data that generate and the error of Generic face model data; Then adopt triangle close classification to build final three-dimensional face model by the depth information of point.
8. a kind of method based on multi-frame video picture construction three-dimensional face model according to claim 7, is characterized in that: in described step e, adopt triangle close classification to build this step of final three-dimensional face model by the depth information of point, it comprises:
E21. from three-dimensional face grid, filter out the triangle that needs segmentation according to default threshold value, and the triangle filtering out is carried out to mark;
E22. the triangle of mark is combined into n gridblock according to neighbouring relations, then independent this n gridblock, be designated as B
b 1 , b 2 , b 3 ..., b n , the part simultaneously three-dimensional face grid not being labeled is designated as
r i ;
E23. will
b i the weight on four summits of middle gridblock is adjusted into respectively 0,1/2,1/2 and 0, thereby right
b i carry out grid interpolation subdividing;
E24. segment not doing
r i carry out interpolation subdividing at boundary, thereby make to be positioned at midpoint in borderline insertion point;
E25. will
bwith
rsynthesize, and whether the grid model of judgement after synthetic meet its all leg-of-mutton length of sides and be all less than default threshold value, if so, using the grid model after synthesizing as final three-dimensional face model, otherwise, return to step e 21.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410253326.5A CN104077804B (en) | 2014-06-09 | 2014-06-09 | A kind of method based on multi-frame video picture construction three-dimensional face model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410253326.5A CN104077804B (en) | 2014-06-09 | 2014-06-09 | A kind of method based on multi-frame video picture construction three-dimensional face model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104077804A true CN104077804A (en) | 2014-10-01 |
| CN104077804B CN104077804B (en) | 2017-03-01 |
Family
ID=51599043
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410253326.5A Expired - Fee Related CN104077804B (en) | 2014-06-09 | 2014-06-09 | A kind of method based on multi-frame video picture construction three-dimensional face model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104077804B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104268578A (en) * | 2014-10-15 | 2015-01-07 | 深圳市晓舟科技有限公司 | Target recognition method implemented by combining small image comparison and fuzzy recognition |
| CN104361131A (en) * | 2014-12-08 | 2015-02-18 | 黑龙江大学 | Establishment method of four-dimensional face model database |
| CN104408416A (en) * | 2014-11-25 | 2015-03-11 | 苏州福丰科技有限公司 | Three-dimensional face recognition monitoring system for gold or jewelry shops |
| CN104574309A (en) * | 2014-12-30 | 2015-04-29 | 北京像素软件科技股份有限公司 | Method and device for subdividing triangular meshes in mobile game application |
| CN104951773A (en) * | 2015-07-12 | 2015-09-30 | 上海微桥电子科技有限公司 | Real-time face recognizing and monitoring system |
| CN105046725A (en) * | 2015-07-10 | 2015-11-11 | 清华大学 | Head shoulder image reconstruction method based on model and object in low bit rate video call |
| CN105046219A (en) * | 2015-07-12 | 2015-11-11 | 上海微桥电子科技有限公司 | Face identification system |
| CN105138954A (en) * | 2015-07-12 | 2015-12-09 | 上海微桥电子科技有限公司 | Image automatic screening, query and identification system |
| CN105184860A (en) * | 2015-09-30 | 2015-12-23 | 南京邮电大学 | Method for reconstructing dense three-dimensional structure and motion field of dynamic face simultaneously |
| CN106500628A (en) * | 2016-10-19 | 2017-03-15 | 杭州思看科技有限公司 | A kind of 3-D scanning method containing multiple different wave length laser instrument and scanner |
| CN106683163A (en) * | 2015-11-06 | 2017-05-17 | 杭州海康威视数字技术股份有限公司 | Imaging method and system used in video monitoring |
| CN106778474A (en) * | 2016-11-14 | 2017-05-31 | 深圳奥比中光科技有限公司 | 3D human body recognition methods and equipment |
| CN107749084A (en) * | 2017-10-24 | 2018-03-02 | 广州增强信息科技有限公司 | A kind of virtual try-in method and system based on 3-dimensional reconstruction technology |
| CN107924579A (en) * | 2015-08-14 | 2018-04-17 | 麦特尔有限公司 | The method for generating personalization 3D head models or 3D body models |
| CN108027983A (en) * | 2015-09-23 | 2018-05-11 | 皇家飞利浦有限公司 | For the generation of the triangular mesh of 3-D view |
| CN108629333A (en) * | 2018-05-25 | 2018-10-09 | 厦门市美亚柏科信息股份有限公司 | A kind of face image processing process of low-light (level), device, equipment and readable medium |
| CN109101915A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | Face and pedestrian and Attribute Recognition network structure design method based on deep learning |
| CN109108968A (en) * | 2018-08-17 | 2019-01-01 | 深圳市三宝创新智能有限公司 | Exchange method, device, equipment and the storage medium of robot head movement adjustment |
| CN109345621A (en) * | 2018-08-28 | 2019-02-15 | 广州智美科技有限公司 | Interactive face three-dimensional modeling method and device |
| CN109685873A (en) * | 2018-12-14 | 2019-04-26 | 广州市百果园信息技术有限公司 | A kind of facial reconstruction method, device, equipment and storage medium |
| CN109753857A (en) * | 2017-11-07 | 2019-05-14 | 北京虹图吉安科技有限公司 | A kind of three-dimensional face identification apparatus and system based on photometric stereo visual imaging |
| CN110135308A (en) * | 2019-04-30 | 2019-08-16 | 天津工业大学 | A kind of direct free kick type identification method based on video analysis |
| CN110136270A (en) * | 2018-02-02 | 2019-08-16 | 北京京东尚科信息技术有限公司 | The method and apparatus of adornment data are held in production |
| CN110393513A (en) * | 2018-04-25 | 2019-11-01 | 财团法人交大思源基金会 | Contactless heartbeat measurement system, method and device thereof |
| CN110462693A (en) * | 2019-06-28 | 2019-11-15 | 深圳市汇顶科技股份有限公司 | Door lock and recognition methods |
| CN110505403A (en) * | 2019-08-20 | 2019-11-26 | 维沃移动通信有限公司 | A kind of video record processing method and device |
| CN110678104A (en) * | 2017-04-03 | 2020-01-10 | 株式会社爱茉莉太平洋 | Matching type facial mask manufacturing system and manufacturing method |
| CN110688905A (en) * | 2019-08-30 | 2020-01-14 | 中山大学 | Three-dimensional object detection and tracking method based on key frame |
| CN111144166A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Method, system and storage medium for establishing abnormal crowd information base |
| CN111179408A (en) * | 2018-11-12 | 2020-05-19 | 北京物语科技有限公司 | Method and apparatus for three-dimensional modeling |
| CN111339958A (en) * | 2020-02-28 | 2020-06-26 | 山东笛卡尔智能科技有限公司 | Monocular vision-based face in-vivo detection method and system |
| CN111369670A (en) * | 2020-03-13 | 2020-07-03 | 江西科骏实业有限公司 | Method for real-time construction of practical training digital twin model |
| CN111435268A (en) * | 2019-01-11 | 2020-07-21 | 合肥虹慧达科技有限公司 | Human-computer interaction method based on image recognition and reconstruction and system and device using same |
| CN112188182A (en) * | 2019-07-02 | 2021-01-05 | 西安诺瓦星云科技股份有限公司 | Stereoscopic display control system, stereoscopic display system and stereoscopic display controller |
| CN112700523A (en) * | 2020-12-31 | 2021-04-23 | 魔珐(上海)信息科技有限公司 | Virtual object face animation generation method and device, storage medium and terminal |
| TWI808336B (en) * | 2020-07-30 | 2023-07-11 | 杰悉科技股份有限公司 | Image display method and image monitoring system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006106465A2 (en) * | 2005-04-07 | 2006-10-12 | Nxp B.V. | Method and device for three-dimentional reconstruction and rendering |
| CN101499128B (en) * | 2008-01-30 | 2011-06-29 | 中国科学院自动化研究所 | Three-dimensional human face action detecting and tracing method based on video stream |
| CN101404091B (en) * | 2008-11-07 | 2011-08-31 | 重庆邮电大学 | Three-dimensional human face reconstruction method and system based on two-step shape modeling |
| CN101593365A (en) * | 2009-06-19 | 2009-12-02 | 电子科技大学 | A kind of method of adjustment of universal three-dimensional human face model |
| CN101625768B (en) * | 2009-07-23 | 2011-11-09 | 东南大学 | Three-dimensional human face reconstruction method based on stereoscopic vision |
| CN101866497A (en) * | 2010-06-18 | 2010-10-20 | 北京交通大学 | Binocular stereo vision based intelligent three-dimensional human face rebuilding method and system |
| CN102254154B (en) * | 2011-07-05 | 2013-06-12 | 南京大学 | Method for authenticating human-face identity based on three-dimensional model reconstruction |
| CN102708385B (en) * | 2012-04-06 | 2014-07-16 | 张丛喆 | Method and system for comparison and recognition of three-dimensional vehicle types in video monitoring scenes |
-
2014
- 2014-06-09 CN CN201410253326.5A patent/CN104077804B/en not_active Expired - Fee Related
Non-Patent Citations (1)
| Title |
|---|
| 张剑: ""融合SFM和动态纹理映射的视频流三维表情重建"", 《计算机辅助设计与图形学学报》 * |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104268578A (en) * | 2014-10-15 | 2015-01-07 | 深圳市晓舟科技有限公司 | Target recognition method implemented by combining small image comparison and fuzzy recognition |
| CN104268578B (en) * | 2014-10-15 | 2017-06-09 | 深圳市晓舟科技有限公司 | The target identification method that a kind of small image is compared and fuzzy diagnosis is combined |
| CN104408416A (en) * | 2014-11-25 | 2015-03-11 | 苏州福丰科技有限公司 | Three-dimensional face recognition monitoring system for gold or jewelry shops |
| CN104361131A (en) * | 2014-12-08 | 2015-02-18 | 黑龙江大学 | Establishment method of four-dimensional face model database |
| CN104361131B (en) * | 2014-12-08 | 2018-06-15 | 黑龙江大学 | The method for building up of four-dimensional faceform's database |
| CN104574309A (en) * | 2014-12-30 | 2015-04-29 | 北京像素软件科技股份有限公司 | Method and device for subdividing triangular meshes in mobile game application |
| CN105046725A (en) * | 2015-07-10 | 2015-11-11 | 清华大学 | Head shoulder image reconstruction method based on model and object in low bit rate video call |
| CN105046219A (en) * | 2015-07-12 | 2015-11-11 | 上海微桥电子科技有限公司 | Face identification system |
| CN105138954A (en) * | 2015-07-12 | 2015-12-09 | 上海微桥电子科技有限公司 | Image automatic screening, query and identification system |
| CN105138954B (en) * | 2015-07-12 | 2019-06-04 | 上海微桥电子科技有限公司 | A kind of image automatic screening inquiry identifying system |
| CN105046219B (en) * | 2015-07-12 | 2018-12-18 | 上海微桥电子科技有限公司 | A kind of face identification system |
| CN104951773B (en) * | 2015-07-12 | 2018-10-02 | 上海微桥电子科技有限公司 | A kind of real-time face recognition monitoring system |
| CN104951773A (en) * | 2015-07-12 | 2015-09-30 | 上海微桥电子科技有限公司 | Real-time face recognizing and monitoring system |
| CN107924579A (en) * | 2015-08-14 | 2018-04-17 | 麦特尔有限公司 | The method for generating personalization 3D head models or 3D body models |
| CN108027983A (en) * | 2015-09-23 | 2018-05-11 | 皇家飞利浦有限公司 | For the generation of the triangular mesh of 3-D view |
| CN105184860A (en) * | 2015-09-30 | 2015-12-23 | 南京邮电大学 | Method for reconstructing dense three-dimensional structure and motion field of dynamic face simultaneously |
| CN106683163B (en) * | 2015-11-06 | 2020-10-27 | 杭州海康威视数字技术股份有限公司 | Imaging method and system for video monitoring |
| CN106683163A (en) * | 2015-11-06 | 2017-05-17 | 杭州海康威视数字技术股份有限公司 | Imaging method and system used in video monitoring |
| CN106500628B (en) * | 2016-10-19 | 2019-02-19 | 杭州思看科技有限公司 | A kind of 3-D scanning method and scanner containing multiple and different long wavelength lasers |
| CN106500628A (en) * | 2016-10-19 | 2017-03-15 | 杭州思看科技有限公司 | A kind of 3-D scanning method containing multiple different wave length laser instrument and scanner |
| CN106778474A (en) * | 2016-11-14 | 2017-05-31 | 深圳奥比中光科技有限公司 | 3D human body recognition methods and equipment |
| CN110678104A (en) * | 2017-04-03 | 2020-01-10 | 株式会社爱茉莉太平洋 | Matching type facial mask manufacturing system and manufacturing method |
| CN107749084A (en) * | 2017-10-24 | 2018-03-02 | 广州增强信息科技有限公司 | A kind of virtual try-in method and system based on 3-dimensional reconstruction technology |
| CN109753857A (en) * | 2017-11-07 | 2019-05-14 | 北京虹图吉安科技有限公司 | A kind of three-dimensional face identification apparatus and system based on photometric stereo visual imaging |
| CN110136270A (en) * | 2018-02-02 | 2019-08-16 | 北京京东尚科信息技术有限公司 | The method and apparatus of adornment data are held in production |
| CN110393513A (en) * | 2018-04-25 | 2019-11-01 | 财团法人交大思源基金会 | Contactless heartbeat measurement system, method and device thereof |
| CN108629333A (en) * | 2018-05-25 | 2018-10-09 | 厦门市美亚柏科信息股份有限公司 | A kind of face image processing process of low-light (level), device, equipment and readable medium |
| CN109101915A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | Face and pedestrian and Attribute Recognition network structure design method based on deep learning |
| CN109108968A (en) * | 2018-08-17 | 2019-01-01 | 深圳市三宝创新智能有限公司 | Exchange method, device, equipment and the storage medium of robot head movement adjustment |
| CN109345621A (en) * | 2018-08-28 | 2019-02-15 | 广州智美科技有限公司 | Interactive face three-dimensional modeling method and device |
| CN111144166A (en) * | 2018-11-02 | 2020-05-12 | 银河水滴科技（北京）有限公司 | Method, system and storage medium for establishing abnormal crowd information base |
| WO2020098566A1 (en) * | 2018-11-12 | 2020-05-22 | 程立苇 | Three-dimensional modeling method and device, and computer readable storage medium |
| CN111179408B (en) * | 2018-11-12 | 2024-04-12 | 北京物语科技有限公司 | Three-dimensional modeling method and equipment |
| CN111179408A (en) * | 2018-11-12 | 2020-05-19 | 北京物语科技有限公司 | Method and apparatus for three-dimensional modeling |
| CN109685873B (en) * | 2018-12-14 | 2023-09-05 | 广州市百果园信息技术有限公司 | Face reconstruction method, device, equipment and storage medium |
| CN109685873A (en) * | 2018-12-14 | 2019-04-26 | 广州市百果园信息技术有限公司 | A kind of facial reconstruction method, device, equipment and storage medium |
| CN111435268A (en) * | 2019-01-11 | 2020-07-21 | 合肥虹慧达科技有限公司 | Human-computer interaction method based on image recognition and reconstruction and system and device using same |
| CN110135308A (en) * | 2019-04-30 | 2019-08-16 | 天津工业大学 | A kind of direct free kick type identification method based on video analysis |
| CN110462693A (en) * | 2019-06-28 | 2019-11-15 | 深圳市汇顶科技股份有限公司 | Door lock and recognition methods |
| CN112188182A (en) * | 2019-07-02 | 2021-01-05 | 西安诺瓦星云科技股份有限公司 | Stereoscopic display control system, stereoscopic display system and stereoscopic display controller |
| CN110505403A (en) * | 2019-08-20 | 2019-11-26 | 维沃移动通信有限公司 | A kind of video record processing method and device |
| CN110688905A (en) * | 2019-08-30 | 2020-01-14 | 中山大学 | Three-dimensional object detection and tracking method based on key frame |
| CN111339958A (en) * | 2020-02-28 | 2020-06-26 | 山东笛卡尔智能科技有限公司 | Monocular vision-based face in-vivo detection method and system |
| CN111339958B (en) * | 2020-02-28 | 2023-08-29 | 南京鑫之派智能科技有限公司 | Face living body detection method and system based on monocular vision |
| CN111369670A (en) * | 2020-03-13 | 2020-07-03 | 江西科骏实业有限公司 | Method for real-time construction of practical training digital twin model |
| CN111369670B (en) * | 2020-03-13 | 2024-10-18 | 江西科骏实业有限公司 | Method for constructing practical training digital twin model in real time |
| TWI808336B (en) * | 2020-07-30 | 2023-07-11 | 杰悉科技股份有限公司 | Image display method and image monitoring system |
| CN112700523A (en) * | 2020-12-31 | 2021-04-23 | 魔珐(上海)信息科技有限公司 | Virtual object face animation generation method and device, storage medium and terminal |
| CN112700523B (en) * | 2020-12-31 | 2022-06-07 | 魔珐(上海)信息科技有限公司 | Virtual object face animation generation method and device, storage medium and terminal |
| WO2022143197A1 (en) * | 2020-12-31 | 2022-07-07 | 魔珐(上海)信息科技有限公司 | Method and apparatus for generating virtual object facial animation, storage medium, and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104077804B (en) | 2017-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104077804A (en) | Method for constructing three-dimensional human face model based on multi-frame video image | |
| CN106802138B (en) | A kind of 3 D scanning system and its scan method | |
| Budd et al. | Global non-rigid alignment of surface sequences | |
| Concha et al. | Using superpixels in monocular SLAM | |
| EP2751777B1 (en) | Method for estimating a camera motion and for determining a three-dimensional model of a real environment | |
| WO2012096747A1 (en) | Forming range maps using periodic illumination patterns | |
| CN101101672A (en) | Stereo vision three-dimensional human face modelling approach based on dummy image | |
| Xu et al. | Dynamic hair capture using spacetime optimization | |
| CN101650834A (en) | Three dimensional reconstruction method of human body surface under complex scene | |
| JP4761670B2 (en) | Moving stereo model generation apparatus and method | |
| Vu et al. | Efficient hybrid tree-based stereo matching with applications to postcapture image refocusing | |
| Wang et al. | Image-based building regularization using structural linear features | |
| Leroy et al. | Volume sweeping: Learning photoconsistency for multi-view shape reconstruction | |
| CN109064533A (en) | A kind of 3D loaming method and system | |
| Ran et al. | High-precision human body acquisition via multi-view binocular stereopsis | |
| Hafiz et al. | Interest point detection in 3D point cloud data using 3D Sobel-Harris operator | |
| Kang et al. | Progressive 3D model acquisition with a commodity hand-held camera | |
| Wu et al. | Photogrammetric reconstruction of free-form objects with curvilinear structures | |
| Delmas et al. | Stereo camera visual odometry for moving urban environments | |
| Kang et al. | 3D urban reconstruction from wide area aerial surveillance video | |
| Xiaojun et al. | High accurate 3D reconstruction method using binocular stereo based on multiple constraints | |
| Abrams et al. | Structure from shadow motion | |
| Quigley et al. | Three dimensional reconstruction of botanical trees with simulatable geometry | |
| Rolin et al. | View synthesis for pose computation | |
| Liu et al. | A wide base line multiple camera system for high performance 3D video and free viewpoint video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170301 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |