[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2024098322A1 - Method for synthesizing 3d human face model by using optical reflection - Google Patents

Method for synthesizing 3d human face model by using optical reflection Download PDF

Info

Publication number
WO2024098322A1
WO2024098322A1 PCT/CN2022/131102 CN2022131102W WO2024098322A1 WO 2024098322 A1 WO2024098322 A1 WO 2024098322A1 CN 2022131102 W CN2022131102 W CN 2022131102W WO 2024098322 A1 WO2024098322 A1 WO 2024098322A1
Authority
WO
WIPO (PCT)
Prior art keywords
reflector
face
degrees
synthesizing
structured light
Prior art date
Application number
PCT/CN2022/131102
Other languages
French (fr)
Chinese (zh)
Inventor
郑炜钢
刘东旭
余旭光
Original Assignee
杭州小肤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州小肤科技有限公司 filed Critical 杭州小肤科技有限公司
Priority to PCT/CN2022/131102 priority Critical patent/WO2024098322A1/en
Publication of WO2024098322A1 publication Critical patent/WO2024098322A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/20Perspective computation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • G06T7/55Depth or shape recovery from multiple images
    • G06T7/593Depth or shape recovery from multiple images from stereo images

Definitions

  • the present invention relates to a method for synthesizing a 3D human face model, and in particular to a method for synthesizing a 3D human face model by utilizing optical reflection.
  • the skin detector is an instrument used to detect the skin. It is equipped with professional detection software to help consumers intuitively and quickly understand the health status of their skin. It has a beautiful interface and practical functions. It should also have several major analysis modules and can accurately analyze the skin oil, moisture, spots, pores, and skin age. The moisture data is directly obtained by obtaining the water data under the skin stratum corneum, and the data accuracy is high.
  • the purpose of the present invention is to provide a method for synthesizing a 3D face model using optical reflection in order to solve the above problems in the prior art.
  • a method for synthesizing a 3D face model using optical reflection comprises the following steps:
  • this method can realize the generation of three-dimensional facial images in a device of the same size, and the use of a structured light camera can obtain more accurate facial depth information, while the single-lens solution can only generate two-dimensional facial images. Therefore, there is a significant difference between the two.
  • the three-dimensional facial image obtained by this method provides a better reference for medical cosmetic surgery, and can obtain information that cannot be obtained by two-dimensional images, and has more comprehensive functions;
  • the two 3D facial images with the smallest absolute value of relative frontal angle are the two 3D facial images closest to the front of the face. They contain lips, nose, and eyes, and their depth information fluctuates greatly, making them easy to align. Therefore, these two images are used as the starting point for alignment, which can facilitate positioning.
  • the number of the optical reflectors is four, corresponding to -15 degrees, 15 degrees, -45 degrees and 45 degrees relative to the front face, respectively, wherein the synthesis step comprises:
  • S600 Rendering the complete face 3D image according to the color map to obtain a 3D face model.
  • the step of synthesizing the image includes:
  • the two composite images can be quickly spliced together to form a complete 3D face model for subsequent rendering.
  • the rendering step includes:
  • the 3D face model can be covered with a color two-dimensional photo to obtain a realistic 3D face model, which can be rotated and observed on a computer device, greatly facilitating the doctor's observation and allowing the doctor to clearly know the height of the nose, the depression of the eyes, the depth of the wrinkles, the size and protrusion of the mouth.
  • a 3D structured light full-face imaging device which includes a head resting area, a 3D structured light rotating module and a plurality of optical reflectors;
  • the 3D structured light rotating module is located in the center in front of the head resting area, and can rotate with itself as the axis through the shaft drive;
  • the optical reflectors are arranged around the 3D structured light rotating module;
  • the number of the optical reflectors is four, corresponding to the positions of -45 degrees, -15 degrees, 15 degrees and 45 degrees of the face in the head resting area.
  • the optical reflector is divided into a first reflector, a second reflector, a third reflector and a fourth reflector, with the connection line between the 3D structured light rotation module and the head resting area as the 0-degree line, the angle between the connection line between the first reflector and the head resting area and the 0-degree line is 90 degrees, the angle between the connection line between the second reflector and the head resting area and the 0-degree line is 45 degrees, the third reflector and the second reflector are mirror-set at the 0-degree line, and the fourth reflector and the first reflector are mirror-set at the 0-degree line.
  • This solution can quickly locate the position of each reflector, making it convenient to install each reflector.
  • This solution only requires rotating the shaft in sequence, that is, the shaft rotates clockwise or counterclockwise, and finally returns to the initial position.
  • Each rotation can be positioned by the motor, and can also be stopped for shooting by controlling the start and stop of the motor. It is easy to control and has low difficulty.
  • the shaft drives the 3D structured light rotation module to turn to the first reflector, the second reflector, the third reflector, and the fourth reflector in sequence.
  • the shaft rotates clockwise or counterclockwise, and finally returns to the initial position.
  • Each rotation can be positioned by the motor, and the motor can be controlled to stop and perform shooting operations. The control is convenient and the implementation difficulty is low.
  • the light shield also includes a light shield and a lighting lamp arranged in the light shield, and the light shield shields the head rest area, the 3D structured light rotation module and the plurality of optical reflectors.
  • the head rest area is provided with a chin rest and a forehead rest, and the face is fixed by the chin rest and the forehead rest.
  • each optical reflector is greater than or equal to 90%.
  • Optical reflectors with a reflectivity of more than 90% can provide better collected images for the 3D structured light rotation module, significantly improving the quality of its final face model.
  • FIG1 is a schematic structural diagram of an embodiment of the device of the present invention.
  • Figure 2 is a schematic diagram of the use status of the device of the present invention Figure 1 ;
  • Figure 3 is a schematic diagram of the use status of the device of the present invention Figure 2 ;
  • FIG4 is a flow chart of an embodiment of the method of the present invention.
  • FIG5 is a point cloud image at -15 degrees obtained by the method of the present invention.
  • FIG6 is a point cloud image of 15 degrees obtained by the method of the present invention.
  • FIG7 is a 45-degree point cloud image obtained by the method of the present invention.
  • FIG8 is a point cloud image at -45 degrees obtained by the method of the present invention.
  • FIG. 9 is a schematic diagram of an image generated by the present invention.
  • 3D structured light The basic principle of 3D structured light technology is to project light with certain structural characteristics onto the object being photographed through a near-infrared laser, and then collect it with a special infrared camera. This light with a certain structure will collect different image phase information due to different depth areas of the object being photographed, and then convert the change of this structure into depth information through the calculation unit to obtain the three-dimensional structure.
  • the three-dimensional structure of the object being photographed is obtained by optical means, and then the obtained information is applied more deeply.
  • a depth image also called a distance image, refers to an image that uses the distance (depth) value from the image collector to each point in the scene as the pixel value.
  • Point cloud When a laser beam hits the surface of an object, the reflected laser will carry information such as direction and distance. If the laser beam is scanned along a certain trajectory, the reflected laser point information will be recorded while scanning. Since the scanning is extremely precise, a large number of laser points can be obtained, thus forming a laser point cloud. Point cloud data is discrete data obtained by scanning and sampling the surface of a real object.
  • Depth images can be calculated into point cloud data after coordinate transformation; point cloud data with rules and necessary information can be inversely calculated into depth images. The two can be transformed into each other under certain conditions.
  • the device used in the method includes a head rest area 1, a 3D structured light rotation module 2 and a plurality of optical reflectors, wherein the head rest area 1 is provided with a chin rest 3 that matches the chin of the human face and a forehead rest 13 that abuts against the forehead of the human face, and a bracket similar to the chin rest 3 that abuts against the top of the head can also be used to fix the head, and a latex pad or other soft cushion can be installed on the chin rest 3, and it can be replaced by itself for easy disinfection.
  • the chin rest 3 of the user 4 is on the chin rest 3, and the 3D structured light rotation module 2 is installed on the rotating shaft 5, and at least includes the 3D structured light rotation module 2 at least includes an RGB camera and a structured light camera.
  • the establishment of the face model is achieved through the RGB camera and the structured light camera.
  • the 3D structured light rotating module 2 is located in the center in front of the head rest area 1, and can rotate with itself as the axis through the drive of the rotating shaft 5.
  • the rotating shaft 5 is driven by a common stepper motor 6 or a servo motor 6, and the motor 6 is controlled by a common computer device.
  • the 3D structured light rotating module 2 is also controlled by the computer device to generate a face model on the computer device.
  • the optical reflector is arranged around the 3D structured light rotating module 2 to reflect the light emitted by the 3D structured light rotating module 2.
  • the optical reflector is a commercially available product and is mounted on an external fixed bracket by means of screws, glue, etc.
  • the fixed bracket is not shown in the attached figure.
  • each optical reflector is greater than or equal to 90%.
  • Optical reflectors with a reflectivity of more than 90% can provide better captured images for the 3D structured light rotation module 2, significantly improving the quality of its final face model.
  • a light shield 7 and a lighting lamp 8 disposed in the light shield 7 are also included, and the light shield 7 is used to shield the head rest area 1, the 3D structured light rotating module 2 and a plurality of optical reflectors, wherein the light shield 7 can be a common black cloth, which can be directly covered on the device, or a cover made of other opaque materials, as shown in FIG1 , and the device is made into a semi-enclosed structure, exposing only an opening for the human head to enter, and a forehead support 13 for supporting the user's forehead.
  • the lighting lamp 8 is turned off. This is mainly to avoid the influence of the lighting lamp 8 on the shooting. This solution can effectively avoid the influence of external light sources, and the internally turned lighting lamp 8 can be turned on as needed.
  • optical reflectors there are four optical reflectors, which correspond to the positions of -45 degrees, -15 degrees, 15 degrees and 45 degrees of the face of the head resting area 1, respectively.
  • the optical reflectors are divided into a first reflector 9, a second reflector 10, a third reflector 11 and a fourth reflector 12.
  • the connecting line between the 3D structured light rotation module 2 and the head resting area 1 is the 0-degree line
  • the angle between the connecting line between the first reflector 9 and the head resting area 1 and the 0-degree line is 90 degrees
  • the angle between the connecting line between the second reflector 10 and the head resting area 1 and the 0-degree line is 45 degrees
  • the third reflector 11 and the second reflector 10 are mirrored at the 0-degree line
  • the fourth reflector 12 and the first reflector 9 are mirrored at the 0-degree line.
  • the installation angles of the reflector are -90 degrees, -45 degrees, 45 degrees and 90 degrees, and the angles that the motor 6 needs to rotate are -45 degrees, -105 degrees, 105 degrees and 45 degrees, respectively.
  • the motor 6 rotates to -45 degrees, it corresponds to the first reflector 9 at -90 degrees, and the -45 degree face data is obtained;
  • the shaft 5 drives the 3D structured light rotating module 2 to turn to the first reflector 9, the second reflector 10, the third reflector 11, and the fourth reflector 12 in sequence, and the shaft 5 is started to rotate to the -45 degree position, and the RGB light, PL, and UV light of the 3D structured light rotating module 2 are turned on in sequence to take pictures, and a group of pictures and corresponding depth data information are obtained, and then the shaft 5 is rotated to the -105 degree, 105 degree and 45 degree positions, and the RGB light, PL, and UV light are turned on in sequence to take pictures, and three different groups of pictures and corresponding depth data information are obtained respectively, and the lighting lamp 8 is turned off when taking pictures.
  • the method for synthesizing a 3D face model using optical reflection includes the following steps:
  • this method can generate three-dimensional facial images in a device of the same size, while the single-lens solution can only generate two-dimensional facial images. Therefore, there is a significant difference between the two.
  • the three-dimensional facial images obtained by this method provide a better reference for medical cosmetic surgery, can obtain information that cannot be obtained from two-dimensional images, and have more comprehensive and rich functions.
  • this method only has one lens, which is obviously lower in cost and simpler in structure.
  • the -15 degree and 15 degree images contain the lips, nose, and eyes of the face.
  • the depth information fluctuates greatly and is easy to align. Therefore, these two images are used as the starting point for alignment.
  • the corresponding grid can cover the 3D face model with a color two-dimensional photo to obtain a realistic 3D face model, which can be rotated and observed on a computer device, which greatly facilitates the doctor's observation and can clearly know the height of the nose, the depression of the eyes, the depth of the wrinkles, the size and protrusion of the mouth.
  • Figure 9 where 1-4 represent the images collected on the first reflector 9, the second reflector 10, the third reflector 11, and the fourth reflector 12, see Figures 5-8 , and the middle of Figure 9 is the merged and rendered image.
  • the original image is a color image with more facial details.
  • the generated face model can be used for analysis of facial skin, such as the depth of wrinkles, the size of acne, etc.
  • Active light source can be used at night.
  • the resolution can reach 1280x1024 and the frame rate can reach 60FPS.
  • one should be understood as “at least one” or “one or more”, that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term “one” should not be understood as a limitation on the quantity.
  • the present invention is not limited to the above-mentioned optimal implementation mode.
  • anyone can derive other various forms of products under the inspiration of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The present invention relates to a method for synthesizing a three-dimensional (3D) human face model by using optical reflection. The solution comprises: acquiring, from optical reflection mirrors, a plurality of 3D human face images and RGB color images at a plurality of angles relative to a front face by rotating a 3D structured light rotating module, wherein the optical reflection mirrors respectively correspond to acquisition positions of the 3D human face images at all the angles; according to the 3D human face images and the RGB color images, generating point cloud images and color maps corresponding to all the angles; synthesizing the corresponding point cloud images in two 3D human face images having a minimum absolute value of the angle relative to the front face into a first synthesized image; sequentially synthesizing the corresponding point cloud images in the 3D human face images of the remaining angles and the first synthesized image; and according to the color maps, carrying out rendering on the synthesized image to obtain a 3D human face model. The method has the advantages that image precision is high, more accurate human face depth information can be obtained, and a good reference is provided for aesthetic plastic surgery.

Description

一种利用光学反射合成3D人脸模型的方法A method for synthesizing 3D face model using optical reflection 技术领域Technical Field
本发明涉及一种合成3D人脸模型的方法,具体涉及一种利用光学反射合成3D人脸模型的方法。The present invention relates to a method for synthesizing a 3D human face model, and in particular to a method for synthesizing a 3D human face model by utilizing optical reflection.
背景技术Background technique
随着时代社会的不断进步,人们越来越关注于自己的皮肤等外貌特征,因此对自己皮肤状态的了解需求也越来越强烈。在医疗美容等领域,如护肤品推荐、肤质分析等方面,人们通常要对全脸的皮肤状况进行一定的检测分析,而后根据分析结果制定相应的策略。通过使用分析仪器来对皮肤进行检测,通过检测结果来推荐合适的护肤品,或者制定相应的护肤策略。其中皮肤检测仪是用来检测皮肤的仪器,它配上专业检测软件可以帮助消费者直观快速了解自己皮肤的健康状况,其具有界面美观、实用功能全面的多种功能,它还应该具有几大分析模块并且可以准确分析出肌肤油份、水分、色斑、毛孔、肌肤年龄,通过获取肌肤角质层下的水份的数据直接获取水份数据,数据准确率高。With the continuous progress of the times and society, people are paying more and more attention to their skin and other appearance features, so the demand for understanding their skin condition is becoming stronger and stronger. In the fields of medical beauty, such as skin care product recommendations and skin quality analysis, people usually have to conduct certain tests and analyses on the skin condition of the whole face, and then formulate corresponding strategies based on the analysis results. The skin is tested by using analytical instruments, and suitable skin care products are recommended based on the test results, or corresponding skin care strategies are formulated. Among them, the skin detector is an instrument used to detect the skin. It is equipped with professional detection software to help consumers intuitively and quickly understand the health status of their skin. It has a beautiful interface and practical functions. It should also have several major analysis modules and can accurately analyze the skin oil, moisture, spots, pores, and skin age. The moisture data is directly obtained by obtaining the water data under the skin stratum corneum, and the data accuracy is high.
但是目前的皮肤检测仪通常只有单摄像头拍摄,或者通过设备转动来拍摄人脸,前者每次只能拍摄一个角度的面部影像,如果需拍摄不同角度的面部影像,要人为的旋转头部且拍摄形成的图片基本上是二维图形,图像不能立体旋转,轮廓分析不精确,后者成本极高,价格昂贵。However, current skin detectors usually only have a single camera for shooting, or they shoot faces by rotating the device. The former can only shoot facial images from one angle each time. If facial images from different angles need to be shot, the head must be rotated manually and the pictures taken are basically two-dimensional. The image cannot be rotated in three dimensions, and the contour analysis is not accurate. The latter is extremely costly and expensive.
综上,亟待一种能够满足消费者日益增长需求的结合光学反射 成像的3D结构光全脸成像装置。In summary, there is an urgent need for a 3D structured light full-face imaging device combined with optical reflective imaging that can meet the growing needs of consumers.
发明内容Summary of the invention
本发明的目的是针对现有技术中存在的上述问题,提供了一种利用光学反射合成3D人脸模型的方法。The purpose of the present invention is to provide a method for synthesizing a 3D face model using optical reflection in order to solve the above problems in the prior art.
为了实现上述发明目的,本发明采用了以下技术方案:一种利用光学反射合成3D人脸模型的方法包括以下步骤:In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme: A method for synthesizing a 3D face model using optical reflection comprises the following steps:
S100、通过旋转3D结构光转动模组从光学反射镜中获取多张相对正脸多个角度的人脸三维图像及RGB彩色图像,其中所述光学反射镜分别对应各个角度的人脸三维图像获取位置;S100, obtaining a plurality of three-dimensional facial images and RGB color images at multiple angles relative to the front face from an optical reflector by rotating a 3D structured light rotation module, wherein the optical reflector corresponds to a position for obtaining a three-dimensional facial image at each angle;
S200、依据人脸三维图像及RGB彩色图像生成对应各个角度的点云图像与彩色贴图。S200, generating point cloud images and color maps corresponding to various angles based on the three-dimensional face image and the RGB color image.
S300、将相对正脸角度绝对值最小的两张人脸三维图像中对应的点云图像进行合成为第一合成图像;S300, synthesizing the corresponding point cloud images in the two three-dimensional face images with the smallest absolute value of the relative frontal face angle into a first synthesized image;
S400、依次将剩余角度的人脸三维图像中对应的点云图像与第一合成图像合成;S400, sequentially synthesizing the point cloud images corresponding to the three-dimensional face images at the remaining angles with the first synthesized image;
S500、根据彩色贴图对合成后的图像进行渲染得到3D人脸模型。S500, rendering the synthesized image according to the color map to obtain a 3D face model.
工作原理及有益效果:1、与现有技术相比,无需将整个设备进行转动来对用户脸部进行拍摄,因此省去了结构庞大的驱动机构,只需要简单小巧的转动机构来转动3D结构光转动模组即可,配合多面光线反射镜即可实现在设备不转动和人脸不转动的情况下对人脸各个角度进行拍摄,克服了整机转动慢、成像时间长的缺点,显著降 低了设备成本,具有结构简单、操作方便、拍摄时间短、用户体验好的优点;Working principle and beneficial effects: 1. Compared with the existing technology, there is no need to rotate the entire device to shoot the user's face, so the bulky driving mechanism is omitted. Only a simple and compact rotating mechanism is needed to rotate the 3D structured light rotating module. With the multi-faceted light reflector, it is possible to shoot the face at all angles without rotating the device or the face, overcoming the shortcomings of slow rotation of the whole device and long imaging time, significantly reducing the equipment cost, and having the advantages of simple structure, easy operation, short shooting time and good user experience;
2、与现有技术的固定单镜头方案相比,本方法可在同样大小的设备中实现生成人脸三维图像,且采用结构光摄像头,可以获得更加精确的人脸深度信息,而单镜头方案只能够生成二维的人脸图像,因此两者具有显著的区别,本方法得到的人脸三维图像为医美整形提供了更为良好的参考,可得到二维图像无法得到的信息,功能更加全面丰富;2. Compared with the fixed single-lens solution of the prior art, this method can realize the generation of three-dimensional facial images in a device of the same size, and the use of a structured light camera can obtain more accurate facial depth information, while the single-lens solution can only generate two-dimensional facial images. Therefore, there is a significant difference between the two. The three-dimensional facial image obtained by this method provides a better reference for medical cosmetic surgery, and can obtain information that cannot be obtained by two-dimensional images, and has more comprehensive functions;
3、与多镜头的方案相比,本方法只有一个镜头,显然成本更低,结构也更加简单。3. Compared with the multi-lens solution, this method only has one lens, which is obviously lower in cost and simpler in structure.
4、其中相对正脸角度绝对值最小的两张人脸三维图像也就是最靠近人脸正面的两张人脸三维图像,其包含了人脸嘴唇、鼻子、眼睛,深度信息起伏较大,容易配准,因此将这两张图像作为配准起点,可方便定位。4. The two 3D facial images with the smallest absolute value of relative frontal angle are the two 3D facial images closest to the front of the face. They contain lips, nose, and eyes, and their depth information fluctuates greatly, making them easy to align. Therefore, these two images are used as the starting point for alignment, which can facilitate positioning.
进一步地,所述光学反射镜的数量为四个,分别对应相对正脸的-15度,15度,-45度及45度,其中合成步骤包括:Furthermore, the number of the optical reflectors is four, corresponding to -15 degrees, 15 degrees, -45 degrees and 45 degrees relative to the front face, respectively, wherein the synthesis step comprises:
S300、将-15度和15度的两张人脸三维图像中对应的点云图像进行合成,得到第一合成图像;S300, synthesizing the corresponding point cloud images in the two three-dimensional face images at -15 degrees and 15 degrees to obtain a first synthesized image;
S400、将-45度或45度的一张人脸三维图像中对应的点云图像与第一合成图像进行合成,得到第二合成图像;S400, synthesizing the point cloud image corresponding to a three-dimensional face image at -45 degrees or 45 degrees with the first synthesized image to obtain a second synthesized image;
S500、将剩下的人脸三维图像中对应的点云图像与第二合成图像进行合成,得到完整人脸3D图像;S500, synthesizing the corresponding point cloud image in the remaining three-dimensional face image with the second synthesized image to obtain a complete 3D face image;
S600、根据彩色贴图进行对完整人脸3D图像渲染得到3D人脸模型。S600: Rendering the complete face 3D image according to the color map to obtain a 3D face model.
进一步地,其中图像的合成步骤包括:Furthermore, the step of synthesizing the image includes:
S410、获取3D结构光转动模组输出的点云数据;S410, acquiring point cloud data output by the 3D structured light rotation module;
S420、依据点云数据进行预设的3D结构光转动模组位置和旋转角度的粗配准;S420, performing a rough alignment of a preset 3D structured light rotation module position and rotation angle according to the point cloud data;
S430、根据点云数据中重合的点进行精配准;S430, performing precise registration according to the overlapping points in the point cloud data;
S440、过滤重合点完成合成拼接。S440, filtering the overlapping points to complete the synthesis splicing.
通过上述步骤,可快速将两个合成图像进行拼接,形成完整的人脸3D模型,供后续渲染使用。Through the above steps, the two composite images can be quickly spliced together to form a complete 3D face model for subsequent rendering.
进一步地,渲染步骤包括:Furthermore, the rendering step includes:
S450、对拼接后的点云数据进行三维重建,生产三角形网格;S450, performing three-dimensional reconstruction on the spliced point cloud data to generate a triangular mesh;
S460、获取3D结构光转动模组中RGB相机获得的贴图图片,将其贴到重建后的三维模型的网格上,生成3D人脸模型。S460, obtaining a texture image obtained by the RGB camera in the 3D structured light rotation module, and pasting it onto the grid of the reconstructed three-dimensional model to generate a 3D face model.
通过上述步骤,可将人脸3D模型覆盖上彩色二维照片,得到逼真的3D人脸模型,可在电脑设备上转动观察,极大地方便医生进行观察,可以清晰知道鼻子的高度,眼睛的凹陷,皱纹的深浅,嘴形的大小和突出情况。Through the above steps, the 3D face model can be covered with a color two-dimensional photo to obtain a realistic 3D face model, which can be rotated and observed on a computer device, greatly facilitating the doctor's observation and allowing the doctor to clearly know the height of the nose, the depression of the eyes, the depth of the wrinkles, the size and protrusion of the mouth.
进一步地,还包括3D结构光全脸成像装置,所述3D结构光全脸成像装置包括头部搁置区、3D结构光转动模组及多个光学反射镜;所述3D结构光转动模组位于头部搁置区前方正中位置,通过转轴驱动可以自身为轴心进行旋转;所述光学反射镜设于3D结构光转 动模组周围;所述光学反射镜的数量为四个,分别对应位于头部搁置区人脸的-45度、-15度、15度及45度位置。通过此装置可快速扫描用户的脸部,从而方便地形成用户的3D人脸模型。Furthermore, it also includes a 3D structured light full-face imaging device, which includes a head resting area, a 3D structured light rotating module and a plurality of optical reflectors; the 3D structured light rotating module is located in the center in front of the head resting area, and can rotate with itself as the axis through the shaft drive; the optical reflectors are arranged around the 3D structured light rotating module; the number of the optical reflectors is four, corresponding to the positions of -45 degrees, -15 degrees, 15 degrees and 45 degrees of the face in the head resting area. Through this device, the user's face can be quickly scanned, thereby conveniently forming a 3D face model of the user.
进一步地,所述光学反射镜分为第一反射镜、第二反射镜、第三反射镜及第四反射镜,以3D结构光转动模组与头部搁置区的连接线为0度线,第一反射镜与头部搁置区的连接线与0度线的夹角为90度,第二反射镜与头部搁置区的连接线与0度线的夹角为45度,第三反射镜与第二反射镜以0度线镜像设置,第四反射镜与第一反射镜以0度线镜像设置。此方案,可快速地对每个反射镜的位置进行定位,方便每个反射镜的安装。此方案,只需要依次转动转轴即可,即转轴顺时针或逆时针转动,最后回到初始位置,每次转动都可通过电机进行定位,也能够通过控制电机的启停来停留进行拍摄操作,控制方便,实现难度低。Furthermore, the optical reflector is divided into a first reflector, a second reflector, a third reflector and a fourth reflector, with the connection line between the 3D structured light rotation module and the head resting area as the 0-degree line, the angle between the connection line between the first reflector and the head resting area and the 0-degree line is 90 degrees, the angle between the connection line between the second reflector and the head resting area and the 0-degree line is 45 degrees, the third reflector and the second reflector are mirror-set at the 0-degree line, and the fourth reflector and the first reflector are mirror-set at the 0-degree line. This solution can quickly locate the position of each reflector, making it convenient to install each reflector. This solution only requires rotating the shaft in sequence, that is, the shaft rotates clockwise or counterclockwise, and finally returns to the initial position. Each rotation can be positioned by the motor, and can also be stopped for shooting by controlling the start and stop of the motor. It is easy to control and has low difficulty.
进一步地,所述转轴驱动3D结构光转动模组依次转向第一反射镜、第二反射镜、第三反射镜、第四反射镜。此方案,只需要依次转动转轴即可,即转轴顺时针或逆时针转动,最后回到初始位置,每次转动都可通过电机进行定位,也能够通过控制电机的启停来停留进行拍摄操作,控制方便,实现难度低。Furthermore, the shaft drives the 3D structured light rotation module to turn to the first reflector, the second reflector, the third reflector, and the fourth reflector in sequence. In this solution, it is only necessary to rotate the shaft in sequence, that is, the shaft rotates clockwise or counterclockwise, and finally returns to the initial position. Each rotation can be positioned by the motor, and the motor can be controlled to stop and perform shooting operations. The control is convenient and the implementation difficulty is low.
进一步地,还包括遮光罩和设于遮光罩内的照明灯,通过所述遮光罩遮挡头部搁置区、3D结构光转动模组及多个光学反射镜。此方案,可有效地免去外部光源的影响,内部开启的照明灯可根据需求打开。Furthermore, it also includes a light shield and a lighting lamp arranged in the light shield, and the light shield shields the head rest area, the 3D structured light rotation module and the plurality of optical reflectors. This solution can effectively avoid the influence of external light sources, and the lighting lamp turned on inside can be turned on as needed.
进一步地,所述头部搁置区设有下巴托和额托,通过下巴托和额托固定人脸。此方案,可方便地对用户的头部进行固定,使用体验好,使得用户的头部能够始终位于最佳的高度,从而生成完整的人脸模型。Furthermore, the head rest area is provided with a chin rest and a forehead rest, and the face is fixed by the chin rest and the forehead rest. This solution can conveniently fix the user's head, and the user experience is good, so that the user's head can always be at the best height, thereby generating a complete face model.
进一步地,每个所述光学反射镜的反射率大于等于90%。拥有90%以上的反射率的光学反射镜可给3D结构光转动模组提供更好的采集图像,显著提升其最终的人脸模型质量。Furthermore, the reflectivity of each optical reflector is greater than or equal to 90%. Optical reflectors with a reflectivity of more than 90% can provide better collected images for the 3D structured light rotation module, significantly improving the quality of its final face model.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是本发明装置的一种实施例的结构示意图; FIG1 is a schematic structural diagram of an embodiment of the device of the present invention;
图2是本发明装置的使用状态示意 图一 Figure 2 is a schematic diagram of the use status of the device of the present invention Figure 1 ;
图3是本发明装置的使用状态示意 图二 Figure 3 is a schematic diagram of the use status of the device of the present invention Figure 2 ;
图4是本发明方法的一种实施例的流程图; FIG4 is a flow chart of an embodiment of the method of the present invention;
图5是本发明方法获取到-15度的点云图像; FIG5 is a point cloud image at -15 degrees obtained by the method of the present invention;
图6是本发明方法获取到15度的点云图像; FIG6 is a point cloud image of 15 degrees obtained by the method of the present invention;
图7是本发明方法获取到45度的点云图像; FIG7 is a 45-degree point cloud image obtained by the method of the present invention;
图8是本发明方法获取到-45度的点云图像; FIG8 is a point cloud image at -45 degrees obtained by the method of the present invention;
图9是本发明生成图像的示意图。 FIG. 9 is a schematic diagram of an image generated by the present invention.
图中,1、头部搁置区;2、3D结构光转动模组;3、下巴托;4、用户;5、转轴;6、电机;7、遮光罩;8、照明灯;9、第一反射镜;10、第二反射镜;11、第三反射镜;12、第四反射镜;13、额托。In the figure, 1. head rest area; 2. 3D structured light rotation module; 3. chin rest; 4. user; 5. shaft; 6. motor; 7. light shield; 8. lighting; 9. first reflector; 10. second reflector; 11. third reflector; 12. fourth reflector; 13. forehead rest.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员所获得的所有其他实施例,都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.
本领域技术人员应理解的是,在本发明的披露中,术语“纵向”、“横向”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”、“内”、“外”等指示的方位或位置关系是基于附图所示的方位或位置关系,其仅是为了便于描述本发明和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此上述术语不能理解为对本发明的限制。Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or position relationship indicated by the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.
为了方便理解,先对3D结构光的原理进行解释:3D结构光技术的基本原理是,通过近红外激光器,将具有一定结构特征的光线投射到被拍摄物体上,再由专门的红外摄像头进行采集。这种具备一定结构的光线,会因被摄物体的不同深度区域,而采集不同的图像相位信息,然后通过运算单元将这种结构的变化换算成深度信息,以此来获得三维结构。简单来说就是,通过光学手段获取被拍摄物体的三维结构,再将获取到的信息进行更深入的应用。For easier understanding, let's first explain the principle of 3D structured light: The basic principle of 3D structured light technology is to project light with certain structural characteristics onto the object being photographed through a near-infrared laser, and then collect it with a special infrared camera. This light with a certain structure will collect different image phase information due to different depth areas of the object being photographed, and then convert the change of this structure into depth information through the calculation unit to obtain the three-dimensional structure. In simple terms, the three-dimensional structure of the object being photographed is obtained by optical means, and then the obtained information is applied more deeply.
深度图像也叫距离影像,是指将从图像采集器到场景中各点的距离(深度)值作为像素值的图像。A depth image, also called a distance image, refers to an image that uses the distance (depth) value from the image collector to each point in the scene as the pixel value.
点云:当一束激光照射到物体表面时,所反射的激光会携带方位、距离等信息。若将激光束按照某种轨迹进行扫描,便会边扫描边记录到反射的激光点信息,由于扫描极为精细,则能够得到大量的激 光点,因而就可形成激光点云。点云数据是对真实物体表面进行扫描采样获得的离散数据。Point cloud: When a laser beam hits the surface of an object, the reflected laser will carry information such as direction and distance. If the laser beam is scanned along a certain trajectory, the reflected laser point information will be recorded while scanning. Since the scanning is extremely precise, a large number of laser points can be obtained, thus forming a laser point cloud. Point cloud data is discrete data obtained by scanning and sampling the surface of a real object.
深度图像经过坐标转换可以计算为点云数据;有规则及必要信息的点云数据可以反算为深度图像。两者在一定条件下可以进行相互转化。Depth images can be calculated into point cloud data after coordinate transformation; point cloud data with rules and necessary information can be inversely calculated into depth images. The two can be transformed into each other under certain conditions.
图1-3所示,本方法中用到的装置包括头部搁置区1、3D结构光转动模组2及多个光学反射镜,其中头部搁置区1处设有吻合人脸下巴的下巴托3和用于抵接人脸额头的额托13,也可以用抵接头顶的类似下巴托3的支架来固定头部,下巴托3上可以安装乳胶垫等软垫,且可以自行更换,方便消毒,使用时用户4的下巴托3在下巴托3上,3D结构光转动模组2安装在转轴5上,且至少包括3D结构光转动模组2至少包括RGB摄像头和结构光摄像头。通过RGB摄像头和结构光摄像头实现人脸模型的建立。 As shown in Fig. 1-3 , the device used in the method includes a head rest area 1, a 3D structured light rotation module 2 and a plurality of optical reflectors, wherein the head rest area 1 is provided with a chin rest 3 that matches the chin of the human face and a forehead rest 13 that abuts against the forehead of the human face, and a bracket similar to the chin rest 3 that abuts against the top of the head can also be used to fix the head, and a latex pad or other soft cushion can be installed on the chin rest 3, and it can be replaced by itself for easy disinfection. When in use, the chin rest 3 of the user 4 is on the chin rest 3, and the 3D structured light rotation module 2 is installed on the rotating shaft 5, and at least includes the 3D structured light rotation module 2 at least includes an RGB camera and a structured light camera. The establishment of the face model is achieved through the RGB camera and the structured light camera.
具体地,3D结构光转动模组2位于头部搁置区1前方正中位置,通过转轴5驱动可以自身为轴心进行旋转,转轴5通过常见的步进电机6或伺服电机6驱动,电机6通过常见的电脑设备控制,3D结构光转动模组2也通过电脑设备控制,在电脑设备上生成人脸模型。Specifically, the 3D structured light rotating module 2 is located in the center in front of the head rest area 1, and can rotate with itself as the axis through the drive of the rotating shaft 5. The rotating shaft 5 is driven by a common stepper motor 6 or a servo motor 6, and the motor 6 is controlled by a common computer device. The 3D structured light rotating module 2 is also controlled by the computer device to generate a face model on the computer device.
具体地,光学反射镜设于3D结构光转动模组2周围,用于反射3D结构光转动模组2发射出的光线,光学反射镜为市面上可购买到的产品,其通过螺丝、胶水等方式安装在外部的固定支架上,在附图中未显示固定支架。Specifically, the optical reflector is arranged around the 3D structured light rotating module 2 to reflect the light emitted by the 3D structured light rotating module 2. The optical reflector is a commercially available product and is mounted on an external fixed bracket by means of screws, glue, etc. The fixed bracket is not shown in the attached figure.
优选地,每个所述光学反射镜的反射率大于等于90%。拥有 90%以上的反射率的光学反射镜可给3D结构光转动模组2提供更好的采集图像,显著提升其最终的人脸模型质量。Preferably, the reflectivity of each optical reflector is greater than or equal to 90%. Optical reflectors with a reflectivity of more than 90% can provide better captured images for the 3D structured light rotation module 2, significantly improving the quality of its final face model.
在本实施例中,还包括遮光罩7和设于遮光罩7内的照明灯8,通过所述遮光罩7遮挡头部搁置区1、3D结构光转动模组2及多个光学反射镜,其中遮光罩7可以是常见的黑布,可直接盖在本装置上,或者是其他不透光材料制成的罩壳,见 图1,将本装置做成半封闭结构,只露出供人头部进入的开口,还有用于抵住用户额头的额托13,当所述3D结构光转动模组2拍摄时,所述照明灯8关闭。主要是为了免去照明灯8对拍摄时产生影响。此方案,可有效地免去外部光源的影响,内部开启的照明灯8可根据需求打开。 In this embodiment, a light shield 7 and a lighting lamp 8 disposed in the light shield 7 are also included, and the light shield 7 is used to shield the head rest area 1, the 3D structured light rotating module 2 and a plurality of optical reflectors, wherein the light shield 7 can be a common black cloth, which can be directly covered on the device, or a cover made of other opaque materials, as shown in FIG1 , and the device is made into a semi-enclosed structure, exposing only an opening for the human head to enter, and a forehead support 13 for supporting the user's forehead. When the 3D structured light rotating module 2 is shooting, the lighting lamp 8 is turned off. This is mainly to avoid the influence of the lighting lamp 8 on the shooting. This solution can effectively avoid the influence of external light sources, and the internally turned lighting lamp 8 can be turned on as needed.
优选地,光学反射镜的数量为四个,分别对应位于头部搁置区1人脸的-45度、-15度、15度及45度位置,光学反射镜分为第一反射镜9、第二反射镜10、第三反射镜11及第四反射镜12,以3D结构光转动模组2与头部搁置区1的连接线为0度线,第一反射镜9与头部搁置区1的连接线与0度线的夹角为90度,第二反射镜10与头部搁置区1的连接线与0度线的夹角为45度,第三反射镜11与第二反射镜10以0度线镜像设置,第四反射镜12与第一反射镜9以0度线镜像设置。此方案,可快速地对每个反射镜的位置进行定位,方便每个反射镜的安装。按照 图3所示,相当于反射镜的安装角度为-90度,-45度,45度及90度,而电机6需要转动的角度依次为-45度,-105度,105度及45度,当电机6转到-45度时对应-90度的第一反射镜9,得到-45度人脸数据; Preferably, there are four optical reflectors, which correspond to the positions of -45 degrees, -15 degrees, 15 degrees and 45 degrees of the face of the head resting area 1, respectively. The optical reflectors are divided into a first reflector 9, a second reflector 10, a third reflector 11 and a fourth reflector 12. The connecting line between the 3D structured light rotation module 2 and the head resting area 1 is the 0-degree line, the angle between the connecting line between the first reflector 9 and the head resting area 1 and the 0-degree line is 90 degrees, the angle between the connecting line between the second reflector 10 and the head resting area 1 and the 0-degree line is 45 degrees, the third reflector 11 and the second reflector 10 are mirrored at the 0-degree line, and the fourth reflector 12 and the first reflector 9 are mirrored at the 0-degree line. This solution can quickly locate the position of each reflector, which is convenient for the installation of each reflector. As shown in FIG3 , the installation angles of the reflector are -90 degrees, -45 degrees, 45 degrees and 90 degrees, and the angles that the motor 6 needs to rotate are -45 degrees, -105 degrees, 105 degrees and 45 degrees, respectively. When the motor 6 rotates to -45 degrees, it corresponds to the first reflector 9 at -90 degrees, and the -45 degree face data is obtained;
当电机6转到-105度时对应-45度的第二反射镜10,得到-15度人脸数据;When the motor 6 rotates to -105 degrees, the second reflector 10 corresponding to -45 degrees is obtained, and the face data of -15 degrees is obtained;
当电机6转到105度时对应45度的第三反射镜11,得到15度人脸数据;When the motor 6 rotates to 105 degrees, the third reflector 11 corresponding to 45 degrees is obtained, and 15 degrees of face data are obtained;
当电机6转到45度时对应90度的第四反射镜12,得到45度人脸数据。When the motor 6 rotates to 45 degrees, the fourth reflector 12 corresponding to 90 degrees is rotated, and 45-degree face data is obtained.
因此在使用时,转轴5驱动3D结构光转动模组2依次转向第一反射镜9、第二反射镜10、第三反射镜11、第四反射镜12,启动转轴5转动转至-45度位置,依次打开3D结构光转动模组2的RGB光、PL、UV光进行拍照,得到一组图片和对应深度数据信息,再让转轴5转到-105度,105度及45度位置,分别依次打开RGB光、PL、UV光进行拍照,分别得到不同的3组图片和对应深度数据信息,在拍照时关闭照明灯8。此方案,只需要依次转动转轴5即可,即转轴5顺时针或逆时针转动,最后回到初始位置,每次转动都可通过电机6进行定位,也能够通过控制电机6的启停来停留进行拍摄操作,控制方便,实现难度低。Therefore, when in use, the shaft 5 drives the 3D structured light rotating module 2 to turn to the first reflector 9, the second reflector 10, the third reflector 11, and the fourth reflector 12 in sequence, and the shaft 5 is started to rotate to the -45 degree position, and the RGB light, PL, and UV light of the 3D structured light rotating module 2 are turned on in sequence to take pictures, and a group of pictures and corresponding depth data information are obtained, and then the shaft 5 is rotated to the -105 degree, 105 degree and 45 degree positions, and the RGB light, PL, and UV light are turned on in sequence to take pictures, and three different groups of pictures and corresponding depth data information are obtained respectively, and the lighting lamp 8 is turned off when taking pictures. In this solution, it is only necessary to rotate the shaft 5 in sequence, that is, the shaft 5 rotates clockwise or counterclockwise, and finally returns to the initial position. Each rotation can be positioned by the motor 6, and the motor 6 can also be controlled to stop and perform shooting operations. The control is convenient and the implementation difficulty is low.
图4所示,本利用光学反射合成3D人脸模型的方法包括以下步骤: As shown in FIG4 , the method for synthesizing a 3D face model using optical reflection includes the following steps:
S000、用户的脸置于装置上的头部搁置区1上,开启装置;S000, the user's face is placed on the head rest area 1 on the device, and the device is turned on;
S100、通过电机6驱动旋转3D结构光转动模组2从光学反射镜中获取包括相对正脸-15度,15度,-45度及45度的人脸三维图像及RGB彩色图像,其中光学反射镜数量为四个,分别对应各个角 度的人脸三维图像获取位置;S100, driving the rotating 3D structured light rotating module 2 by the motor 6 to obtain the three-dimensional face image and RGB color image including -15 degrees, 15 degrees, -45 degrees and 45 degrees relative to the front face from the optical reflector, wherein the number of the optical reflectors is four, corresponding to the three-dimensional face image acquisition position at each angle respectively;
与现有技术相比,无需将整个设备进行转动来对用户脸部进行拍摄,因此省去了结构庞大的驱动机构,只需要简单小巧的转动机构来转动3D结构光转动模组2即可,配合多面光线反射镜即可实现在设备不转动和人脸不转动的情况下对人脸各个角度进行拍摄,显著降低了设备成本,具有结构简单、操作方便、拍摄时间短、用户体验好的优点。Compared with the prior art, there is no need to rotate the entire device to capture the user's face, thus eliminating the need for a bulky driving mechanism. Only a simple and compact rotation mechanism is needed to rotate the 3D structured light rotation module 2. In conjunction with a multi-faceted light reflector, the face can be captured at all angles without rotating the device or the face, significantly reducing the cost of the equipment. It has the advantages of simple structure, easy operation, short shooting time, and good user experience.
与现有技术的固定单镜头方案相比,本方法可在同样大小的设备中实现生成人脸三维图像,而单镜头方案只能够生成二维的人脸图像,因此两者具有显著的区别,本方法得到的人脸三维图像为医美整形提供了更为良好的参考,可得到二维图像无法得到的信息,功能更加全面丰富。Compared with the fixed single-lens solution in the prior art, this method can generate three-dimensional facial images in a device of the same size, while the single-lens solution can only generate two-dimensional facial images. Therefore, there is a significant difference between the two. The three-dimensional facial images obtained by this method provide a better reference for medical cosmetic surgery, can obtain information that cannot be obtained from two-dimensional images, and have more comprehensive and rich functions.
与多镜头的方案相比,本方法只有一个镜头,显然成本更低,结构也更加简单。Compared with the multi-lens solution, this method only has one lens, which is obviously lower in cost and simpler in structure.
S200、通过3D结构光转动模组2和RGB彩色相机的内外参参数,生成对应各个角度的3D点云图像与彩色贴图,生成的算法为现有技术;S200, generating 3D point cloud images and color maps corresponding to various angles through the internal and external parameters of the 3D structured light rotation module 2 and the RGB color camera, and the generated algorithm is the existing technology;
S300、将-15度和15度的两张人脸三维图像中对应的点云图像进行合成,得到第一合成图像;S300, synthesizing the corresponding point cloud images in the two three-dimensional face images at -15 degrees and 15 degrees to obtain a first synthesized image;
-15度和15度图像包含人脸嘴唇、鼻子、眼睛,深度信息起伏较大,容易配准,所以将这两幅图像作为配准起点The -15 degree and 15 degree images contain the lips, nose, and eyes of the face. The depth information fluctuates greatly and is easy to align. Therefore, these two images are used as the starting point for alignment.
S400、将-45度的两张人脸三维图像中对应的点云图像进行合 成,得到第一合成图像,得到第二合成图像;S400, synthesizing the corresponding point cloud images in the two -45 degree three-dimensional images of the face to obtain a first synthesized image and a second synthesized image;
S500、将45度的两张人脸三维图像中对应的点云图像与第二合成图像进行合成,完整人脸3D图像;S500, synthesizing the corresponding point cloud images in the two 45-degree three-dimensional face images with the second synthesized image to obtain a complete 3D face image;
S600、获取3D结构光转动模组2输出的点云数据,对对应点云数据进行n的下采样,降低点云数据中点的个数,相当于过滤操作,提高合成速度;S600, obtaining the point cloud data output by the 3D structured light rotation module 2, performing down-sampling of the corresponding point cloud data by n, reducing the number of points in the point cloud data, which is equivalent to a filtering operation, and improving the synthesis speed;
S610、依据点云数据进行预设的3D结构光转动模组2位置和旋转角度的粗配准,具体是根据3D结构光转动模组2的拍摄距离与角度,将选中的点云数据进行平移与旋转,使选中的点云数据处于空间坐标(0,0)位置,完成粗配准;S610, performing a rough alignment of the preset position and rotation angle of the 3D structured light rotation module 2 according to the point cloud data, specifically, translating and rotating the selected point cloud data according to the shooting distance and angle of the 3D structured light rotation module 2, so that the selected point cloud data is at the spatial coordinate position (0, 0), and completing the rough alignment;
S620、根据点云数据中重合的点进行精配准,具体通过设置现有的配准算法的参数来进行;S620, performing precise registration according to the overlapping points in the point cloud data, specifically by setting parameters of an existing registration algorithm;
S630、过滤重合点完成合成拼接;S630, filtering the coincident points to complete the synthesis splicing;
S640、对拼接后的点云数据进行三维重建,生产三角形网格;S640, performing three-dimensional reconstruction on the spliced point cloud data to generate a triangular mesh;
S650、获取3D结构光转动模组2中RGB相机获得的贴图图片,将其贴到重建后的三维模型的网格上,生成3D人脸模型。通过3D结构光转动模组2中的结构光摄像头获取到的数据,我们能准确知道图像中每个点离摄像头距离,这样加上该点在RGB相机的2d图像中的(x,y)坐标,就能获取图像中每个点的三维空间坐标,对应网格可将人脸3D模型覆盖上彩色二维照片,得到逼真的3D人脸模型,可在电脑设备上转动观察,极大地方便医生进行观察,可以清晰知道鼻子的高度,眼睛的凹陷,皱纹的深浅,嘴形的大小和突出情况。S650, obtain the texture image obtained by the RGB camera in the 3D structured light rotation module 2, paste it on the grid of the reconstructed three-dimensional model, and generate a 3D face model. Through the data obtained by the structured light camera in the 3D structured light rotation module 2, we can accurately know the distance of each point in the image from the camera. In this way, adding the (x, y) coordinates of the point in the 2d image of the RGB camera, we can obtain the three-dimensional spatial coordinates of each point in the image. The corresponding grid can cover the 3D face model with a color two-dimensional photo to obtain a realistic 3D face model, which can be rotated and observed on a computer device, which greatly facilitates the doctor's observation and can clearly know the height of the nose, the depression of the eyes, the depth of the wrinkles, the size and protrusion of the mouth.
参见 图9,图中1-4分别代表第一反射镜9、第二反射镜10、第三反射镜11、第四反射镜12上采集到的图像,见 图5-8,而 图9中间为合并渲染后的图像,原图为彩色图像且具有较多面部细节,生成的人脸模型可以用于人脸皮肤的分析,如皱纹深浅、痘的大小等。 See Figure 9 , where 1-4 represent the images collected on the first reflector 9, the second reflector 10, the third reflector 11, and the fourth reflector 12, see Figures 5-8 , and the middle of Figure 9 is the merged and rendered image. The original image is a color image with more facial details. The generated face model can be used for analysis of facial skin, such as the depth of wrinkles, the size of acne, etc.
本方案采用结构光(散斑)的优点主要有:The advantages of using structured light (speckle) in this solution are mainly:
1)方案成熟,相机基线可以做的比较小,方便小型化。1) The solution is mature and the camera baseline can be made relatively small, which facilitates miniaturization.
2)资源消耗较低,单帧IR图就可计算出深度图,功耗低。2) Low resource consumption: a single frame of IR image can be used to calculate the depth map, with low power consumption.
3)主动光源,夜晚也可使用。3) Active light source, can be used at night.
4)在一定范围内精度高,分辨率高,分辨率可达1280x1024,帧率可达60FPS。4) High accuracy and resolution within a certain range, the resolution can reach 1280x1024 and the frame rate can reach 60FPS.
本发明未详述部分为现有技术,故本发明未对其进行详述。The parts not described in detail in the present invention are prior art, so the present invention does not describe them in detail.
可以理解的是,术语“一”应理解为“至少一”或“一个或多个”,即在一个实施例中,一个元件的数量可以为一个,而在另外的实施例中,该元件的数量可以为多个,术语“一”不能理解为对数量的限制。It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.
尽管本文较多地使用了头部搁置区1、3D结构光转动模组2、下巴托3、用户4、转轴5、电机6、遮光罩7、照明灯8、第一反射镜9、第二反射镜10、第三反射镜11、第四反射镜12、额托13等术语,但并不排除使用其它术语的可能性。使用这些术语仅仅是为了更方便地描述和解释本发明的本质;把它们解释成任何一种附加的限制都是与本发明精神相违背的。Although this article uses more terms such as head rest area 1, 3D structured light rotation module 2, chin rest 3, user 4, rotation axis 5, motor 6, light shield 7, lighting lamp 8, first reflector 9, second reflector 10, third reflector 11, fourth reflector 12, forehead rest 13, etc., it does not exclude the possibility of using other terms. These terms are used only to more conveniently describe and explain the essence of the present invention; interpreting them as any additional restrictions is contrary to the spirit of the present invention.
本发明不局限于上述最佳实施方式,任何人在本发明的启示下都可得出其他各种形式的产品,但不论在其形状或结构上作任何变化, 凡是具有与本申请相同或相近似的技术方案,均落在本发明的保护范围之内。The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that are the same or similar to those of the present application fall within the protection scope of the present invention.

Claims (9)

  1. 一种利用光学反射合成3D人脸模型的方法,其特征在于,包括以下步骤:通过旋转3D结构光转动模组从光学反射镜中获取多张相对正脸多个角度的人脸三维图像及RGB彩色图像,其中所述光学反射镜分别对应各个角度的人脸三维图像获取位置;依据人脸三维图像及RGB彩色图像生成对应各个角度的点云图像与彩色贴图;将相对正脸角度绝对值最小的两张人脸三维图像中对应的点云图像进行合成为第一合成图像;依次将剩余角度的人脸三维图像中对应的点云图像与第一合成图像合成;根据彩色贴图对合成后的图像进行渲染得到3D人脸模型;所述光学反射镜的数量为四个,分别对应相对正脸的-15度,15度,-45度及45度,其中合成步骤包括:将-15度和15度的两张人脸三维图像中对应的点云图像进行合成,得到第一合成图像;将-45度或45度的一张人脸三维图像中对应的点云图像与第一合成图像进行合成,得到第二合成图像;将剩下的人脸三维图像中对应的点云图像与第二合成图像进行合成,得到完整人脸3D图像;根据彩色贴图对完整人脸3D图像进行渲染得到3D人脸模型。A method for synthesizing a 3D face model using optical reflection, characterized in that it includes the following steps: obtaining multiple three-dimensional face images and RGB color images at multiple angles relative to the front face from an optical reflector by rotating a 3D structured light rotation module, wherein the optical reflector corresponds to the position of obtaining the three-dimensional face image at each angle; generating point cloud images and color maps corresponding to each angle based on the three-dimensional face image and the RGB color image; synthesizing the point cloud images corresponding to the two three-dimensional face images with the smallest absolute value of the angle relative to the front face into a first synthesized image; sequentially synthesizing the point cloud images corresponding to the three-dimensional face images at the remaining angles with the first synthesized image; and synthesizing the point cloud images corresponding to the three-dimensional face images according to the color maps. The synthesized image is rendered to obtain a 3D face model; the number of the optical reflectors is four, corresponding to -15 degrees, 15 degrees, -45 degrees and 45 degrees relative to the front face, respectively, wherein the synthesis step comprises: synthesizing the corresponding point cloud images in the two three-dimensional face images of -15 degrees and 15 degrees to obtain a first synthesized image; synthesizing the corresponding point cloud image in a three-dimensional face image of -45 degrees or 45 degrees with the first synthesized image to obtain a second synthesized image; synthesizing the corresponding point cloud image in the remaining three-dimensional face image with the second synthesized image to obtain a complete face 3D image; rendering the complete face 3D image according to the color map to obtain a 3D face model.
  2. 根据权利要求1所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,其中图像的合成步骤包括:获取3D结构光转动模组输出的点云数据;依据点云数据进行预设的3D结构光转动模组位置和旋转角度的粗配准;根据点云数据中重合的点进行精配准;过滤重合点完成合成拼接。According to a method for synthesizing a 3D face model using optical reflection according to claim 1, it is characterized in that the image synthesis step includes: obtaining point cloud data output by a 3D structured light rotation module; performing coarse alignment of a preset 3D structured light rotation module position and rotation angle based on the point cloud data; performing fine alignment based on overlapping points in the point cloud data; and filtering overlapping points to complete synthetic splicing.
  3. 根据权利要求2所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,渲染步骤包括:对拼接后的点云数据进行三维重建,生 产三角形网格;获取3D结构光转动模组中RGB相机获得的贴图图片,将其贴到重建后的三维模型的网格上,生成3D人脸模型。According to the method of synthesizing a 3D face model using optical reflection according to claim 2, it is characterized in that the rendering step includes: performing three-dimensional reconstruction on the spliced point cloud data to produce a triangular mesh; obtaining a texture image obtained by an RGB camera in a 3D structured light rotation module, and pasting it on the mesh of the reconstructed three-dimensional model to generate a 3D face model.
  4. 根据权利要求1所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,还包括3D结构光全脸成像装置,所述3D结构光全脸成像装置包括头部搁置区、3D结构光转动模组及多个光学反射镜;所述3D结构光转动模组位于头部搁置区前方正中位置,通过转轴驱动可以自身为轴心进行旋转;所述光学反射镜设于3D结构光转动模组周围。According to a method for synthesizing a 3D face model using optical reflection according to claim 1, it is characterized in that it also includes a 3D structured light full-face imaging device, the 3D structured light full-face imaging device includes a head placement area, a 3D structured light rotation module and a plurality of optical reflectors; the 3D structured light rotation module is located in the center in front of the head placement area, and can rotate with itself as the axis through a rotating shaft drive; the optical reflectors are arranged around the 3D structured light rotation module.
  5. 根据权利要求4所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,每个所述光学反射镜竖直设置,且分为第一反射镜、第二反射镜、第三反射镜及第四反射镜,分别对应位于头部搁置区人脸的-45度、-15度、15度及45度位置;以3D结构光转动模组中心与头部搁置区中心的连接线为0度线,第一反射镜和第四反射镜分别位于头部搁置区的左前方位置和右前方位置,第二反射镜和第三反射镜分别位于3D结构光转动模组的左前方位置和右前方位置;第三反射镜与第二反射镜以0度线为中心线镜像设置且互相垂直,第四反射镜与第一反射镜以0度线为中心线镜像设置且互相平行;当该转轴转到-45度时,3D结构光转动模组能够朝向并对应第一反射镜,从第一反射镜得到-45度人脸数据;当该转轴转到-105度时,3D结构光转动模组能够朝向并对应第二反射镜,从第二反射镜得到-15度人脸数据;当该转轴转到105度时,3D结构光转动模组能够朝向并对应第三反射镜,从第三反射镜得到15度人脸数据;当该转轴转到45度时, 3D结构光转动模组能够朝向并对应第四反射镜,从第四反射镜得到45度人脸数据。According to a method for synthesizing a 3D face model using optical reflection according to claim 4, it is characterized in that each of the optical reflectors is vertically arranged and is divided into a first reflector, a second reflector, a third reflector and a fourth reflector, which correspond to -45 degrees, -15 degrees, 15 degrees and 45 degrees of the face in the head rest area respectively; with the connecting line between the center of the 3D structured light rotation module and the center of the head rest area as the 0 degree line, the first reflector and the fourth reflector are respectively located at the left front position and the right front position of the head rest area, the second reflector and the third reflector are respectively located at the left front position and the right front position of the 3D structured light rotation module; the third reflector and The second reflector is mirrored with the 0-degree line as the center line and is perpendicular to each other. The fourth reflector and the first reflector are mirrored with the 0-degree line as the center line and are parallel to each other. When the rotating shaft is rotated to -45 degrees, the 3D structured light rotation module can face and correspond to the first reflector, and obtain -45-degree facial data from the first reflector. When the rotating shaft is rotated to -105 degrees, the 3D structured light rotation module can face and correspond to the second reflector, and obtain -15-degree facial data from the second reflector. When the rotating shaft is rotated to 105 degrees, the 3D structured light rotation module can face and correspond to the third reflector, and obtain 15-degree facial data from the third reflector. When the rotating shaft is rotated to 45 degrees, the 3D structured light rotation module can face and correspond to the fourth reflector, and obtain 45-degree facial data from the fourth reflector.
  6. 根据权利要求5所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,所述转轴驱动3D结构光转动模组依次转向第一反射镜、第二反射镜、第三反射镜、第四反射镜。According to the method for synthesizing a 3D face model using optical reflection according to claim 5, it is characterized in that the rotating shaft drives the 3D structured light rotation module to turn to the first reflector, the second reflector, the third reflector, and the fourth reflector in sequence.
  7. 根据权利要求6所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,还包括遮光罩和设于遮光罩内的照明灯,通过所述遮光罩遮挡头部搁置区、3D结构光转动模组及多个光学反射镜。According to claim 6, a method for synthesizing a 3D face model using optical reflection is characterized in that it also includes a light shield and a lighting lamp arranged in the light shield, and the light shield is used to shield the head rest area, the 3D structured light rotation module and the multiple optical reflectors.
  8. 根据权利要求5-7任意一项所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,所述头部搁置区设有下巴托和额托,通过下巴托和额托固定人脸。According to a method for synthesizing a 3D face model using optical reflection according to any one of claims 5 to 7, it is characterized in that the head rest area is provided with a chin rest and a forehead rest, and the face is fixed by the chin rest and the forehead rest.
  9. 根据权利要求1-7任意一项所述的一种利用光学反射合成3D人脸模型的方法,其特征在于,每个所述光学反射镜的反射率大于等于90%。The method for synthesizing a 3D face model using optical reflection according to any one of claims 1 to 7, characterized in that the reflectivity of each of the optical reflectors is greater than or equal to 90%.
PCT/CN2022/131102 2022-11-10 2022-11-10 Method for synthesizing 3d human face model by using optical reflection WO2024098322A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/131102 WO2024098322A1 (en) 2022-11-10 2022-11-10 Method for synthesizing 3d human face model by using optical reflection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/131102 WO2024098322A1 (en) 2022-11-10 2022-11-10 Method for synthesizing 3d human face model by using optical reflection

Publications (1)

Publication Number Publication Date
WO2024098322A1 true WO2024098322A1 (en) 2024-05-16

Family

ID=91031589

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/131102 WO2024098322A1 (en) 2022-11-10 2022-11-10 Method for synthesizing 3d human face model by using optical reflection

Country Status (1)

Country Link
WO (1) WO2024098322A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102206108B1 (en) * 2019-09-20 2021-01-21 광운대학교 산학협력단 A point cloud registration method based on RGB-D camera for shooting volumetric objects
KR20210058320A (en) * 2019-11-14 2021-05-24 한국전자통신연구원 Method for generation 3d model using single input image and apparatus using the same
CN113379893A (en) * 2021-05-27 2021-09-10 杭州小肤科技有限公司 Method for synthesizing 3D face model by utilizing optical reflection
CN113485058A (en) * 2021-06-18 2021-10-08 苏州小优智能科技有限公司 Compact high-precision three-dimensional face imaging device and three-dimensional face imaging method
CN113902852A (en) * 2021-10-18 2022-01-07 深圳追一科技有限公司 Face three-dimensional reconstruction method and device, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102206108B1 (en) * 2019-09-20 2021-01-21 광운대학교 산학협력단 A point cloud registration method based on RGB-D camera for shooting volumetric objects
KR20210058320A (en) * 2019-11-14 2021-05-24 한국전자통신연구원 Method for generation 3d model using single input image and apparatus using the same
CN113379893A (en) * 2021-05-27 2021-09-10 杭州小肤科技有限公司 Method for synthesizing 3D face model by utilizing optical reflection
CN113485058A (en) * 2021-06-18 2021-10-08 苏州小优智能科技有限公司 Compact high-precision three-dimensional face imaging device and three-dimensional face imaging method
CN113902852A (en) * 2021-10-18 2022-01-07 深圳追一科技有限公司 Face three-dimensional reconstruction method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US20230224446A1 (en) Method and apparatus for colour imaging a three-dimensional structure
JP7086330B2 (en) Generation of one or more luminosity edges to form a three-dimensional model of an object
CN106802138B (en) A kind of 3 D scanning system and its scan method
JP6544696B2 (en) Dental computed tomography
KR101608848B1 (en) System and method for generating a multi-dimensional image
US6594539B1 (en) Three-dimensional dental imaging method and apparatus having a reflective member
US4825263A (en) Optical method and apparatus for determining three-dimensional changes in facial contours
EP1941843B1 (en) Method and apparatus for colour imaging a three-dimensional structure
CN102679959B (en) Omnibearing 3D (Three-Dimensional) modeling system based on initiative omnidirectional vision sensor
JP2013538592A (en) Time-dependent three-dimensional measuring device using color optical impression
CN113379893B (en) Method for synthesizing 3D face model by utilizing optical reflection
JP2003202216A (en) Method, device, system and program for three-dimensional image processing
WO2024036784A1 (en) Ocular parameter evaluation apparatus
WO2024098322A1 (en) Method for synthesizing 3d human face model by using optical reflection
CN107479180A (en) A kind of micro- stereoscopic image acquisition device
CN106325035A (en) Application method of holographic intelligent glasses
CN113163188B (en) 3D structured light full-face imaging device combined with optical reflection imaging
WO2024098321A1 (en) 3d structured light full-face imaging apparatus combined with optical reflection imaging
KR20120009877A (en) Apparatus for measuring skin structure
CN118799491A (en) Portable surface scanning method and system based on laser radar
CN118505901A (en) Aligned three-dimensional face model sequence reconstruction method and system
TUNCAY et al. news No. 4 June 2000-Supplement
CN113902791A (en) Three-dimensional reconstruction method and device based on liquid lens depth focusing
CN113040713A (en) Oral cavity scanner and oral cavity scanning method
Keller et al. Switched pattern laser projection for real-time depth extraction and visualization through endoscopes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22964792

Country of ref document: EP

Kind code of ref document: A1