WO2023000703A1

WO2023000703A1 - Image acquisition system, three-dimensional reconstruction method and apparatus, device and storage medium

Info

Publication number: WO2023000703A1
Application number: PCT/CN2022/082917
Authority: WO
Inventors: 毋戈; 王海君; 陈相礼; 柴学智
Original assignee: 北京百度网讯科技有限公司
Priority date: 2021-07-23
Filing date: 2022-03-25
Publication date: 2023-01-26
Also published as: CN113724368B; CN113724368A

Abstract

The present disclosure relates to the technical field of computer vision, and provides an image acquisition system, a three-dimensional reconstruction method and apparatus, a device and a storage medium. A specific implementation solution is as follows: acquiring a first image set of a target object, wherein the first image set is collected by means of an image acquisition system when a plurality of supplementary light devices emit supplementary light patterns, any two first images in the first image set have an overlapping region, and each first image comprises part or all of a preset pattern; extracting feature points of each first image in the first image set; according to the feature points of each first image, performing feature point matching on at least two first images which have an overlapping region to obtain a matching feature point set; and performing three-dimensional reconstruction on the target object on the basis of the matching feature point set to obtain a three-dimensional reconstruction model. According to the present implementation solution, the success rate for three-dimensionally reconstructing objects which lack texture may be increased.

Description

Image acquisition system, three-dimensional reconstruction method, device, equipment and storage medium

cross reference

This patent application claims the priority of the Chinese patent application with the application number 202110836891.4 and the title of the invention "image acquisition system, three-dimensional reconstruction method, device, equipment and storage medium" submitted on July 23, 2021. The full text of the application Incorporated into this application by reference.

technical field

The present disclosure relates to the field of computer technology, in particular to the field of computer vision technology, and in particular to an image acquisition system, a three-dimensional reconstruction method, device, equipment, and storage medium.

Background technique

3D modeling is an important research field in computer graphics and computer vision. Its goal is to capture the 3D shape and appearance of objects and scenes with high quality to simulate 3D interaction and perception in digital spaces.

Contents of the invention

The disclosure provides an image acquisition system, a three-dimensional reconstruction method, a device, a device, and a storage medium.

According to the first aspect, an image acquisition system is provided, including: a shooting platform; a plurality of supplementary light devices arranged around the shooting platform, the relative positions of the plurality of supplementary light devices and the shooting platform are fixed, and the plurality of supplementary light devices are connected to the first A controller is connected, and is configured to issue a supplementary light pattern at the same time when receiving the supplementary light instruction sent by the first controller, and perform supplementary light on the target object fixed on the shooting platform; and at least one image that can rotate around the shooting platform Acquisition device, at least one image acquisition device is connected to the second controller, and is configured to respond to the acquisition instruction sent by the second controller to photograph the target object at a plurality of preset acquisition positions, so as to acquire images of the target object .

According to the second aspect, there is provided a three-dimensional reconstruction method, including: acquiring a first set of images of the target object, and the first set of images is sent through the image acquisition system as described in the first aspect when multiple supplementary light devices emit supplementary light patterns Acquisition, any two first images in the first image set have overlapping areas, and each first image includes part or all of the preset pattern; extracting feature points of each first image in the first image set; according to each first image performing feature point matching on at least two first images with overlapping regions to obtain a matching feature point set; based on the matching feature point set, performing 3D reconstruction on the target object to obtain a 3D reconstruction model.

According to a third aspect, there is provided a three-dimensional reconstruction device, including: a first image acquisition unit configured to acquire a first set of images of a target object, and the first set of images is multi-purpose through the image acquisition system described in the first aspect. Collecting when a supplementary light device emits a supplementary light pattern, any two first images in the first image set have overlapping areas, and each first image includes part or all of the preset pattern; the feature point extraction unit is configured to extract the first image. The feature points of each first image in an image set; the feature point matching unit is configured to perform feature point matching on at least two first images with overlapping regions according to the feature points of each first image, to obtain a matching feature point set The 3D reconstruction unit is configured to perform 3D reconstruction on the target object based on the matching feature point set to obtain a 3D reconstruction model.

According to a fourth aspect, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by at least one processor. Executed by a processor, so that at least one processor can execute the three-dimensional reconstruction method described in the second aspect.

According to a fifth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to make a computer execute the three-dimensional reconstruction method as described in the second aspect.

According to a sixth aspect, a computer program product includes a computer program, and when the computer program is executed by a processor, the three-dimensional reconstruction method as described in the second aspect is implemented.

According to the technology of the present disclosure, by introducing light textures and using an image acquisition system to acquire compliant images, the three-dimensional reconstruction process can be optimized, and objects lacking surface texture features can be effectively reconstructed in three dimensions at a relatively low cost.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

Figure 2a shows a top view of an image acquisition system according to the present disclosure;

Fig. 2b shows a schematic perspective view of an image acquisition system according to the present disclosure;

FIG. 3 is a flowchart of an embodiment of a three-dimensional reconstruction method according to the present disclosure;

FIG. 4 is a flowchart of another embodiment of a three-dimensional reconstruction method according to the present disclosure;

FIG. 5 is a comparison diagram of a target object, a reconstruction failure, and a successful reconstruction according to the three-dimensional reconstruction method of the present disclosure;

Fig. 6 is a schematic structural diagram of an embodiment of a three-dimensional reconstruction device according to the present disclosure;

FIG. 7 is a block diagram of an electronic device for implementing the three-dimensional reconstruction method of the embodiment of the present disclosure.

detailed description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that, in the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and embodiments.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the image acquisition system, three-dimensional reconstruction method or three-dimensional reconstruction apparatus of the present disclosure can be applied.

As shown in FIG. 1 , a system architecture 100 may include an image acquisition system 101 , a network 102 and a terminal device 103 . The network 102 is used as a medium for providing a communication link between the image acquisition system 101 and the terminal device 103 . Network 102 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

The image acquisition system 101 may be a system for acquiring images of a target object, which includes a plurality of hardware. Multiple pieces of hardware work together to capture compliant images. The image collection system 101 can send the multiple collected images to the terminal device 103 through the network 102 .

The terminal device 103 may process the multiple received images to perform 3D reconstruction to obtain a 3D reconstruction model. Various communication client applications, such as image processing applications, can be installed on the terminal device 103 . The user can view the above three-dimensional reconstruction model through the display screen of the terminal device 103 .

The terminal device 103 may be hardware or software. When the terminal device 103 is hardware, it may be various electronic devices, including but not limited to smartphones, tablet computers, e-book readers, vehicle-mounted computers, laptop computers, desktop computers, and the like. When the terminal device 103 is software, it can be installed in the electronic devices listed above. It can be implemented as a plurality of software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

It should be noted that the three-dimensional reconstruction method provided by the embodiment of the present disclosure is generally executed by the terminal device 103 . Correspondingly, the three-dimensional reconstruction apparatus is generally set in the terminal device 103 .

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Referring to FIG. 2a and FIG. 2b, FIG. 2a shows a top view of the image acquisition system according to the present disclosure, and FIG. 2b shows a perspective view of the image acquisition system according to the present disclosure. As shown in FIG. 2 a and FIG. 2 b , in this embodiment, the image acquisition system may include a plurality of supplementary light devices 201 , a photographing stage 202 and at least one image acquisition device 203 .

Wherein, a plurality of supplementary light devices 201 are arranged around the shooting platform 202 , and their relative positions to the shooting platform 202 are fixed. Specifically, the plurality of supplementary light devices 201 and the photographing stage 202 can be fixed by the same fixing device, so that the relative positions between the plurality of supplementary light devices 201 and the photographing stage 202 are fixed.

At least one image capture device 203 can rotate around the shooting platform 202 . Specifically, at least one image acquisition device 203 may be fixedly connected to the photographing platform 202 through a rotatable connection. Alternatively, the photographing platform 202 may be fixedly connected to at least one image acquisition device 203 through a rotatable connection. The relative rotation between at least one image capture device 203 and the capture platform 202 can be realized by rotating the capture platform 202, and the relative rotation between at least one image capture device 203 and the capture platform 202 can also be realized by rotating the at least one image capture device 203.

Multiple supplementary light devices 201 can be connected with the first controller (not shown in the figure). The first controller may send supplementary light instructions to multiple supplementary light devices 201 . The plurality of supplementary light devices 201 can send supplementary light patterns at the same time when or after receiving the supplementary light instruction sent by the first controller, so as to implement supplementary light for the target object fixed on the shooting stage. Multiple supplementary light devices 201 can emit preset patterns, or emit laser spots.

At least one image acquisition device 203 may be connected to a second controller (not shown in the figure). The second controller may send an acquisition instruction to at least one image acquisition device 203 . At least one image acquisition device 203 can photograph the target object fixed on the shooting platform at multiple preset acquisition positions when or after receiving the acquisition instruction sent by the second controller, so as to acquire the image of the target object .

The image acquisition system provided by the above-mentioned embodiments of the present disclosure can supplement light to a target object fixed on a shooting platform through a light supplement device, thereby providing light texture for objects lacking surface texture features, and improving the efficiency and accuracy of 3D reconstruction.

In some optional implementation manners of this embodiment, there is an overlapping area between the supplementary light ranges of two adjacent supplementary light devices among the plurality of supplementary light devices 201 .

In this implementation manner, the image acquisition system may include four supplementary light devices. The above four supplementary light devices can be distributed on a circular track around the shooting platform. Wherein, the supplementary light ranges of two adjacent supplementary light devices overlap. In this way, it is more convenient to perform feature point matching and three-dimensional reconstruction on images collected at the junction of two adjacent supplementary light devices.

In some optional implementations of this embodiment, the above-mentioned multiple supplementary light devices 201 may be evenly arranged around the shooting platform 202, and the supplementary light patterns emitted by the multiple supplementary light devices 201 cover the entire surface of the target object.

In this implementation manner, a plurality of supplementary light devices 201 may be uniformly arranged around the shooting platform 202 . If multiple supplementary light devices 201 are distributed on a circular track around the shooting platform, the central angles between two adjacent supplementary light devices are the same. Moreover, in order to ensure the integrity of the three-dimensional reconstruction of the target object, it is necessary to make the supplementary light patterns emitted by the plurality of supplementary light devices 201 cover the entire surface of the target object.

In some optional implementations of this embodiment, the above preset multiple collection positions include multiple collection position sets, and the collection positions in each collection position set are located on the same circular orbit.

In this implementation manner, the preset multiple collection locations may be divided into multiple collection location sets. Multiple collection locations in each collection location set are located on the same circular orbit. The distribution of different circular orbits differs from the relative height of the target object. By setting these collection positions at different heights and different angles, the collection of images of the entire surface of the target object can be realized.

In some optional implementation manners of this embodiment, a central angle between two adjacent collection positions in each collection position set is within a preset angle range.

In this implementation manner, in order to ensure a certain degree of overlap between images of two adjacent collection locations, the central angle between two adjacent collection locations in a single collection location set may be within a preset angle range. It can be understood that if the central angle between two adjacent collection positions is relatively large, the coincidence degree between the images of the two adjacent collection positions is small, making the effect of three-dimensional reconstruction of the target object poor. If the central angle between two adjacent collection positions is small, the coincidence degree between the images of the two adjacent collection positions will be relatively large, which greatly increases the amount of calculation. Therefore, in this implementation manner, the central angle between the two acquisition positions can be set within a preset angle range, thereby ensuring the rationality of the number of acquired images and the accuracy of the three-dimensional reconstruction effect.

In some optional implementation manners of this embodiment, the photographing table 202 and the plurality of supplementary light devices 201 may rotate synchronously.

In this implementation manner, in order to fully capture images from various angles of the target object fixed on the shooting platform 202, the relative positions between the shooting platform and each image acquisition device may be changed. Specifically, the change of the relative position can be realized by fixing the position of each image acquisition device and rotating the photographing table and multiple supplementary light devices. It is also possible to change the relative positions by fixing the positions of the shooting table and multiple supplementary light devices, and rotating each image acquisition device.

In some optional implementation manners of this embodiment, the colors of the supplementary light patterns emitted by the above-mentioned plurality of supplementary light devices 201 can be switched.

In this implementation manner, multiple supplementary light devices can emit visible light of various colors. The executive body can set the color of the supplementary light pattern according to the color of the surface of the target object. The color of the supplementary light image and the color of the surface of the target object have a higher contrast, thereby facilitating the extraction of feature points and further facilitating the three-dimensional reconstruction.

In some optional implementation manners of this embodiment, the supplementary light patterns emitted by the plurality of supplementary light devices 201 do not illuminate the optical lens of at least one image acquisition device 203 .

In this implementation, in order to prevent the supplementary light patterns emitted by multiple supplementary light devices 201 from irradiating the optical lens of each image acquisition device 203, causing the quality of the captured image to decline, it is necessary to make the supplementary light patterns issued by multiple supplementary light devices 201 The optical lens of each image capture device 203 cannot be irradiated.

In some optional implementations of this embodiment, each image acquisition device can properly adjust the angle of the image acquisition device when acquiring an image of a target object, so that the target object can always be kept at the center of the camera field of view, which can Reduce the error caused by the distortion of the image edge to the 3D reconstruction.

Continue referring to FIG. 3 , which shows a flow 300 of an embodiment of the three-dimensional reconstruction method according to the present disclosure. As shown in Figure 3, the method of this embodiment may include the following steps:

Step 301, acquire a first set of images of a target object.

In this embodiment, the execution subject of the three-dimensional reconstruction method (for example, the terminal device 103 shown in FIG. 1 ) may first acquire the first image set of the target object. Here, the first set of images may be collected by the image acquisition system described in the embodiment of FIG. 2a and FIG. 2b when multiple supplementary light devices turn on the supplementary light to irradiate the supplementary light pattern to the target object. At least two first images in the first image set have overlapping areas. Moreover, each first image includes part or all of the preset pattern.

Step 302, extract feature points of each first image in the first image set.

After obtaining the first image set, the execution subject may perform feature point extraction on each first image in the first image set. Here, the feature points may be points with special characteristics such as edge points and corner points. Specifically, the executive body can use existing feature point extraction algorithms to extract feature points, such as SRUF (Speeded Up Robust Features, accelerated robust features), SIFT (Scale-invariant feature transform, scale-invariant feature transform), AKAZE (KAZE's Accelerated version), deep learning network, etc.

Step 303, according to the feature points of each first image, perform feature point matching on at least two first images with overlapping areas, to obtain a matching feature point set.

After obtaining the feature points of each first image, the execution subject may perform feature point matching on at least two first images with overlapping areas to obtain matched feature points. These matched feature points are used to determine the relationship between at least two first images with overlapping regions, which is convenient for candidate grid reconstruction. When performing feature point matching, the execution subject can use but not limited to ANN (approximate nearest neighbor, approximate nearest neighbor search), Optical Flow and other methods for matching.

Step 304, based on the matching feature point set, perform 3D reconstruction on the target object to obtain a 3D reconstruction model.

After obtaining the set of matching feature points, the execution subject can use the set of matching feature points to perform 3D reconstruction. Specifically, the execution subject may determine the three-dimensional coordinates of each feature point in the matching feature point set, perform three-dimensional reconstruction according to each three-dimensional coordinate, and obtain a three-dimensional reconstruction model.

The 3D reconstruction method provided by the above-mentioned embodiments of the present disclosure, by introducing light textures, combining software and hardware, and optimizing the 3D reconstruction process, can perform effective 3D reconstruction on objects lacking surface texture features at a relatively low cost, and can significantly expand The scope of application of image-based 3D reconstruction algorithms.

Continue to refer to FIG. 4 , which shows a flow 400 of another embodiment of the three-dimensional reconstruction method according to the present disclosure. As shown in Figure 4, the method of this embodiment may include the following steps:

Step 401, acquire a first image set and a second image set of a target object.

In this embodiment, the above-mentioned first image set and second image set can be acquired by the image acquisition system described in the embodiment of Fig. 2a and Fig. 2b. Wherein, each first image in the first image set is in one-to-one correspondence with each second image in the second image set. The corresponding first and second images are captured at the same capture location. And the first image is collected when the supplementary light device emits the supplementary light pattern, and the second image is collected when the supplementary light device does not emit the supplementary light pattern.

Step 402, extract feature points of each first image in the first image set.

Step 403, according to the feature points of each first image, perform feature point matching on at least two first images with overlapping areas to obtain a matching feature point set.

Step 404, determining the three-dimensional coordinates of each matching feature point in the matching feature point set and the camera pose of each first image.

In this embodiment, the execution subject may determine the three-dimensional coordinates of each matching feature point in the matching feature point set after obtaining the matching feature point set. Specifically, the execution subject can first determine the coordinates of the feature points in the image coordinate system, and then combine the conversion relationship between the image coordinate system and the world coordinate system to convert the two-dimensional coordinates in the image coordinate system into the coordinates in the world coordinate system. 3D coordinates. At the same time, the execution subject can also determine the camera pose of each first image according to the position and angle of each image acquisition device, and the internal parameters and external parameters of the image acquisition device.

Step 405, calculate a dense point cloud according to the three-dimensional coordinates of each matching feature point and the camera pose of each first image.

After the execution subject determines the three-dimensional coordinates of each matching feature point, it can be used as a sparse point cloud. Combining the camera poses of each first image, the three-dimensional coordinates of each pixel in the first image can be determined to obtain a dense point cloud. Specifically, when calculating dense point clouds, the execution subject can use but not limited to CMVS (Clustering Multi-View Stereo, multi-view clustering), PMVS (Patch-based Multi-View Stereo, multi-view clustering based on patches) , Deep learning (MVS-Net, etc.) algorithm.

Step 406, perform surface mesh reconstruction according to the dense point cloud, and determine a 3D reconstruction model.

Surface mesh reconstruction can be performed through the above dense point cloud. Specifically, the execution subject can construct triangular patches according to the 3D coordinates of each point in the dense point cloud, and connect the triangular patches to obtain a 3D reconstruction model. The executive body can also use the Poisson reconstruction algorithm to perform three-dimensional reconstruction.

Step 407, determine a texture image according to the second image set.

In this embodiment, the execution subject may also determine the texture image according to the second image set. Specifically, the execution subject may use a deep learning algorithm to determine the texture image.

Step 408, fusing the texture image into the 3D reconstruction model.

Finally, the executive body can fuse the texture image to the 3D reconstructed model. When merging, the 3D reconstruction model can be fused according to the position corresponding to each pixel in the texture image.

The three-dimensional reconstruction method provided by the above-mentioned embodiments of the present disclosure can improve the reconstruction effect of an object lacking texture.

FIG. 5 shows images of target objects, images of failed reconstructions, and results of 3D reconstruction using the 3D reconstruction method of this embodiment. It can be seen from the comparison that the 3D reconstruction method of this embodiment effectively improves the success rate of 3D reconstruction of objects lacking in texture.

Further referring to FIG. 6, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a three-dimensional reconstruction device, which corresponds to the method embodiments shown in FIG. 2a and FIG. 2b. The device can be specifically applied to various electronic devices.

As shown in FIG. 6 , the 3D reconstruction device 600 of this embodiment includes: a first image acquisition unit 601 , a feature point extraction unit 602 , a feature point matching unit 603 and a 3D reconstruction unit 604 .

The first image acquiring unit 601 is configured to acquire a first image set of a target object. The first image set is collected by the image acquisition system described in the embodiment of Fig. 2a and Fig. 2b when a plurality of supplementary light devices emit supplementary light patterns, any two first images in the first image set have overlapping areas, and each second An image includes part or all of the preset pattern.

The feature point extraction unit 602 is configured to extract feature points of each first image in the first image set.

The feature point matching unit 603 is configured to perform feature point matching on at least two first images with overlapping regions according to the feature points of each first image, to obtain a matching feature point set.

The 3D reconstruction unit 604 is configured to perform 3D reconstruction of the target object based on the matching feature point set to obtain a 3D reconstruction model.

In some optional implementation manners of this embodiment, the apparatus 600 may further include not shown in FIG. 6 : a second image acquisition unit, a texture image determination unit, and a texture image fusion unit.

The second image acquisition unit is configured to acquire a second image set, and the second image set is acquired by the image acquisition system as described in the embodiment of Fig. 2a and Fig. 2b when the plurality of supplementary light devices do not emit supplementary light patterns.

The texture image determining unit is configured to determine the texture image according to the second set of images.

The texture image fusion unit is configured to fuse the texture image into the three-dimensional reconstruction model.

In some optional implementations of this embodiment, the 3D reconstruction unit 604 may be further configured to: determine the 3D coordinates of each matching feature point in the matching feature point set and the camera pose of each first image; The 3D coordinates of the first image and the camera pose of each first image are calculated to calculate the dense point cloud; the surface mesh is reconstructed according to the dense point cloud to determine the 3D reconstruction model.

It should be understood that the units 601 to 604 recorded in the three-dimensional reconstruction apparatus 600 respectively correspond to the steps in the method described with reference to FIG. 3 . Therefore, the operations and features described above for the three-dimensional reconstruction method are also applicable to the device 600 and the units contained therein, and will not be repeated here.

In the technical solution of this disclosure, the acquisition, storage and application of user personal information involved are in compliance with relevant laws and regulations, and do not violate public order and good customs.

According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

FIG. 7 shows a block diagram of an electronic device 700 for performing a three-dimensional reconstruction method according to an embodiment of the present disclosure. Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 7 , an electronic device 700 includes a processor 701 that can execute according to a computer program stored in a read-only memory (ROM) 702 or loaded from a memory 708 into a random access memory (RAM) 703. Various appropriate actions and treatments. In the RAM 703, various programs and data necessary for the operation of the electronic device 700 can also be stored. The processor 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An I/O interface (input/output interface) 705 is also connected to the bus 704 .

Multiple components in the electronic device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a memory 708, such as a magnetic disk, an optical disk, etc. ; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the electronic device 700 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Processor 701 may be various general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 701 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 701 executes various methods and processes described above, such as a three-dimensional reconstruction method. For example, in some embodiments, the three-dimensional reconstruction method may be implemented as a computer software program tangibly embodied on a machine-readable storage medium, such as memory 708. In some embodiments, part or all of the computer program can be loaded and/or installed on the electronic device 700 via the ROM 702 and/or the communication unit 709. When the computer program is loaded into RAM 703 and executed by processor 701, one or more steps of the three-dimensional reconstruction method described above can be performed. Alternatively, in other embodiments, the processor 701 may be configured in any other appropriate way (for example, by means of firmware) to execute the three-dimensional reconstruction method.

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.

Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. The above program code can be packaged into a computer program product. These program codes or computer program products may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the program codes, when executed by the processor 701, make the flow diagrams and/or block diagrams specified The function/operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The machine-readable storage medium may be a machine-readable signal storage medium or a machine-readable storage medium. A machine-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.

The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.

A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the problem of traditional physical host and VPS service ("Virtual Private Server", or "VPS") Among them, there are defects such as difficult management and weak business scalability. The server can also be a server of a distributed system, or a server combined with a blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of the present disclosure can be achieved, no limitation is imposed herein.

The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall fall within the protection scope of the present disclosure.

Claims

An image acquisition system, comprising:

shooting stage;

A plurality of supplementary light devices arranged around the shooting platform, the relative positions of the plurality of supplementary light devices and the shooting platform are fixed, the plurality of supplementary light devices are connected to the first controller, and are configured to receive At the same time as the supplementary light instruction sent by the first controller, a supplementary light pattern is issued to supplement the light on the target object fixed on the shooting platform; and

At least one image acquisition device that can rotate around the shooting platform, the at least one image acquisition device is connected to the second controller, and is configured to respond to the acquisition instruction sent by the second controller, within a preset number of The target object is photographed at the collection position to collect an image of the target object.
The image acquisition system according to claim 1, wherein there is an overlapping area between the supplementary light ranges of adjacent supplementary light devices among the plurality of supplementary light devices.
The image acquisition system according to claim 1, wherein the preset plurality of acquisition positions includes a plurality of acquisition position sets, and the acquisition positions in each acquisition position set are located on the same circular orbit.
The image acquisition system according to claim 3, wherein the central angle between two adjacent acquisition positions in each acquisition position set is within a preset angle range.
The image acquisition system according to claim 1, wherein the plurality of supplementary light devices are evenly arranged around the shooting platform, and the supplementary light patterns emitted by the plurality of supplementary light devices cover the target object full surface.
The image acquisition system according to claim 1, wherein the shooting table and the plurality of supplementary light devices can rotate synchronously.
The image acquisition system according to claim 1, wherein the colors of the supplementary light patterns emitted by the plurality of supplementary light devices are switchable.
The image acquisition system according to claim 1, wherein the supplementary light patterns emitted by the plurality of supplementary light devices do not illuminate the optical lens of the at least one image acquisition device.
A three-dimensional reconstruction method, comprising:

Acquiring a first set of images of the target object, the first set of images is collected by the image acquisition system according to any one of claims 1-8 when multiple supplementary light devices emit supplementary light patterns, the first collection of images Any two of the first images in there are overlapping areas, and each first image includes part or all of the preset pattern;

extracting feature points of each first image in the first image set;

According to the feature points of each first image, perform feature point matching on at least two first images with overlapping regions to obtain a matching feature point set;

Based on the set of matching feature points, three-dimensional reconstruction is performed on the target object to obtain a three-dimensional reconstruction model.
The three-dimensional reconstruction method according to claim 9, wherein the method further comprises:

Acquiring a second set of images, the second set of images is collected by the image acquisition system according to any one of claims 1-8 when a plurality of supplementary light devices do not emit a supplementary light pattern;

determining a texture image according to the second set of images;

The texture image is fused to the three-dimensional reconstruction model.
The three-dimensional reconstruction method according to claim 9, wherein said performing three-dimensional reconstruction on said target object based on said matching feature point set to obtain a three-dimensional reconstruction model comprises:

Determining the three-dimensional coordinates of each matching feature point in the set of matching feature points and the camera pose of each first image;

Calculate the dense point cloud according to the three-dimensional coordinates of each matching feature point and the camera pose of each first image;

Perform surface grid reconstruction according to the dense point cloud to determine a three-dimensional reconstruction model.
A three-dimensional reconstruction device, comprising:

The first image acquisition unit is configured to acquire a first image collection of the target object, and the first image collection emits supplementary light patterns on multiple supplementary light devices through the image acquisition system according to any one of claims 1-8 Acquisition at the same time, any two first images in the first image set have overlapping areas, and each first image includes part or all of the preset pattern;

a feature point extraction unit configured to extract feature points of each first image in the first image set;

The feature point matching unit is configured to perform feature point matching on at least two first images with overlapping regions according to the feature points of each first image, to obtain a matching feature point set;

The 3D reconstruction unit is configured to perform 3D reconstruction on the target object based on the matching feature point set to obtain a 3D reconstruction model.
The three-dimensional reconstruction device according to claim 12, wherein the three-dimensional reconstruction device further comprises:

The second image acquisition unit is configured to acquire a second set of images, the second set of images is acquired by the image acquisition system according to any one of claims 1-8 when a plurality of supplementary light devices do not emit supplementary light patterns ;

a texture image determining unit configured to determine a texture image according to the second set of images;

A texture image fusion unit configured to fuse the texture image to the three-dimensional reconstruction model.
The three-dimensional reconstruction device according to claim 12, wherein the three-dimensional reconstruction unit is further configured to:

Determining the three-dimensional coordinates of each matching feature point in the set of matching feature points and the camera pose of each first image;

Calculate the dense point cloud according to the three-dimensional coordinates of each matching feature point and the camera pose of each first image;

Perform surface grid reconstruction according to the dense point cloud to determine a three-dimensional reconstruction model.
An electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 9-11. 3D reconstruction method.
A non-transitory computer-readable storage medium storing computer instructions, the computer instructions are used to make the computer execute the three-dimensional reconstruction method described in any one of claims 9-11.
A computer program product comprising a computer program which, when executed by a processor, implements the three-dimensional reconstruction method according to any one of claims 9-11.