CN112653881A - 3D image self-adjusting method and system and 3D image depth processing method and system - Google Patents
3D image self-adjusting method and system and 3D image depth processing method and system Download PDFInfo
- Publication number
- CN112653881A CN112653881A CN202011018176.1A CN202011018176A CN112653881A CN 112653881 A CN112653881 A CN 112653881A CN 202011018176 A CN202011018176 A CN 202011018176A CN 112653881 A CN112653881 A CN 112653881A
- Authority
- CN
- China
- Prior art keywords
- image
- processing module
- signal processing
- depth
- shot object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/167—Synchronising or controlling image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a 3D image self-adjusting method and system and a 3D image depth processing method and system, comprising a driving circuit, a vertical cavity surface emitting laser, a TOF sensor, a DMA controller, a RAM1 and an image signal processing module, wherein the driving circuit is connected with the vertical cavity surface emitting laser and drives the vertical cavity surface emitting laser to project light onto a shot object; the TOF sensor collects a 3D image of a shot object, the 3D image is moved into the RAM1 through an AHB system bus by using the DMA controller, the image signal processing module carries out statistical analysis according to the 3D image in the RAM1, the quality of the 3D image is calculated, and the parameters of the TOF sensor or the power of a driving circuit are adjusted according to the image definition index. The invention can collect 3D image data meeting the standard, avoid the dependence on the main control CPU and avoid high power consumption and high cost brought by a high-performance CPU.
Description
Technical Field
The invention relates to a 3D image self-adjusting method and system and a 3D image depth processing method and system, and belongs to the technical field of 3D image processing.
Background
The existing 3D TOF module directly transmits the collected original image data to a main control CPU for processing, and the 3D data, especially the original data volume of the TOF is large, so that the main control CPU is required to quickly receive the data, and simultaneously the main control CPU is required to send a command to dynamically adjust parameters such as the transmitting power of the 3D TOF module after quickly analyzing the TOF original data, so that the image data processing method has high requirements on the performance of the main control CPU, the problems are not great in a high-performance main control application scene taking a mobile phone as an example, and meanwhile, the high-performance requirements of the CPU bring about power consumption increase and cost increase.
Disclosure of Invention
The invention provides a 3D image self-adjusting method and system and a 3D image depth processing method and system in order to overcome the defects in the prior art.
The invention can be realized by adopting the following technical scheme:
a 3D image self-adjustment method comprising the steps of:
s1, putting light on the shot object, and carrying out image acquisition on the shot object to form a 3D image and storing the image;
s2, carrying out statistical analysis on the 3D image, calculating the quality of the 3D image, and adjusting the intensity and the acquisition parameters of light according to the image definition index by the image signal processing module.
Preferably, the 3D image self-adjusting method further comprises the steps of:
and S3, temperature drift compensation is carried out on the intensity of the light and the acquisition parameters according to the real-time environment temperature.
A3D image self-adjusting system comprises a driving circuit, a vertical cavity surface emitting laser, a TOF sensor, a DMA controller, a RAM1 and an image signal processing module,
the driving circuit is connected with the vertical cavity surface emitting laser and drives the vertical cavity surface emitting laser to project light onto a shot object;
the TOF sensor collects a 3D image of a shot object, the 3D image is moved into a RAM1 by a DMA controller through an AHB system bus,
the image signal processing module performs statistical analysis according to the 3D image in the RAM1, calculates the quality of the 3D image, and adjusts the parameters of the TOF sensor or the power of the driving circuit according to the image definition index.
Preferably, the 3D image self-adjusting system further includes a temperature sensor, the temperature sensor is connected to the image signal processing module, and transmits the real-time ambient temperature to the image signal processing module, and the image signal processing module performs temperature drift compensation on the emission optical device and the TOF sensor according to the real-time ambient temperature.
A3D image depth processing method comprises the following steps:
s1, putting light on the shot object, and carrying out image acquisition on the shot object to form a 3D image and storing the image;
s2, carrying out statistical analysis on the 3D image, calculating the quality of the 3D image, and adjusting the intensity and the acquisition parameters of light according to the image definition index by the image signal processing module.
And S3, performing depth calculation on the 3D image by using an algorithm accelerator to obtain depth data, restoring an infrared image and storing the infrared image.
Preferably, in the step S2, the temperature drift compensation is performed on the light intensity and the acquisition parameters according to the real-time ambient temperature.
A3D image depth processing system comprises the 3D image self-adjusting system and a depth algorithm calculating device, wherein the depth algorithm calculating device comprises a CPU Core, an algorithm accelerator and a RAM2, the algorithm accelerator and the RAM2 are electrically connected with the CPU Core through an AHB system bus, an image information processing module is electrically connected with the CPU Core, the CPU Core controls the algorithm accelerator to perform depth calculation on a 3D image of the RAM1 to obtain depth data, and the depth data is restored to form an infrared image and stored in the RAM 2.
Preferably, the algorithmic accelerators include trigonometric function accelerators, convolution accelerators and Hamming distance acceleration engines.
Compared with the prior art, the invention has the beneficial effects that: according to the invention, the image signal processing module is used for analyzing and counting the 3D image acquired by the TOF sensor, and the parameters of the TOF sensor or the power of the driving circuit are adjusted according to the image definition index, so that on one hand, 3D image data meeting the standard can be acquired, on the other hand, dependence on a main control CPU can be avoided, and high power consumption and high cost caused by a high-performance CPU are avoided; in addition, the CPU Core control algorithm accelerator is used for carrying out depth algorithm calculation on the 3D image to obtain depth data, dependence of the algorithm on CPU dominant frequency is avoided, the use threshold of the CPU is effectively reduced, the power consumption of the CPU is reduced, and the cost of the whole system is reduced.
Drawings
FIG. 1 is a block diagram of the architecture of the 3D image depth processing system of the present invention;
FIG. 2 is a workflow diagram of the 3D image depth processing system of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings.
Example 1
As shown in fig. 1 and 2, the 3D image self-adjustment system of the present embodiment includes a drive circuit 1, a vertical cavity surface emitting laser (VSCEL)2, a TOF sensor 3, a DMA controller 4, a RAM 15, and an image signal processing module (ISP) 6; the driving circuit 1 is connected with the vertical cavity surface emitting laser 2 and drives the vertical cavity surface emitting laser 2 to project light onto a shot object; the TOF sensor 3 collects a 3D image of a shot object, the 3D image is moved into the RAM 15 through an AHB system bus by using the DMA controller 4; the image signal processing module 6 performs statistical analysis according to the 3D image in the RAM 15, calculates the quality of the 3D image, and adjusts the parameters of the TOF sensor 3 or the power of the driving circuit 1 according to the image sharpness index. The 3D image self-adjusting system further comprises a temperature sensor 7, the temperature sensor 7 is connected with the image signal processing module 6 and transmits the real-time environment temperature to the image signal processing module 6, and the image signal processing module 6 performs temperature drift compensation on the vertical cavity surface emitting laser 2 and the TOF sensor 3 according to the real-time environment temperature. Wherein, the TOF sensor 3 selects 320 pixels by 240 pixels and above area array; the vertical cavity surface emitting laser 2 selects 940nm wavelength, and the divergence angle selects 72 degrees x55 degrees, 3W and above power; the drive circuit, the DMA controller, the RAM1, the image signal processing module, and the like employ common components.
The working process of the 3D image self-adjusting system is as follows:
1. the vertical cavity surface emitting laser throws light on a shot object, the TOF sensor collects images of the shot object to form a 3D image, and the 3D image is moved into the RAM1 by the DMA controller through an AHB system bus;
2. the image signal processing module carries out statistical analysis on the 3D image, calculates the quality of the 3D image, and adjusts the intensity and the acquisition parameters of light according to the image definition index.
In the process, according to the real-time environment temperature acquired by the temperature sensor, the image signal processing module performs temperature drift compensation on the light intensity of the vertical cavity surface emitting laser and the acquisition parameters of the TOF sensor according to the real-time environment temperature.
Example 2
On the basis of embodiment 1, the 3D image depth processing system of embodiment 2 includes the above-mentioned 3D image self-adjustment system, and a depth algorithm calculation device, the depth algorithm calculation device includes a CPU Core 8, an algorithm accelerator 9, and a RAM 210, the algorithm accelerator 9 and the RAM 210 are electrically connected to the CPU Core 8 through an AHB system bus, the image information processing module 6 is electrically connected to the CPU Core 8, and the CPU Core 8 controls the operation of the image information processing module 6; the CPU Core 8 control algorithm accelerator 9 performs depth algorithm calculation on the 3D image of the RAM 15 to obtain depth data, restores an infrared image and stores the infrared image in the RAM 210.
The CPU Core 8 adopts a common chip Core with lower power, and an SCM clock generator of an auxiliary control circuit adopts a built-in ring oscillation design so as to achieve the purpose of minimizing peripheral devices of the system. The SCM clock generator and the clock generating unit are mainly composed of analog circuits, and the generated clock is used for providing system and peripheral work. The clock generation unit integrates 1 internal oscillator and 1 phase-locked loop, the internal oscillator is high-frequency high-precision ring oscillation HFROSC respectively, and the 1 phase-locked loop can be connected with a 12MHz external crystal oscillator in a hanging mode. The high-frequency high-precision ring oscillation HFROSC is mainly used as a main clock source for system and peripheral work, and the nominal values of output frequencies are four stages, namely 12MHz, 48MHz, 96MHz and 144 MHz; calibration can be carried out at each stage of frequency, calibration values are given in the Wafer test stage and recorded in the embedded OTP calibration domain, and the calibration values are written into the calibration register at the time of power-on boot. After the HFROSC is calibrated in a factory, the precision reaches +/-1.5 percent, and the HFROSC can be dynamically calibrated through an integrated ring oscillation automatic calibration unit so as to meet the requirement of higher clock precision. The external 12MHz crystal oscillator is used for generating an accurate 12MHz clock for system and peripheral work, and can also be used as a reference clock of the ring oscillation automatic calibration unit to calibrate high-frequency high-precision ring oscillation HFROSC. The peripheral unit of the CPU mainly comprises MIPI CSI-2 sending and receiving, and comprises universal CVP, SPI, I2C, UART, GPIO and other interface systems, after the system is started from the ROM, the algorithm program is loaded and operated through an external AP, the SQI interface is preferably reserved, and the algorithm program can be stored and operated through an external serial flash. The algorithm accelerator preferably mainly comprises a trigonometric function accelerator, a convolution accelerator and a Hamming distance acceleration engine, corresponding assembly instructions are designed, the depth algorithm is optimized, and a large number of convolution operations are called and executed at high speed through the assembly instructions. The convolution accelerator provides hardware acceleration processing capacity for convolution operation in an image processing algorithm, and can support two modes of two-dimensional convolution and one-dimensional convolution, wherein the one-dimensional convolution can support convolution operation in the transverse direction and the longitudinal direction.
The working process of the 3D image depth processing system is as follows:
1. the vertical cavity surface emitting laser throws light on a shot object, the TOF sensor collects images of the shot object to form a 3D image, and the 3D image is moved into the RAM1 by the DMA controller through an AHB system bus;
2. the image signal processing module carries out statistical analysis on the 3D image, calculates the quality of the 3D image, and adjusts the intensity and the acquisition parameters of light according to the image definition index.
In the process, according to the real-time environment temperature acquired by the temperature sensor, the image signal processing module performs temperature drift compensation on the light intensity of the vertical cavity surface emitting laser and the acquisition parameters of the TOF sensor according to the real-time environment temperature;
3. the CPU controls an algorithm accelerator to perform depth algorithm calculation on the 3D image of the RAM1 to obtain depth data, and the depth data are restored to obtain an infrared image and stored in the RAM 2.
The present invention has been described in connection with the preferred embodiments, but the present invention is not limited to the embodiments disclosed above, and is intended to cover various modifications, equivalent combinations, which are within the spirit of the invention.
Claims (8)
1. A3D image self-adjusting method is characterized by comprising the following steps: the method comprises the following steps:
s1, putting light on the shot object, and carrying out image acquisition on the shot object to form a 3D image and storing the image;
s2, carrying out statistical analysis on the 3D image, calculating the quality of the 3D image, and adjusting the intensity and the acquisition parameters of light according to the image definition index by the image signal processing module.
2. 3D image self-adjustment method according to claim 1, characterized in that: further comprising the steps of:
and S3, temperature drift compensation is carried out on the intensity of the light and the acquisition parameters according to the real-time environment temperature.
3. A3D image depth processing method is characterized in that: the method comprises the following steps:
s1, putting light on the shot object, and carrying out image acquisition on the shot object to form a 3D image and storing the image;
s2, carrying out statistical analysis on the 3D image, calculating the quality of the 3D image, and adjusting the intensity and the acquisition parameters of light according to the image definition index by the image signal processing module.
And S3, performing depth calculation on the 3D image by using an algorithm accelerator to obtain depth data, restoring an infrared image and storing the infrared image.
4. The 3D image depth processing method according to claim 3, characterized in that: in step S2, temperature drift compensation is performed on the intensity of light and the acquisition parameter according to the real-time ambient temperature.
5. A 3D image self-adjustment system characterized by: comprises a driving circuit, a vertical cavity surface emitting laser, a TOF sensor, a DMA controller, a RAM1 and an image signal processing module,
the driving circuit is connected with the vertical cavity surface emitting laser and drives the vertical cavity surface emitting laser to project light onto a shot object;
the TOF sensor collects a 3D image of a shot object, the 3D image is moved into a RAM1 by a DMA controller through an AHB system bus,
the image signal processing module performs statistical analysis according to the 3D image in the RAM1, calculates the quality of the 3D image, and adjusts the parameters of the TOF sensor or the power of the driving circuit according to the image definition index.
6. The 3D image self-adjustment system according to claim 5, characterized in that: the 3D image self-adjusting system further comprises a temperature sensor, the temperature sensor is connected with the image signal processing module and transmits the real-time environment temperature to the image signal processing module, and the image signal processing module carries out temperature drift compensation on the transmitting laser and the TOF sensor according to the real-time environment temperature.
7. A 3D image depth processing system characterized by: the 3D image self-adjusting system comprising the 3D image of claim 5 or 6, and a depth algorithm calculating device comprising a CPU Core, an algorithm accelerator and a RAM2, wherein the algorithm accelerator and the RAM2 are electrically connected with the CPU Core through an AHB system bus, the image information processing module is electrically connected with the CPU Core, and the CPU Core controls the algorithm accelerator to perform depth calculation on the 3D image of the RAM1 to obtain depth data, restore an infrared image and store the depth data in the RAM 2.
8. The 3D image depth processing system according to claim 7, wherein: the algorithm accelerator includes a trigonometric function accelerator, a convolution accelerator, and a hamming distance acceleration engine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011018176.1A CN112653881B (en) | 2020-09-24 | 2020-09-24 | 3D image self-adjusting method and system and 3D image depth processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011018176.1A CN112653881B (en) | 2020-09-24 | 2020-09-24 | 3D image self-adjusting method and system and 3D image depth processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112653881A true CN112653881A (en) | 2021-04-13 |
| CN112653881B CN112653881B (en) | 2023-05-09 |
Family
ID=75346461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011018176.1A Active CN112653881B (en) | 2020-09-24 | 2020-09-24 | 3D image self-adjusting method and system and 3D image depth processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112653881B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726247A (en) * | 2008-10-31 | 2010-06-09 | 鸿富锦精密工业（深圳）有限公司 | Light irradiation regulating method and computer system thereof |
| US20160377417A1 (en) * | 2015-06-23 | 2016-12-29 | Hand Held Products, Inc. | Dual-projector three-dimensional scanner |
| CN110719454A (en) * | 2019-10-15 | 2020-01-21 | 浙江晶鲸科技有限公司 | Self-adaptive driving device and driving method for electric control infrared light emitter |
| CN110836647A (en) * | 2018-08-17 | 2020-02-25 | 鉴微科技股份有限公司 | Three-dimensional scanning system |
| CN111352098A (en) * | 2020-02-21 | 2020-06-30 | 深圳奥比中光科技有限公司 | Temperature drift calibration method and system for ToF camera |
| CN111462707A (en) * | 2020-04-27 | 2020-07-28 | 北京航空航天大学 | Modeling compensation-based temperature drift suppression method for liquid crystal power stabilizing system |
-
2020
- 2020-09-24 CN CN202011018176.1A patent/CN112653881B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101726247A (en) * | 2008-10-31 | 2010-06-09 | 鸿富锦精密工业（深圳）有限公司 | Light irradiation regulating method and computer system thereof |
| US20160377417A1 (en) * | 2015-06-23 | 2016-12-29 | Hand Held Products, Inc. | Dual-projector three-dimensional scanner |
| CN110836647A (en) * | 2018-08-17 | 2020-02-25 | 鉴微科技股份有限公司 | Three-dimensional scanning system |
| CN110719454A (en) * | 2019-10-15 | 2020-01-21 | 浙江晶鲸科技有限公司 | Self-adaptive driving device and driving method for electric control infrared light emitter |
| CN111352098A (en) * | 2020-02-21 | 2020-06-30 | 深圳奥比中光科技有限公司 | Temperature drift calibration method and system for ToF camera |
| CN111462707A (en) * | 2020-04-27 | 2020-07-28 | 北京航空航天大学 | Modeling compensation-based temperature drift suppression method for liquid crystal power stabilizing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112653881B (en) | 2023-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3422699B1 (en) | Camera module and control method | |
| US11182917B2 (en) | Stereo camera depth determination using hardware accelerator | |
| US11887321B2 (en) | Three-dimensional scanning image acquisition and processing methods and apparatuses, and three-dimensional scanning device | |
| CN110645998B (en) | Dynamic object-free map segmentation establishing method based on laser point cloud | |
| CN106296716A (en) | The power regulating method of light source, depth measurement method and device | |
| CN110191266B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN107220954B (en) | 3D scanning system based on mobile terminal | |
| CN112153306B (en) | Image acquisition system, method and device, electronic equipment and wearable equipment | |
| US11733771B2 (en) | High-precision anti-interference VR system and operation method | |
| CN110491316A (en) | A kind of projector and its method for controlling projection | |
| CN110007289B (en) | Motion artifact reduction method based on time-of-flight depth camera | |
| EP3007359A1 (en) | 3d image sensor module and electronic apparatus including the same | |
| CN109008909B (en) | Low-power-consumption capsule endoscope image acquisition and three-dimensional reconstruction system | |
| CN114627249A (en) | Three-dimensional scanning system and three-dimensional scanning method | |
| CN114862828A (en) | Light spot searching method and device, computer readable medium and electronic equipment | |
| CN112653881B (en) | 3D image self-adjusting method and system and 3D image depth processing method and system | |
| CN109905691A (en) | Depth image acquisition device and depth image acquisition system and its image processing method | |
| CN115546379A (en) | Data processing method and device and computer equipment | |
| CN110336993A (en) | Depth camera head controlling method, device, electronic equipment and storage medium | |
| CN111896974A (en) | Time delay control system and method for TOF module swing error calibration | |
| CN103140049A (en) | System and method of chip mounter quick positioning based on field programmable gate array (FPGA) | |
| CN118297936A (en) | Workpiece defect detection method and system based on three-dimensional reconstruction and image processing computer | |
| Zhang et al. | MiniTracker: a lightweight CNN-based system for visual object tracking on embedded device | |
| CN109889690B (en) | Method for improving frame rate of depth image and depth camera group | |
| CN212965422U (en) | Delay plate for calibrating swing error of TOF module |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |