CN104869287A - Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor - Google Patents
Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor Download PDFInfo
- Publication number
- CN104869287A CN104869287A CN201510251716.3A CN201510251716A CN104869287A CN 104869287 A CN104869287 A CN 104869287A CN 201510251716 A CN201510251716 A CN 201510251716A CN 104869287 A CN104869287 A CN 104869287A
- Authority
- CN
- China
- Prior art keywords
- frame
- video
- gpu
- motion compensation
- noise reduction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Studio Devices (AREA)
Abstract
The invention discloses a video shooting noise reduction method based on a mobile apparatus GPU and an angular velocity sensor. The method specifically comprises following steps: collecting and acquiring high-definition videos through a camera and caching multiple previous video frames; performing video single-frame intra-frame noise-reduction on cached current video frames; performing inter-frame motion compensation on the multiple cached previous video frames; superposing the multiple previous video frames which are subjected to motion compensation, wherein pixels having color distance within thresholds are superposed in a manner that superposition mixture ratios relative to the previous frames are selected through threshold calculation, and the pixels having relatively large color distance are directly superposed based on intra-frame spatial domain noise reduction results; then outputting the videos. By employing the method, the high-definition videos can be processed in a real time manner on a mobile apparatus, and static and dynamic scenes can achieve excellent noise reduction effects by the use of calculation performance of the angular velocity sensor and the GPU and with the cooperation of inter-frame time domain superposition noise reduction and intra-frame spatial domain noise reduction.
Description
Technical field
The present invention relates to Video processing, particularly relate to a kind of video capture noise-reduction method based on mobile device GPU and angular-rate sensor.
Background technology
The mobile devices such as present mobile phone or panel computer have become the common tool of video clip shooting.The sensor devices noise that this type of mobile device camera uses is larger, and under especially dark needs the scene of high ISO, shooting video noise out clearly, makes shooting effect have a greatly reduced quality.Carry out real-time noise-reducing to HD video and need larger amount of calculation, common capture apparatus uses average, intermediate value usually, KNN, NLM etc. carry out noise reduction process to each two field picture, although processing speed can lose some details soon.Meanwhile, the mobile device that Performance comparision is low is difficult to the noise by inter motion compensation filtering photographed scene.
Summary of the invention
Object of the present invention is just to provide a kind of video capture noise-reduction method based on mobile device GPU and angular-rate sensor, utilize the data of angular-rate sensor, in conjunction with the physical parameter of camera lens and sensor devices, calculate to obtain global motion compensation parameter, motion compensation, superposition noise reduction are carried out to frame of video, effectively can solve above-mentioned deficiency of the prior art.
The present invention is directed to the deficiencies in the prior art, provide following technical scheme:
Video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention, it is characterized in that, concrete steps are as follows:
101, camera collection is utilized to obtain HD video, the multiple frame of video in the past of buffer-stored;
102, the current video frame of buffer-stored is carried out noise reduction in video single frames frame;
103, multiple frame of video in the past of buffer-stored are carried out inter motion compensation, comprise the following steps;
A () utilizes angular-rate sensor collection to obtain angular velocity data, and angular velocity data is carried out integration, calculates interframe global motion compensation parameter,
B () transfers multiple frame of video in the past, carry out motion compensation according to global motion compensation parameter;
104, the past frame of video of multiple motion compensation is superposed, the pixel of color distance in threshold value is selected with threshold calculations and is superposed with the mixed proportion that superposes of front frame, the result superposition of the pixel that color distance is larger direct interior spatial domain frame by frame noise reduction, output video afterwards;
Particular content is as follows:
Comprise spatial domain noise reduction module and GPU interframe laminating module in high definition video collecting module, frame buffer module, angular velocity detecting module, angular speed integration module, GPU inter motion compensation module, GPU frame.
Further, described step 103 adopts 3 axis angular rate sensors to gather angular velocity data on 3 directions.
Again further, angular velocity data is converted to quaternion algebra according to carrying out integration by described step 103.
Further, described step 103 calculates interframe global motion compensation parameter according to angular velocity data and camera physical parameter.
Compared with prior art, the invention has the advantages that:
Video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention, HD video can be realized on the mobile apparatus to process in real time, utilize the calculated performance of angular-rate sensor and GPU, in conjunction with spatial domain noise reduction in interframe time domain superposition noise reduction and frame, extraordinary noise reduction is all had to Static and dynamic scene, realize most of scene and do not lose details, especially the noise reduction process of static scene neither loses image quality and also add details.
Other features and advantages of the present invention will be set forth in the following description, and, partly become apparent from specification, or understand by implementing the present invention.Object of the present invention and other advantages realize by structure specifically noted in write specification, claims and accompanying drawing and obtain.
Below by drawings and Examples, technical scheme of the present invention is described in further detail.
Accompanying drawing explanation
Accompanying drawing is used to provide a further understanding of the present invention, and forms a part for specification, together with embodiments of the present invention for explaining the present invention, is not construed as limiting the invention.In the accompanying drawings:
Fig. 1 is the flow chart of the video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention;
Fig. 2 is the system block diagram of the video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention.
Embodiment
Below in conjunction with accompanying drawing, the invention will be further described.
embodiment:
Below in conjunction with accompanying drawing, the preferred embodiments of the present invention are described, should be appreciated that preferred embodiment described herein is only for instruction and explanation of the present invention, is not intended to limit the present invention.
Shown in Fig. 1, Fig. 2.
The system of the video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention, comprise high definition video collecting module, frame buffer module, angular velocity detecting module, angular speed integration module, GPU inter motion compensation module, spatial domain noise reduction module and GPU interframe laminating module in GPU frame, described high definition video collecting module is connected with frame buffer module, described frame buffer module also with spatial domain noise reduction module in GPU frame and GPU inter motion compensation model calling, with angular velocity detecting module and GPU inter motion compensation model calling while of described angular speed integration module, described GPU inter motion compensation module is also all connected with GPU interframe laminating module with spatial domain noise reduction module in GPU frame.
High definition video collecting module acquires HD video also outputs to frame buffer module, frame buffer module stores multiple frame of video in the past, and current video frame is outputted to spatial domain noise reduction module in GPU frame and carry out noise reduction in video single frames frame, multiple frame of video is in the past outputted to GPU inter motion compensation module, the interframe global motion compensation parameter that GPU inter motion compensation module calculates according to angular velocity detecting module and angular speed integration module, multiple frame of video is in the past carried out to inter motion compensation and result outputted to GPU interframe laminating module, GPU interframe laminating module in conjunction with multiple motion compensation past frame of video and in frame the current video frame of noise reduction carry out superposing rear output video.
The concrete noise reduction step of the video capture noise-reduction method based on mobile device GPU and angular-rate sensor of the present invention is as follows:
Step 101, camera collection is utilized to obtain HD video, the multiple in the past frame of video of buffer-stored;
High definition video collecting module in charge is from camera collection HD video and output to frame buffer module, frame buffer module is responsible for preserving multiple frame of video in the past, and current video frame is outputted to spatial domain noise reduction module in GPU frame, multiple frame of video is in the past outputted to GPU inter motion compensation module.
Step 102, the current video frame of buffer-stored is carried out noise reduction in video single frames frame;
In GPU frame, spatial domain noise reduction module adopts average, intermediate value, KNN and NLM scheduling algorithm to carry out noise reduction in video single frames frame.
Step 103, multiple in the past frame of video of buffer-stored are carried out inter motion compensation, comprise the following steps;
A () utilizes angular-rate sensor collection to obtain angular velocity data, and angular velocity data is carried out integration, calculates interframe global motion compensation parameter,
The data of angular velocity detecting module Real-time Collection angular-rate sensor, obtain the angular velocity data on 3 directions, and carry out interpolation when being necessary to data.After angular speed integration module obtains the angular velocity information on 3 directions, the local coordinate system converting hypercomplex number representative to carries out integration, realize the prediction of interframe movement direction on 3 directions and angle, simultaneously in conjunction with the physical parameter of camera lens and sensor devices, finally calculate interframe global motion compensation parameter.
B () transfers multiple frame of video in the past, carry out motion compensation according to global motion compensation parameter;
GPU inter motion compensation module, according to interframe global motion compensation parameter, carries out inter motion compensation, with scene of aliging to multiple frame of video in the past.
Step 104, the past frame of video of multiple motion compensation to be superposed, the pixel of color distance in threshold value is selected with threshold calculations and is superposed with the mixed proportion that superposes of front frame, the result superposition of the pixel that color distance is larger direct interior spatial domain frame by frame noise reduction, output video afterwards.
The past frame of video of multiple motion compensation superposes by GPU interframe laminating module, consider that the object that may have rapid movement in scene cannot carry out global motion compensation, when superposing, need the color distance of compared pixels, the pixel of distance in threshold value is thought what noise produced, and with this calculate a factor decide with front frame superpose mixed proportion.For the pixel that color distance is larger, then directly superpose the result of spatial domain noise reduction in frame, finally export de-noising video.
The present invention utilizes the data of angular-rate sensor, learn movement angle and the direction of camera lens between two frames in real time, in conjunction with the physical parameter of camera lens and sensor devices, directly can calculate to obtain global motion compensation parameter, calculate interframe time domain by GPU motion compensation and superposed average again and superpose noise reduction, simultaneously in conjunction with spatial domain noise reduction in video single frames frame, all have extraordinary noise reduction to Static and dynamic scene, especially the noise reduction process of static scene neither loses image quality and also add details.
The present invention can realize HD video on the mobile apparatus and process in real time, utilize the calculated performance of angular-rate sensor and GPU, in conjunction with spatial domain noise reduction in interframe time domain superposition noise reduction and frame, all there is extraordinary noise reduction to Static and dynamic scene, realize most of scene and do not lose details.
Finally illustrate: the foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, although with reference to previous embodiment to invention has been detailed description, for a person skilled in the art, it still can be modified to the technical scheme described in foregoing embodiments, or carries out equivalent replacement to wherein portion of techniques feature.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (4)
1., based on a video capture noise-reduction method for mobile device GPU and angular-rate sensor, it is characterized in that, concrete steps are as follows:
101, camera collection is utilized to obtain HD video, the multiple frame of video in the past of buffer-stored;
102, the current video frame of buffer-stored is carried out noise reduction in video single frames frame;
103, multiple frame of video in the past of buffer-stored are carried out inter motion compensation, comprise the following steps;
A () utilizes angular-rate sensor collection to obtain angular velocity data, and angular velocity data is carried out integration, calculates interframe global motion compensation parameter,
B () transfers multiple frame of video in the past, carry out motion compensation according to global motion compensation parameter;
104, the past frame of video of multiple motion compensation is superposed, the pixel of color distance in threshold value is selected with threshold calculations and is superposed with the mixed proportion that superposes of front frame, the result superposition of the pixel that color distance is larger direct interior spatial domain frame by frame noise reduction, output video afterwards;
Particular content is as follows:
Comprise spatial domain noise reduction module and GPU interframe laminating module in high definition video collecting module, frame buffer module, angular velocity detecting module, angular speed integration module, GPU inter motion compensation module, GPU frame.
2. according to claim 1 based on the video capture noise-reduction method of mobile device GPU and angular-rate sensor, it is characterized in that: described step 103 adopts 3 axis angular rate sensors to gather angular velocity data on 3 directions.
3. according to claim 1 or 2 based on the video capture noise-reduction method of mobile device GPU and angular-rate sensor, it is characterized in that: angular velocity data is converted to quaternion algebra according to carrying out integration by described step 103.
4. according to claim 3 based on the video capture noise-reduction method of mobile device GPU and angular-rate sensor, it is characterized in that: described step 103 calculates interframe global motion compensation parameter according to angular velocity data and camera physical parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510251716.3A CN104869287A (en) | 2015-05-18 | 2015-05-18 | Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510251716.3A CN104869287A (en) | 2015-05-18 | 2015-05-18 | Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104869287A true CN104869287A (en) | 2015-08-26 |
Family
ID=53914777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510251716.3A Pending CN104869287A (en) | 2015-05-18 | 2015-05-18 | Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104869287A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018152977A1 (en) * | 2017-02-27 | 2018-08-30 | 中兴通讯股份有限公司 | Image noise reduction method and terminal, and computer storage medium |
| CN109672931A (en) * | 2018-12-20 | 2019-04-23 | 北京百度网讯科技有限公司 | Method and apparatus for handling video frame |
| CN112788199A (en) * | 2019-11-08 | 2021-05-11 | 海信视像科技股份有限公司 | Spatial domain noise reduction method and device for video image and storage medium |
| CN112866506A (en) * | 2019-11-08 | 2021-05-28 | 海信视像科技股份有限公司 | Time domain noise reduction method and device for video image and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02248173A (en) * | 1989-03-22 | 1990-10-03 | Canon Inc | Moving picture/still picture converter |
| US20060132612A1 (en) * | 2004-11-26 | 2006-06-22 | Hideo Kawahara | Motion picture taking apparatus and method |
| CN101262559A (en) * | 2008-03-28 | 2008-09-10 | 北京中星微电子有限公司 | A method and device for eliminating sequential image noise |
| CN101355644A (en) * | 2007-09-11 | 2009-01-28 | 豪威科技有限公司 | Image sensor device and method for embedding image steady data into image data |
| CN101379457A (en) * | 2006-09-21 | 2009-03-04 | 索尼计算机娱乐公司 | Operation device control apparatus, operation device control method, information storage medium, and operation device |
| US20100053343A1 (en) * | 2008-08-26 | 2010-03-04 | Samsung Electro-Mechanics Co., Ltd. | Apparatus for correcting motion caused by hand shake |
| US20100079605A1 (en) * | 2008-09-29 | 2010-04-01 | William Marsh Rice University | Sensor-Assisted Motion Estimation for Efficient Video Encoding |
| CN101964863A (en) * | 2010-05-07 | 2011-02-02 | 镇江唐桥微电子有限公司 | Self-adaptive time-space domain video image denoising method |
| US20110109755A1 (en) * | 2009-11-12 | 2011-05-12 | Joshi Neel S | Hardware assisted image deblurring |
| CN102289306A (en) * | 2011-08-30 | 2011-12-21 | 江苏惠通集团有限责任公司 | Attitude sensing equipment and positioning method thereof as well as method and device for controlling mouse pointer |
| US20130044228A1 (en) * | 2011-08-15 | 2013-02-21 | Apple Inc. | Motion-Based Video Stabilization |
| CN103905692A (en) * | 2012-12-26 | 2014-07-02 | 苏州赛源微电子有限公司 | Simple 3D noise reduction algorithm base on motion detection |
-
2015
- 2015-05-18 CN CN201510251716.3A patent/CN104869287A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02248173A (en) * | 1989-03-22 | 1990-10-03 | Canon Inc | Moving picture/still picture converter |
| US20060132612A1 (en) * | 2004-11-26 | 2006-06-22 | Hideo Kawahara | Motion picture taking apparatus and method |
| CN101379457A (en) * | 2006-09-21 | 2009-03-04 | 索尼计算机娱乐公司 | Operation device control apparatus, operation device control method, information storage medium, and operation device |
| CN101355644A (en) * | 2007-09-11 | 2009-01-28 | 豪威科技有限公司 | Image sensor device and method for embedding image steady data into image data |
| CN101262559A (en) * | 2008-03-28 | 2008-09-10 | 北京中星微电子有限公司 | A method and device for eliminating sequential image noise |
| US20100053343A1 (en) * | 2008-08-26 | 2010-03-04 | Samsung Electro-Mechanics Co., Ltd. | Apparatus for correcting motion caused by hand shake |
| US20100079605A1 (en) * | 2008-09-29 | 2010-04-01 | William Marsh Rice University | Sensor-Assisted Motion Estimation for Efficient Video Encoding |
| US20110109755A1 (en) * | 2009-11-12 | 2011-05-12 | Joshi Neel S | Hardware assisted image deblurring |
| CN101964863A (en) * | 2010-05-07 | 2011-02-02 | 镇江唐桥微电子有限公司 | Self-adaptive time-space domain video image denoising method |
| US20130044228A1 (en) * | 2011-08-15 | 2013-02-21 | Apple Inc. | Motion-Based Video Stabilization |
| CN102289306A (en) * | 2011-08-30 | 2011-12-21 | 江苏惠通集团有限责任公司 | Attitude sensing equipment and positioning method thereof as well as method and device for controlling mouse pointer |
| CN103905692A (en) * | 2012-12-26 | 2014-07-02 | 苏州赛源微电子有限公司 | Simple 3D noise reduction algorithm base on motion detection |
Non-Patent Citations (1)
| Title |
|---|
| JOSHI N.等: "Image deblurring using inertial measurement sensors", 《ACM TRANSACTIONS ON GRAPHICS (TOG)- PROCEEDINGS OF ACM SIGGRAPH 2010》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018152977A1 (en) * | 2017-02-27 | 2018-08-30 | 中兴通讯股份有限公司 | Image noise reduction method and terminal, and computer storage medium |
| CN108513043A (en) * | 2017-02-27 | 2018-09-07 | 中兴通讯股份有限公司 | A kind of image denoising method and terminal |
| CN109672931A (en) * | 2018-12-20 | 2019-04-23 | 北京百度网讯科技有限公司 | Method and apparatus for handling video frame |
| US11195248B2 (en) | 2018-12-20 | 2021-12-07 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for processing pixel data of a video frame |
| CN112788199A (en) * | 2019-11-08 | 2021-05-11 | 海信视像科技股份有限公司 | Spatial domain noise reduction method and device for video image and storage medium |
| CN112866506A (en) * | 2019-11-08 | 2021-05-28 | 海信视像科技股份有限公司 | Time domain noise reduction method and device for video image and storage medium |
| CN112788199B (en) * | 2019-11-08 | 2023-04-07 | 海信视像科技股份有限公司 | Spatial domain noise reduction method and device for video image and storage medium |
| CN112866506B (en) * | 2019-11-08 | 2023-08-15 | 海信视像科技股份有限公司 | Time domain noise reduction method and device for video image and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3216216B1 (en) | Methods and systems for multi-view high-speed motion capture | |
| CN101448077B (en) | Self-adapting video image 3D denoise method | |
| US9615039B2 (en) | Systems and methods for reducing noise in video streams | |
| JP2022071177A (en) | Multiplexed high dynamic range image | |
| JP7319390B2 (en) | Image/video deblurring using convolutional neural networks with application to SFM/SLAM with blurry images/videos | |
| CN104301596B (en) | A kind of method for processing video frequency and device | |
| CN108174057B (en) | Method and device for rapidly reducing noise of picture by utilizing video image inter-frame difference | |
| TWI459810B (en) | Image processing apparatus and processing method thereof | |
| WO2021232965A1 (en) | Video noise reduction method and apparatus, mobile terminal and storage medium | |
| CN104506755B (en) | HD video based on FPGA automates defogging method in real time | |
| CN114586337B (en) | Video anti-shake optimization processing method and device and electronic equipment | |
| US11599974B2 (en) | Joint rolling shutter correction and image deblurring | |
| CN107360344B (en) | Surveillance video fast defogging method | |
| CN104869287A (en) | Video shooting noise reduction method based on mobile apparatus GPU and angular velocity sensor | |
| CN102067607B (en) | Image synthesizing and encoding method, image synthesizing and encoding device, and imaging system | |
| CN104469251A (en) | Image acquisition method and electronic equipment | |
| CN205320214U (en) | 3DVR panoramic video image device | |
| WO2020117379A1 (en) | High dynamic range anti-ghosting and fusion | |
| CN107392879B (en) | A low-illumination surveillance image enhancement method based on reference frames | |
| CN103903229A (en) | Night image enhancement method and device | |
| US9336460B2 (en) | Adaptive motion instability detection in video | |
| Kang et al. | Retinomorphic sensing: A novel paradigm for future multimedia computing | |
| CN113205011B (en) | Image mask determining method and device, storage medium and electronic equipment | |
| CN115546043A (en) | Video processing method and related equipment | |
| CN102377953A (en) | Filtering and denoising system and filtering and denoising method for video data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150826 |