CN109658459A - Camera calibration method, device, electronic equipment and computer readable storage medium - Google Patents
Camera calibration method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN109658459A CN109658459A CN201811453896.3A CN201811453896A CN109658459A CN 109658459 A CN109658459 A CN 109658459A CN 201811453896 A CN201811453896 A CN 201811453896A CN 109658459 A CN109658459 A CN 109658459A
- Authority
- CN
- China
- Prior art keywords
- camera
- image
- group
- calibration
- uncalibrated image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 70
- 238000012360 testing method Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 16
- 239000000284 extract Substances 0.000 claims description 11
- 238000003384 imaging method Methods 0.000 description 17
- 241000208340 Araliaceae Species 0.000 description 9
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 9
- 235000003140 Panax quinquefolius Nutrition 0.000 description 9
- 235000008434 ginseng Nutrition 0.000 description 9
- 238000012937 correction Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000006641 stabilisation Effects 0.000 description 2
- 238000011105 stabilization Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 239000000872 buffer Substances 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
This application involves a kind of camera calibration method, device, electronic equipment and computer readable storage mediums.The described method includes: being shot by the first camera and second camera, obtain first group of uncalibrated image, calibration processing is carried out to the first camera and second camera according to first group of uncalibrated image, obtain the first calibration information, it is shot by the first camera and third camera, obtain second group of uncalibrated image, wherein, third camera is depth camera, calibration processing is carried out to the first camera and third camera according to second group of uncalibrated image, the second calibration information is obtained, the accuracy of camera calibration can be improved.
Description
Technical field
This application involves image technology fields, more particularly to a kind of camera calibration method, device, electronic equipment and meter
Calculation machine readable storage medium storing program for executing.
Background technique
Camera needs the calibration information for being demarcated to obtain camera to camera before factory, thus camera
Image can be handled according to calibration information, such that treated image restores the object in three-dimensional space.
Summary of the invention
The embodiment of the present application provides a kind of camera calibration method, device, electronic equipment and computer readable storage medium,
The accuracy of camera calibration can be improved.
A kind of camera calibration method, comprising:
It is shot by the first camera and second camera, obtains first group of uncalibrated image；
Calibration processing is carried out to first camera and second camera according to first group of uncalibrated image, obtains
One calibration information；
It is shot by first camera and third camera, obtains second group of uncalibrated image, wherein described
Three cameras are depth camera；
Calibration processing is carried out to first camera and third camera according to second group of uncalibrated image, obtains
Two calibration informations.
A kind of camera calibration device, comprising:
First image collection module obtains first group of mark for being shot by the first camera and second camera
Determine image；
First demarcating module, for according to first group of uncalibrated image to first camera and second camera into
Rower is handled surely, obtains the first calibration information；
Second image collection module obtains second for being shot by first camera and third camera
Group uncalibrated image, wherein the third camera is depth camera；
Second demarcating module, for according to second group of uncalibrated image to first camera and third camera into
Rower is handled surely, obtains the second calibration information.
A kind of electronic equipment, including camera, memory and processor store computer program in the memory,
When the computer program is executed by the processor, so that the processor executes following steps:
It is shot by the first camera and second camera, obtains first group of uncalibrated image；
Calibration processing is carried out to first camera and second camera according to first group of uncalibrated image, obtains
One calibration information；
It is shot by first camera and third camera, obtains second group of uncalibrated image, wherein described
Three cameras are depth camera；
Calibration processing is carried out to first camera and third camera according to second group of uncalibrated image, obtains
Two calibration informations.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor
Following steps are realized when row:
It is shot by the first camera and second camera, obtains first group of uncalibrated image；
Calibration processing is carried out to first camera and second camera according to first group of uncalibrated image, obtains
One calibration information；
It is shot by first camera and third camera, obtains second group of uncalibrated image, wherein described
Three cameras are depth camera；
Calibration processing is carried out to first camera and third camera according to second group of uncalibrated image, obtains
Two calibration informations.
Above-mentioned camera calibration method, device, electronic equipment and computer readable storage medium, by the first camera and
Second camera is shot, and first group of uncalibrated image is obtained, and is taken the photograph according to first group of uncalibrated image to the first camera and second
As head carries out calibration processing, the first calibration information is obtained, is shot by the first camera and third camera, obtains second
Group uncalibrated image, wherein third camera is depth camera, is taken the photograph according to second group of uncalibrated image to the first camera and third
As head carries out calibration processing, the second calibration information is obtained, the accuracy of camera calibration can be improved.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of application for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the applied environment figure of camera calibration method in one embodiment；
Fig. 2 is the flow chart of camera calibration method in one embodiment；
Fig. 3 is the flow chart that the first calibration information is tested in one embodiment；
Fig. 4 is the flow chart that calibration test is carried out in one embodiment；
Fig. 5 is the flow chart of camera calibration device in one embodiment；
Fig. 6 is the internal structure block diagram of electronic equipment in one embodiment；
Fig. 7 is the schematic diagram of image processing circuit in one embodiment.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood
The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, and
It is not used in restriction the application.
It is appreciated that term " first " used in this application, " second " etc. can be used to describe various elements herein,
But these elements should not be limited by these terms.These terms are only used to distinguish the first element from the other element.Citing comes
It says, in the case where not departing from scope of the present application, the first camera can be known as second camera, and similarly, can incite somebody to action
Second camera is known as the first camera.First camera and second camera both camera, but it is not same takes the photograph
As head.
Fig. 1 is the application environment schematic diagram of camera calibration method in one embodiment.As shown in Figure 1, the application environment
Including electronic equipment 110 and scaling board 120.Chart pattern is had on scaling board (chart plate) 120, scaling board 120 can carry out
Rotation, keeps the pose of different angle.Electronic equipment 110 can shoot the chart pattern comprising scaling board 120 by camera
Image.The first camera, second camera and third camera in electronic equipment 110 comprising being arranged with random order.Electricity
Sub- equipment 110 can be shot by the first camera and second camera, first group of uncalibrated image be obtained, according to first group
Uncalibrated image carries out calibration processing to the first camera and second camera, obtains the first calibration information, passes through the first camera
It is shot with third camera, obtains second group of uncalibrated image, according to second group of uncalibrated image to the first camera and third
Camera carries out calibration processing, obtains the second calibration information.Wherein, the first camera can be colored (RGB) camera shooting, and second takes the photograph
It can be, but not limited to be colour imagery shot, black and white camera, focal length camera, wide-angle camera, depth as head and third camera
Spend one of camera or a variety of.Scaling board 120 can be three-dimensional scaling plate, be also possible to two-dimensional calibrations plate.Electronic equipment
110 can be but be not limited to various mobile phones, computer, portable equipment etc..
Fig. 2 is the flow chart of camera calibration method in one embodiment.Camera calibration method in the present embodiment, with
It is described for running on the electronic equipment in Fig. 1.As shown in Fig. 2, the camera calibration method includes step 202 to step
Rapid 208.Wherein:
Step 202, it is shot by the first camera and second camera, obtains first group of uncalibrated image.
Electronic equipment may include the first camera, second camera and third camera.Each camera can be interior
It is placed in the camera of electronic equipment, is also possible to be placed outside the camera of electronic equipment；The camera for including in electronic equipment can
To be front camera, it is also possible to rear camera.First camera can be any camera, herein with no restrictions.Example
Such as, the first camera and second camera can all be colour imagery shots, be also possible to colour imagery shot and black and white camera
Combination, can also be focal length camera and the combination of black and white camera etc., without being limited thereto.
Electronic equipment is shot by the first camera and second camera, and specifically, electronic equipment can pass through
One camera and second camera shoot scaling board, and acquisition includes first group of uncalibrated image of uncalibrated image.First
The uncalibrated image of the first camera and second camera shooting is contained in group uncalibrated image.Wherein, scaling board can be two dimension
Scaling board is also possible to three-dimensional scaling plate.Three-dimensional scaling plate refers to the scaling board comprising at least three calibration faces.When scaling board is
It include at least three different calibration patterns in every uncalibrated image that electronic equipment obtains when three-dimensional scaling plate.
Step 204, calibration processing is carried out to the first camera and second camera according to first group of uncalibrated image, obtains
One calibration information.
Calibration processing refers to the operation that the parameter in geometrical model to camera imaging is solved, by camera at
The geometrical model of picture can make the object in the image restoring space of shooting.Calibration information, which refers to, carries out calibration processing to camera
Camera parameter obtained afterwards etc., calibration information can be used for being corrected the image that camera acquires, thus after making correction
Image can restore the object in space.Specifically, calibration information may include the internal reference of camera, outer ginseng, distortion factor
Deng.First calibration information is the first camera and the corresponding calibration information of second camera.It include the in first calibration information
The Bi-objective of one camera and the corresponding monocular calibration information of second camera and the first camera and second camera
Determine information, wherein monocular calibration information may include internal reference, outer ginseng, distortion factor, and binocular calibration information may include first taking the photograph
As the outer ginseng between head and second camera.
Electronic equipment carries out calibration processing to the first camera and second camera according to first group of uncalibrated image, specifically
Ground, electronic equipment can use traditional camera standardization, camera self-calibrating method, between traditional scaling method and self-calibration
Zhang Zhengyou scaling method between method etc. carries out calibration processing to camera, obtains the first camera and second camera is corresponding
The first calibration information.Electronic equipment can obtain the first mark shot in first group of uncalibrated image by the first camera respectively
Determine image, searches the characteristic point of the first uncalibrated image, the corresponding internal reference of the first camera, outer ginseng and distortion are sought according to characteristic point
Coefficient；Similarly, it is corresponding can also to obtain second camera by the second uncalibrated image that second camera is shot for electronic equipment
Internal reference, outer ginseng and distortion factor, and then calculate first according to the corresponding outer ginseng of the first camera and second camera and take the photograph
As the outer ginseng between head and second camera.
Step 206, it is shot by the first camera and third camera, obtains second group of uncalibrated image, wherein the
Three cameras are depth camera.
Third camera can be depth camera.Specifically, third camera can be structure light video camera head, can also be with
It is TOF (Time of flight, flight time ranging) camera etc., can also be that other can obtain the camera shooting of depth information
Head.Structure light video camera head is by projecting controllable luminous point, striation or smooth surface structure to testee surface；And it receives controllable
Luminous point, striation or smooth surface structure reflected light, and depth image is obtained according to the deformation quantity of transmitting light.TOF camera passes through
Emit near infrared light to scene；The near infrared ray for receiving reflection is obtained by calculating near infrared ray time difference or the phase difference of reflection
Take the depth information of scene；The profile of scene is represented into different distance in different colors, to obtain depth image.Electronic equipment can
To shoot by the first camera and third camera to scaling board, first is contained in second obtained group uncalibrated image
The uncalibrated image that camera and third camera are shot respectively.Wherein, the uncalibrated image of third camera shooting can be visually
Image, i.e., may include the scene information of camera shooting in uncalibrated image, such as can be color image, gray level image etc..
Step 208, calibration processing is carried out to the first camera and third camera according to second group of uncalibrated image, obtains
Two calibration informations.
Second calibration information contains the first camera and the corresponding internal reference of third camera, outer ginseng and distortion system
The several and outer ginseng between the first camera and third camera.It is taken the photograph according to first group of uncalibrated image to first with electronic equipment
As to carry out calibration processing similar for head and second camera, electronic equipment can according to second group of uncalibrated image to the first camera with
Third camera carries out calibration processing, obtains the second calibration information.The calibration of depth camera is contained in second calibration information
Information can be used for being corrected the depth image that depth camera is shot, allow the depth image after correction accurate
Indicate the depth information of the image of the first camera acquisition.
In one embodiment, electronic equipment can also first pass through the first camera and third camera is shot, and obtain
To second group of uncalibrated image, calibration processing is carried out to the first camera and third camera according to second group of uncalibrated image, is obtained
Second calibration information, then shot by the first camera and second camera, first group of uncalibrated image is obtained, according to first
Group uncalibrated image carries out calibration processing to the first camera and second camera, the first calibration information is obtained, herein not to camera shooting
The fixed sequence of leader is defined.
In the embodiment of the present application, electronic equipment is shot by the first camera and second camera, obtains first group
Uncalibrated image carries out calibration processing to the first camera and second camera according to first group of uncalibrated image, obtains the first calibration
Information is shot by the first camera and third camera, obtains second group of uncalibrated image, wherein third camera is
Depth camera carries out calibration processing to the first camera and third camera according to second group of uncalibrated image, obtains the second mark
Determine information.Due to three cameras that can include to electronic equipment, and one of camera is that depth camera is marked
Fixed processing, obtains the first calibration information and the second calibration information comprising depth camera calibration information, camera can be improved
The accuracy of calibration.And then in the use process of camera, corresponding calibration information can be obtained according to the camera used
Processing is corrected to the image of acquisition, when needing to use the depth information of subject in image, third can be used
Camera sampling depth image, and being corrected according to the depth image that the second calibration information acquires third camera, can be with
Improve the accuracy of image procossing.
In one embodiment, it is clapped in the camera calibration method provided by the first camera and second camera
It takes the photograph, the process for obtaining first group of uncalibrated image includes: mark by the first camera and second camera at least three angles
Fixed board is shot, and first group of uncalibrated image is obtained；It is shot by the first camera and third camera, obtains second group
The process of uncalibrated image includes: to be shot by the first camera and third camera to the scaling board of at least three angles,
Obtain second group of uncalibrated image.
When scaling board is two-dimensional calibrations plate, i.e., only has the plate in a calibration face, scaling board can pass through rotary shaft
Rotate multiple angles.As shown in Figure 1, one of angle can be 0 degree, other angles at least three angles of scaling board
For ± θ degree.θ value can determine according to actual needs on the basis of decoupling between guaranteeing posture.The angle of scaling board can lead to
It crosses electronic equipment to be controlled, for example, electronic equipment can be controlled after the completion of scaling board of the electronic equipment to first angle is shot
Scaling board processed is adjusted to second angle, and then electronic equipment is shot by scaling board of the camera to second angle.One
In a little embodiments, the angle of scaling board can also be controlled etc. by artificial or other electronic equipments, without being limited thereto.Electronics
Equipment can shoot the scaling board of at least three angles to obtain first group of mark by the first camera and second camera
Determine image, and the scaling board of at least three angles is shot by the first camera and third camera, obtains second group
Uncalibrated image.For example, electronic equipment is by the first camera and second camera to 0 degree, -20 degree ,+20 degree, -30 degree ,+30 degree
The scaling board of five angles is shot, and the first camera and the corresponding first group of uncalibrated image of second camera are obtained.Wherein,
The uncalibrated image of the scaling board shooting of the first camera and second camera to different angle is contained in first group of uncalibrated image.
In one embodiment, third camera is depth camera in the camera calibration method provided, then obtains the
The step of one camera and second group of uncalibrated image of third camera shooting may include: to be taken the photograph by the first camera and third
As head shoots scaling board, the corresponding third uncalibrated image of the first camera and corresponding 4th mark of third camera are obtained
Determine image, wherein the 4th uncalibrated image is gray level image；Obtain second group comprising third uncalibrated image and the 4th uncalibrated image
Uncalibrated image.
Gray level image refers to the image of only one sample color of pixel.Specifically, gray level image is in black and white
Between there are the color depths of many grades.Electronic equipment can be gray level image by the image that third camera acquires.At this
To apply in embodiment, electronic equipment shoots scaling board by the first camera and third camera, and available first
The third uncalibrated image of camera shooting and the 4th uncalibrated image of third camera shooting, the 4th uncalibrated image is to include
The gray level image of the calibration pattern of scaling board.Electronic equipment is available to be made of third uncalibrated image and the 4th uncalibrated image
Second group of uncalibrated image.It is shot by the first camera and third camera, obtains the ash comprising the shooting of third camera
Second group of uncalibrated image for spending image, carries out calibration processing to the first camera and third camera, camera shooting leader can be improved
Fixed accuracy.
Fig. 3 is the flow chart that the first calibration information is tested in one embodiment.As shown in figure 3, in one embodiment, mentioning
Shot in the camera calibration method of confession by the first camera and third camera, obtain second group of uncalibrated image it
Before, further includes:
Step 302, the first uncalibrated image and second camera of the first camera shooting in first group of uncalibrated image are obtained
Second uncalibrated image of shooting.
The first uncalibrated image and second that the first camera is shot in the available first group of uncalibrated image of electronic equipment is taken the photograph
The second uncalibrated image shot as head.When what is shot there are electronic equipment in the scaling board of different angle in first group of uncalibrated image
When uncalibrated image, electronic equipment is available under the scaling board of equal angular, the first uncalibrated image of the first camera shooting
With the second uncalibrated image of second camera shooting.
Step 304, processing is corrected to the first uncalibrated image and the second uncalibrated image according to the first calibration information, obtained
Corresponding first object image and the second target image.
Electronic equipment is corrected processing to the first uncalibrated image and the second uncalibrated image according to the first calibration information, specifically
Ground, electronic equipment can be according to the monocular calibration informations and binocular calibration information that the first calibration information includes to the first uncalibrated image
It is corrected processing with the second uncalibrated image, obtain that treated first object image and the second target image.Wherein, the first mesh
Characteristic point horizontal alignment in logo image and the second target image.
Step 306, the first depth information is calculated according to first object image and the second target image.
Depth information refers to the distance between subject and camera in image.Electronic equipment is according to first object figure
Picture and the second target image calculate the first depth information, and specifically, electronic equipment can search first object image and the second mesh
Corresponding characteristic point in logo image obtains subject according to characteristic point corresponding in first object image and the second target image
Parallax of the body between the first camera and second camera, to convert parallax to the depth information of subject.
Step 308, it when the first depth information and predetermined depth information matches, then executes and passes through the first camera and third
Camera is shot, and the operation of second group of uncalibrated image is obtained.
Predetermined depth information can be the distance value of the camera and subject that measure in advance.Specifically, Ke Yigen
Electronic equipment and scaling board are placed according to predetermined depth information, so that electronic equipment is shot by the first camera and second camera
First uncalibrated image and the second uncalibrated image are corrected processing to the first uncalibrated image and the second uncalibrated image and obtain the first mesh
Logo image and the second target image, then between first object image and subject and scaling board in the second target image
Distance Theory should be identical as predetermined depth information.Electronic equipment can be calculated by first object image and the second target image
It is matched to the first depth information, then by the first depth information with predetermined depth information, specifically, electronic equipment can calculate
The matching degree of first depth information and predetermined depth information then determines the first depth information and presets when matching degree is greater than threshold value
Depth information matches.Electronic equipment can be according to depth of each pixel in the first depth information and predetermined depth information
Difference calculates matching degree, is less than in advance for example, the average value of the depth difference of each pixel, variance, depth difference can be calculated
If number of the pixel of difference etc. calculates matching degree, without being limited thereto.
First depth information and predetermined depth information matches, then illustrate electronic equipment to the first camera and second camera
It is accurate to carry out the first calibration information that calibration is handled, electronic equipment is according to the first calibration information to the first uncalibrated image and
Two uncalibrated images are corrected that treated first object image and the second target image can accurately restore three-dimensional space
Object.When the first depth information and predetermined depth information matches, execution is clapped by the first camera and third camera
It takes the photograph, obtains the operation of second group of uncalibrated image；When the first depth information and predetermined depth information mismatch, electronic equipment can be with
It is shot, the first camera and second camera are carried out at calibration again through the first camera and second camera
Reason avoids continuing in the case where the first camera and second camera calibration inaccuracy to the first camera and third camera
It is demarcated, the efficiency of camera calibration can be improved.
In one embodiment, third camera is depth camera in the camera calibration method provided, and this method is also
It may include: when obtaining first group of uncalibrated image, to be taken the photograph by third when being shot by the first camera and second camera
As head is shot, corresponding depth image is obtained；Using the depth information for including in depth image as predetermined depth information.
Third camera is depth camera.The depth image that third camera can acquire, wherein wrapped in depth image
Contain by the depth information between camera shooting object and camera.Electronic equipment can pass through the first camera and second camera
It is shot, when obtaining first group of uncalibrated image, is shot by third camera, obtained and the first uncalibrated image and second
The corresponding depth image of uncalibrated image can be to avoid people using the depth information for including in depth image as predetermined depth information
For measurement electronic equipment and scaling board apart from when the measurement error that generates, improve the accuracy of predetermined depth information.
Fig. 4 is the flow chart that calibration test is carried out in one embodiment.As shown in figure 4, in one embodiment, second group
4th uncalibrated image of third uncalibrated image and the shooting of third camera comprising the shooting of the first camera in uncalibrated image, then should
Can also include: in camera calibration method
Step 402, processing is corrected to third uncalibrated image and the 4th uncalibrated image according to the second calibration information, obtained
Third target image and the 4th target image.
Electronic equipment is corrected processing to third uncalibrated image and the 4th uncalibrated image according to the second calibration information, specifically
Ground, electronic equipment can be according to the monocular calibration informations and binocular calibration information that the second calibration information includes to third uncalibrated image
It is corrected processing with the 4th uncalibrated image, obtain that treated third target image and the 4th target image.Wherein, third mesh
Characteristic point horizontal alignment in logo image and the 4th target image.
Step 404, the first pixel in first object image is extracted, extracts the first pixel in the second target image pair
The second pixel answered obtains the first parallax according to the first pixel and the second pixel.
Electronic equipment can be used scale invariant feature conversion (Scale-invariant feature transform,
SIFT) method or accelerate robust features (Speed Up Robust Features, SURF) method extract first object image and
Pixel in second target image.Specifically, electronic equipment can extract the first pixel in first object image, utilize
Stereo Matching Algorithm extracts the first pixel corresponding second pixel in the second target image, and then according to the first pixel
The first parallax is calculated with the second pixel.
Step 406, the third pixel in third target image is extracted, extracts third pixel in the 4th target image pair
The 4th pixel answered obtains the second parallax according to third pixel and the 4th pixel.
Electronic equipment can be used scale invariant feature conversion method or robust features method accelerated to extract third target figure
Pixel in picture and the 4th target image.Specifically, electronic equipment can extract the third pixel in third target image,
Third pixel corresponding 4th pixel in the 4th target image is extracted using Stereo Matching Algorithm, and then according to third picture
Vegetarian refreshments and the 4th pixel calculate the second parallax.
Step 408, the disparity difference for calculating the first parallax and the second parallax, when disparity difference is less than difference threshold, then
Generate the standby signal that calibration test passes through.
Specifically, the absolute difference of available first parallax of electronic equipment and the second parallax, will as disparity difference
Absolute difference and difference threshold are compared, and wherein difference threshold is that engineer sets in advance in camera module calibration process
It sets, herein with no restriction, the setting of preset threshold determines as the case may be.If disparity difference is less than difference threshold, table
Show that the actual error of camera calibration result in allowable range of error, and then generates the standby signal that passes through of calibration test, it should
Standby signal is used to prompt the calibration test processes unit of electronic equipment, and camera calibration result is set by calibration test, electronics
It is standby to be saved the first calibration information that camera calibration is handled and the second calibration information according to the standby signal.
In one embodiment, electronic equipment can also be clapped by the first camera, second camera and third camera
New image is taken the photograph, the image of each camera acquisition can not limit to and be the image shot to scaling board, thus
Processing is corrected to the image of shooting according to the first calibration information and the second calibration information, between the image after obtaining correction
Parallax and then verifying, which are demarcated, to be tested whether to pass through.
In one embodiment, the camera calibration method provided can also include: when disparity difference is more than or equal to poor
When being worth threshold value, then the standby signal of calibration test crash is generated；And carry out second of calibration processing.
When disparity difference is greater than difference threshold, then it represents that the actual error of camera calibration result allows model beyond error
It enclosing, i.e., electronic equipment carries out the first calibration information obtained after calibration processing and the second calibration information inaccuracy to camera, into
And the standby signal of calibration test crash is generated, which is used to prompt the calibration test processes unit of electronic equipment, takes the photograph
It is tested as head calibration result does not pass through calibration, then needs to carry out second of calibration processing to camera.
Electronic equipment is corrected processing to uncalibrated image by the calibration information handled according to calibration, obtains correction
Whether the parallax between target image obtained afterwards, parallax information is compared, can to examine camera calibration result qualified
To improve the accuracy of camera calibration.
It should be understood that although each step in the flow chart of Fig. 2-4 is successively shown according to the instruction of arrow,
These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps
Execution there is no stringent sequences to limit, these steps can execute in other order.Moreover, at least one in Fig. 2-4
Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps
Completion is executed, but can be executed at different times, the execution sequence in these sub-steps or stage is also not necessarily successively
It carries out, but can be at least part of the sub-step or stage of other steps or other steps in turn or alternately
It executes.
Fig. 5 is the structural block diagram of the camera calibration device of one embodiment.As shown in figure 5, the camera calibration device
Including the first image collection module 502, the first demarcating module 504, the second image collection module 506, the second demarcating module 508.
Wherein:
First image collection module 502 obtains first group for being shot by the first camera and second camera
Uncalibrated image；
First demarcating module 504, for being marked according to first group of uncalibrated image to the first camera and second camera
Fixed processing, obtains the first calibration information；
Second image collection module 506 obtains second group for being shot by the first camera and third camera
Uncalibrated image；
Second demarcating module 508, for being marked according to second group of uncalibrated image to the first camera and third camera
Fixed processing, obtains the second calibration information.
Camera calibration device provided by the embodiments of the present application, is shot by the first camera and second camera,
First group of uncalibrated image is obtained, calibration processing is carried out to the first camera and second camera according to first group of uncalibrated image, is obtained
It to the first calibration information, is shot by the first camera and third camera, obtains second group of uncalibrated image, wherein the
Three cameras are depth camera, carry out calibration processing to the first camera and third camera according to second group of uncalibrated image,
Obtain the second calibration information, available includes to carry out calibration treated calibration information to depth camera, be can be improved
The accuracy of camera calibration.
In one embodiment, the first image collection module 502 can be also used for imaging by the first camera and second
Head shoots the scaling board of at least three angles, obtains first group of uncalibrated image；Second image collection module 506 can be with
For shooting by the first camera and third camera to the scaling board of at least three angles, second group of calibration maps is obtained
Picture.
In one embodiment, the camera calibration device provided can also include calibration test module 510, wherein demarcating
Test module 510 can be used for obtaining the first uncalibrated image that the first camera is shot in first group of uncalibrated image and the second camera shooting
Second uncalibrated image of head shooting；Place is corrected to the first uncalibrated image and the second uncalibrated image according to the first calibration information
Reason, obtains corresponding first object image and the second target image；The is calculated according to first object image and the second target image
One depth information；When the first depth information and predetermined depth information matches, then executes and imaged by the first camera and third
Head is shot, and the operation of second group of uncalibrated image is obtained.
In one embodiment, calibration test module 510 can be also used for when by the first camera and second camera
It is shot, when obtaining first group of uncalibrated image, is shot by third camera, obtain corresponding depth image；It will be deep
The depth information for including in degree image is as predetermined depth information；When the first depth information and predetermined depth information matches, then
Execution is shot by the first camera and third camera, obtains the operation of second group of uncalibrated image.
In one embodiment, calibration test module 510 can be also used for according to the second calibration information to third calibration maps
Picture and the 4th uncalibrated image are corrected processing, obtain third target image and the 4th target image；Extract first object image
In the first pixel, extract the first pixel in corresponding second pixel of the second target image, according to the first pixel with
Second pixel obtains the first parallax；The third pixel in third target image is extracted, extracts third pixel in the 4th mesh
Corresponding 4th pixel of logo image obtains the second parallax according to third pixel and the 4th pixel；Calculate the first parallax with
The disparity difference of second parallax then generates the standby signal that calibration test passes through when disparity difference is less than difference threshold.
In one embodiment, calibration test module 510 can be also used for being greater than or equal to difference threshold when disparity difference
When, then generate the standby signal of calibration test crash；And carry out second of calibration processing.
In one embodiment, the second image collection module 506 can be also used for imaging by the first camera and third
Head shoots scaling board, obtains the corresponding third uncalibrated image of the first camera and corresponding 4th calibration of third camera
Image, wherein the 4th uncalibrated image is gray level image；Obtain second group of mark comprising third uncalibrated image and the 4th uncalibrated image
Determine image.
The division of modules is only used for for example, in other embodiments, can incite somebody to action in above-mentioned camera calibration device
Camera calibration device is divided into different modules as required, to complete all or part of function of above-mentioned camera calibration device
Energy.
Fig. 6 is the schematic diagram of internal structure of electronic equipment in one embodiment.As shown in fig. 6, the electronic equipment includes logical
Cross the processor and memory of system bus connection.Wherein, which supports entire electricity for providing calculating and control ability
The operation of sub- equipment.Memory may include non-volatile memory medium and built-in storage.Non-volatile memory medium is stored with behaviour
Make system and computer program.The computer program can be performed by processor, to be mentioned for realizing following each embodiment
A kind of camera calibration method supplied.Built-in storage provides height for the operating system computer program in non-volatile memory medium
The running environment of speed caching.The electronic equipment can be mobile phone, tablet computer or personal digital assistant or wearable device etc..
Realizing for the modules in camera calibration device provided in the embodiment of the present application can be computer program
Form.The computer program can be run in terminal or server.The program module that the computer program is constituted is storable in end
On the memory of end or server.When the computer program is executed by processor, method described in the embodiment of the present application is realized
The step of.
The embodiment of the present application also provides a kind of electronic equipment.It include image processing circuit in above-mentioned electronic equipment, at image
Reason circuit can use hardware and or software component realization, it may include define ISP (Image Signal Processing, figure
As signal processing) the various processing units of pipeline.Fig. 7 is the schematic diagram of image processing circuit in one embodiment.Such as Fig. 7 institute
Show, for purposes of illustration only, only showing the various aspects of image processing techniques relevant to the embodiment of the present application.
As shown in fig. 7, image processing circuit includes ISP processor 740 and control logic device 750.Imaging device 710 captures
Image data handled first by ISP processor 740, ISP processor 740 to image data analyzed with capture can be used for really
The image statistics of fixed and/or imaging device 710 one or more control parameters.Imaging device 710 may include having one
The camera of a or multiple lens 712 and imaging sensor 714.Imaging sensor 714 may include colour filter array (such as
Bayer filter), imaging sensor 714 can obtain the luminous intensity captured with each imaging pixel of imaging sensor 714 and wavelength
Information, and the one group of raw image data that can be handled by ISP processor 740 is provided.Sensor 720 (such as gyroscope) can be based on biography
The parameter (such as stabilization parameter) of the image procossing of acquisition is supplied to ISP processor 740 by 720 interface type of sensor.Sensor 720
Interface can use SMIA (Standard Mobile Imaging Architecture, Standard Mobile Imager framework) interface,
The combination of other serial or parallel camera interfaces or above-mentioned interface.
In addition, raw image data can also be sent to sensor 720 by imaging sensor 714, sensor 720 can be based on biography
Raw image data is supplied to ISP processor 740 to 720 interface type of sensor or sensor 720 deposits raw image data
It stores up in video memory 730.
ISP processor 740 handles raw image data pixel by pixel in various formats.For example, each image pixel can
Bit depth with 8,10,12 or 14 bits, ISP processor 740 can carry out raw image data at one or more images
Reason operation, statistical information of the collection about image data.Wherein, image processing operations can be by identical or different bit depth precision
It carries out.
ISP processor 740 can also receive image data from video memory 730.For example, 720 interface of sensor will be original
Image data is sent to video memory 730, and the raw image data in video memory 730 is available to ISP processor 740
It is for processing.Video memory 730 can be independent special in a part, storage equipment or electronic equipment of memory device
It with memory, and may include DMA (Direct Memory Access, direct direct memory access (DMA)) feature.
When receiving from 714 interface of imaging sensor or from 720 interface of sensor or from video memory 730
When raw image data, ISP processor 740 can carry out one or more image processing operations, such as time-domain filtering.Treated schemes
As data can be transmitted to video memory 730, to carry out other processing before shown.ISP processor 740 is from image
Memory 730 receives processing data, and carries out in original domain and in RGB and YCbCr color space to the processing data
Image real time transfer.Treated that image data may be output to display 770 for ISP processor 740, for user's viewing and/or
It is further processed by graphics engine or GPU (Graphics Processing Unit, graphics processor).In addition, ISP processor
740 output also can be transmitted to video memory 730, and display 770 can read image data from video memory 730.?
In one embodiment, video memory 730 can be configured to realize one or more frame buffers.In addition, ISP processor 740
Output can be transmitted to encoder/decoder 760, so as to encoding/decoding image data.The image data of coding can be saved,
And it is decompressed before being shown in 770 equipment of display.Encoder/decoder 760 can be real by CPU or GPU or coprocessor
It is existing.
The statistical data that ISP processor 740 determines, which can be transmitted, gives control logic device Unit 750.For example, statistical data can wrap
Include the image sensings such as automatic exposure, automatic white balance, automatic focusing, flicker detection, black level compensation, 712 shadow correction of lens
714 statistical information of device.Control logic device 750 may include the processor and/or micro-control for executing one or more routines (such as firmware)
Device processed, one or more routines can statistical data based on the received, determine the control parameter and ISP processor of imaging device 710
740 control parameter.For example, the control parameter of imaging device 710 may include 720 control parameter of sensor (such as gain, exposure
The time of integration, stabilization parameter of control etc.), camera flash control parameter, 712 control parameter of lens (such as focus or zoom
With focal length) or these parameters combination.ISP control parameter may include for automatic white balance and color adjustment (for example, in RGB
During processing) 712 shadow correction parameter of gain level and color correction matrix and lens.
In the embodiment of the present application, image processing circuit may include at least three imaging devices (camera) 710, use
Image processing techniques can realize above-mentioned camera calibration method in Fig. 7.
The embodiment of the present application also provides a kind of computer readable storage mediums.One or more is executable comprising computer
The non-volatile computer readable storage medium storing program for executing of instruction, when the computer executable instructions are executed by one or more processors
When, so that the step of processor executes camera calibration method.
A kind of computer program product comprising instruction, when run on a computer, so that computer executes camera shooting
Head scaling method.
It may include non-to any reference of memory, storage, database or other media used in the embodiment of the present application
Volatibility and/or volatile memory.Suitable nonvolatile memory may include read-only memory (ROM), programming ROM
(PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include
Random access memory (RAM), it is used as external cache.By way of illustration and not limitation, RAM in a variety of forms may be used
, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDR SDRAM),
Enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM
(RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM).
The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously
The limitation to the application the scope of the patents therefore cannot be interpreted as.It should be pointed out that for those of ordinary skill in the art
For, without departing from the concept of this application, various modifications and improvements can be made, these belong to the guarantor of the application
Protect range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.
Claims (10)
1. a kind of camera calibration method characterized by comprising
It is shot by the first camera and second camera, obtains first group of uncalibrated image；
Calibration processing is carried out to first camera and second camera according to first group of uncalibrated image, obtains the first mark
Determine information；
It is shot by first camera and third camera, obtains second group of uncalibrated image, wherein the third is taken the photograph
As head is depth camera；
Calibration processing is carried out to first camera and third camera according to second group of uncalibrated image, obtains the second mark
Determine information.
2. passing through the first camera and second camera is clapped the method according to claim 1, wherein described
It takes the photograph, obtains first group of uncalibrated image, comprising:
The scaling board of at least three angles is shot by first camera and second camera, obtains described first
Group uncalibrated image；
It is described to be shot by first camera and third camera, obtain second group of uncalibrated image, comprising:
The scaling board of at least three angles is shot by first camera and third camera, obtains described second
Group uncalibrated image.
3. the method according to claim 1, wherein it is described by first camera and third camera into
Row shooting, before obtaining second group of uncalibrated image, further includes:
Obtain the first uncalibrated image and the second camera of the shooting of the first camera described in first group of uncalibrated image
Second uncalibrated image of shooting；
Processing is corrected to first uncalibrated image and the second uncalibrated image according to first calibration information, is corresponded to
First object image and the second target image；
The first depth information is calculated according to the first object image and the second target image；
When first depth information and predetermined depth information matches, then executes and imaged by first camera and third
Head is shot, and the operation of second group of uncalibrated image is obtained.
4. according to the method described in claim 3, it is characterized in that, the method also includes:
It is shot when by the first camera and second camera, when obtaining first group of uncalibrated image, is taken the photograph by the third
As head is shot, corresponding depth image is obtained；
Using the depth information for including in the depth image as the predetermined depth information.
5. the method according to claim 1, wherein including first camera shooting in second group of uncalibrated image
The third uncalibrated image of head shooting and the 4th uncalibrated image of third camera shooting, the method also includes:
Processing is corrected to the third uncalibrated image and the 4th uncalibrated image according to second calibration information, obtains third
Target image and the 4th target image；
The first pixel in the first object image is extracted, extracts first pixel in second target image pair
The second pixel answered obtains the first parallax according to first pixel and the second pixel；
The third pixel in the third target image is extracted, extracts the third pixel in the 4th target image pair
The 4th pixel answered obtains the second parallax according to the third pixel and the 4th pixel；
The disparity difference for calculating first parallax and the second parallax is then generated when the disparity difference is less than difference threshold
The standby signal that calibration test passes through.
6. according to the method described in claim 5, it is characterized in that, the method also includes:
When the disparity difference is greater than or equal to difference threshold, then the standby signal of calibration test crash is generated；And carry out the
Secondary calibration processing.
7. the method according to claim 1, wherein what the first camera of the acquisition and third camera were shot
Second group of uncalibrated image include:
Scaling board is shot by first camera and third camera, obtains first camera corresponding
Three uncalibrated images and corresponding 4th uncalibrated image of the third camera, wherein the 4th uncalibrated image is gray level image；
Obtain second group of uncalibrated image comprising the third uncalibrated image and the 4th uncalibrated image.
8. a kind of camera calibration device characterized by comprising
First image collection module obtains first group of calibration maps for being shot by the first camera and second camera
Picture；
First demarcating module, for being marked according to first group of uncalibrated image to first camera and second camera
Fixed processing, obtains the first calibration information；
Second image collection module obtains second group of mark for being shot by first camera and third camera
Determine image；
Second demarcating module, for being marked according to second group of uncalibrated image to first camera and third camera
Fixed processing, obtains the second calibration information.
9. a kind of electronic equipment, including camera, memory and processor, computer program is stored in the memory, institute
When stating computer program and being executed by the processor, so that the processor is executed as described in any one of claims 1 to 7
The step of camera calibration method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The step of method as described in any one of claims 1 to 7 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811453896.3A CN109658459B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811453896.3A CN109658459B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109658459A true CN109658459A (en) | 2019-04-19 |
| CN109658459B CN109658459B (en) | 2020-11-24 |
Family
ID=66111980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811453896.3A Active CN109658459B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109658459B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111832425A (en) * | 2020-06-22 | 2020-10-27 | 五邑大学 | Target ranging method and device for agricultural trolley and storage medium |
| CN112598751A (en) * | 2020-12-23 | 2021-04-02 | Oppo（重庆）智能科技有限公司 | Calibration method and device, terminal and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610185A (en) * | 2017-10-12 | 2018-01-19 | 长沙全度影像科技有限公司 | A kind of fisheye camera fast calibration device and scaling method |
| CN107730462A (en) * | 2017-09-30 | 2018-02-23 | 努比亚技术有限公司 | A kind of image processing method, terminal and computer-readable recording medium |
| CN108288294A (en) * | 2018-01-17 | 2018-07-17 | 视缘(上海)智能科技有限公司 | A kind of outer ginseng scaling method of a 3D phases group of planes |
-
2018
- 2018-11-30 CN CN201811453896.3A patent/CN109658459B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107730462A (en) * | 2017-09-30 | 2018-02-23 | 努比亚技术有限公司 | A kind of image processing method, terminal and computer-readable recording medium |
| CN107610185A (en) * | 2017-10-12 | 2018-01-19 | 长沙全度影像科技有限公司 | A kind of fisheye camera fast calibration device and scaling method |
| CN108288294A (en) * | 2018-01-17 | 2018-07-17 | 视缘(上海)智能科技有限公司 | A kind of outer ginseng scaling method of a 3D phases group of planes |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111832425A (en) * | 2020-06-22 | 2020-10-27 | 五邑大学 | Target ranging method and device for agricultural trolley and storage medium |
| CN112598751A (en) * | 2020-12-23 | 2021-04-02 | Oppo（重庆）智能科技有限公司 | Calibration method and device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109658459B (en) | 2020-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111091063B (en) | Living body detection method, device and system | |
| CN110689581B (en) | Structured light module calibration method, electronic device and computer readable storage medium | |
| CN109712192B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN101630406B (en) | Camera calibration method and camera calibration device | |
| CN109598764A (en) | Camera calibration method and device, electronic equipment, computer readable storage medium | |
| CN110536057A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN109598763A (en) | Camera calibration method, device, electronic equipment and computer readable storage medium | |
| JP2019510234A (en) | Depth information acquisition method and apparatus, and image acquisition device | |
| CN109584312A (en) | Camera calibration method, device, electronic equipment and computer readable storage medium | |
| CN109685853B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| WO2019232793A1 (en) | Two-camera calibration method, electronic device and computer-readable storage medium | |
| CN109600548A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN109559353B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN112004029B (en) | Exposure processing method, exposure processing device, electronic apparatus, and computer-readable storage medium | |
| CN111385461B (en) | Panoramic shooting method and device, camera and mobile terminal | |
| CN111882655A (en) | Method, apparatus, system, computer device and storage medium for three-dimensional reconstruction | |
| CN109660718A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN109963080A (en) | Image-pickup method, device, electronic equipment and computer storage medium | |
| CN108924426A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN109559352B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109951641A (en) | Image capturing method and device, electronic equipment, computer readable storage medium | |
| CN109697737A (en) | Camera calibration method, device, electronic equipment and computer readable storage medium | |
| CN109658459A (en) | Camera calibration method, device, electronic equipment and computer readable storage medium | |
| CN109584311B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109671028A (en) | Image processing method and device, electronic equipment, computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210615 Address after: Room 01, 8th floor, No.1 Lane 61, shengxia Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai, 200120 Patentee after: Zheku Technology (Shanghai) Co.,Ltd. Address before: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Patentee before: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20241017 Address after: 6th Floor, No.1 Chongqing Road, Banqiao District, Xinbei City, Taiwan, China, China Patentee after: Weiguang Co.,Ltd. Country or region after: Samoa Address before: Room 01, 8th floor, No.1 Lane 61, shengxia Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai, 200120 Patentee before: Zheku Technology (Shanghai) Co.,Ltd. Country or region before: China |