CN117528259B - Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium - Google Patents
Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium Download PDFInfo
- Publication number
- CN117528259B CN117528259B CN202410025694.8A CN202410025694A CN117528259B CN 117528259 B CN117528259 B CN 117528259B CN 202410025694 A CN202410025694 A CN 202410025694A CN 117528259 B CN117528259 B CN 117528259B
- Authority
- CN
- China
- Prior art keywords
- light supplementing
- cradle head
- shooting
- light
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000001502 supplementing effect Effects 0.000 title claims abstract description 233
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000003860 storage Methods 0.000 title claims abstract description 21
- 230000008859 change Effects 0.000 claims abstract description 106
- 206010034719 Personality change Diseases 0.000 claims abstract description 42
- 238000005286 illumination Methods 0.000 claims description 37
- 238000009826 distribution Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 29
- 238000007781 pre-processing Methods 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 13
- 230000005484 gravity Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000005520 cutting process Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 238000010183 spectrum analysis Methods 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/71—Circuitry for evaluating the brightness variation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/681—Motion detection
- H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/685—Vibration or motion blur correction performed by mechanical compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Studio Devices (AREA)
Abstract
The invention provides an intelligent shooting light supplementing method, device and equipment for a cradle head and a storage medium, wherein the method comprises the following steps: when the cradle head is opened, shooting is carried out through the image pickup equipment to obtain a first shooting image, and the light supplementing equipment is controlled to conduct primary light supplementing according to the first shooting image; when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a micro light supplementing zone; if yes, calculating the shake compensation quantity of the cradle head according to the attitude change quantity; and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity. When the cradle head moves in the posture, the method judges whether micro-light supplementing is needed according to the calculated posture change quantity. The problem of excessive light filling or insufficient light filling is avoided through a preset micro light filling interval, and in addition, the micro light filling amount is calculated by using the attitude change amount and the jitter compensation amount, so that the image can always keep proper brightness and contrast.
Description
Technical Field
The invention relates to the field of travel analysis, in particular to an intelligent shooting light supplementing method, device and equipment for a cradle head and a storage medium.
Background
In the current society, the application of camera equipment and cloud deck technology is more and more widespread, from security monitoring to movie production, from unmanned aerial vehicle shooting to video conference, they play an important role in various fields. However, as technology advances and demand continues to grow, we are also faced with challenges. In the existing imaging device and pan-tilt technology, there is a general problem that when the imaging device and the pan-tilt are moved in a gesture, the brightness and contrast of an output image may be fluctuated or unstable due to factors such as a change of a light condition or a tiny shake of the pan-tilt. In order to solve this problem, some technical solutions have been proposed, for example, to adjust the brightness of the light supplementing device with a preset brightness compensation value to maintain the stability of the output image. However, the prior art often cannot accurately calculate the attitude change amount and finely control the micro-light-compensating interval, so that the light-compensating effect cannot reach the optimal state.
Disclosure of Invention
The invention mainly aims to solve the technical problem that the light supplementing effect cannot reach the optimal state due to slight shake generated by shooting by using a cradle head in the prior art.
The first aspect of the present invention provides an intelligent shooting light filling method for a pan-tilt, wherein light filling equipment and image capturing equipment are mounted on the pan-tilt, and the intelligent shooting light filling method for the pan-tilt comprises:
when the cradle head is opened, shooting through the image pickup equipment to obtain a first shooting image, and controlling the light supplementing equipment to conduct primary light supplementing according to the first shooting image;
when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a preset micro light supplementing zone or not;
if the attitude change quantity is in a preset micro-light supplementing interval, calculating the shake compensation quantity of the cradle head according to the attitude change quantity;
and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity.
Optionally, in a first implementation manner of the first aspect of the present invention, when the pan-tilt is turned on, shooting the shooting object by the image capturing device, and controlling the light supplementing device to supplement light once according to the first shooting image includes:
When the cradle head is opened, shooting an object through the image shooting equipment to obtain a first shooting image, and cutting the first shooting image to obtain a foreground image and a background image in the first shooting image;
respectively carrying out illumination distribution identification on the foreground image and the background image to obtain first illumination distribution data and second illumination distribution data;
and calculating initial light supplementing quantity of the shooting equipment based on the first illumination distribution data and the second illumination distribution data, and controlling the light supplementing equipment to conduct primary light supplementing according to the initial light supplementing quantity.
Optionally, in a second implementation manner of the first aspect of the present invention, a gyroscope and an accelerometer are installed on the pan-tilt; when the cradle head moves in a posture, calculating the posture change amount of the cradle head comprises:
when the cradle head moves in a gesture, acquiring the rotation speed of the cradle head in each axial direction in a machine body coordinate axis through the gyroscope, and acquiring the current gravity direction information of the cradle head through an accelerometer;
calculating the rotation angles of the cradle head in each axial direction in the machine body coordinate axis according to the rotation speed and the gravity direction information;
And taking the rotation angles of the axial directions as the posture change quantity of the cradle head.
Optionally, in a third implementation manner of the first aspect of the present invention, when the pan-tilt generates the gesture movement, calculating a gesture variation of the pan-tilt, and determining whether the gesture variation is within a preset micro light filling interval, further includes:
if the posture change amount is not in the preset micro-light supplementing interval, judging whether the posture change amount is larger than the maximum value of the micro-light supplementing interval or not;
if the attitude change quantity is larger than the maximum value of the micro light supplementing section, acquiring a second shooting image obtained by shooting by the shooting equipment;
comparing the first shooting image with the second shooting image, and judging whether a shooting object change occurs between the first shooting image and the second shooting image;
and if the shot object changes, controlling the light supplementing device to conduct secondary light supplementing based on the second shot image.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the calculating the shake compensation amount of the pan-tilt according to the attitude change amount includes:
filtering the attitude change quantity, and obtaining the change frequency and the corresponding change amplitude of the attitude change quantity after the filtering;
Judging whether the cradle head shakes or not according to the change frequency and the change amplitude;
if shake exists, acquiring a target pose and a current pose of the holder, and calculating an attitude error of the holder according to the target pose and the current pose;
and calculating the shake compensation amount of the cradle head based on a preset compensation algorithm and the attitude error.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the calculating a micro-light-compensating amount of the light-compensating device based on the posture change amount and the shake compensation amount, and controlling the light-compensating device to perform secondary light compensation according to the micro-light-compensating amount includes:
performing data preprocessing on the attitude change amount and the jitter compensation amount to obtain preprocessed data;
inputting the preprocessing data into a preset light supplementing quantity prediction model, and calculating the micro light supplementing quantity of the light supplementing equipment through the light supplementing quantity prediction model;
and controlling the light supplementing equipment to carry out secondary light supplementing according to the micro light supplementing quantity.
Optionally, in a sixth implementation manner of the first aspect of the present invention, before the calculating, based on the attitude change amount and the shake compensation amount, a micro-light compensation amount of the light compensation device, and controlling the light compensation device according to the micro-light compensation amount to perform secondary light compensation, the method further includes:
Acquiring the historical posture change quantity and the historical jitter compensation quantity of the cradle head, and setting light supplementing quantity corresponding to the preset historical posture change quantity and the historical jitter compensation quantity;
preprocessing the historical posture change amount and the historical jitter compensation amount, and dividing time steps to obtain a phase posture change amount and a phase jitter compensation amount corresponding to a plurality of time steps;
generating a time sequence according to the phase posture change quantity and the phase jitter compensation quantity corresponding to the time steps, and inputting the time sequence into a preset initial model, wherein the initial model is a long-time and short-time memory network based on an attention model;
extracting features of the time sequence through an input layer of the initial model to obtain sequence features, and calculating attention weights of time steps corresponding to the time sequence based on the attention mechanism;
calculating a context vector of the time sequence according to the attention weight and the sequence characteristic, and inputting the context vector into the long-short-time memory network to obtain a predicted light supplementing quantity;
calculating a preset loss function according to the set light supplementing quantity and the predicted light supplementing quantity to obtain a loss function value, and judging whether the loss function value is smaller than a preset loss threshold value or not;
If not, updating the model parameters of the initial model according to the loss function value, and returning to the step of inputting the time sequence into a preset initial model until the loss function value is smaller than a preset loss threshold value;
if yes, the initial model is used as a light supplementing quantity prediction model of the cradle head.
The second aspect of the present invention provides an intelligent shooting light filling device for a pan-tilt, on which light filling equipment and image capturing equipment are mounted, the intelligent shooting light filling device for a pan-tilt comprising:
the initial light supplementing module is used for shooting through the camera equipment when the cradle head is started to obtain a first shooting image, and controlling the light supplementing equipment to conduct primary light supplementing according to the first shooting image;
the attitude calculation module is used for calculating the attitude variation of the cradle head when the cradle head moves in an attitude, and judging whether the attitude variation is in a preset micro-light supplementing zone or not;
the compensation amount calculating module is used for calculating the shake compensation amount of the cradle head according to the attitude variation if the attitude variation is in a preset micro-light supplementing interval;
and the micro-light supplementing model is used for calculating the micro-light supplementing quantity of the light supplementing device based on the attitude change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity.
The third aspect of the present invention provides an intelligent shooting light supplementing device for a pan-tilt, comprising: a memory and at least one processor, the memory having instructions stored therein, the memory and the at least one processor being interconnected by a line; and the at least one processor calls the instruction in the memory so that the intelligent shooting light filling equipment of the cradle head executes the steps of the intelligent shooting light filling method of the cradle head.
A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored therein, which when run on a computer, cause the computer to perform the steps of the intelligent shooting light filling method of a pan-tilt as described above.
According to the intelligent shooting light supplementing method, device, equipment and storage medium of the cradle head, when the cradle head is started, shooting is carried out through the shooting equipment to obtain a first shooting image, and the light supplementing equipment is controlled to conduct primary light supplementing according to the first shooting image; when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a micro light supplementing zone; if yes, calculating the shake compensation quantity of the cradle head according to the attitude change quantity; and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity. When the cradle head moves in the posture, the method judges whether micro-light supplementing is needed according to the calculated posture change quantity. The problem of excessive light filling or insufficient light filling is avoided through a preset micro light filling interval, and in addition, the micro light filling amount is calculated by using the attitude change amount and the jitter compensation amount, so that the image can always keep proper brightness and contrast.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
Fig. 1 is a schematic diagram of an embodiment of an intelligent shooting light supplementing method for a pan-tilt head according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an embodiment of an intelligent photographing light supplementing device of a pan-tilt head according to an embodiment of the present invention;
fig. 3 is a schematic diagram of another embodiment of an intelligent shooting light supplementing device of a pan-tilt in an embodiment of the present invention;
fig. 4 is a schematic diagram of an embodiment of an intelligent photographing light supplementing device for a pan-tilt in an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "comprising" and "having" and any variations thereof, as used in the embodiments of the present invention, are intended to cover non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or apparatus.
For the convenience of understanding the present embodiment, the embodiment of the present invention discloses an intelligent shooting light supplementing method for a pan-tilt head. As shown in fig. 1, the method for intelligent shooting and light supplementing of the pan-tilt comprises the following steps:
101. when the cradle head is opened, shooting is carried out through the image pickup equipment to obtain a first shooting image, and the light supplementing equipment is controlled to conduct primary light supplementing according to the first shooting image;
in this embodiment, when the cradle head is turned on, shooting the shooting object by the image capturing device, and controlling the light supplementing device to supplement light once according to the first shooting image includes: when the cradle head is opened, shooting an object through the image shooting equipment to obtain a first shooting image, and cutting the first shooting image to obtain a foreground image and a background image in the first shooting image; respectively carrying out illumination distribution identification on the foreground image and the background image to obtain first illumination distribution data and second illumination distribution data; and calculating initial light supplementing quantity of the shooting equipment based on the first illumination distribution data and the second illumination distribution data, and controlling the light supplementing equipment to conduct primary light supplementing according to the initial light supplementing quantity.
Specifically, when the cradle head is opened, the image capturing device records image information of the whole scene, and selects an image which is initially captured as a first captured image, and cuts the first captured image to obtain a corresponding foreground image and a background image, wherein the foreground image generally comprises a main captured object, the background image generally comprises surrounding environment and other unimportant elements, and the cutting of the foreground and the background is mainly due to the change of factors such as the position, the shape, the size and the like of the object in the scene, and the illumination distribution of the foreground and the background may be different. For foreground objects, the illumination distribution may be determined primarily by the incident light and the reflective properties of the object itself; the background area may be affected by various factors such as ambient light, shadows, reflection, etc. Thus, cutting the image into foreground and background can better analyze and model their respective illumination distributions. In the image cutting process, the first shot image needs to be preprocessed, including denoising, graying, edge detection and other operations, so that the accuracy of subsequent cutting is improved. Next, a suitable image segmentation algorithm is selected, such as a segmentation method based on pixel color, texture, shape, etc. These algorithms can divide the image feature into different regions, resulting in image portions of foreground and background. Then, the segmentation processing is carried out, the foreground and the background in the first shot image are effectively separated through the computer vision technology and the image processing algorithm, and finally, necessary post-processing such as edge smoothing, cavity filling and the like is carried out on the segmented foreground image and background image, so that the segmentation effect is ensured to be more accurate and natural.
Specifically, when the illumination distribution of the foreground image and the background image is identified, the illumination direction and intensity can be inferred for the foreground image by using the characteristics of high light, shadow and the like in the image. This can be achieved by calculating local gradient information in the image to obtain the normal direction of the surface and deducing the angle of incidence and intensity of the illumination from the normal direction. Or modeling and recognition of illumination distribution for foreground images using statistical analysis or machine learning based methods, such as illumination estimation and analysis with illumination spheres, illumination models, etc. The invention is not limited, and the illumination distribution of the background image is generally influenced by a plurality of factors such as ambient light, shadow, reflection and the like, so that the illumination distribution identification can be more complex, the illumination distribution situation can be deduced by utilizing the characteristics such as color, texture, shadow and the like in the background image, and the illumination can be deduced and modeled by combining the depth information. After the illumination distribution identification is carried out on the foreground image and the background image through the steps, the respective illumination distribution data of the foreground image and the background image can be obtained. Such data may include information such as direction, intensity, color, and distribution of the illumination.
102. When the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a preset micro light supplementing interval;
in one embodiment of the invention, a gyroscope and an accelerometer are mounted on the cradle head; when the cradle head moves in a posture, calculating the posture change amount of the cradle head comprises: when the cradle head moves in a gesture, acquiring the rotation speed of the cradle head in each axial direction in a machine body coordinate axis through the gyroscope, and acquiring the current gravity direction information of the cradle head through an accelerometer; calculating the rotation angles of the cradle head in each axial direction in the machine body coordinate axis according to the rotation speed and the gravity direction information; and taking the rotation angles of the axial directions as the posture change quantity of the cradle head.
Specifically, when the cradle head is opened, the above sensors, including the gyroscope and the accelerometer, need to be initialized, so that the gyroscope and the accelerometer can be ensured to work normally, and the gyroscope and the accelerometer can be calibrated to eliminate any deviation or error. And acquiring the rotation speeds of the cradle head in each axial direction in the machine body coordinate axis through the gyroscope. Gyroscopes typically provide angular velocity measurements in three axial directions (e.g., X, Y and Z-axis). These values represent the rotational speed of the pan/tilt head about the axes. And acquiring current gravity direction information of the cradle head through an accelerometer. The accelerometer can measure acceleration of the cradle head in each axial direction. The accelerometer may provide a gravity direction vector pointing to the earth's center due to the force of gravity. And calculating the rotation angles of the cradle head in each axial direction in the machine body coordinate axis through the rotation speed and the gravity direction information. The method can be realized by using an attitude estimation algorithm, and a Kalman filtering method, a complementary filtering method or a quaternion method is adopted in a common algorithm, so that the rotation angles of the cradle head in all axial directions in a machine body coordinate axis can be obtained according to the method.
Further, when the cradle head moves in posture, calculating a posture change amount of the cradle head, and judging whether the posture change amount is behind a preset micro-light supplementing interval, the method further comprises: if the posture change amount is not in the preset micro-light supplementing interval, judging whether the posture change amount is larger than the maximum value of the micro-light supplementing interval or not; if the attitude change quantity is larger than the maximum value of the micro light supplementing section, acquiring a second shooting image obtained by shooting by the shooting equipment; comparing the first shooting image with the second shooting image, and judging whether a shooting object change occurs between the first shooting image and the second shooting image; and if the shot object changes, controlling the light supplementing device to conduct secondary light supplementing based on the second shot image.
Specifically, the micro-light-compensating section is set to determine whether the cradle head needs to perform micro-light-compensating adjustment. When the measured posture change amount falls within the micro light filling section, the change can be considered to be tiny, at the moment, tiny light filling adjustment is needed to keep the shot light filling stable, and when the posture change amount exceeds the micro light filling section, the state that the cradle head has changed greatly is indicated, and light filling is needed to be carried out again according to the shot image so as to adapt to the new shooting angle and condition. The arrangement of the micro-light supplementing interval can help the system to carry out timely fine adjustment when the gesture changes in a small amplitude, and meanwhile unnecessary reactions to the gesture changes in a larger amplitude are avoided. Thus, the relationship between the response speed to the change of the posture of the cradle head and the system stability can be balanced better. And the posture offset corresponding to the minimum value smaller than the micro-light supplementing interval indicates that the posture change of the cradle head is too slight and even micro-light supplementing adjustment is not needed.
103. If the posture change quantity is in a preset micro-light supplementing interval, calculating the shake compensation quantity of the cradle head according to the posture change quantity;
in one embodiment of the present invention, the calculating the shake compensation amount of the pan-tilt according to the attitude change amount includes: filtering the attitude change quantity, and obtaining the change frequency and the corresponding change amplitude of the attitude change quantity after the filtering; judging whether the cradle head shakes or not according to the change frequency and the change amplitude; if shake exists, acquiring a target pose and a current pose of the holder, and calculating an attitude error of the holder according to the target pose and the current pose; and calculating the shake compensation amount of the cradle head based on a preset compensation algorithm and the attitude error.
Specifically, the filtering process is to eliminate instability caused by sensor noise, mechanical vibration, and the like, and common filtering algorithms include a kalman filter, a low-pass filter, and the like. The algorithms can carry out smoothing processing on the original attitude change quantity, and reduce the influence of noise and mutation on the result. The change amount of the gesture after the filtering treatment can be obtained by spectrum analysis or time domain analysis. The spectrum analysis may use a Fast Fourier Transform (FFT) method, and the time domain analysis may acquire information of the change frequency and the change amplitude by observing the change condition of the posture change amount with time. By comparing the change frequency and the change amplitude with a preset threshold value, whether the gesture change is in a normal range can be judged. If the change frequency and the change amplitude exceed the preset threshold values, it can be judged that jitter exists. If shake exists, current attitude information of the holder is obtained through a sensor, wherein the current attitude information comprises an attitude angle and a position, and then an attitude error between a target pose and the current pose, comprising the deviation of the attitude angle and the position, is calculated. This can be achieved by a simple subtraction operation. And according to a preset compensation algorithm (such as a PID controller, a fuzzy controller and the like), and combining the attitude errors, calculating the jitter compensation quantity required by the cradle head. For example, the compensation amount of the pan-tilt can be adjusted according to the magnitude and the variation trend of the attitude error so as to return to the stable state as much as possible. The specific calculation of the compensation amount by the PID can be performed by calculating the proportional term by using the proportional controller, i.e. generating a compensation amount according to the magnitude of the attitude error. The calculation formula of the proportion term is as follows: proportional term = proportional gain x attitude error. The proportional gain is an adjustable parameter used to control the degree of reaction of the proportional term to the attitude error. Higher proportional gain may make compensation more sensitive, but instability may also be introduced. Next, an integral controller is used to calculate an integral term, i.e., a compensation amount is generated based on the accumulated amount of the attitude error. The calculation formula of the integral term is as follows: integral term = integral gain x cumulative attitude error. The integral gain is another adjustable parameter for controlling the degree of reaction of the integral term to the posing error. The integral term can eliminate persistent static errors, but excessive integral gain can cause system oscillations. The differential controller is then used to calculate the differential term, i.e. to generate a compensation amount based on the rate of change of the attitude error. The calculation formula of the differential term is as follows: differential term = differential gain x rate of change of attitude error. The differential gain is a third adjustable parameter for controlling the degree of reaction of the differential term to the rate of change of attitude error. The differential term can suppress rapidly changing errors, but excessive differential gain may introduce noise amplification. And finally, adding proportional, integral and differential terms to obtain the total compensation quantity. Total compensation = proportional term + integral term + differential term.
104. And calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity.
In one embodiment of the present invention, the calculating the micro-light-compensation amount of the light-compensation device based on the posture change amount and the shake compensation amount, and controlling the light-compensation device to perform the secondary light-compensation according to the micro-light-compensation amount includes: performing data preprocessing on the attitude change amount and the jitter compensation amount to obtain preprocessed data; inputting the preprocessing data into a preset light supplementing quantity prediction model, and calculating the micro light supplementing quantity of the light supplementing equipment through the light supplementing quantity prediction model; and controlling the light supplementing equipment to carry out secondary light supplementing according to the micro light supplementing quantity.
Specifically, necessary data preprocessing is performed on the attitude change amount and the jitter compensation amount, including data cleaning, normalization, outlier processing, and the like, so as to ensure the accuracy and stability of the input data. The preprocessed data is input into a pre-constructed light replenishment quantity prediction model, which may be a machine learning based regression model or other prediction algorithm. The model will predict the corresponding amount of light complement using the input attitude change amount and shake compensation amount data. And calculating the micro-light compensation quantity required by the light compensation equipment through a light compensation quantity prediction model, wherein the micro-light compensation quantity is dynamically adjusted according to the current posture change condition and the jitter compensation requirement. And controlling the light supplementing equipment to perform secondary light supplementing operation according to the calculated micro light supplementing quantity so as to perform fine light supplementing adjustment according to the current posture change condition, thereby improving the imaging quality.
Further, before the calculating the micro-light-compensating amount of the light compensating device based on the posture change amount and the shake compensation amount, and controlling the light compensating device to perform secondary light compensating according to the micro-light-compensating amount, the method further includes: acquiring the historical posture change quantity and the historical jitter compensation quantity of the cradle head, and setting light supplementing quantity corresponding to the preset historical posture change quantity and the historical jitter compensation quantity; preprocessing the historical posture change amount and the historical jitter compensation amount, and dividing time steps to obtain a phase posture change amount and a phase jitter compensation amount corresponding to a plurality of time steps; generating a time sequence according to the phase posture change quantity and the phase jitter compensation quantity corresponding to the time steps, and inputting the time sequence into a preset initial model, wherein the initial model is a long-time and short-time memory network based on an attention model; extracting features of the time sequence through an input layer of the initial model to obtain sequence features, and calculating attention weights of time steps corresponding to the time sequence based on the attention mechanism; calculating a context vector of the time sequence according to the attention weight and the sequence characteristic, and inputting the context vector into the long-short-time memory network to obtain a predicted light supplementing quantity; calculating a preset loss function according to the set light supplementing quantity and the predicted light supplementing quantity to obtain a loss function value, and judging whether the loss function value is smaller than a preset loss threshold value or not; if not, updating the model parameters of the initial model according to the loss function value, and returning to the step of inputting the time sequence into a preset initial model until the loss function value is smaller than a preset loss threshold value; if yes, the initial model is used as a light supplementing quantity prediction model of the cradle head.
Specifically, in the model training process, a training data set containing the posture change amount, the shake compensation amount, and the micro-compensation amount is prepared. The dataset is divided into an input sequence and an output target. An LSTM model based on an attention mechanism is constructed. The model is typically composed of one or more LSTM layers and an attention layer. The attention layer is responsible for calculating the attention weight for each time step in the input sequence. At each time step, the attention layer calculates an attention weight for each time step based on the characteristics of the input sequence. These weights reflect the degree of attention of the model to the different time steps in the input sequence. The attention layer calculates a context vector based on the attention weights and the characteristics of the input sequence. The context vector is obtained by a weighted average of the input sequence, wherein the attention weights are used to indicate the importance of the individual time steps. The context vector is used as an input of the LSTM model to predict the micro-light quantity. The output layer of the LSTM model may be set as a single neuron with a suitable activation function for predicting micro-patches. The model predictions and loss of true micro-patch are calculated and back-propagation algorithms are used to optimize the model parameters to minimize the loss function. Common optimization algorithms include random gradient descent (SGD), adam, etc. The above steps are repeated until the model converges or reaches a specified number of training rounds. In each training round, the model propagates forward according to the input sequence, calculates the loss and updates the parameters. Through the attention mechanism, the model can adjust the attention degree of different time steps according to the input weight. In particular, the attention mechanism focuses more attention on time steps that are more important to the current task by calculating the attention weight for each time step. This helps the model to better understand the inherent structure of the input sequence and improves the modeling ability of the model on the input sequence. The calculation of the attention weights is typically based on features of the input sequence, and different methods may be used to calculate the weights, for example using neural networks, attention functions or other mechanisms. These methods typically take into account interactions and correlations between time steps in the input sequence, resulting in weights for each time step. By adjusting the attention weight, the model can learn which time steps are more important to the prediction task during the training process and use the information of these critical time steps to make predictions. This can improve the modeling ability of the model for the input sequence and can better adapt to different input data.
In the embodiment, when the cradle head is opened, shooting is performed through the image pickup device to obtain a first shooting image, and the light supplementing device is controlled to conduct primary light supplementing according to the first shooting image; when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a micro light supplementing zone; if yes, calculating the shake compensation quantity of the cradle head according to the attitude change quantity; and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity. When the cradle head moves in the posture, the method judges whether micro-light supplementing is needed according to the calculated posture change quantity. The problem of excessive light filling or insufficient light filling is avoided through a preset micro light filling interval, and in addition, the micro light filling amount is calculated by using the attitude change amount and the jitter compensation amount, so that the image can always keep proper brightness and contrast.
The foregoing describes an intelligent shooting light filling method for a pan-tilt in an embodiment of the present invention, and the following describes an intelligent shooting light filling device for a pan-tilt in an embodiment of the present invention, where light filling equipment and image capturing equipment are mounted on the pan-tilt, please refer to fig. 2, and one embodiment of the intelligent shooting light filling device for a pan-tilt in an embodiment of the present invention includes:
An initial light supplementing module 201, configured to, when the pan-tilt is turned on, perform shooting through the image capturing device to obtain a first shot image, and control the light supplementing device to perform primary light supplementing according to the first shot image;
the gesture calculation module 202 is configured to calculate a gesture variation of the pan-tilt and determine whether the gesture variation is in a preset micro-light supplementing interval when the pan-tilt moves in a gesture;
the compensation amount calculating module 203 is configured to calculate a shake compensation amount of the pan-tilt according to the posture variation if the posture variation is in a preset micro-light supplementing interval;
and a micro-light supplementing mode 204, configured to calculate a micro-light supplementing amount of the light supplementing device based on the posture change amount and the shake compensation amount, and control the light supplementing device to perform secondary light supplementing according to the micro-light supplementing amount.
In the embodiment of the invention, the intelligent shooting and light supplementing device of the cradle head runs the intelligent shooting and light supplementing method of the cradle head, when the cradle head is started, the intelligent shooting and light supplementing device of the cradle head shoots through the shooting equipment to obtain a first shooting image, and the light supplementing equipment is controlled to conduct primary light supplementing according to the first shooting image; when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a micro light supplementing zone; if yes, calculating the shake compensation quantity of the cradle head according to the attitude change quantity; and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity. When the cradle head moves in the posture, the method judges whether micro-light supplementing is needed according to the calculated posture change quantity. The problem of excessive light filling or insufficient light filling is avoided through a preset micro light filling interval, and in addition, the micro light filling amount is calculated by using the attitude change amount and the jitter compensation amount, so that the image can always keep proper brightness and contrast.
Referring to fig. 3, a second embodiment of an intelligent shooting light filling apparatus for a pan/tilt head according to an embodiment of the present invention includes:
an initial light supplementing module 201, configured to, when the pan-tilt is turned on, perform shooting through the image capturing device to obtain a first shot image, and control the light supplementing device to perform primary light supplementing according to the first shot image;
the gesture calculation module 202 is configured to calculate a gesture variation of the pan-tilt and determine whether the gesture variation is in a preset micro-light supplementing interval when the pan-tilt moves in a gesture;
the compensation amount calculating module 203 is configured to calculate a shake compensation amount of the pan-tilt according to the posture variation if the posture variation is in a preset micro-light supplementing interval;
and a micro-light supplementing mode 204, configured to calculate a micro-light supplementing amount of the light supplementing device based on the posture change amount and the shake compensation amount, and control the light supplementing device to perform secondary light supplementing according to the micro-light supplementing amount.
In one embodiment of the present invention, the initial light compensating module 201 is specifically configured to:
when the cradle head is opened, shooting an object through the image shooting equipment to obtain a first shooting image, and cutting the first shooting image to obtain a foreground image and a background image in the first shooting image;
Respectively carrying out illumination distribution identification on the foreground image and the background image to obtain first illumination distribution data and second illumination distribution data;
and calculating initial light supplementing quantity of the shooting equipment based on the first illumination distribution data and the second illumination distribution data, and controlling the light supplementing equipment to conduct primary light supplementing according to the initial light supplementing quantity.
In one embodiment of the invention, a gyroscope and an accelerometer are mounted on the cradle head; the gesture calculation module 202 is specifically configured to:
when the cradle head moves in a gesture, acquiring the rotation speed of the cradle head in each axial direction in a machine body coordinate axis through the gyroscope, and acquiring the current gravity direction information of the cradle head through an accelerometer;
calculating the rotation angles of the cradle head in each axial direction in the machine body coordinate axis according to the rotation speed and the gravity direction information;
and taking the rotation angles of the axial directions as the posture change quantity of the cradle head.
In an embodiment of the present invention, the intelligent shooting light filling device of the pan-tilt further includes a changing light filling module 205, where the changing light filling module 205 is specifically configured to:
if the posture change amount is not in the preset micro-light supplementing interval, judging whether the posture change amount is larger than the maximum value of the micro-light supplementing interval or not;
If the attitude change quantity is larger than the maximum value of the micro light supplementing section, acquiring a second shooting image obtained by shooting by the shooting equipment;
comparing the first shooting image with the second shooting image, and judging whether a shooting object change occurs between the first shooting image and the second shooting image;
and if the shot object changes, controlling the light supplementing device to conduct secondary light supplementing based on the second shot image.
In one embodiment of the present invention, the compensation calculating module 203 is specifically configured to:
filtering the attitude change quantity, and obtaining the change frequency and the corresponding change amplitude of the attitude change quantity after the filtering;
judging whether the cradle head shakes or not according to the change frequency and the change amplitude;
if shake exists, acquiring a target pose and a current pose of the holder, and calculating an attitude error of the holder according to the target pose and the current pose;
and calculating the shake compensation amount of the cradle head based on a preset compensation algorithm and the attitude error.
In one embodiment of the present invention, the micro-complementary light model 204 is specifically used for:
performing data preprocessing on the attitude change amount and the jitter compensation amount to obtain preprocessed data;
Inputting the preprocessing data into a preset light supplementing quantity prediction model, and calculating the micro light supplementing quantity of the light supplementing equipment through the light supplementing quantity prediction model;
and controlling the light supplementing equipment to carry out secondary light supplementing according to the micro light supplementing quantity.
In an embodiment of the present invention, the intelligent shooting light filling device of the pan-tilt further includes a model training module 206, where the model training module 206 is specifically configured to:
acquiring the historical posture change quantity and the historical jitter compensation quantity of the cradle head, and setting light supplementing quantity corresponding to the preset historical posture change quantity and the historical jitter compensation quantity;
preprocessing the historical posture change amount and the historical jitter compensation amount, and dividing time steps to obtain a phase posture change amount and a phase jitter compensation amount corresponding to a plurality of time steps;
generating a time sequence according to the phase posture change quantity and the phase jitter compensation quantity corresponding to the time steps, and inputting the time sequence into a preset initial model, wherein the initial model is a long-time and short-time memory network based on an attention model;
extracting features of the time sequence through an input layer of the initial model to obtain sequence features, and calculating attention weights of time steps corresponding to the time sequence based on the attention mechanism;
Calculating a context vector of the time sequence according to the attention weight and the sequence characteristic, and inputting the context vector into the long-short-time memory network to obtain a predicted light supplementing quantity;
calculating a preset loss function according to the set light supplementing quantity and the predicted light supplementing quantity to obtain a loss function value, and judging whether the loss function value is smaller than a preset loss threshold value or not;
if not, updating the model parameters of the initial model according to the loss function value, and returning to the step of inputting the time sequence into a preset initial model until the loss function value is smaller than a preset loss threshold value;
if yes, the initial model is used as a light supplementing quantity prediction model of the cradle head.
The embodiment describes the specific functions of each module and the unit constitution of part of the modules in detail on the basis of the previous embodiment, and when the cradle head is opened, the shooting is performed through the shooting equipment to obtain a first shooting image, and the light supplementing equipment is controlled to conduct primary light supplementing according to the first shooting image through each module and each unit in the modules; when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a micro light supplementing zone; if yes, calculating the shake compensation quantity of the cradle head according to the attitude change quantity; and calculating micro-light supplementing quantity of the light supplementing device based on the posture change quantity and the jitter compensation quantity, and controlling the light supplementing device to conduct secondary light supplementing according to the micro-light supplementing quantity. When the cradle head moves in the posture, the method judges whether micro-light supplementing is needed according to the calculated posture change quantity. The problem of excessive light filling or insufficient light filling is avoided through a preset micro light filling interval, and in addition, the micro light filling amount is calculated by using the attitude change amount and the jitter compensation amount, so that the image can always keep proper brightness and contrast.
The intelligent shooting light filling device of the middle cradle head in the embodiment of the invention is described in detail from the angle of modularized functional entity in the above fig. 2 and 3, and the intelligent shooting light filling device of the cradle head in the embodiment of the invention is described in detail from the angle of hardware processing.
Fig. 4 is a schematic structural diagram of an intelligent shooting light filling apparatus for a pan-tilt, where the intelligent shooting light filling apparatus 400 for a pan-tilt may have a relatively large difference due to different configurations or performances, and may include one or more processors (central processing units, CPU) 410 (e.g., one or more processors) and a memory 420, and one or more storage media 430 (e.g., one or more mass storage devices) storing application 433 or data 432. Wherein memory 420 and storage medium 430 may be transitory or persistent storage. The program stored in the storage medium 430 may include one or more modules (not shown), each of which may include a series of instruction operations in the smart shooting light compensating apparatus 400 for a cradle head. Still further, the processor 410 may be configured to communicate with the storage medium 430, and execute a series of instruction operations in the storage medium 430 on the smart shooting light filling apparatus 400 of the pan-tilt, so as to implement the steps of the smart shooting light filling method of the pan-tilt.
The intelligent shooting light filling apparatus 400 of the pan-tilt may further include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input/output interfaces 460, and/or one or more operating systems 431, such as Windows service, mac OS X, unix, linux, freeBSD, etc. It will be appreciated by those skilled in the art that the configuration of the smart photographing and light supplementing apparatus for a cradle head shown in fig. 4 does not constitute a limitation of the smart photographing and light supplementing apparatus for a cradle head provided by the present invention, and may include more or fewer components than shown, or may combine certain components, or may have different component arrangements.
The invention also provides a computer readable storage medium, which can be a nonvolatile computer readable storage medium, and can also be a volatile computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions run on a computer, the instructions cause the computer to execute the steps of the intelligent shooting light supplementing method of the cradle head.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system or apparatus and unit described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. The intelligent shooting light filling method of the cradle head is characterized in that light filling equipment and image pickup equipment are mounted on the cradle head, and the intelligent shooting light filling method of the cradle head comprises the following steps:
when the cradle head is opened, shooting through the image pickup equipment to obtain a first shooting image, and controlling the light supplementing equipment to conduct primary light supplementing according to the first shooting image;
when the cradle head moves in the gesture, calculating the gesture variation of the cradle head, and judging whether the gesture variation is in a preset micro light supplementing zone or not;
if the attitude change quantity is in a preset micro-light supplementing interval, calculating the shake compensation quantity of the cradle head according to the attitude change quantity;
performing data preprocessing on the attitude change amount and the jitter compensation amount to obtain preprocessed data; inputting the preprocessing data into a preset light supplementing quantity prediction model, and calculating the micro light supplementing quantity of the light supplementing equipment through the light supplementing quantity prediction model; controlling the light supplementing equipment to conduct secondary light supplementing according to the micro light supplementing quantity, and before preprocessing the attitude change quantity and the jitter compensation quantity to obtain preprocessed data, further comprising:
Acquiring the historical posture change quantity and the historical jitter compensation quantity of the cradle head, and setting light supplementing quantity corresponding to the preset historical posture change quantity and the historical jitter compensation quantity; preprocessing the historical posture change amount and the historical jitter compensation amount, and dividing time steps to obtain a phase posture change amount and a phase jitter compensation amount corresponding to a plurality of time steps; generating a time sequence according to the phase posture change quantity and the phase jitter compensation quantity corresponding to the time steps, and inputting the time sequence into a preset initial model, wherein the initial model is a long-time and short-time memory network based on an attention model; extracting features of the time sequence through an input layer of the initial model to obtain sequence features, and calculating attention weights of time steps corresponding to the time sequence based on the attention mechanism; calculating a context vector of the time sequence according to the attention weight and the sequence characteristic, and inputting the context vector into the long-short-time memory network to obtain a predicted light supplementing quantity; calculating a preset loss function according to the set light supplementing quantity and the predicted light supplementing quantity to obtain a loss function value, and judging whether the loss function value is smaller than a preset loss threshold value or not; if not, updating the model parameters of the initial model according to the loss function value, and returning to the step of inputting the time sequence into a preset initial model until the loss function value is smaller than a preset loss threshold value; if yes, the initial model is used as a light supplementing quantity prediction model of the cradle head.
2. The intelligent shooting light filling method of a pan-tilt according to claim 1, wherein when the pan-tilt is turned on, shooting a shooting object by the image capturing apparatus, and controlling the light filling apparatus to perform light filling once according to a first shooting image comprises:
when the cradle head is opened, shooting an object through the image shooting equipment to obtain a first shooting image, and cutting the first shooting image to obtain a foreground image and a background image in the first shooting image;
respectively carrying out illumination distribution identification on the foreground image and the background image to obtain first illumination distribution data and second illumination distribution data;
and calculating initial light supplementing quantity of the shooting equipment based on the first illumination distribution data and the second illumination distribution data, and controlling the light supplementing equipment to conduct primary light supplementing according to the initial light supplementing quantity.
3. The intelligent shooting light supplementing method of a tripod head according to claim 1, wherein a gyroscope and an accelerometer are installed on the tripod head; when the cradle head moves in a posture, calculating the posture change amount of the cradle head comprises:
when the cradle head moves in a gesture, acquiring the rotation speed of the cradle head in each axial direction in a machine body coordinate axis through the gyroscope, and acquiring the current gravity direction information of the cradle head through an accelerometer;
Calculating the rotation angles of the cradle head in each axial direction in the machine body coordinate axis according to the rotation speed and the gravity direction information;
and taking the rotation angles of the axial directions as the posture change quantity of the cradle head.
4. The intelligent shooting light filling method of a pan-tilt according to claim 1, wherein when the pan-tilt is moved in posture, calculating a posture change amount of the pan-tilt, and judging whether the posture change amount is after a preset micro light filling section, further comprising:
if the posture change amount is not in the preset micro-light supplementing interval, judging whether the posture change amount is larger than the maximum value of the micro-light supplementing interval or not;
if the attitude change quantity is larger than the maximum value of the micro light supplementing section, acquiring a second shooting image obtained by shooting by the shooting equipment;
comparing the first shooting image with the second shooting image, and judging whether a shooting object change occurs between the first shooting image and the second shooting image;
and if the shot object changes, controlling the light supplementing device to conduct secondary light supplementing based on the second shot image.
5. The intelligent shooting light filling method of a pan-tilt according to claim 1, wherein the calculating the shake compensation amount of the pan-tilt according to the attitude change amount includes:
Filtering the attitude change quantity, and obtaining the change frequency and the corresponding change amplitude of the attitude change quantity after the filtering;
judging whether the cradle head has jitter according to the change frequency and the change amplitude;
if shake exists, acquiring a target pose and a current pose of the holder, and calculating an attitude error of the holder according to the target pose and the current pose;
and calculating the shake compensation amount of the cradle head based on a preset compensation algorithm and the attitude error.
6. The utility model provides a light filling device is shot to intelligence of cloud platform, its characterized in that, carry on light filling equipment and camera equipment on the cloud platform, the light filling device is shot to intelligence of cloud platform includes:
the initial light supplementing module is used for shooting through the camera equipment when the cradle head is started to obtain a first shooting image, and controlling the light supplementing equipment to conduct primary light supplementing according to the first shooting image;
the attitude calculation module is used for calculating the attitude variation of the cradle head when the cradle head moves in an attitude, and judging whether the attitude variation is in a preset micro-light supplementing zone or not;
the compensation amount calculating module is used for calculating the shake compensation amount of the cradle head according to the attitude variation if the attitude variation is in a preset micro-light supplementing interval;
The micro light supplementing model is used for carrying out data preprocessing on the attitude change quantity and the jitter compensation quantity to obtain preprocessed data; inputting the preprocessing data into a preset light supplementing quantity prediction model, and calculating the micro light supplementing quantity of the light supplementing equipment through the light supplementing quantity prediction model; controlling the light supplementing equipment to conduct secondary light supplementing according to the micro light supplementing quantity, and before preprocessing the attitude change quantity and the jitter compensation quantity to obtain preprocessed data, further comprising:
acquiring the historical posture change quantity and the historical jitter compensation quantity of the cradle head, and setting light supplementing quantity corresponding to the preset historical posture change quantity and the historical jitter compensation quantity; preprocessing the historical posture change amount and the historical jitter compensation amount, and dividing time steps to obtain a phase posture change amount and a phase jitter compensation amount corresponding to a plurality of time steps; generating a time sequence according to the phase posture change quantity and the phase jitter compensation quantity corresponding to the time steps, and inputting the time sequence into a preset initial model, wherein the initial model is a long-time and short-time memory network based on an attention model; extracting features of the time sequence through an input layer of the initial model to obtain sequence features, and calculating attention weights of time steps corresponding to the time sequence based on the attention mechanism; calculating a context vector of the time sequence according to the attention weight and the sequence characteristic, and inputting the context vector into the long-short-time memory network to obtain a predicted light supplementing quantity; calculating a preset loss function according to the set light supplementing quantity and the predicted light supplementing quantity to obtain a loss function value, and judging whether the loss function value is smaller than a preset loss threshold value or not; if not, updating the model parameters of the initial model according to the loss function value, and returning to the step of inputting the time sequence into a preset initial model until the loss function value is smaller than a preset loss threshold value; if yes, the initial model is used as a light supplementing quantity prediction model of the cradle head.
7. The utility model provides a light filling equipment is shot to intelligence of cloud platform, its characterized in that, light filling equipment is shot to intelligence of cloud platform includes: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the smart shooting light filling apparatus of the pan-tilt to perform the steps of the smart shooting light filling method of the pan-tilt of any one of claims 1-5.
8. A computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the steps of the intelligent shooting light filling method of a pan-tilt as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410025694.8A CN117528259B (en) | 2024-01-08 | 2024-01-08 | Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410025694.8A CN117528259B (en) | 2024-01-08 | 2024-01-08 | Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117528259A CN117528259A (en) | 2024-02-06 |
| CN117528259B true CN117528259B (en) | 2024-03-26 |
Family
ID=89749891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410025694.8A Active CN117528259B (en) | 2024-01-08 | 2024-01-08 | Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117528259B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117896618B (en) * | 2024-03-15 | 2024-05-14 | 深圳市浩瀚卓越科技有限公司 | Anti-shake method, device, equipment and storage medium for cradle head shooting |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020019295A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Image acquisition method, imaging apparatus, and photographing system |
| CN114040113A (en) * | 2021-11-26 | 2022-02-11 | 维沃移动通信有限公司 | Image processing method and device |
| CN115047890A (en) * | 2022-08-17 | 2022-09-13 | 北京大学 | Unmanned ship control method, unmanned ship control device and computer-readable storage medium |
-
2024
- 2024-01-08 CN CN202410025694.8A patent/CN117528259B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020019295A1 (en) * | 2018-07-27 | 2020-01-30 | 深圳市大疆创新科技有限公司 | Image acquisition method, imaging apparatus, and photographing system |
| CN114040113A (en) * | 2021-11-26 | 2022-02-11 | 维沃移动通信有限公司 | Image processing method and device |
| CN115047890A (en) * | 2022-08-17 | 2022-09-13 | 北京大学 | Unmanned ship control method, unmanned ship control device and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117528259A (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190720B (en) | Intelligent agent reinforcement learning method, device, equipment and medium | |
| CN105955308B (en) | The control method and device of a kind of aircraft | |
| CN117528259B (en) | Intelligent shooting light supplementing method, device and equipment for cradle head and storage medium | |
| JP7131994B2 (en) | Self-position estimation device, self-position estimation method, self-position estimation program, learning device, learning method and learning program | |
| JP2014081863A (en) | Information processing device, information processing method and program | |
| CN109816698A (en) | Unmanned plane visual target tracking method based on dimension self-adaption core correlation filtering | |
| JP7390322B2 (en) | Motion sensor with drift correction | |
| US11263796B1 (en) | Binocular pose prediction | |
| CN114338994A (en) | Optical anti-shake method, optical anti-shake apparatus, electronic device, and computer-readable storage medium | |
| US11763508B2 (en) | Disambiguation of poses | |
| CN112699536A (en) | Self-adaptive target ship track prediction and locking method | |
| CN114800500B (en) | Flexible constant force control method and system for polishing robot | |
| CN118365707B (en) | Pose determination method, device, equipment and storage medium based on visual tracking | |
| CN110989691B (en) | Cloud deck control method and device, storage medium, electronic equipment and unmanned aerial vehicle | |
| CN117615255B (en) | Shooting tracking method, device, equipment and storage medium based on cradle head | |
| EP2357605B1 (en) | Stabilisation method and computer system | |
| Yang et al. | A Novel Motion‐Intelligence‐Based Control Algorithm for Object Tracking by Controlling PAN‐Tilt Automatically | |
| CN115797490B (en) | Graph construction method and system based on laser vision fusion | |
| US12056896B2 (en) | Prior informed pose and scale estimation | |
| CN114527776B (en) | Unmanned aerial vehicle anti-wind-disturbance control method, device, terminal and storage medium | |
| CN117457062A (en) | Molecular position determining method, device, electronic equipment and storage medium | |
| CN115615421A (en) | Unmanned aerial vehicle positioning method and system based on binocular vision inertial odometer | |
| CN114022949A (en) | Event camera motion compensation method and device based on motion model | |
| US20210255695A1 (en) | Information processing apparatus, information processing method, program, and floor modeling system | |
| CN117495900B (en) | Multi-target visual tracking method based on camera motion trend estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |