Disclosure of Invention
In view of the problems in the prior art, the present invention provides a method, a system, a device and a medium for controlling camera brightness, which mainly solve the problem of over-exposure or insufficient light supplement.
In order to achieve the above and other objects, the present invention adopts the following technical solutions.
An image capturing luminance control method includes:
acquiring the brightness of a visible light image, and carrying out face detection on the visible light image to acquire the brightness of a corresponding infrared light image face area;
and carrying out exposure control on an image sensing unit according to the brightness of the visible light image and the brightness of the face area of the infrared light image.
Optionally, the image acquired by the image sensing unit is subjected to interpolation processing, and a visible light image and a corresponding infrared light image are acquired respectively.
Optionally, the interpolation process comprises one of bilinear interpolation, bicubic interpolation or median filtering.
Optionally, a visible light brightness threshold range is set, and when the visible light image brightness does not reach the visible light brightness threshold range, visible light brightness compensation is performed.
Optionally, an infrared light brightness threshold range is set, and when the brightness of the face area of the infrared image does not reach the infrared light brightness threshold range, infrared light brightness compensation is performed.
Optionally, the exposure control includes brightness compensation, exposure time control, gain control.
Optionally, the position of the human face in the visible light image is detected, and then the human face area of the infrared light image at the corresponding position in the infrared light image is obtained.
Optionally, obtaining a weighted statistic value of the brightness compensation through the brightness of the visible light image, the brightness of the face area of the infrared light image, and the brightness weight of the visible light image, the brightness weight of the face area of the visible light image, and the brightness weight of the face area of the infrared light image, which are respectively set; the sum of the visible light image brightness weight, the visible light image face area brightness weight and the infrared light image face area brightness weight is 256.
Optionally, the brightness of the visible light image face area and the corresponding brightness of the infrared light image face area are obtained, and the brightness weight of the infrared light image face area is set according to the ratio of the two brightnesses.
Optionally, the brightness weight of the face region of the visible light image is set according to the ratio between the brightness of the visible light image and the brightness of the face region of the visible light image.
Optionally, weights corresponding to the red, green, and blue three-primary-color components of each pixel of the visible light image are set, and then a weighted statistic of all pixels is calculated according to the set corresponding weights, so as to obtain the brightness of the visible light image.
A camera brightness control system comprising:
the brightness acquisition module is used for acquiring the brightness of a visible light image, carrying out face detection on the visible light image and acquiring the corresponding brightness of a face area of an infrared light image;
and the exposure control module is used for carrying out exposure control on the image sensing unit according to the brightness of the visible light image and the brightness of the human face area of the infrared light image.
Optionally, an interpolation module is included, configured to perform interpolation processing on the image acquired by the image sensing unit, and acquire the visible light image and the corresponding infrared light image respectively.
Optionally, the system comprises a visible light threshold setting module, configured to set a visible light threshold range, and when the visible light image brightness does not reach the visible light threshold range, perform visible light brightness compensation.
Optionally, the infrared light image processing device comprises an infrared light threshold setting module, configured to set an infrared light brightness threshold range, and perform infrared light brightness compensation when the brightness of the face area of the infrared light image does not reach the infrared light brightness threshold range.
Optionally, the brightness acquiring module includes a face detecting unit, configured to detect a position of a face in the visible light image, and further acquire a face region of the infrared light image at a corresponding position in the infrared light image.
Optionally, the system comprises a brightness weight setting module, configured to obtain a weighted statistical value of the brightness compensation according to the brightness of the visible light image, the brightness of the face area of the infrared light image, and the brightness of the visible light image, the brightness of the face area of the visible light image, and the brightness of the face area of the infrared light image, which are respectively set; the sum of the visible light image brightness weight, the visible light image face area brightness weight and the infrared light image face area brightness weight is 256.
Optionally, the brightness weight setting module includes an infrared face region weight setting unit, configured to obtain the brightness of the visible light image face region and the brightness of the corresponding infrared light image face region, and set the brightness weight of the infrared light image face region according to a ratio of the two brightnesses.
Optionally, the brightness weight setting module includes a visible light face region weight setting unit, configured to set the brightness weight of the visible light image face region according to a ratio between the brightness of the visible light image and the brightness of the infrared light image face region.
Optionally, the brightness obtaining module includes a pixel component weight setting unit, configured to set weights corresponding to red, green, and blue three-primary-color components of each pixel of the visible light image, and further calculate a weighted statistic of all pixels according to the set corresponding weights, so as to obtain the brightness of the visible light image.
An apparatus, comprising:
one or more processors; and
one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the camera brightness control method.
One or more machine-readable media having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform the camera brightness control method.
As described above, the present invention provides an image capturing luminance control method, system, device, and medium, having the following advantageous effects.
By carrying out combined analysis on the visible light image and the infrared light image, the exposure precision can be effectively improved.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Referring to fig. 1, the present invention provides a method for controlling brightness of a camera, comprising steps S01-S02.
In step S01, acquiring the brightness of a visible light image, and performing face detection on the visible light image to acquire the brightness of a corresponding face area of an infrared light image;
in an embodiment, the image of the designated area is acquired through an image sensing unit, which may be an RGB-IR image sensor, and an optical signal is converted into an electrical signal for storage, and then the electrical signal is exposed, and an interpolation algorithm is used to interpolate the image according to a Color Filter Array (CFA for short) of the image sensor. Taking the most common Bayer CFA as an example, in a typical 2X2 Bayer CFA basic cell, the green component G accounts for 1/2 of the total number of pixels, and the red component R and the blue component B each account for 1/4. CFA color sampling is firstly carried out according to a Bayer CFA mode, and then interpolation is carried out by adopting an interpolation algorithm. The interpolation algorithm may comprise one of bilinear interpolation, bicubic interpolation, or median filter interpolation.
After interpolation processing, two paths of images including a visible light image and a corresponding infrared light image can be respectively obtained. And carrying out brightness statistics on the visible light image. Each pixel in the visible light image includes R, G, B channel components, and the weight of R, G, B three-color components at each pixel is preset. And counting three-color components of all pixel points to obtain the brightness of the visible light image. Specifically, the brightness of the visible light image may be calculated by means of weighted average, and the calculation formula of the brightness of the visible light image may be represented as:
wherein, Y
RGBRepresenting the visible image brightness; wr, Wg and Wb are weight coefficients respectively; ri, Gi and Bi are respectively components of R, G, B channels of i pixels, and n represents the total number of pixels;
and adjusting the brightness of the visible light supplementary lamp according to the acquired visible light image brightness, performing brightness compensation, and further performing exposure control on the image sensing unit.
In one embodiment, the exposure control includes brightness compensation, exposure time control, exposure gain control, and the like.
The visible light brightness threshold range can be set, when the brightness of the obtained visible light image is lower than the lower limit of the visible light brightness threshold range, the brightness of the visible light fill-in lamp can be improved, the exposure time and the gain are adjusted, and then the exposure process of the image sensing unit is controlled; when the brightness of the obtained visible light image is higher than the upper limit of the visible light brightness threshold range, the brightness of the visible light supplementary lamp can be reduced, and the exposure time and the gain are adjusted.
In an embodiment, the face detection is performed on the visible light image at the same time, and the position of the face in the visible light image can be determined by performing the real-time face detection through a deep neural network algorithm, a template matching algorithm and the like.
According to the position of the obtained human face in the visible light image, the position of the human face in the corresponding infrared light image can be obtained, and then the human face area of the infrared light image is extracted from the infrared light image, so that the statistics can be carried out on the brightness of the human face area of the infrared light image.
In step S02, exposure control is performed on the image sensing unit according to the visible light image brightness and the infrared light image face region brightness:
the infrared light brightness threshold range can be set, when the brightness of the face area of the infrared light image is lower than the lower limit of the set threshold range, the brightness of an infrared light supplement lamp can be improved, and the exposure time and the gain are adjusted; when the brightness of the face area of the infrared light image is higher than the upper limit of the set threshold range, the brightness of an infrared light supplement lamp can be reduced, the exposure time and the gain are adjusted, and then exposure control is carried out on the image sensing unit.
And comprehensively considering the brightness of the visible light image and the brightness of the face area of the infrared light image, and performing combined control on the exposure process. Specifically, the brightness weight of the visible light image, the brightness weight of the face area of the visible light image and the brightness weight of the face area of the infrared light image can be respectively set, the sum of the weights of the visible light image, the face area of the visible light image and the face area of the infrared light image is preset to be 256, and the brightness of exposure control is obtained through weighted average of the three.
Specifically, a weighted statistic value of the brightness compensation is obtained through the brightness of the visible light image, the brightness of the face area of the infrared light image, and the brightness weight of the visible light image, the brightness weight of the face area of the visible light image and the brightness weight of the face area of the infrared light image which are respectively arranged; the sum of the visible light image brightness weight, the visible light image face area brightness weight and the infrared light image face area brightness weight is 256.
Further, the air conditioner is provided with a fan,
step 1, respectively setting a visible light image brightness weight, a visible light image face area brightness weight and an infrared light image face area brightness weight, and presetting the sum of the weights of the visible light image brightness weight, the visible light image face area brightness weight and the infrared light image face area brightness weight to be 256;
step 2, acquiring the brightness of the visible light image according to a calculation formula of the brightness of the visible light image;
wherein, Y
RGBRepresenting the visible image brightness; wr, Wg and Wb are weight coefficients respectively; ri, Gi and Bi are respectively components of R, G, B channels of i pixels, and n represents the total number of pixels;
step 3, expressing a formula (formula I) according to the brightness weight of the face area of the visible light image, combining the brightness weight of the face area of the visible light image set in the step 1 and the brightness Y of the visible light image calculated in the step 2
RGBObtaining the brightness Y of the face region of the visible light image
FaceRGB
And 4, step 4: according to the brightness weight expression formula (formula II) of the face area of the infrared light image and the Y obtained in the step 3
FaceRGBAnd the brightness weight of the face area of the infrared light image set in the step 1 is used for obtaining the brightness of the face area of the infrared light image.
And 5: calculation of weighted statistics for luminance compensation
The calculation formula of the weighted statistic of the luminance compensation can be expressed as:
by combining the brightness of the visible light image and the brightness of the face area of the infrared light image, the brightness compensation is carried out on the exposure process, and the weighted statistical value for the brightness compensation is obtained, so that the exposure is more accurate, and the problems of overexposure or insufficient light supplement are avoided.
By combining the corresponding brightness of the visible light image and the infrared image in the face area, the exposure can be more accurate, and the problems of overexposure or insufficient light supplement are avoided.
Referring to fig. 2, the present embodiment provides a system for controlling image capturing brightness, which is used to execute the method for controlling image capturing brightness in the foregoing method embodiment. Since the technical principle of the system embodiment is similar to that of the method embodiment, repeated description of the same technical details is omitted.
In an embodiment, the image capturing brightness control system includes a brightness acquiring module 10 and an exposure control module 11, wherein the brightness acquiring module 10 is configured to assist in executing the step S01 described in the foregoing method embodiment, and the exposure control module 11 is configured to execute the step S02 described in the foregoing method embodiment.
In one embodiment, the system is provided with an interpolation module, and the interpolation module performs interpolation processing on the image collected by the image sensing unit to obtain a color image. The interpolation process can adopt a nearest neighbor interpolation method to map the points in the target image to the original image phase, and find the nearest integer points as the interpolated output.
In one embodiment, the system is provided with a visible light threshold setting module for setting a visible light threshold range, and performing visible light brightness compensation when the visible light image brightness does not reach the visible light threshold range. The visible light threshold setting module can also be used for setting the brightness threshold range of the face area of the visible light image. When the brightness of the visible light image or the brightness of the human face area of the visible light image exceeds the set corresponding threshold range, the calculation of the visible light brightness compensation can be triggered, and further the brightness compensation is carried out on the exposure process.
In an embodiment, the system is provided with an infrared light threshold setting module for setting an infrared light brightness threshold range, and when the brightness of the face area of the infrared image does not reach the set infrared light brightness threshold range, sending a trigger signal to perform infrared light brightness compensation.
In an embodiment, the brightness acquiring module 10 includes a face detecting unit, configured to detect a position of a face in the visible light image, and further acquire a face area of the infrared light image at a corresponding position in the infrared light image. The face detection framework can be trained by a Multi-Task Cascade convolutional neural network (MTCNN for short), and the face detection framework is integrated into the face detection unit.
In an embodiment, the system is provided with a brightness weight setting module, configured to obtain a weighted statistical value of the brightness compensation according to the brightness of the visible light image, the brightness of the face area of the infrared light image, and the brightness of the visible light image, the brightness of the face area of the visible light image, and the brightness of the face area of the infrared light image, which are respectively set; the sum of the visible light image brightness weight, the visible light image face area brightness weight and the infrared light image face area brightness weight is 256.
In one embodiment, the brightness weight setting module comprises an infrared face area weight setting unit and a visible light face area weight setting unit, wherein the infrared face area weight setting unit is used for acquiring the brightness of a visible light image face area and the brightness of a corresponding infrared light image face area, and setting the weight of an infrared face image according to the ratio of the brightness of the visible light image face area to the brightness of the infrared light image face area; the visible light face area weight setting unit is used for setting the visible light image face area brightness weight according to the ratio of the visible light image brightness to the infrared light image face area brightness.
In an embodiment, the luminance obtaining module 10 may further include a pixel component weight setting unit, configured to set weights corresponding to three primary color components of red, green, and blue of each pixel of the visible light image, and the luminance obtaining module 10 may calculate a weighted average of all pixels according to the set corresponding weights, so as to obtain the luminance of the visible light image.
An embodiment of the present application further provides an apparatus, which may include: one or more processors; and one or more machine readable media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the method of fig. 1. In practical applications, the device may be used as a terminal device, and may also be used as a server, where examples of the terminal device may include: the mobile terminal includes a smart phone, a tablet computer, an electronic book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop, a vehicle-mounted computer, a desktop computer, a set-top box, an intelligent television, a wearable device, and the like.
The embodiment of the present application further provides a non-volatile readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a device, the device may execute instructions (instructions) of steps included in the method for controlling brightness of an image pickup in fig. 1 according to the embodiment of the present application.
Fig. 3 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present application. As shown, the terminal device may include: an input device 1100, a first processor 1101, an output device 1102, a first memory 1103, and at least one communication bus 1104. The communication bus 1104 is used to implement communication connections between the elements. The first memory 1103 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and the first memory 1103 may store various programs for performing various processing functions and implementing the method steps of the present embodiment.
Alternatively, the first processor 1101 may be, for example, a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and the processor 1101 is coupled to the input device 1100 and the output device 1102 through a wired or wireless connection.
Optionally, the input device 1100 may include a variety of input devices, such as at least one of a user-oriented user interface, a device-oriented device interface, a software programmable interface, a camera, and a sensor. Optionally, the device interface facing the device may be a wired interface for data transmission between devices, or may be a hardware plug-in interface (e.g., a USB interface, a serial port, etc.) for data transmission between devices; optionally, the user-facing user interface may be, for example, a user-facing control key, a voice input device for receiving voice input, and a touch sensing device (e.g., a touch screen with a touch sensing function, a touch pad, etc.) for receiving user touch input; optionally, the programmable interface of the software may be, for example, an entry for a user to edit or modify a program, such as an input pin interface or an input interface of a chip; the output devices 1102 may include output devices such as a display, audio, and the like.
In this embodiment, the processor of the terminal device includes a function for executing each module of the speech recognition apparatus in each device, and specific functions and technical effects may refer to the above embodiments, which are not described herein again.
Fig. 4 is a schematic hardware structure diagram of a terminal device according to another embodiment of the present application. Fig. 4 is a specific embodiment of fig. 3 in an implementation process. As shown, the terminal device of the present embodiment may include a second processor 1201 and a second memory 1202.
The second processor 1201 executes the computer program code stored in the second memory 1202 to implement the method described in fig. 1 in the above embodiment.
The second memory 1202 is configured to store various types of data to support operations at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, pictures, videos, and so forth. The second memory 1202 may include a Random Access Memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
Optionally, the first processor 1201 is provided in the processing assembly 1200. The terminal device may further include: communication component 1203, power component 1204, multimedia component 1205, speech component 1206, input/output interfaces 1207, and/or sensor component 1208. The specific components included in the terminal device are set according to actual requirements, which is not limited in this embodiment.
The processing component 1200 generally controls the overall operation of the terminal device. The processing assembly 1200 may include one or more second processors 1201 to execute instructions to perform all or part of the steps of the method illustrated in fig. 1 described above. Further, the processing component 1200 can include one or more modules that facilitate interaction between the processing component 1200 and other components. For example, the processing component 1200 can include a multimedia module to facilitate interaction between the multimedia component 1205 and the processing component 1200.
The power supply component 1204 provides power to the various components of the terminal device. The power components 1204 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the terminal device.
The multimedia components 1205 include a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
The voice component 1206 is configured to output and/or input voice signals. For example, the voice component 1206 includes a Microphone (MIC) configured to receive external voice signals when the terminal device is in an operational mode, such as a voice recognition mode. The received speech signal may further be stored in the second memory 1202 or transmitted via the communication component 1203. In some embodiments, the speech component 1206 further comprises a speaker for outputting speech signals.
The input/output interface 1207 provides an interface between the processing component 1200 and peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include, but are not limited to: a volume button, a start button, and a lock button.
The sensor component 1208 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor component 1208 may detect an open/closed state of the terminal device, relative positioning of the components, presence or absence of user contact with the terminal device. The sensor assembly 1208 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor assembly 1208 may also include a camera or the like.
The communication component 1203 is configured to facilitate communications between the terminal device and other devices in a wired or wireless manner. The terminal device may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot therein for inserting a SIM card therein, so that the terminal device may log onto a GPRS network to establish communication with the server via the internet.
As can be seen from the above, the communication component 1203, the voice component 1206, the input/output interface 1207 and the sensor component 1208 referred to in the embodiment of fig. 4 can be implemented as the input device in the embodiment of fig. 3.
In summary, the method, the system, the device and the medium for controlling the camera shooting brightness can sense the brightness change in real time, adjust the exposure time, the exposure gain, the brightness compensation and the like by performing combined analysis on the visible light image and the infrared light image and controlling the exposure process according to the brightness of the face area, thereby realizing accurate exposure and effectively improving the exposure precision. Therefore, the invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.