CN115996320B - Event camera adaptive threshold adjustment method, device, equipment and storage medium - Google Patents
Event camera adaptive threshold adjustment method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115996320B CN115996320B CN202310285291.2A CN202310285291A CN115996320B CN 115996320 B CN115996320 B CN 115996320B CN 202310285291 A CN202310285291 A CN 202310285291A CN 115996320 B CN115996320 B CN 115996320B
- Authority
- CN
- China
- Prior art keywords
- current moment
- event camera
- illuminance
- state vector
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000003044 adaptive effect Effects 0.000 title claims description 26
- 239000013598 vector Substances 0.000 claims abstract description 141
- 230000008859 change Effects 0.000 claims abstract description 85
- 230000008569 process Effects 0.000 claims abstract description 29
- 239000011159 matrix material Substances 0.000 claims description 141
- 238000012546 transfer Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000009795 derivation Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 8
- 238000005286 illumination Methods 0.000 description 6
- 230000004044 response Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000005282 brightening Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Studio Devices (AREA)
Abstract
The invention relates to the technical field of event cameras, in particular to an event camera self-adaptive threshold adjustment method, which comprises the following steps: acquiring illuminance of an illuminance acquisition window of the event camera at the current moment and a state vector of the event camera at the current moment in the running process of the event camera; based on the illuminance at the current moment and the state vector at the current moment, obtaining a compensated state vector at the current moment; and after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches the preset number, obtaining a target change threshold of the event camera according to the plurality of compensated state vectors, and adjusting the change threshold used by the event camera in the running process to be the target change threshold. According to the method, the change threshold value of the event camera is dynamically and adaptively adjusted, so that the event camera is matched with different application environments, and the use effect and the user experience of a user are improved.
Description
Technical Field
The present invention relates to the field of event cameras, and in particular, to a method, an apparatus, a device, and a storage medium for adjusting an adaptive threshold of an event camera.
Background
An Event Camera (Event Camera) is a novel type of biological heuristic vision sensor, also known as dynamic vision sensor (Dynamic Vision Sensor, DVS). The event camera has the characteristics of high response speed, high dynamic range, only capturing dynamic change, low power consumption and the like, can overcome the defects of low frame rate, large delay, small dynamic response range and the like of the conventional visual camera, and can also play a powerful role in challenging scenes such as high brightness, high contrast and the like and high-speed and high dynamic range visual application. The vision algorithm based on the event camera has wide application in the fields of synchronous positioning, target tracking, three-dimensional reconstruction, target recognition, optical flow estimation and the like.
The event camera is used as a biological heuristic sensor, the pixel structure is more complex, and the working principle of the event camera is quite different from that of a traditional camera. The event camera simulates retina of living things in nature, does not output at a fixed rate, only outputs relevant information of local pixel brightness change, and particularly outputs based on illumination change on each pixel driven by asynchronous events. The pixels of the event camera are mutually independent, and the event camera asynchronously detects and responds to the logarithmic change of the brightness of the pixels, namely asynchronously detects and responds to the events corresponding to the pixels. Triggering a DVS event when the illumination brightness change of a certain pixel reaches a specified threshold value, outputting a corresponding asynchronous event by the pixel, realizing asynchronous sparse output of coordinates, time stamps and event polarities of the pixel corresponding to the DVS event with microsecond resolution, and outputting high and low levels according to the enhancement and the weakening of the illumination brightness, wherein the high level represents positive polarity of the event, and the low level identifies negative polarity of the event. DVS events are typically triggered by two conditions, one by a change in background illumination and one by a relative motion of the event camera and the background.
Wherein the specified threshold is an event camera change threshold C (Contrast sensitivity). The specified threshold is typically a fixed value that is set manually, and is currently set mainly empirically. However, considering the influence of different factors such as the environment applied by the event camera and the condition triggered by the event, if the designated threshold value is set too low, the event stream of the event camera has great redundancy and excessive noise; if the specified threshold is set too high, the event camera response is made sluggish. Therefore, when the fixed threshold value is set empirically in the prior art, the event camera is enabled to influence the use effect and the user experience of the user by the fixed threshold value under different application environments.
Disclosure of Invention
The embodiment of the application can at least solve the technical problems by providing a method, a device, equipment and a storage medium for adjusting the self-adaptive threshold of an event camera.
In a first aspect, an embodiment of the present invention provides a method for adjusting an adaptive threshold of an event camera, including:
acquiring illuminance of an illuminance acquisition window of an event camera at the current moment and a state vector of the illuminance acquisition window at the current moment in the running process of the event camera;
based on the illuminance at the current moment and the state vector at the current moment, performing compensation processing on the illuminance acquisition window to obtain a compensated state vector at the current moment;
and after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches a preset number, obtaining a target change threshold of the event camera according to the plurality of compensated state vectors, and adjusting the change threshold used by the event camera in the running process to be the target change threshold.
Preferably, the acquiring the state vector of the illuminance collection window of the event camera at the current moment includes:
and obtaining a state vector of the current moment according to the illuminance of the current moment and the historical change threshold of the event camera.
Preferably, the obtaining the compensated state vector at the current time based on the illuminance at the current time and the state vector at the current time includes:
obtaining a gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment;
and obtaining the compensated state vector at the current moment according to the gain matrix at the current moment.
Preferably, the obtaining the gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment includes:
updating the state matrix of the current moment according to the state vector of the current moment, and obtaining the covariance matrix of the current moment according to the updated state matrix, the covariance matrix of the last moment of the current moment and the covariance matrix of Gaussian noise of the current moment;
and obtaining a gain matrix of the current moment according to the covariance matrix of the current moment.
Preferably, the obtaining the gain matrix of the current moment according to the covariance matrix of the current moment includes:
updating the observation matrix at the current moment according to the illuminance at the current moment to obtain an updated observation matrix;
and obtaining a gain matrix of the current moment according to the updated observation matrix, the covariance matrix of the current moment and the covariance matrix of the Gaussian noise of the event camera.
Preferably, the obtaining the target change threshold of the event camera according to the plurality of compensated state vectors includes:
and averaging the plurality of compensated state vectors to obtain an average value of the plurality of compensated state vectors, and taking the average value as the target change threshold.
Preferably, the acquiring the illuminance of the illuminance collection window of the event camera at the current moment includes:
and obtaining the illuminance at the current moment through the observation equation at the current moment.
Based on the same inventive concept, the present invention also provides an event camera adaptive threshold adjustment device, acting on the above event camera adaptive threshold adjustment method, the device comprising:
the acquisition module is used for acquiring the illuminance of the illuminance acquisition window of the event camera at the current moment and the state vector at the current moment in the running process of the event camera;
the compensation module is used for obtaining a compensated state vector at the current moment based on the illuminance at the current moment and the state vector at the current moment;
and the target module is used for continuously acquiring the compensated state vector of the illuminance acquisition window at the next moment after acquiring the compensated state vector at the current moment, acquiring a target change threshold of the event camera according to the plurality of compensated state vectors after the number of the acquired plurality of compensated state vectors reaches a preset number, and adjusting the change threshold used by the event camera in the running process to be the target change threshold.
Based on the same inventive concept, in a third aspect, the present invention provides an event camera device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the event camera adaptive threshold adjustment method when executing the program.
Based on the same inventive concept, in a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of an event camera adaptive threshold adjustment method.
One or more technical solutions in the embodiments of the present invention at least have the following technical effects or advantages:
in the embodiment of the invention, during the running process of the event camera, the illuminance and the state vector of the illuminance acquisition window of the event camera at the current moment are firstly acquired. Here, the illuminance of the illuminance collection window of the event camera at the current moment is measured in real time through the observation equation, and the state vector at the current moment is determined according to the illuminance at the current moment, so as to make a tamping basis for the state vector after compensation which is obtained later.
Then, based on the illuminance at the current time and the state vector, a compensated state vector at the current time is obtained. And then, after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches the preset number, obtaining a target change threshold of the event camera according to the plurality of compensated state vectors, and adjusting the target change threshold to be a change threshold used in the operation process of the event camera. Here, based on the compensated state vectors at a plurality of continuous moments, a target change threshold of the event camera is determined, so that the change threshold of the event camera is adaptively adjusted, the accuracy of the change threshold is improved, the operation efficiency of the event camera is improved, the operation stability of the event camera is ensured, and the rationality of setting the change threshold of the event camera is ensured.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also throughout the drawings, like reference numerals are used to designate like parts. In the drawings:
fig. 1 is a schematic flow chart of steps of an event camera adaptive threshold adjustment method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating another step of an adaptive threshold adjustment method for an event camera according to an embodiment of the present invention;
FIG. 3 shows a block diagram of an event camera adaptive threshold adjustment device in an embodiment of the invention;
fig. 4 shows a schematic structural diagram of an event camera device in an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Embodiment one: a first embodiment of the present invention provides a method for adjusting adaptive threshold of an event camera, as shown in FIG. 1, including:
s101, acquiring illuminance of an illuminance acquisition window of an event camera at the current moment and a state vector of the illuminance acquisition window at the current moment in the running process of the event camera;
s102, obtaining a compensated state vector at the current moment based on illuminance at the current moment and the state vector at the current moment;
and S103, after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches the preset number, obtaining a target change threshold of the event camera according to the plurality of compensated state vectors, and adjusting the change threshold used by the event camera in the running process to be the target change threshold.
Next, the specific implementation steps of the event camera adaptive threshold adjustment method provided in this embodiment will be described in detail with reference to fig. 1:
first, step S01 is executed, in which, during operation of the event camera, illuminance of an illuminance collection window of the event camera at a current time and a state vector of the event camera at the current time are obtained.
Specifically, after the event camera is powered on and started, the event camera is formally operated. In the running process of the event camera, the illuminance of the environment where the event camera is located at the current moment is collected through the illuminance collection window of the event camera, the collected illuminance at the current moment is obtained through an observation equation, the observation equation is shown in a formula (1), the process of quickly obtaining the illuminance of the illuminance collection window of the event camera at the current moment is achieved, and the process of obtaining the illuminance at the current moment through the observation equation has the advantages of being high in accuracy, high in calculation speed and the like. The observation equation is the observation equation in the kalman filter algorithm.
wherein ,xn Representing the illuminance of the nth pixel point of the illuminance collection window, wherein N is the total number of the pixel points of the illuminance collection window, and h k The illuminance of the window at the current moment is collected for the illuminance.
After the illuminance of the illuminance collection window at the current time is obtained, a state vector of the illuminance collection window at the current time is obtained through a Contrast Detector (abbreviated as CD, contrast detector) vector transfer equation shown in formula (2).
wherein ,fk And acquiring a state vector of the window at the current moment for illuminance. CD vector transfer equations include a CD on threshold and a CD off threshold,
weight value for open state of CD vector transfer equation,/->
Weight value for the off state of the CD vector transfer equation,/->
and />
Are all arranged according to the actual requirements. />
Is the CD open threshold at time k, +.>
Is the CD closure threshold at time k. />
Noise value for CD open threshold, +.>
Is the noise value of the CD closure threshold.
Global CD on threshold for event camera, +.>
Is a global CD closure threshold for the event camera.
When the following is performedWhen the illumination change of a certain pixel of the event camera reaches a set change threshold, the pixel outputs a corresponding asynchronous event, and the positive polarity and the negative polarity of the event are respectively represented according to the high and low output levels of the illumination. Such an event-based asynchronous output is used to simulate the transmission of neural signals in a biological vision system, event e triggered k Expressed as:
wherein ,
is the coordinates of the event in the pixel plane, +.>
Is a time stamp,/->
For the polarity of the event, +.>
Indicating a darkening event, +.>
Indicating a lightening event. Then, the CD on threshold is a threshold at which a brightening event can be detected, and the CD off threshold is a threshold at which a darkening event can be detected.
In this embodiment, the illuminance of the illuminance collection window of the event camera at the current time is measured in real time through the observation equation, and the state vector at the current time is determined according to the illuminance at the current time, so as to make a ramming basis for the state vector after compensation obtained later.
Next, step S102 is executed to obtain a compensated state vector at the current time based on the illuminance at the current time and the state vector at the current time.
Specifically, a gain matrix at the current time is obtained according to illuminance at the current time and a state vector at the current time. And obtaining the compensated state vector at the current moment according to the gain matrix at the current moment. The specific process of obtaining the gain matrix at the current moment is as follows:
a. according to the state vector at the current moment, updating the state matrix at the current moment to obtain an updated state matrix F k As shown in equation (4). Meanwhile, according to the illuminance at the current moment, updating the observation matrix at the current moment to obtain an updated observation matrix H k As shown in equation (5). The observation matrix and the state matrix are jacobian matrices.
Wherein, for f k Each element in the list is subjected to respective one-time derivation to obtain F k 。
Wherein, for h k Each element in the list is subjected to respective one-time derivation to obtain H k 。
b. And obtaining the covariance matrix of the current moment according to the updated state matrix, the covariance matrix of the last moment of the current moment and the covariance matrix of Gaussian noise of the current moment, as shown in a formula (6).
wherein ,
the covariance matrix of Gaussian noise at the current moment is expressed and is used for measuring the accuracy of a CD vector transfer equation, and the more accurate the equation model is, the smaller the value of the equation model is. P (P) k For the covariance matrix at time k, P k-1 The covariance matrix at the previous moment is the covariance matrix after compensation at the previous moment.
c. And obtaining a gain matrix at the current moment according to the updated observation matrix, the covariance matrix at the current moment and the covariance matrix of Gaussian noise of the event camera, as shown in a formula (7).
wherein ,
is a gain matrix, R k The covariance matrix, which is gaussian noise of the event camera, represents the error of the sensor measurements of the event camera.
In this embodiment, the state matrix at the current time is updated by the state vector at the current time, and the observation matrix at the current time is updated by the illuminance at the current time. And obtaining the covariance matrix of the current moment through the updated state matrix, the covariance matrix of the last moment of the current moment and the covariance matrix of Gaussian noise of the current moment. Then, according to the updated observation matrix, the covariance matrix at the current moment and the Gaussian noise covariance matrix of the event camera, the gain matrix at the current moment is obtained, so that a time domain state space with visual physical significance is adopted, a relatively simple recurrence algorithm is used for calculating the gain matrix at the current moment, the gain matrix is convenient to realize on a computer, the calculated amount is small, the calculation speed is increased, and the response speed is improved.
After obtaining the gain matrix at the current moment, obtaining the compensated state vector at the current moment according to the gain matrix at the current moment
As shown in equation (8). Meanwhile, according to the gain matrix at the current moment and the covariance matrix at the current moment, obtaining a compensated covariance matrix at the current moment +.>
。
wherein ,
is f k Is included in the list.
In this embodiment, based on the illuminance at the current time and the state vector at the current time, which are high in accuracy, correction and compensation are performed on the state vector at the current time, so that a compensated state vector at the current time is quickly obtained, and the accuracy of the compensated state vector at the current time is improved.
Then, step S103 is executed, after the compensated state vector at the current moment is obtained, the compensated state vector of the illuminance collection window at the next moment is continuously obtained until the number of the obtained plurality of compensated state vectors reaches the preset number, the target change threshold of the event camera is obtained according to the plurality of compensated state vectors, and the change threshold used by the event camera in the running process is adjusted to be the target change threshold.
Specifically, after the compensated state vector at the current time is obtained, steps S101-S102 are performed in the next time to obtain the compensated state vector at the next time, and so on. Thus, a compensated state vector, i.e. a plurality of compensated state vectors, can be obtained for each of a plurality of successive moments. When the number of the plurality of compensated state vectors reaches a preset number, carrying out average operation on the plurality of compensated state vectors to obtain an average value of the plurality of compensated state vectors, and taking the average value as a target change threshold value. And after the target change threshold value is obtained, the change threshold value used by the event camera in the running process is adjusted to be the target change threshold value. Wherein, the preset quantity is set according to the actual demand.
Therefore, based on the compensated state vectors at a plurality of continuous moments, the target change threshold of the event camera is determined, so that the change threshold of the event camera is adaptively adjusted, the accuracy of the change threshold is improved, the operation efficiency of the event camera is improved, and the operation stability of the event camera is guaranteed. The adjustment method provided by the invention can enable the event camera to adopt different change thresholds according to the environment conditions in different application environments, and further enable the event camera to adopt proper change thresholds in each specific use environment, so as to avoid slow response of the event camera caused by too high change threshold in each specific use environment and avoid great redundancy and excessive noise caused by too low event camera in different environments. In addition, the method and the device automatically update the change threshold value adaptively according to the use environment of the event camera, have high operation efficiency and quick response, can always ensure the operation stability of the event camera, and well improve the use effect and the user experience of a user.
After the target change threshold of the event camera is obtained, taking the target change threshold as the historical change threshold of the event camera, and performing S101-S103 again in a circulating way, namely obtaining a preset number of compensated state vectors again, so as to obtain the next target change threshold of the event camera until the event camera is powered off. In the process of executing S101-S103, a historical change threshold (i.e., a target change threshold) is needed to be calculated and executed. In this way, in the running process of the event camera, the change threshold value of the event camera is dynamically and adaptively adjusted in real time, the accuracy of the change threshold value is improved, and the setting reasonability of the change threshold value of the event camera is ensured. The event camera automatically matches different application environments through updating the change threshold in real time, so that the use effect and the user experience of a user are improved.
The following describes in detail the event camera adaptive threshold adjustment method of the present embodiment by taking an actually applied event camera as an example:
as shown in fig. 2, after the event camera is powered on and turned on, the event camera is initialized, that is, a change threshold M0, a covariance matrix P0, a state matrix, and an observation matrix of the event camera are initialized. And acquiring the illuminance h1 of the illuminance acquisition window of the event camera at the current moment, and obtaining a state vector f1 at the current moment according to h1 and M0. At the position ofAt the current moment, updating the state matrix according to F1 to obtain an updated state matrix F1, and simultaneously updating the observation matrix according to H1 to obtain an updated observation matrix H1. And obtaining a covariance matrix P1 of the current moment according to F1 and P0 and the covariance matrix Q1 of Gaussian noise of the current moment. And obtaining a gain matrix K1' at the current moment according to P1 and H1 and the covariance matrix R1 of Gaussian noise of the event camera. Then according to K1', obtaining the compensated state matrix at the current moment
And the compensated covariance matrix +.>
Namely the first +.>
and />
。
And continuously acquiring illuminance h2 at the next moment, and obtaining a state vector f2 at the next moment according to h2 and M0. And under the condition that the next moment is the current moment, updating the state matrix according to F2 to obtain an updated state matrix F2, and updating the observation matrix according to H2 to obtain an updated observation matrix H2. According to F2, Q2 and
and obtaining a covariance matrix P2 at the current moment. And obtaining a gain matrix K2' at the current moment according to the P2, the R2 and the H2. Then according to K2', obtaining the compensated state matrix at the current moment
And the compensated covariance matrix +.>
Namely second +.>
and />
. And so on.
After the number of the obtained 10 state matrixes after compensation reaches the preset number of 10, the obtained 10 state matrixes after compensation are averaged, and the obtained average value is the target change threshold M1. At this time, the change threshold of the event camera is M1, and the above process is repeatedly executed by M1 to acquire 10 compensated state matrices again, so as to obtain M2, and so on, until the event camera is turned off.
In the process of acquiring the 11 th compensated state matrix by using M1, the illuminance h11 is acquired, and the state vector f11 is obtained according to h11 and M1. At this moment, the state matrix is updated according to F11 to obtain an updated state matrix F11, and the observation matrix is updated according to H11 to obtain an updated observation matrix H11. According to F11, Q11 and
a covariance matrix P11 is obtained. And according to P11, R11 and H11, gain matrix K11' is obtained. Then according to K11', obtaining the compensated state matrix at the current moment
And the compensated covariance matrix +.>
Namely second +.>
and />
。
In the process of acquiring the 12 th compensated state matrix by using M1, the illuminance h12 is acquired, and the state vector f12 is obtained according to h12 and M1. At this moment, the state matrix is updated according to F12 to obtain an updated state matrix F12, and the observation matrix is updated according to H12 to obtain an updated observation matrix H12. According to F12, Q12 and
obtaining covariance matrix P12. And obtaining a gain matrix K12' according to P12, R12 and H12. Then according to K12', obtaining the compensated state matrix at the current moment
And the compensated covariance matrix +.>
Namely second +.>
and />
. And so on.
In this way, the method of the embodiment enables the event camera to automatically and dynamically adjust the change threshold value of the event camera in the starting operation process, adjusts the change threshold value in real time according to the applied environment, improves the accuracy of the change threshold value, ensures the setting reasonability of the change threshold value of the event camera, and improves the operation efficiency of the event camera. And the event camera is used for automatically matching different application environments by updating the change threshold in real time, so that the use effect and the user experience of a user are improved.
One or more technical solutions in the embodiments of the present invention at least have the following technical effects or advantages:
in this embodiment, during the operation of the event camera, the illuminance and the state vector of the illuminance collection window of the event camera at the current time are first obtained. Here, the illuminance of the illuminance collection window of the event camera at the current moment is measured in real time through the observation equation, and the state vector at the current moment is determined according to the illuminance at the current moment, so as to make a tamping basis for the state vector after compensation which is obtained later.
And then, based on the illuminance at the current moment and the state vector, performing compensation processing on the illuminance acquisition window to obtain a compensated state vector at the current moment. And then, after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches the preset number, obtaining a target change threshold value of the event camera according to the plurality of compensated state vectors, and adjusting the target change threshold value to be a change threshold value used in the operation process of the event camera. Here, based on the compensated state vectors at a plurality of continuous moments, a target change threshold of the event camera is determined, so that the change threshold of the event camera is adaptively adjusted, the accuracy of the change threshold is improved, the operation efficiency of the event camera is improved, the operation stability of the event camera is ensured, and the rationality of setting the change threshold of the event camera is ensured. Meanwhile, the event camera is used for automatically matching different application environments through updating the change threshold value in real time, so that the use effect and the user experience of a user are improved.
Embodiment two: based on the same inventive concept, the second embodiment of the present invention further provides an event camera adaptive threshold adjustment device, as shown in fig. 3, including:
an obtaining module 201, configured to obtain illuminance of an illuminance collection window of an event camera at a current time and a state vector of the current time during an operation process of the event camera;
the compensation module 202 is configured to perform compensation processing on the illuminance collection window based on the illuminance at the current time and the state vector at the current time, so as to obtain a compensated state vector at the current time;
and the target module 203 is configured to, after obtaining the compensated state vector at the current time, continuously obtain the compensated state vector of the illuminance collection window at the next time, until the number of the obtained plurality of compensated state vectors reaches a preset number, obtain a target change threshold of the event camera according to the plurality of compensated state vectors, and adjust the change threshold used by the event camera in the operation process to the target change threshold.
As an optional embodiment, the acquiring a state vector of the illuminance collection window of the event camera at the current moment includes:
and obtaining a state vector of the current moment according to the illuminance of the current moment and the historical change threshold of the event camera.
As an optional embodiment, the obtaining the compensated state vector at the current time based on the illuminance at the current time and the state vector at the current time includes:
obtaining a gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment;
and obtaining the compensated state vector at the current moment according to the gain matrix at the current moment.
As an optional embodiment, the obtaining the gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment includes:
updating the state matrix of the current moment according to the state vector of the current moment, and obtaining the covariance matrix of the current moment according to the updated state matrix, the covariance matrix of the last moment of the current moment and the covariance matrix of Gaussian noise of the current moment;
and obtaining a gain matrix of the current moment according to the covariance matrix of the current moment.
As an optional embodiment, the obtaining the gain matrix of the current time according to the covariance matrix of the current time includes:
updating the observation matrix at the current moment according to the illuminance at the current moment to obtain an updated observation matrix;
and obtaining a gain matrix of the current moment according to the updated observation matrix, the covariance matrix of the current moment and the covariance matrix of the Gaussian noise of the event camera.
As an optional embodiment, the obtaining the target change threshold of the event camera according to the plurality of compensated state vectors includes:
and averaging the plurality of compensated state vectors to obtain an average value of the plurality of compensated state vectors, and taking the average value as the target change threshold.
As an optional embodiment, the acquiring the illuminance of the illuminance collection window of the event camera at the current moment includes:
and obtaining the illuminance at the current moment through the observation equation at the current moment.
Since the adaptive threshold adjustment device for an event camera according to the present embodiment is a device for implementing the adaptive threshold adjustment method for an event camera according to the first embodiment of the present application, based on the adaptive threshold adjustment method for an event camera according to the first embodiment of the present application, those skilled in the art can understand the specific implementation of the adaptive threshold adjustment device for an event camera according to the present embodiment and various modifications thereof, so how to implement the adaptive threshold adjustment device for an event camera according to the first embodiment of the present application will not be described in detail herein. The device used by those skilled in the art to implement the event camera adaptive threshold adjustment method in the first embodiment of the present application is within the scope of protection intended in the present application.
Embodiment III: based on the same inventive concept, the third embodiment of the present invention also provides an event camera device, as shown in fig. 4, comprising a memory 302, a processor 301 and a computer program stored on the memory 302 and executable on the processor 301, wherein the processor 301 implements the steps of any one of the above-mentioned event camera adaptive threshold adjustment methods when executing the program.
Embodiment four: based on the same inventive concept, the fourth embodiment of the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any one of the event camera adaptive threshold adjustment methods described in the previous embodiment.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (5)
1. An event camera adaptive threshold adjustment method, comprising:
in the running process of an event camera, acquiring the illuminance of an illuminance acquisition window of the event camera at the current moment and a state vector of the current moment, wherein the method comprises the following steps:
obtaining illuminance at the current moment through an observation equation at the current moment, wherein the observation equation is an observation equation in a Kalman filtering algorithm;
obtaining a state vector of the current moment according to the illuminance of the current moment and the historical change threshold of the event camera, wherein the state vector is shown in the following formula:
wherein ,
for the illuminance at the present moment,f k the method is characterized in that the state vector of the illuminance acquisition window at the current moment is a CD vector transfer equation, wherein the CD vector transfer equation comprises a CD opening threshold value and a CD closing threshold value, < >>
Weight value for open state of CD vector transfer equation,/->
Weight value, y, for the off state of the CD vector transfer equation 1 Is the CD opening threshold at time k, y 2 Is the CD-off threshold at time k, +.>
Noise value for CD open threshold, +.>
Noise value for CD-off threshold, +.>
Global CD on threshold for event camera, +.>
A global CD close threshold for an event camera;
based on the illuminance at the current time and the state vector at the current time, obtaining a compensated state vector at the current time includes:
obtaining a gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment;
obtaining a compensated state vector at the current moment according to the gain matrix at the current moment;
the obtaining the gain matrix of the current moment according to the illuminance of the current moment and the state vector of the current moment includes:
updating the current moment according to the state vector of the current momentTo obtain an updated state matrix F k The following formula is shown:
wherein, for f k Each element in the list is subjected to respective one-time derivation to obtain F k；
And obtaining the covariance matrix of the current moment according to the updated state matrix, the covariance matrix of the last moment of the current moment and the covariance matrix of the Gaussian noise of the current moment, wherein the covariance matrix of the current moment is shown in the following formula:
, wherein ,/>
Covariance matrix representing gaussian noise at the current moment,/->
For measuring accuracy of CD vector transfer equation, P k For the covariance matrix at time k, P k-1 The covariance matrix of the last moment;
obtaining a gain matrix of the current moment according to the covariance matrix of the current moment, wherein the gain matrix is shown in the following formula:
where K' is the gain matrix, R k A covariance matrix of Gaussian noise of the event camera, representing errors of sensor measurement of the event camera, H k Is the updated observation matrix;
and after the compensated state vector at the current moment is obtained, continuously obtaining the compensated state vector of the illuminance collection window at the next moment until the number of the obtained plurality of compensated state vectors reaches a preset number, obtaining a target change threshold of the event camera according to the plurality of compensated state vectors, and adjusting the change threshold used by the event camera in the running process to be the target change threshold.
2. The method of claim 1, wherein the obtaining the gain matrix for the current time based on the covariance matrix for the current time comprises:
updating the observation matrix at the current moment according to the illuminance at the current moment to obtain an updated observation matrix;
and obtaining a gain matrix of the current moment according to the updated observation matrix, the covariance matrix of the current moment and the covariance matrix of the Gaussian noise of the event camera.
3. An event camera adaptive threshold adjustment apparatus, acting on an event camera adaptive threshold adjustment method according to any of claims 1-2, the apparatus comprising:
the acquisition module is used for acquiring the illuminance of the illuminance acquisition window of the event camera at the current moment and the state vector at the current moment in the running process of the event camera;
the compensation module is used for obtaining a compensated state vector at the current moment based on the illuminance at the current moment and the state vector at the current moment;
and the target module is used for continuously acquiring the compensated state vector of the illuminance acquisition window at the next moment after acquiring the compensated state vector at the current moment, acquiring a target change threshold of the event camera according to the plurality of compensated state vectors after the number of the acquired plurality of compensated state vectors reaches a preset number, and adjusting the change threshold used by the event camera in the running process to be the target change threshold.
4. An event camera device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method steps of any of claims 1-2 when the program is executed.
5. A computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor realizes the method steps of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310285291.2A CN115996320B (en) | 2023-03-22 | 2023-03-22 | Event camera adaptive threshold adjustment method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310285291.2A CN115996320B (en) | 2023-03-22 | 2023-03-22 | Event camera adaptive threshold adjustment method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115996320A CN115996320A (en) | 2023-04-21 |
| CN115996320B true CN115996320B (en) | 2023-06-16 |
Family
ID=85993775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310285291.2A Active CN115996320B (en) | 2023-03-22 | 2023-03-22 | Event camera adaptive threshold adjustment method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115996320B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113810611A (en) * | 2021-09-17 | 2021-12-17 | 北京航空航天大学 | Method and device for data simulation of event camera |
| CN115278059A (en) * | 2022-06-27 | 2022-11-01 | 深圳锐视智芯科技有限公司 | A parameter configuration method and related device |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201612528D0 (en) * | 2016-07-19 | 2016-08-31 | Machines With Vision Ltd | Vehicle localisation using the ground or road surface |
| US10845601B1 (en) * | 2018-02-07 | 2020-11-24 | Apple Inc. | AR/VR controller with event camera |
| CN109005329B (en) * | 2018-09-19 | 2020-08-11 | 广东工业大学 | Pixel unit, image sensor and camera |
| CN112399032B (en) * | 2019-08-13 | 2022-05-31 | 天津大学青岛海洋技术研究院 | Optical flow acquisition method of pulse type image sensor based on detector |
| CN113038040B (en) * | 2021-03-03 | 2023-03-24 | 深圳市丛矽微电子科技有限公司 | Event camera, threshold adjusting method and system |
| CN113888607A (en) * | 2021-09-02 | 2022-01-04 | 中国电子科技南湖研究院 | Target detection and tracking method and system based on event camera and storage medium |
| CN114359714B (en) * | 2021-12-15 | 2025-05-27 | 中国电子科技南湖研究院 | Unmanned human obstacle avoidance method, device and intelligent unmanned human based on event camera |
| CN115022621B (en) * | 2022-06-27 | 2025-01-21 | 深圳锐视智芯科技有限公司 | Event camera testing method, device, equipment and readable storage medium |
| CN115375581B (en) * | 2022-09-05 | 2025-06-27 | 东南大学 | Evaluation method of denoising effect of dynamic visual event stream based on event spatiotemporal synchronization |
-
2023
- 2023-03-22 CN CN202310285291.2A patent/CN115996320B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113810611A (en) * | 2021-09-17 | 2021-12-17 | 北京航空航天大学 | Method and device for data simulation of event camera |
| CN115278059A (en) * | 2022-06-27 | 2022-11-01 | 深圳锐视智芯科技有限公司 | A parameter configuration method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115996320A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11770619B2 (en) | Generating static images with an event camera | |
| WO2022241874A1 (en) | Infrared thermal imaging monocular vision ranging method and related assembly | |
| WO2018077121A1 (en) | Method for recognizing target object in image, method for recognizing food article in refrigerator and system | |
| CN112640417A (en) | Matching relation determining method, reprojection error calculating method and related device | |
| CN109272520B (en) | An adaptive infrared focal plane non-uniformity correction method based on joint motion guidance and edge detection | |
| CN110197126A (en) | A kind of target tracking method, device and portable terminal | |
| CN105631895A (en) | Temporal-spatial context video target tracking method combining particle filtering | |
| CN111798485B (en) | Event camera optical flow estimation method and system enhanced by IMU | |
| CN110166707A (en) | Image processing method, device, electronic device, and storage medium | |
| CN107403414B (en) | A kind of image area selecting method and system being conducive to fuzzy kernel estimates | |
| CN109029435B (en) | A method for improving the precision of inertial-geomagnetic combination dynamic attitude determination | |
| JP2017028511A5 (en) | ||
| WO2009016256A1 (en) | Ultra-compact aperture controlled depth from defocus range sensor | |
| CN111340732B (en) | A low-light video image enhancement method and device | |
| WO2017045468A1 (en) | Method and device for correcting position error | |
| CN113763276A (en) | Underwater imaging method and system | |
| CN104866821B (en) | Video object tracking based on machine learning | |
| CN116363163A (en) | Space object detection and tracking method, system and storage medium based on event camera | |
| CN115996320B (en) | Event camera adaptive threshold adjustment method, device, equipment and storage medium | |
| CN102547157A (en) | Adaptive phase calibration method of correlated double sampling | |
| CN110111341B (en) | Image foreground obtaining method, device and equipment | |
| CN103278247A (en) | Background noise characteristic estimation method for point source Hartmann wavefront detector | |
| JP4563982B2 (en) | Motion estimation method, apparatus, program thereof, and recording medium thereof | |
| CN111798484A (en) | Method and system for continuous dense optical flow estimation based on event camera | |
| CN111382646B (en) | Living body identification method, storage medium and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230727 Address after: 201805 Room 508, No. 1080, Moyu South Road, Anting, Jiading District, Shanghai J4553 Patentee after: Shanghai Beihu Ice Silicon Technology Co.,Ltd. Address before: 310, Building 1, Shenzhen New Generation Industrial Park, 136 Zhongkang Road, Meidu Community, Meilin Street, Futian District, Shenzhen City, Guangdong Province, 518049 Patentee before: SHENZHEN JIUTIAN RUIXIN TECHNOLOGY Co.,Ltd. |