CN110718068B - Road monitoring camera installation angle estimation method - Google Patents
Road monitoring camera installation angle estimation method Download PDFInfo
- Publication number
- CN110718068B CN110718068B CN201910931121.0A CN201910931121A CN110718068B CN 110718068 B CN110718068 B CN 110718068B CN 201910931121 A CN201910931121 A CN 201910931121A CN 110718068 B CN110718068 B CN 110718068B
- Authority
- CN
- China
- Prior art keywords
- camera
- image
- installation angle
- straight line
- angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000009434 installation Methods 0.000 title claims abstract description 47
- 238000012544 monitoring process Methods 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 abstract description 5
- 230000001965 increasing effect Effects 0.000 abstract description 4
- 230000009466 transformation Effects 0.000 abstract description 3
- 238000011900 installation process Methods 0.000 abstract description 2
- 238000002372 labelling Methods 0.000 abstract description 2
- 238000005259 measurement Methods 0.000 description 6
- 238000007664 blowing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for estimating the installation angle of a road monitoring camera, and belongs to the field of road monitoring. The method comprises the following steps: detecting horizontal vanishing points of the images acquired by the camera based on parallel lines; and acquiring the installation angle of the road monitoring camera by utilizing the relation between the horizontal vanishing point in the image and the installation angle of the camera. The invention combines the priori knowledge of the external conditions of the road monitoring scene and the projection transformation principle in the imaging principle, utilizes the relation between the vanishing point and the installation angle of the video camera, and calculates two concerned installation angles through a mathematical formula by taking the characteristic that only two angles have freedom degrees in the installation process of the security camera as an auxiliary. Therefore, after the monitoring camera is installed, the installation angle of the camera can be automatically calculated on the basis of not increasing any operation flow. The installation angle is automatically identified at intervals of fixed time so as to adapt to the influence of natural environment factors on the installation angle of the camera and achieve the purpose of automatic labeling.
Description
Technical Field
The invention belongs to the field of road monitoring, and particularly relates to a method for estimating the installation angle of a road monitoring camera.
Background
In recent years, with the rapid development of economy, the quantity of motor vehicles kept in urban and rural areas in China is rapidly increased, and criminals and security cases related to vehicles are also increased year by year. Under the condition, the advanced scientific and technological means are effectively utilized to improve the urban and rural road traffic management level, hit and prevent car-related cases, deter criminals and improve the social security management level, so that the method becomes an important work of all levels of public security traffic management departments. Many functions of a road monitoring camera need to calibrate external parameters of the camera, such as the installation height of the camera, the pitch angle, the yaw angle, the roll angle and other parameters of the camera, the measurement of the installation angle is difficult in the process of camera deployment, and in road monitoring, the installation angle of the camera can be changed due to the influence of external environment, such as wind blowing, pole shaking and the like, so that the condition of inaccuracy appears behind the earlier-measured angle.
At present, the following measurement methods are generally used for measuring the installation angle of a camera: 1. measuring by a level equiangular measuring tool; 2. measuring by installing equipment such as a gyroscope in the camera; 3. the reference object is manually searched in the camera, and then the installation angle of the camera is calculated through the conversion relation.
However, the first method is to measure the angle by a measuring tool, so that the engineering implementation is very difficult, and automatic updating cannot be achieved after one measurement. In the second method, the number of security monitoring cameras with gyroscopes is very small, and the security monitoring cameras are difficult to spread out for use in a large area. The third method requires a clear reference in the camera field angle, which is not necessarily available in most cameras. And calibration can be carried out only once during installation, the security monitoring camera works in outdoor scenes all the year round, the installation angle of the camera can be changed due to wind blowing, high-temperature deformation and the like, and the method cannot automatically identify the installation angle to adapt to the application scenes.
Disclosure of Invention
Aiming at the defects of high cost and incapability of self-adaption of an angle measuring method in the prior art, the invention provides a road monitoring camera installation angle estimation method, aiming at automatically calibrating the installation angle of a camera under the condition of no need of human intervention, providing automatically extracted parameters for distance measurement, speed measurement and the like of a three-dimensional camera, and automatically calibrating the installation angle of the camera in real time at the lowest cost.
To achieve the above object, according to a first aspect of the present invention, there is provided a road monitoring camera installation angle estimation method, including the steps of:
s1, detecting horizontal vanishing points of an image acquired by a camera based on parallel lines;
and S2, acquiring the installation angle of the road monitoring camera by utilizing the relation between the horizontal vanishing point in the image and the installation angle of the camera.
Specifically, step S1 includes the following sub-steps:
s11, detecting straight line segments in the enhanced image;
s12, removing horizontal line segments from the detected straight line segment set;
s13, solving intersection points by using the rest straight line segments pairwise to obtain intersection point areas of the straight line segments;
s14, obtaining a central point of the intersection point area, and taking the central point as a horizontal vanishing point of the image;
and S15, carrying out normalization operation on the vanishing points by utilizing the width and the height of the image and the image center information to obtain normalized vanishing points.
Specifically, step S1 includes the following sub-steps:
s111, acquiring a Y channel from the input image, and enhancing the lane line based on the Y channel;
and S112, detecting straight line segments in the enhanced image by using an LSD algorithm.
Specifically, the lane line enhancement based on the Y channel is as follows:
E(x,y)=2P(x,y)-P(x-N,y)-P(x+N,y)
wherein E (x, Y) represents the pixel with x abscissa and Y ordinate after the lane line is enhanced, P (x, Y) represents the pixel with x abscissa and Y ordinate in the Y channel before the lane line is enhanced, and the width of the lane line in the image is 2 × N.
Specifically, in step S12, it is determined whether the straight line segment is a straight line in the horizontal direction according to the included angle between the straight line segment and the horizontal direction: if the absolute value of the included angle between the straight line segment and the image horizontal direction is less than alpha, the straight line segment is considered to be a horizontal line segment, and the value range of the threshold value alpha is [10 degrees ], 30 degrees ].
Specifically, in step S14, the center point of the intersection region is obtained using the MeanShift algorithm.
Specifically, the relationship between the horizontal vanishing point in the image and the installation angle of the camera is used to obtain the installation angle of the road monitoring camera, which specifically includes:
wherein, phi represents a yaw angle,representing the pitch angle, Ψ representing the roll angle, and vpx and vpy representing the location of the horizontal vanishing point in the image in the normalized image, respectively.
Specifically, the steps S1 to S2 are repeated at regular intervals for the image stream acquired by the camera, and automatic identification of the installation angle of the road monitoring camera is realized.
To achieve the above object, according to a second aspect of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the road monitoring camera installation angle estimation method according to the first aspect.
Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:
the invention combines the priori knowledge of the external conditions of the road monitoring scene and the projection transformation principle in the imaging principle, utilizes the relation between the vanishing point and the installation angle of the camera, and calculates two concerned installation angles through a mathematical formula by taking the characteristic of freedom degrees of two angles (namely the roll angle of the camera is 0) in the installation process of the security camera as an auxiliary. Therefore, after the monitoring camera is installed, the installation angle of the camera can be automatically calculated on the basis of not increasing any operation flow, and the installed security monitoring camera is also suitable. Because the method automatically calculates the installation angle, the installation angle can be automatically identified at intervals of fixed time so as to adapt to the influence of natural environment factors on the installation angle of the camera and achieve the purpose of automatic labeling.
Drawings
Fig. 1 is a flowchart of a method for estimating an installation angle of a road surveillance camera according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of vanishing points in an image according to an embodiment of the present invention;
fig. 3 is a flowchart of vanishing point detection provided in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The invention combines the prior knowledge of the external conditions of the road monitoring scene and the projection transformation principle in the imaging principle, provides an algorithm for automatically identifying the installation angle of the road monitoring camera, can automatically calibrate the installation angle of the camera without human intervention, and provides automatically extracted parameters for distance measurement, speed measurement and the like of the three-dimensional camera.
As shown in fig. 1, the present invention provides a method for estimating a mounting angle of a road surveillance camera, the method comprising the steps of:
and S1, detecting horizontal vanishing points of the image acquired by the camera based on parallel lines.
The method is applied to a road security monitoring camera, and mainly takes cameras without roll angles, such as a gun camera, a dome camera and the like. The vanishing point in the image refers to the intersection point of parallel lines of the physical world in the image as shown in fig. 2, therefore, the parallel lines in the image are required to be present for detecting the vanishing point in the image, and a great number of parallel lines, such as road traffic sign lines, road edges, fences and the like, can be used as raw materials for detecting the vanishing point in road monitoring.
As shown in fig. 3, step S1 includes the following sub-steps:
and S11, detecting straight line segments in the enhanced image.
And S111, acquiring a Y channel from the input image, and enhancing by using the lane line.
The inherent characteristics of road monitoring are utilized to detect the enhanced road mark in the road monitoring image in the image, thereby conveniently extracting the straight line segment from the image.
The gray value of all the lane lines in the Y channel is higher than that of the road surface background pixels, the lane lines are in a strip shape, and the width of the lane lines is relatively fixed. Therefore, based on this characteristic, the present invention enhances the lane line by the following method:
assuming that the width of the lane line in the image is 2 × N, P (x, Y) represents a pixel with x abscissa and Y ordinate in the Y channel before the lane line is enhanced, and E (x, Y) represents a pixel with x abscissa and Y ordinate after the lane line is enhanced, E (x, Y) is 2P (x, Y) -P (x-N, Y) -P (x + N, Y). After the enhancement by the method, the lane line part will be highlighted in the image.
And S112, detecting straight Line segments in the enhanced image by using an LSD (Line Segment Detector) algorithm.
A great number of parallel lines in a road monitoring scene can be utilized, and the parallel lines can be automatically extracted by using computer vision knowledge.
The invention preferably detects straight line segments in the enhanced image by using an LSD algorithm, and the detected straight line is more accurate and more suitable for lane lines with alternate virtual and real.
And S12, removing the horizontal line segments from the detected straight line segment set.
Since it is necessary to find the vanishing point, it is necessary to eliminate the influence of the horizontal or nearly horizontal straight line segment. And judging whether the straight line section is a straight line in the horizontal direction or not according to the included angle between the straight line section and the horizontal direction. And eliminating the horizontal line segments from the detected straight line segments, thereby acquiring intersection points (namely vanishing points) of parallel lines of the physical world in the image by utilizing the identified parallel line characteristics of the road traffic.
If the absolute value of the included angle between the straight line segment and the image horizontal direction is less than 15 degrees, the straight line segment is regarded as a horizontal line segment; and if the absolute value of the included angle between the straight line segment and the vertical direction is less than 15 degrees, judging as the vertical line segment.
And S13, solving intersection points by using the rest straight line segments pairwise to obtain intersection point areas of the straight line segments.
And S14, obtaining a central point of the intersection point area, and taking the central point as a horizontal vanishing point of the image.
Vanishing points in the image can be estimated using parallel lines of the physical world.
The invention preferably selects the MeanShift algorithm to obtain the center point of the intersection point area, and the obtained center point has higher accuracy, is less influenced by initialization and is less interfered by noise.
And S15, carrying out normalization operation on the vanishing points by utilizing the width and the height of the image and the image center information to obtain normalized vanishing points.
And S2, acquiring the installation angle of the road monitoring camera by utilizing the relation between the horizontal vanishing point in the image and the installation angle of the camera.
By utilizing the relationship between the vanishing point in the imaging principle and the camera angle, the equation with infinite solutions can be changed into an equation with feasible solutions by acquiring under the condition that the roll angle of the camera is assumed to be 0, so that the installation angle of the camera can be obtained.
The imaging principle is utilized, and the relation between the horizontal vanishing point in the image and the installation angle of the camera is shown as the following formula:
wherein, phi represents a yaw angle,the pitch angle is represented, psi represents the roll angle, the camera support for general road monitoring only has two degrees of freedom, namely, the pitch angle and the yaw angle, therefore, as long as the camera is installed, the roll angle of the camera can be ensured to be 0 degree, and the following formula can be obtained by substituting the known conditions into the above formula:
where vpx and vpy represent the location of the horizontal vanishing point in the image in the normalized image, respectively, and there are only two unknowns in the equation that can form two equations, so that the equation has a solution that can be solved to obtain:
it can be known from the above equation that the three-dimensional mounting angle of the camera can be obtained only by locating the normalized position of the horizontal vanishing point in the image. The three-dimensional mounting angle of the camera can be expressed as:
it will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for estimating the installation angle of a road monitoring camera is characterized by comprising the following steps:
s1, detecting horizontal vanishing points of an image acquired by a camera based on parallel lines;
s2, acquiring the installation angle of the road monitoring camera by utilizing the relation between the horizontal vanishing point and the installation angle of the camera in the image, wherein the method specifically comprises the following steps:
where Φ represents yaw angle, Θ represents pitch angle, Ψ represents roll angle, and vpx and vpy represent the location of the horizontal vanishing point in the image in the normalized image, respectively.
2. The method of claim 1, wherein step S1 includes the sub-steps of:
s11, detecting straight line segments in the enhanced image;
s12, removing horizontal line segments from the detected straight line segment set;
s13, solving intersection points by using the rest straight line segments pairwise to obtain intersection point areas of the straight line segments;
s14, obtaining a central point of the intersection point area, and taking the central point as a horizontal vanishing point of the image;
and S15, carrying out normalization operation on the vanishing points by utilizing the width and the height of the image and the image center information to obtain normalized vanishing points.
3. The method of claim 2, wherein step S1 includes the sub-steps of:
s111, acquiring a Y channel from the input image, and enhancing the lane line based on the Y channel;
and S112, detecting straight line segments in the enhanced image by using an LSD algorithm.
4. The method of claim 3, wherein the Y-channel based lane line enhancement is as follows:
E(x,y)=2P(x,y)-P(x-N,y)-P(x+N,y)
wherein E (x, Y) represents the pixel with x abscissa and Y ordinate after the lane line is enhanced, P (x, Y) represents the pixel with x abscissa and Y ordinate in the Y channel before the lane line is enhanced, and the width of the lane line in the image is 2 × N.
5. The method as claimed in claim 2, wherein in step S12, it is determined whether the straight line segment is a straight line in the horizontal direction according to the included angle between the straight line segment and the horizontal: if the absolute value of the included angle between the straight line segment and the image horizontal direction is less than alpha, the straight line segment is considered to be a horizontal line segment, and the value range of the threshold value alpha is [10 degrees ], 30 degrees ].
6. The method of claim 2, wherein in step S14, the center point of the intersection region is obtained using a MeanShift algorithm.
7. The method as claimed in claim 1, wherein the steps S1-S2 are repeated at regular intervals for the stream of images captured by the camera, thereby achieving automatic recognition of the installation angle of the road monitoring camera.
8. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the road monitoring camera installation angle estimation method according to any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910931121.0A CN110718068B (en) | 2019-09-27 | 2019-09-27 | Road monitoring camera installation angle estimation method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910931121.0A CN110718068B (en) | 2019-09-27 | 2019-09-27 | Road monitoring camera installation angle estimation method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110718068A CN110718068A (en) | 2020-01-21 |
CN110718068B true CN110718068B (en) | 2020-12-08 |
Family
ID=69212068
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910931121.0A Expired - Fee Related CN110718068B (en) | 2019-09-27 | 2019-09-27 | Road monitoring camera installation angle estimation method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110718068B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112017249A (en) * | 2020-08-18 | 2020-12-01 | 东莞正扬电子机械有限公司 | Vehicle-mounted camera roll angle obtaining and mounting angle correcting method and device |
CN113206994B (en) * | 2021-03-30 | 2023-03-28 | 华为技术有限公司 | Camera automatic engineering survey method and device, electronic equipment, chip and storage medium |
CN113706630B (en) * | 2021-08-26 | 2024-02-06 | 西安电子科技大学 | Method for calibrating pitch angle of camera based on group of horizontal parallel lines |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106462968A (en) * | 2014-05-14 | 2017-02-22 | 大众汽车有限公司 | Method and apparatus for calibrating a camera system in a motor vehicle |
CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
CN109948552A (en) * | 2019-03-20 | 2019-06-28 | 四川大学 | It is a kind of complexity traffic environment in lane detection method |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE112013002200T5 (en) * | 2012-04-27 | 2015-01-08 | Adobe Systems Incorporated | Automatic adjustment of images |
CN102831426B (en) * | 2012-08-30 | 2014-11-05 | 杭州电子科技大学 | Road environment self-adaptive straight-lane detection method |
CN107133985B (en) * | 2017-04-20 | 2020-05-12 | 常州智行科技有限公司 | Automatic calibration method for vehicle-mounted camera based on lane line vanishing point |
CN107492123B (en) * | 2017-07-07 | 2020-01-14 | 长安大学 | Road monitoring camera self-calibration method using road surface information |
CN109961483B (en) * | 2017-12-22 | 2022-03-18 | 比亚迪股份有限公司 | Calibration method and system for automobile and monocular camera |
CN109345593B (en) * | 2018-09-04 | 2022-04-26 | 海信集团有限公司 | Camera posture detection method and device |
CN109685855B (en) * | 2018-12-05 | 2022-10-14 | 长安大学 | Camera calibration optimization method under road cloud monitoring platform |
-
2019
- 2019-09-27 CN CN201910931121.0A patent/CN110718068B/en not_active Expired - Fee Related
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106462968A (en) * | 2014-05-14 | 2017-02-22 | 大众汽车有限公司 | Method and apparatus for calibrating a camera system in a motor vehicle |
CN106558080A (en) * | 2016-11-14 | 2017-04-05 | 天津津航技术物理研究所 | Join on-line proving system and method outside a kind of monocular camera |
CN109948552A (en) * | 2019-03-20 | 2019-06-28 | 四川大学 | It is a kind of complexity traffic environment in lane detection method |
Also Published As
Publication number | Publication date |
---|---|
CN110718068A (en) | 2020-01-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11854272B2 (en) | Hazard detection from a camera in a scene with moving shadows | |
JP5747549B2 (en) | Signal detector and program | |
JP4717760B2 (en) | Object recognition device and video object positioning device | |
CN110718068B (en) | Road monitoring camera installation angle estimation method | |
CN109099929B (en) | Intelligent vehicle positioning device and method based on scene fingerprints | |
CN111723801A (en) | Method and system for detecting and correcting target in fisheye camera picture | |
CN104933398A (en) | vehicle identification system and method | |
CN110349172B (en) | Power transmission line external damage prevention early warning method based on image processing and binocular stereo ranging | |
CN113029185B (en) | Road marking change detection method and system in crowdsourcing type high-precision map updating | |
CN114719873B (en) | Low-cost fine map automatic generation method and device and readable medium | |
CN114283126B (en) | Method for detecting deviation of monitoring equipment of power transmission line | |
KR101381580B1 (en) | Method and system for detecting position of vehicle in image of influenced various illumination environment | |
CN113971697A (en) | Air-ground cooperative vehicle positioning and orienting method | |
KR102259981B1 (en) | Visual range measurement and alarm system based on video analysis and method thereof | |
CN117710905A (en) | Video monitoring information processing method, system, device and storage medium | |
Ji et al. | Manhole cover detection using vehicle-based multi-sensor data | |
CN111860084B (en) | Image feature matching and positioning method and device and positioning system | |
Laureshyn et al. | Automated video analysis as a tool for analysing road user behaviour | |
CN107301399B (en) | Water surface bridge identification method based on overlook image | |
JP5533988B2 (en) | Image processing system | |
CN113744345A (en) | Camera tilt detection method, device, electronic device, and storage medium | |
CN107907109B (en) | Angle identification method based on improved image processing and small unmanned aerial vehicle | |
CN110659577A (en) | Blind road obstacle detection method for smart phone platform | |
CN116539635B (en) | Unmanned aerial vehicle shield tunnel inspection method based on calculation vision and SLAM | |
GB2523598A (en) | Method for determining the position of a client device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201208 |