CN108622176B - Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition - Google Patents
Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition Download PDFInfo
- Publication number
- CN108622176B CN108622176B CN201810497789.4A CN201810497789A CN108622176B CN 108622176 B CN108622176 B CN 108622176B CN 201810497789 A CN201810497789 A CN 201810497789A CN 108622176 B CN108622176 B CN 108622176B
- Authority
- CN
- China
- Prior art keywords
- unit
- steering wheel
- near infrared
- light source
- gesture recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 23
- 238000005286 illumination Methods 0.000 claims abstract description 49
- 238000003384 imaging method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000010363 phase shift Effects 0.000 claims abstract description 8
- 239000000178 monomer Substances 0.000 claims description 12
- 238000005259 measurement Methods 0.000 claims description 11
- 238000005516 engineering process Methods 0.000 claims description 4
- 238000000034 method Methods 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
- B62D—MOTOR VEHICLES; TRAILERS
- B62D1/00—Steering controls, i.e. means for initiating a change of direction of the vehicle
- B62D1/02—Steering controls, i.e. means for initiating a change of direction of the vehicle vehicle-mounted
- B62D1/04—Hand wheels
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
- B62D—MOTOR VEHICLES; TRAILERS
- B62D1/00—Steering controls, i.e. means for initiating a change of direction of the vehicle
- B62D1/24—Steering controls, i.e. means for initiating a change of direction of the vehicle not vehicle-mounted
- B62D1/28—Steering controls, i.e. means for initiating a change of direction of the vehicle not vehicle-mounted non-mechanical, e.g. following a line or other known markers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Landscapes
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The invention discloses a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, and relates to the field of human-vehicle interaction interfaces. The system comprises a near infrared illumination light source unit, a TOF imaging array sensor unit, a lens module unit, a TOP control unit, a three-dimensional point cloud image processing unit and an automobile steering wheel, wherein the near infrared illumination light source unit is used for illuminating a gesture recognition area; the TOP control unit modulates the near infrared illumination light source unit, simultaneously controls the TOF imaging array sensor unit to measure the phase shift between gesture illumination and reflection and convert the phase shift into distance depth information for gesture recognition, and the TOF imaging array sensor unit completes analysis through the three-dimensional point cloud image processing unit and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN/LIN bus. The method effectively improves the flexibility of application of gesture operation in the vehicle, and can perform remote and non-contact control, so that the human-vehicle interaction mode is more natural and convenient.
Description
Technical Field
The invention relates to the field of human-vehicle interaction interfaces, in particular to a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition.
Background
At present, the man-vehicle interaction mode on the multifunctional steering wheel is mainly based on mechanical keys, a deflector rod and a knob. These input devices have certain disadvantages, and the input habits and the natural communication habits of the person themselves are different.
Disclosure of Invention
In order to solve the defects of the traditional mechanical keys, the deflector rods and the knobs on the multifunctional steering wheel, the invention provides the multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, the flexibility of application of gesture operation in the automobile is effectively improved by adopting a non-contact gesture mode, and the related functions of the automobile body can be controlled remotely and in a non-contact mode, so that the human-vehicle interaction mode is more natural and more convenient, and the development trend of human-vehicle interaction of the intelligent automobile in the future is met.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition comprises a near infrared illumination light source unit, a TOF imaging array sensor unit, a TOP control unit, a three-dimensional point cloud image processing unit and an automobile steering wheel, wherein the near infrared illumination light source unit and the TOF imaging array sensor unit are respectively arranged on two sides of the automobile steering wheel; the near infrared illumination light source unit illuminates the gesture recognition area; the TOP control unit modulates the near infrared illumination light source unit, simultaneously controls the TOF imaging array sensor unit to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit, and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN bus or the LIN bus.
Further, after the near infrared light emitted by the near infrared illumination light source unit is reflected by the gesture recognition, the reflected near infrared light is received by the TOF imaging array sensor unit.
Further, the near-infrared illumination light source unit includes a plurality of illumination monomers; the TOF imaging array sensor unit simultaneously measures a plurality of illumination monomers at a time, the interval between each illumination monomer is corresponding to the phase of the illumination monomer number, and the phase difference between illumination and reflection is utilized to calculate the distance depth information of gesture recognition.
Further, the TOP control unit adopts a multi-frequency technology to modulate the near infrared illumination light source unit, so that the problems of measurement accuracy and measurement distance are solved, namely, one or more modulation frequencies are added to work, and the optimal detection accuracy is obtained under the condition of enough measurement distance. Each modulation frequency will calculate a corresponding measured distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies.
Further, the TOP control unit modulates the near-infrared illumination light source unit, and simultaneously controls the TOF imaging array sensor unit to receive the reflected near-infrared light to generate a three-dimensional point cloud image, so that distance and depth information of each pixel point can be provided and displayed in a gray form, and the gesture approaches the TOF imaging array sensor unit more when the gray value is higher.
Further, the three-dimensional point cloud image processing unit separates the gesture part from other backgrounds of the image according to distance and depth information (as an important characteristic of gesture recognition) obtained by the TOF imaging array sensor unit.
Further, the multifunctional steering wheel human-vehicle interaction system further comprises a lens module unit, and the lens module unit adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit, so that the gesture recognition area range meets the requirements of actual application scenes.
The beneficial effects are that:
1. According to the invention, traditional mechanical keys, a deflector rod and a knob on the multifunctional steering wheel are replaced by a gesture control mode, so that light function control, audio function control, video and audio function control, wiper function control, instrument panel function control, constant-speed cruising and telephone function control are realized.
2. The invention adopts the video input mode to control the vehicle body in a long distance and non-contact way, and realizes the gesture control of the related functions of the vehicle body; compared with the traditional mode, the gesture recognition can achieve higher accuracy in weak light and strong light environments, so that interaction between people and vehicles is more natural and convenient.
3. The TOF gesture recognition system is high in integration level and compact in structure, the calculation intensity brought by the recognition algorithm can be realized in an embedded system, meanwhile, the requirements of detection distance and measurement accuracy are met in the in-vehicle environment, the response time is the fastest in a vision system, and the system accords with the development trend of human-vehicle interaction of the intelligent automobile in the future.
Drawings
FIG. 1 is a front view of a multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition in accordance with an embodiment of the present invention;
FIG. 2 is a left side view of a multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a lens module unit according to an embodiment of the invention;
FIG. 4 is a waveform diagram of phase difference measurement between illumination and reflection in accordance with an embodiment of the present invention;
FIG. 5 is a three-dimensional point cloud of an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating segmentation of a gesture portion from other background of an image according to an embodiment of the present invention;
In the figure: the device comprises a 1-infrared illumination light source unit, a 2-TOF imaging array sensor unit, a 3-lens module unit, a 4-TOP control unit, a 5-three-dimensional point cloud image processing unit and a 6-automobile steering wheel.
Detailed Description
The invention will be further described with reference to the drawings and examples.
The embodiment provides a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, which is shown in fig. 1-2, and comprises a near infrared illumination light source unit 1, a TOF imaging array sensor unit 2, a lens module unit 3, a TOP control unit 4, a three-dimensional point cloud image processing unit 5 and an automobile steering wheel 6, wherein the near infrared illumination light source unit 1 and the TOF imaging array sensor unit 2 are respectively arranged at two sides of the automobile steering wheel 6; the near infrared illumination light source unit 1 illuminates the gesture recognition area; the lens module unit 3 (figure 3) adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit 2 so that the gesture recognition area range meets the requirement of the actual application scene; the TOP control unit 4 modulates the near infrared illumination light source unit 1, simultaneously controls the TOF imaging array sensor unit 2 to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit 5, and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN bus or the LIN bus.
In the present embodiment, near infrared light emitted from a near infrared illumination light source unit 1 (model: lumentum 350mW,854nm TO-56 Packaged Laser Diode 22045504) is reflected by a recognition gesture, and the reflected near infrared light is received by a TOF imaging array sensor unit 2 (model: 320× 240 Array OPT8241).
In the present embodiment, the near-infrared illumination light source unit 1 includes a plurality of illumination monomers; the TOF imaging array sensor unit 2 simultaneously measures a plurality of illumination monomers at a time, each of the illumination monomers is spaced by a phase corresponding to the number of the illumination monomers, and the distance depth information of gesture recognition is calculated by using the phase difference between illumination and reflection. Specifically: as shown in fig. 4, the distance depth d of gesture recognition (formula (2) below) is calculated using the phase angle Φ (formula (1) below) between illumination and reflection.
Wherein c is the speed of light; q1, Q2, Q3, and Q4 are the amounts of electric charges sampled by the TOF imaging array sensor unit 2.
In the present embodiment, the TOP control unit 4 modulates the near-infrared illumination light source unit 1 by using the multi-frequency technology, which gives consideration to both measurement accuracy and measurement distance problems, that is, one or more modulation frequencies are added to operate so that the optimal detection accuracy is obtained under the condition of sufficient measurement distance. Each modulation frequency will calculate a corresponding measured distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies. Specifically: the distance measurement variance of the continuous square wave light source modulation detection gesture recognition can be expressed as the following formula (3):
where c d is the modulation contrast, the ability of the TOF imaging array sensor unit 2 to separate and collect photoelectrons is described.
Wherein the reflected light intensity L is a function of the light power, and can be expressed as the following formula (4):
wherein the offset S is a function of the ambient light and the system offset, and can be expressed as the following formula (5):
from equation (3), it can be deduced that the reflected light brightness, the modulation frequency and the modulation contrast are three factors affecting the distance measurement accuracy.
In this embodiment, the TOP control unit 4 (model: 3D ToF Controller OPT9221) modulates the near-infrared illumination light source unit 1, and at the same time controls the TOF imaging array sensor unit 2 to receive the reflected near-infrared light, so as to generate a three-dimensional point cloud image, and may provide depth information of each pixel point, as shown in fig. 5, and perform display in a gray scale form, where the higher the gray scale value, the closer the gesture is to the TOF imaging array sensor unit 2.
In the present embodiment, the three-dimensional point cloud image processing unit 5 (model: high-performance ARMAM437 x) separates the gesture sites from other backgrounds of the image according to the distance depth information (as an important feature of gesture recognition) obtained by the TOF imaging array sensor unit 2, as shown in fig. 6.
It should be apparent to those skilled in the art that various modifications or variations can be made in the present invention without requiring any inventive effort by those skilled in the art based on the technical solutions of the present invention.
Claims (3)
1. A multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition is characterized in that: the multifunctional steering wheel human-vehicle interaction system comprises a near infrared illumination light source unit (1), a TOF imaging array sensor unit (2), a TOP control unit (4), a three-dimensional point cloud image processing unit (5) and an automobile steering wheel (6), wherein the near infrared illumination light source unit (1) and the TOF imaging array sensor unit (2) are respectively arranged on two sides of the automobile steering wheel (6); the near infrared illumination light source unit (1) illuminates a gesture recognition area; the TOP control unit (4) modulates the near infrared illumination light source unit (1), simultaneously controls the TOF imaging array sensor unit (2) to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit (5), and outputs a corresponding steering wheel multifunctional control signal to a vehicle body controller through a CAN bus or a LIN bus;
The near-infrared illumination light source unit (1) includes a plurality of illumination monomers; the TOF imaging array sensor unit (2) simultaneously measures a plurality of illumination monomers at one time, the interval between each illumination monomer is corresponding to the phase of the illumination monomer number, and the distance depth information of gesture recognition is calculated by utilizing the phase difference between illumination and reflection;
The TOP control unit (4) modulates the near infrared illumination light source unit (1) by adopting a multi-frequency technology; the multi-frequency technology works by adding one or more modulation frequencies, each modulation frequency is calculated to obtain a corresponding measurement distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies;
the multifunctional steering wheel human-vehicle interaction system further comprises a lens module unit (3), wherein the lens module unit (3) adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit (2);
The three-dimensional point cloud image processing unit (5) separates gesture parts from other backgrounds of the images according to the distance and depth information obtained by the TOF imaging array sensor unit (2).
2. The multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition of claim 1, wherein: after the near infrared light emitted by the near infrared illumination light source unit (1) is reflected by the gesture recognition, the reflected near infrared light is received by the TOF imaging array sensor unit (2).
3. The multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition of claim 1, wherein: the TOP control unit (4) modulates the near infrared illumination light source unit (1) and simultaneously controls the TOF imaging array sensor unit (2) to receive the reflected near infrared light so as to generate a three-dimensional point cloud image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810497789.4A CN108622176B (en) | 2018-05-23 | 2018-05-23 | Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810497789.4A CN108622176B (en) | 2018-05-23 | 2018-05-23 | Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108622176A CN108622176A (en) | 2018-10-09 |
CN108622176B true CN108622176B (en) | 2024-10-29 |
Family
ID=63694081
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810497789.4A Active CN108622176B (en) | 2018-05-23 | 2018-05-23 | Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108622176B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110822719A (en) * | 2019-10-08 | 2020-02-21 | 广东万博电气有限公司 | Gesture-controlled electric water heater |
CN111679439B (en) * | 2020-08-11 | 2020-12-18 | 上海鲲游光电科技有限公司 | Optical field modulator and modulation method thereof |
CN116009275B (en) * | 2022-12-30 | 2024-07-19 | 福州大学 | Naked eye 3D display device capable of recognizing dynamic gestures |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102016636A (en) * | 2009-03-05 | 2011-04-13 | 松下电器产业株式会社 | Distance measuring device, distance measuring method, program and integrated circuit |
KR101328441B1 (en) * | 2012-06-07 | 2013-11-20 | 자동차부품연구원 | Interface device of steering wheeling of vehicles |
CN103998949A (en) * | 2012-01-10 | 2014-08-20 | 索弗特凯耐提克传感器股份有限公司 | Improvements in or relating to the processing of time-of-flight signals |
CN104755308A (en) * | 2013-06-29 | 2015-07-01 | 奥迪股份公司 | Motor vehicle control interface with gesture recognition |
CN105730496A (en) * | 2014-12-31 | 2016-07-06 | 哈曼国际工业有限公司 | Force measuring steering wheel |
CN108038453A (en) * | 2017-12-15 | 2018-05-15 | 罗派智能控制技术(上海)有限公司 | A kind of driver's state-detection and identifying system based on RGBD |
CN208411849U (en) * | 2018-05-23 | 2019-01-22 | 常州星宇车灯股份有限公司 | A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8775023B2 (en) * | 2009-02-15 | 2014-07-08 | Neanode Inc. | Light-based touch controls on a steering wheel and dashboard |
DE102013207664A1 (en) * | 2013-04-26 | 2014-10-30 | Zumtobel Lighting Gmbh | Arrangement and method for recognizing gestures |
CN103513491B (en) * | 2013-09-25 | 2016-04-20 | 南京大学 | A kind of method and apparatus producing optical frequency com based on Parabolic Phase modulation |
CN104122995A (en) * | 2014-05-28 | 2014-10-29 | 重庆大学 | Method for simulating car race game steering wheel by utilizing visual information |
DE102014116292A1 (en) * | 2014-11-07 | 2016-05-12 | Visteon Global Technologies, Inc. | System for transmitting information in a motor vehicle |
DE102015015248A1 (en) * | 2015-11-18 | 2017-05-18 | Elmos Semiconductor Aktiengesellschaft | Simple gesture recognition device |
CN105718065B (en) * | 2016-01-27 | 2018-08-21 | 北京交通大学 | Interactive inspection analogue system under bus or train route cooperative surroundings |
CN206921118U (en) * | 2017-03-09 | 2018-01-23 | 深圳奥比中光科技有限公司 | Double-wavelength images acquisition system |
-
2018
- 2018-05-23 CN CN201810497789.4A patent/CN108622176B/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102016636A (en) * | 2009-03-05 | 2011-04-13 | 松下电器产业株式会社 | Distance measuring device, distance measuring method, program and integrated circuit |
CN103998949A (en) * | 2012-01-10 | 2014-08-20 | 索弗特凯耐提克传感器股份有限公司 | Improvements in or relating to the processing of time-of-flight signals |
KR101328441B1 (en) * | 2012-06-07 | 2013-11-20 | 자동차부품연구원 | Interface device of steering wheeling of vehicles |
CN104755308A (en) * | 2013-06-29 | 2015-07-01 | 奥迪股份公司 | Motor vehicle control interface with gesture recognition |
CN105730496A (en) * | 2014-12-31 | 2016-07-06 | 哈曼国际工业有限公司 | Force measuring steering wheel |
CN108038453A (en) * | 2017-12-15 | 2018-05-15 | 罗派智能控制技术(上海)有限公司 | A kind of driver's state-detection and identifying system based on RGBD |
CN208411849U (en) * | 2018-05-23 | 2019-01-22 | 常州星宇车灯股份有限公司 | A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification |
Also Published As
Publication number | Publication date |
---|---|
CN108622176A (en) | 2018-10-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111257866B (en) | Target detection method, device and system for linkage of vehicle-mounted camera and vehicle-mounted radar | |
CN108622176B (en) | Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition | |
EP3049285B1 (en) | Driver assistance system for displaying surroundings of a vehicle | |
CN103852067B (en) | The method for adjusting the operating parameter of flight time (TOF) measuring system | |
CN105807912B (en) | Vehicle, method for controlling the same, and gesture recognition apparatus therein | |
CN103852754B (en) | The method of AF panel in flight time (TOF) measuring system | |
KR101143176B1 (en) | Method and Apparatus for Recognizing Parking Slot Marking by Using Bird's Eye View and Parking Assist System Using Same | |
CN107749194B (en) | Lane changing assisting method and mobile terminal | |
JP2016506572A (en) | Infotainment system | |
KR20050099623A (en) | Device for a motor vehicle used for the three-dimensional detection of a scene inside or outside said motor vehicle | |
JPH10208047A (en) | On-vehicle traveling environment recognizing device | |
CN109840014A (en) | Virtual touch identification device and method for correcting its identification error | |
JP7517335B2 (en) | Signal processing device, signal processing method, and ranging module | |
KR101808714B1 (en) | Vehicle Center Fascia Control Method Based On Gesture Recognition By Depth Information And Virtual Touch Sensor | |
JP3961584B2 (en) | Lane marking detector | |
CN109963144A (en) | A kind of vehicle-mounted identifying system based on AR-HUD | |
CN112889272B (en) | Depth image acquisition method, depth image acquisition device and electronic device | |
WO2021065494A1 (en) | Distance measurement sensor, signal processing method, and distance measurement module | |
CN208411849U (en) | A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification | |
CN114868148A (en) | Information processing device, information processing system, information processing program, and information processing method | |
KR101809916B1 (en) | Vehicle, controlling method thereof and gesture recognition apparatus therein | |
CN105323539A (en) | Automotive safety system and operating method thereof | |
CN212992427U (en) | Image acquisition module | |
WO2021065500A1 (en) | Distance measurement sensor, signal processing method, and distance measurement module | |
JP2002314990A (en) | System for visually confirming periphery of vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant |