[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN108622176B - Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition - Google Patents

Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition Download PDF

Info

Publication number
CN108622176B
CN108622176B CN201810497789.4A CN201810497789A CN108622176B CN 108622176 B CN108622176 B CN 108622176B CN 201810497789 A CN201810497789 A CN 201810497789A CN 108622176 B CN108622176 B CN 108622176B
Authority
CN
China
Prior art keywords
unit
steering wheel
near infrared
light source
gesture recognition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810497789.4A
Other languages
Chinese (zh)
Other versions
CN108622176A (en
Inventor
刘正全
邓亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Xingyu Automotive Lighting Systems Co Ltd
Original Assignee
Changzhou Xingyu Automotive Lighting Systems Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Xingyu Automotive Lighting Systems Co Ltd filed Critical Changzhou Xingyu Automotive Lighting Systems Co Ltd
Priority to CN201810497789.4A priority Critical patent/CN108622176B/en
Publication of CN108622176A publication Critical patent/CN108622176A/en
Application granted granted Critical
Publication of CN108622176B publication Critical patent/CN108622176B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D1/00Steering controls, i.e. means for initiating a change of direction of the vehicle
    • B62D1/02Steering controls, i.e. means for initiating a change of direction of the vehicle vehicle-mounted
    • B62D1/04Hand wheels
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D1/00Steering controls, i.e. means for initiating a change of direction of the vehicle
    • B62D1/24Steering controls, i.e. means for initiating a change of direction of the vehicle not vehicle-mounted
    • B62D1/28Steering controls, i.e. means for initiating a change of direction of the vehicle not vehicle-mounted non-mechanical, e.g. following a line or other known markers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures

Landscapes

  • Engineering & Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Measurement Of Optical Distance (AREA)

Abstract

The invention discloses a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, and relates to the field of human-vehicle interaction interfaces. The system comprises a near infrared illumination light source unit, a TOF imaging array sensor unit, a lens module unit, a TOP control unit, a three-dimensional point cloud image processing unit and an automobile steering wheel, wherein the near infrared illumination light source unit is used for illuminating a gesture recognition area; the TOP control unit modulates the near infrared illumination light source unit, simultaneously controls the TOF imaging array sensor unit to measure the phase shift between gesture illumination and reflection and convert the phase shift into distance depth information for gesture recognition, and the TOF imaging array sensor unit completes analysis through the three-dimensional point cloud image processing unit and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN/LIN bus. The method effectively improves the flexibility of application of gesture operation in the vehicle, and can perform remote and non-contact control, so that the human-vehicle interaction mode is more natural and convenient.

Description

Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition
Technical Field
The invention relates to the field of human-vehicle interaction interfaces, in particular to a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition.
Background
At present, the man-vehicle interaction mode on the multifunctional steering wheel is mainly based on mechanical keys, a deflector rod and a knob. These input devices have certain disadvantages, and the input habits and the natural communication habits of the person themselves are different.
Disclosure of Invention
In order to solve the defects of the traditional mechanical keys, the deflector rods and the knobs on the multifunctional steering wheel, the invention provides the multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, the flexibility of application of gesture operation in the automobile is effectively improved by adopting a non-contact gesture mode, and the related functions of the automobile body can be controlled remotely and in a non-contact mode, so that the human-vehicle interaction mode is more natural and more convenient, and the development trend of human-vehicle interaction of the intelligent automobile in the future is met.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition comprises a near infrared illumination light source unit, a TOF imaging array sensor unit, a TOP control unit, a three-dimensional point cloud image processing unit and an automobile steering wheel, wherein the near infrared illumination light source unit and the TOF imaging array sensor unit are respectively arranged on two sides of the automobile steering wheel; the near infrared illumination light source unit illuminates the gesture recognition area; the TOP control unit modulates the near infrared illumination light source unit, simultaneously controls the TOF imaging array sensor unit to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit, and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN bus or the LIN bus.
Further, after the near infrared light emitted by the near infrared illumination light source unit is reflected by the gesture recognition, the reflected near infrared light is received by the TOF imaging array sensor unit.
Further, the near-infrared illumination light source unit includes a plurality of illumination monomers; the TOF imaging array sensor unit simultaneously measures a plurality of illumination monomers at a time, the interval between each illumination monomer is corresponding to the phase of the illumination monomer number, and the phase difference between illumination and reflection is utilized to calculate the distance depth information of gesture recognition.
Further, the TOP control unit adopts a multi-frequency technology to modulate the near infrared illumination light source unit, so that the problems of measurement accuracy and measurement distance are solved, namely, one or more modulation frequencies are added to work, and the optimal detection accuracy is obtained under the condition of enough measurement distance. Each modulation frequency will calculate a corresponding measured distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies.
Further, the TOP control unit modulates the near-infrared illumination light source unit, and simultaneously controls the TOF imaging array sensor unit to receive the reflected near-infrared light to generate a three-dimensional point cloud image, so that distance and depth information of each pixel point can be provided and displayed in a gray form, and the gesture approaches the TOF imaging array sensor unit more when the gray value is higher.
Further, the three-dimensional point cloud image processing unit separates the gesture part from other backgrounds of the image according to distance and depth information (as an important characteristic of gesture recognition) obtained by the TOF imaging array sensor unit.
Further, the multifunctional steering wheel human-vehicle interaction system further comprises a lens module unit, and the lens module unit adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit, so that the gesture recognition area range meets the requirements of actual application scenes.
The beneficial effects are that:
1. According to the invention, traditional mechanical keys, a deflector rod and a knob on the multifunctional steering wheel are replaced by a gesture control mode, so that light function control, audio function control, video and audio function control, wiper function control, instrument panel function control, constant-speed cruising and telephone function control are realized.
2. The invention adopts the video input mode to control the vehicle body in a long distance and non-contact way, and realizes the gesture control of the related functions of the vehicle body; compared with the traditional mode, the gesture recognition can achieve higher accuracy in weak light and strong light environments, so that interaction between people and vehicles is more natural and convenient.
3. The TOF gesture recognition system is high in integration level and compact in structure, the calculation intensity brought by the recognition algorithm can be realized in an embedded system, meanwhile, the requirements of detection distance and measurement accuracy are met in the in-vehicle environment, the response time is the fastest in a vision system, and the system accords with the development trend of human-vehicle interaction of the intelligent automobile in the future.
Drawings
FIG. 1 is a front view of a multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition in accordance with an embodiment of the present invention;
FIG. 2 is a left side view of a multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a lens module unit according to an embodiment of the invention;
FIG. 4 is a waveform diagram of phase difference measurement between illumination and reflection in accordance with an embodiment of the present invention;
FIG. 5 is a three-dimensional point cloud of an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating segmentation of a gesture portion from other background of an image according to an embodiment of the present invention;
In the figure: the device comprises a 1-infrared illumination light source unit, a 2-TOF imaging array sensor unit, a 3-lens module unit, a 4-TOP control unit, a 5-three-dimensional point cloud image processing unit and a 6-automobile steering wheel.
Detailed Description
The invention will be further described with reference to the drawings and examples.
The embodiment provides a multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition, which is shown in fig. 1-2, and comprises a near infrared illumination light source unit 1, a TOF imaging array sensor unit 2, a lens module unit 3, a TOP control unit 4, a three-dimensional point cloud image processing unit 5 and an automobile steering wheel 6, wherein the near infrared illumination light source unit 1 and the TOF imaging array sensor unit 2 are respectively arranged at two sides of the automobile steering wheel 6; the near infrared illumination light source unit 1 illuminates the gesture recognition area; the lens module unit 3 (figure 3) adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit 2 so that the gesture recognition area range meets the requirement of the actual application scene; the TOP control unit 4 modulates the near infrared illumination light source unit 1, simultaneously controls the TOF imaging array sensor unit 2 to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit 5, and outputs a corresponding steering wheel multifunctional control signal to the vehicle body controller through the CAN bus or the LIN bus.
In the present embodiment, near infrared light emitted from a near infrared illumination light source unit 1 (model: lumentum 350mW,854nm TO-56 Packaged Laser Diode 22045504) is reflected by a recognition gesture, and the reflected near infrared light is received by a TOF imaging array sensor unit 2 (model: 320× 240 Array OPT8241).
In the present embodiment, the near-infrared illumination light source unit 1 includes a plurality of illumination monomers; the TOF imaging array sensor unit 2 simultaneously measures a plurality of illumination monomers at a time, each of the illumination monomers is spaced by a phase corresponding to the number of the illumination monomers, and the distance depth information of gesture recognition is calculated by using the phase difference between illumination and reflection. Specifically: as shown in fig. 4, the distance depth d of gesture recognition (formula (2) below) is calculated using the phase angle Φ (formula (1) below) between illumination and reflection.
Wherein c is the speed of light; q1, Q2, Q3, and Q4 are the amounts of electric charges sampled by the TOF imaging array sensor unit 2.
In the present embodiment, the TOP control unit 4 modulates the near-infrared illumination light source unit 1 by using the multi-frequency technology, which gives consideration to both measurement accuracy and measurement distance problems, that is, one or more modulation frequencies are added to operate so that the optimal detection accuracy is obtained under the condition of sufficient measurement distance. Each modulation frequency will calculate a corresponding measured distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies. Specifically: the distance measurement variance of the continuous square wave light source modulation detection gesture recognition can be expressed as the following formula (3):
where c d is the modulation contrast, the ability of the TOF imaging array sensor unit 2 to separate and collect photoelectrons is described.
Wherein the reflected light intensity L is a function of the light power, and can be expressed as the following formula (4):
wherein the offset S is a function of the ambient light and the system offset, and can be expressed as the following formula (5):
from equation (3), it can be deduced that the reflected light brightness, the modulation frequency and the modulation contrast are three factors affecting the distance measurement accuracy.
In this embodiment, the TOP control unit 4 (model: 3D ToF Controller OPT9221) modulates the near-infrared illumination light source unit 1, and at the same time controls the TOF imaging array sensor unit 2 to receive the reflected near-infrared light, so as to generate a three-dimensional point cloud image, and may provide depth information of each pixel point, as shown in fig. 5, and perform display in a gray scale form, where the higher the gray scale value, the closer the gesture is to the TOF imaging array sensor unit 2.
In the present embodiment, the three-dimensional point cloud image processing unit 5 (model: high-performance ARMAM437 x) separates the gesture sites from other backgrounds of the image according to the distance depth information (as an important feature of gesture recognition) obtained by the TOF imaging array sensor unit 2, as shown in fig. 6.
It should be apparent to those skilled in the art that various modifications or variations can be made in the present invention without requiring any inventive effort by those skilled in the art based on the technical solutions of the present invention.

Claims (3)

1. A multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition is characterized in that: the multifunctional steering wheel human-vehicle interaction system comprises a near infrared illumination light source unit (1), a TOF imaging array sensor unit (2), a TOP control unit (4), a three-dimensional point cloud image processing unit (5) and an automobile steering wheel (6), wherein the near infrared illumination light source unit (1) and the TOF imaging array sensor unit (2) are respectively arranged on two sides of the automobile steering wheel (6); the near infrared illumination light source unit (1) illuminates a gesture recognition area; the TOP control unit (4) modulates the near infrared illumination light source unit (1), simultaneously controls the TOF imaging array sensor unit (2) to measure the phase shift between gesture illumination and reflection, converts the phase shift into distance depth information for gesture recognition, finishes analysis through the three-dimensional point cloud image processing unit (5), and outputs a corresponding steering wheel multifunctional control signal to a vehicle body controller through a CAN bus or a LIN bus;
The near-infrared illumination light source unit (1) includes a plurality of illumination monomers; the TOF imaging array sensor unit (2) simultaneously measures a plurality of illumination monomers at one time, the interval between each illumination monomer is corresponding to the phase of the illumination monomer number, and the distance depth information of gesture recognition is calculated by utilizing the phase difference between illumination and reflection;
The TOP control unit (4) modulates the near infrared illumination light source unit (1) by adopting a multi-frequency technology; the multi-frequency technology works by adding one or more modulation frequencies, each modulation frequency is calculated to obtain a corresponding measurement distance, and the actual gesture depth position is determined by a plurality of distances measured by the modulation frequencies;
the multifunctional steering wheel human-vehicle interaction system further comprises a lens module unit (3), wherein the lens module unit (3) adjusts the focal length of the gesture image picked up by the TOF imaging array sensor unit (2);
The three-dimensional point cloud image processing unit (5) separates gesture parts from other backgrounds of the images according to the distance and depth information obtained by the TOF imaging array sensor unit (2).
2. The multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition of claim 1, wherein: after the near infrared light emitted by the near infrared illumination light source unit (1) is reflected by the gesture recognition, the reflected near infrared light is received by the TOF imaging array sensor unit (2).
3. The multi-functional steering wheel human-vehicle interaction system based on TOF gesture recognition of claim 1, wherein: the TOP control unit (4) modulates the near infrared illumination light source unit (1) and simultaneously controls the TOF imaging array sensor unit (2) to receive the reflected near infrared light so as to generate a three-dimensional point cloud image.
CN201810497789.4A 2018-05-23 2018-05-23 Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition Active CN108622176B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810497789.4A CN108622176B (en) 2018-05-23 2018-05-23 Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810497789.4A CN108622176B (en) 2018-05-23 2018-05-23 Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition

Publications (2)

Publication Number Publication Date
CN108622176A CN108622176A (en) 2018-10-09
CN108622176B true CN108622176B (en) 2024-10-29

Family

ID=63694081

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810497789.4A Active CN108622176B (en) 2018-05-23 2018-05-23 Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition

Country Status (1)

Country Link
CN (1) CN108622176B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110822719A (en) * 2019-10-08 2020-02-21 广东万博电气有限公司 Gesture-controlled electric water heater
CN111679439B (en) * 2020-08-11 2020-12-18 上海鲲游光电科技有限公司 Optical field modulator and modulation method thereof
CN116009275B (en) * 2022-12-30 2024-07-19 福州大学 Naked eye 3D display device capable of recognizing dynamic gestures

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102016636A (en) * 2009-03-05 2011-04-13 松下电器产业株式会社 Distance measuring device, distance measuring method, program and integrated circuit
KR101328441B1 (en) * 2012-06-07 2013-11-20 자동차부품연구원 Interface device of steering wheeling of vehicles
CN103998949A (en) * 2012-01-10 2014-08-20 索弗特凯耐提克传感器股份有限公司 Improvements in or relating to the processing of time-of-flight signals
CN104755308A (en) * 2013-06-29 2015-07-01 奥迪股份公司 Motor vehicle control interface with gesture recognition
CN105730496A (en) * 2014-12-31 2016-07-06 哈曼国际工业有限公司 Force measuring steering wheel
CN108038453A (en) * 2017-12-15 2018-05-15 罗派智能控制技术(上海)有限公司 A kind of driver's state-detection and identifying system based on RGBD
CN208411849U (en) * 2018-05-23 2019-01-22 常州星宇车灯股份有限公司 A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8775023B2 (en) * 2009-02-15 2014-07-08 Neanode Inc. Light-based touch controls on a steering wheel and dashboard
DE102013207664A1 (en) * 2013-04-26 2014-10-30 Zumtobel Lighting Gmbh Arrangement and method for recognizing gestures
CN103513491B (en) * 2013-09-25 2016-04-20 南京大学 A kind of method and apparatus producing optical frequency com based on Parabolic Phase modulation
CN104122995A (en) * 2014-05-28 2014-10-29 重庆大学 Method for simulating car race game steering wheel by utilizing visual information
DE102014116292A1 (en) * 2014-11-07 2016-05-12 Visteon Global Technologies, Inc. System for transmitting information in a motor vehicle
DE102015015248A1 (en) * 2015-11-18 2017-05-18 Elmos Semiconductor Aktiengesellschaft Simple gesture recognition device
CN105718065B (en) * 2016-01-27 2018-08-21 北京交通大学 Interactive inspection analogue system under bus or train route cooperative surroundings
CN206921118U (en) * 2017-03-09 2018-01-23 深圳奥比中光科技有限公司 Double-wavelength images acquisition system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102016636A (en) * 2009-03-05 2011-04-13 松下电器产业株式会社 Distance measuring device, distance measuring method, program and integrated circuit
CN103998949A (en) * 2012-01-10 2014-08-20 索弗特凯耐提克传感器股份有限公司 Improvements in or relating to the processing of time-of-flight signals
KR101328441B1 (en) * 2012-06-07 2013-11-20 자동차부품연구원 Interface device of steering wheeling of vehicles
CN104755308A (en) * 2013-06-29 2015-07-01 奥迪股份公司 Motor vehicle control interface with gesture recognition
CN105730496A (en) * 2014-12-31 2016-07-06 哈曼国际工业有限公司 Force measuring steering wheel
CN108038453A (en) * 2017-12-15 2018-05-15 罗派智能控制技术(上海)有限公司 A kind of driver's state-detection and identifying system based on RGBD
CN208411849U (en) * 2018-05-23 2019-01-22 常州星宇车灯股份有限公司 A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification

Also Published As

Publication number Publication date
CN108622176A (en) 2018-10-09

Similar Documents

Publication Publication Date Title
CN111257866B (en) Target detection method, device and system for linkage of vehicle-mounted camera and vehicle-mounted radar
CN108622176B (en) Multifunctional steering wheel human-vehicle interaction system based on TOF gesture recognition
EP3049285B1 (en) Driver assistance system for displaying surroundings of a vehicle
CN103852067B (en) The method for adjusting the operating parameter of flight time (TOF) measuring system
CN105807912B (en) Vehicle, method for controlling the same, and gesture recognition apparatus therein
CN103852754B (en) The method of AF panel in flight time (TOF) measuring system
KR101143176B1 (en) Method and Apparatus for Recognizing Parking Slot Marking by Using Bird's Eye View and Parking Assist System Using Same
CN107749194B (en) Lane changing assisting method and mobile terminal
JP2016506572A (en) Infotainment system
KR20050099623A (en) Device for a motor vehicle used for the three-dimensional detection of a scene inside or outside said motor vehicle
JPH10208047A (en) On-vehicle traveling environment recognizing device
CN109840014A (en) Virtual touch identification device and method for correcting its identification error
JP7517335B2 (en) Signal processing device, signal processing method, and ranging module
KR101808714B1 (en) Vehicle Center Fascia Control Method Based On Gesture Recognition By Depth Information And Virtual Touch Sensor
JP3961584B2 (en) Lane marking detector
CN109963144A (en) A kind of vehicle-mounted identifying system based on AR-HUD
CN112889272B (en) Depth image acquisition method, depth image acquisition device and electronic device
WO2021065494A1 (en) Distance measurement sensor, signal processing method, and distance measurement module
CN208411849U (en) A kind of multi-functional steering wheel people-car interaction system based on TOF gesture identification
CN114868148A (en) Information processing device, information processing system, information processing program, and information processing method
KR101809916B1 (en) Vehicle, controlling method thereof and gesture recognition apparatus therein
CN105323539A (en) Automotive safety system and operating method thereof
CN212992427U (en) Image acquisition module
WO2021065500A1 (en) Distance measurement sensor, signal processing method, and distance measurement module
JP2002314990A (en) System for visually confirming periphery of vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant