CN106926800B - The vehicle-mounted visual perception system of multi-cam adaptation - Google Patents
The vehicle-mounted visual perception system of multi-cam adaptation Download PDFInfo
- Publication number
- CN106926800B CN106926800B CN201710193302.9A CN201710193302A CN106926800B CN 106926800 B CN106926800 B CN 106926800B CN 201710193302 A CN201710193302 A CN 201710193302A CN 106926800 B CN106926800 B CN 106926800B
- Authority
- CN
- China
- Prior art keywords
- camera
- vehicle
- unit
- data
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
- B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Automation & Control Theory (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of vehicle-mounted visual perception systems of multi-cam adaptation, including vehicle-mounted camera signal transmission unit, DSP processing unit, camera types judging unit, vision data storage unit and image information pretreatment unit；The adaptation of a variety of camera types including infrared camera, monocular cam, monocular wide-angle camera, binocular solid camera, binocular image splicing camera, more lens image spliced panoramic cameras may be implemented in the present invention, and establish a kind of universal vehicle-mounted visual processes scheme, various compatible vision prescription is provided for car steering auxiliary system, can equally reduce time cost and economic cost that a variety of vehicle-mounted visions are carried.
Description
Technical field
The present invention relates in vehicle-mounted visual perception technical field, in particular to a kind of vehicle-mounted visual impression of multi-cam adaptation
Know system.
Background technique
In recent years, increasingly mature with car steering ancillary technique, various automobile miscellaneous functions are answered more and more
With on volume production automobile.Car steering ancillary technique is the necessary technology that automobile develops from " mechanization " to " intelligence "
Stage；It can provide safety guarantee for driver's driving behavior, while improve comfort, safety, the fuel oil of vehicle driving
Economy.In driving ancillary technique and unmanned technology, environment sensing is its important core component.Environment sensing
Technology refers to vehicle by including the coherent signal of the sensors such as camera, ultrasonic radar, millimetre-wave radar, laser radar to week
Collarette border is perceived, and provides important evidence for the control decision of vehicle.Wherein, compared to other environmental sensors, camera
Environmental information more abundant, and cheap, reliable performance can be provided, are often considered as unmanned environment sensing skill
One irreplaceable direction of art.
Vehicle-mounted vision system is collectively referred to as by a kind of context aware systems that sensor forms of camera.Vehicle-mounted vision system
Function have very much, including driving recording, anti-collision early warning, ACC, lane detection etc..Existing vehicle-mounted vision system only provides list
One function.If driver it is expected to realize a variety of drive assistance functions, generally require that more set different function are installed on vehicle
The vehicle-mounted vision system of type inevitably causes a large amount of wastes of hardware and software economic cost.The vehicle-mounted view of multiple types simultaneously
The compatibling problem and data redundancy issue of feel system, also the strong influence stability of vehicle-mounted vision system, safety and
Economy.
In existing vehicle-mounted visual perception system, camera broad categories, including infrared/pure visible image capturing head, fixed-focus/change
Burnt camera, common/wide-angle camera, monocular/binocular camera, plane/stereo camera etc..These cameras respectively have respectively
Emphasize advantage, but have certain similarity again in terms of function treatment.Existing vehicle-mounted vision system is not to multiple types
Camera is adapted to, to be suitable for a variety of different types of cameras.
In conclusion existing vehicle-mounted visual perception system is in multi-class camera adaptation and multimode function integrated approach
Aspect has defect, needs to improve.
Summary of the invention
In view of this, the purpose of the present invention is to provide a kind of vehicle-mounted visual perception systems of multi-cam adaptation, to drive
It sails ancillary technique and unmanned technology and more stable, safer, the more integrated vehicle-mounted vision technique of function is provided.
The vehicle-mounted visual perception system of multi-cam adaptation of the invention, comprising:
Vehicle-mounted camera signal transmission unit, signal conversion and signal transmission for vehicle-mounted hardware sensitive component；
DSP processing unit is connected with vehicle-mounted camera signal transmission unit, the signal for vehicle-mounted camera digital signal
Filtering and A/D conversion；
Camera types judging unit is connected with camera types judging unit, for identification the hardware of vehicle-mounted camera
Type simultaneously provides camera types information for system function；
Vision data storage unit is connected with DSP processing unit, for storing vehicle-mounted camera data as driving note
Record, the data-interface that storage image preprocessing result data is merged as sensor；
Image information pretreatment unit is connected with camera types judging unit and vision data storage unit, and being used for will
Camera data are pre-processed according to camera types, to image carry out defogging-optimization processing, Shannon entropy detection, gray processing,
Binaryzation and image segmentation based on edge detection.
Further, which further includes image processing unit, described image processing unit and image information pretreatment unit
It is connected；Described image processing unit includes:
Road surface estimation unit estimates vehicle running surface for the image data after being optimized according to defogging, is classified；
Visual odometry unit, for carrying out phase to vehicle driving according to defogging optimization processing data/greyscale image data
Pose is resolved；
Box counting algorithm unit, for carrying out visual signature resolving to defogging optimization processing data, greyscale image data,
Including feature detection, feature description, calculating feature is respectively Haar, HOG, FAST, ORB, BRIEF or LBP, the practical solution of feature
Type is calculated to be specified by camera process demand；
Vision map constructing unit is merged for establishing visual signature map office for BRIEF feature with visual odometry
For visual environment SLAM, processing is optimized to vision map, carries out real-time winding detection；
Image identification unit, for carrying out vision to vehicle, pedestrian, traffic mark, lane line according to known-image-features
Identification；
Picture charge pattern unit, for position to carry out spy in different frame picture to vehicle, pedestrian, traffic mark, lane line
Sign matching calculates, estimates state, position at the time of vehicle, pedestrian, traffic mark, lane line are under world coordinate system；And
Visual signature library storage element, the visual signature data calculated for storing box counting algorithm unit, storage
Mode is the feature database storing mode according to 3D structural remodeling under timing, provides calculating data-interface for vision map constructing unit.
Further, which further includes System Back-end, and the System Back-end is connected with image processing unit；After the system
End includes:
Structured road construction unit, for utilizing road surface estimation unit, vision map constructing unit, image identification unit
Calculation result building structure road model, Optimized model result；
Running environment detection unit, for according to road surface estimation unit, vision map constructing unit, image identification unit,
Picture charge pattern cell data calculation result, detection is penetrated in range L rice from rung travels environmental change；And
System data interaction interface unit be used for for it is man-machine drive altogether, human-computer interaction, vehicle control decision location data extract,
Vehicle control policy setting data are extracted and Data Fusion of Sensor decision level data are extracted and provide data-interface.
Further, the L is 100-200.
Further, the camera types judging unit includes:
Vehicle-mounted visual signal input module judges data-interface for providing camera types, by DSP processing unit processes
Consequential signal, infrared generator signal input vehicle-mounted camera type judging unit；
Camera types judgment module, for judging camera types comprising infrared/pure visible image capturing head judges mould
Block, camera quantity detection module, camera wide-angle detection module and camera view detection module；And
Camera types message output module, for exporting camera judging result.
Further, the system further include:
Vehicle-mounted visual information input unit, is located between camera types judging unit and image information pretreatment unit,
It is used for transmission camera types information and camera installation site information data；And
Vehicle-mounted visual signal input unit, is located between camera types judging unit and image information pretreatment unit,
It is used for transmission original camera data.
Beneficial effects of the present invention: the vehicle-mounted visual perception system of multi-cam adaptation of the invention can be adapted to a variety of
Camera types complete the multiple module function of vehicle-mounted vision；In this process, engineering staff only needs few artificial work
Industry and manual intervention save a large amount of manpower, time cost；The present invention may be implemented to include that infrared camera, monocular are taken the photograph
As head, monocular wide-angle camera, binocular solid camera, binocular image splicing camera, the camera shooting of more lens image spliced panoramics
The adaptation of a variety of camera types including head, and a kind of universal vehicle-mounted visual processes scheme is established, it is car steering
Auxiliary system provides multiplicity compatible vision prescription, can equally reduce time cost and economy that a variety of vehicle-mounted visions are carried
Cost.
Detailed description of the invention
The invention will be further described with reference to the accompanying drawings and examples:
Fig. 1 is structural block diagram of the invention；
Fig. 2 is the structural block diagram of the camera types judging unit of invention；
Fig. 3 is the structural block diagram of image information pretreatment unit and image information pretreatment unit of the invention；
Fig. 4 is that single vehicle mounted infrared camera image handles thread flow figure；
Fig. 5 is single vehicle-mounted pure visible image capturing head image procossing thread flow figure；
Fig. 6 is multi-cam panoramic picture differentiation process flow diagram；
Fig. 7 is binocular stereo vision image difference alienation process flow diagram；
Fig. 8 is monocular wide-angle camera image difference alienation process flow diagram.
Specific embodiment
This vehicle-mounted visual perception system includes: that vehicle-mounted vision hardware/software system front end, vehicle-mounted vision camera type are sentenced
Disconnected unit, image pre-processing unit, image processing unit, vehicle-mounted vision system rear end.
Wherein, vehicle-mounted vision hardware/software system front end includes vehicle-mounted camera hardware installation arrangement, replaces, vehicle-mounted to take the photograph
As the conversion of head sensitive component signal, A/D conversion, Digital Signal Processing DSP, camera infrared emittance working condition；It is vehicle-mounted
Vision camera type judging unit includes camera infrared detection, the judgement of camera number, the detection of camera wide-angle, camera
Visual field detection；Image pre-processing unit include the calibration of camera internal reference number, image defogging, the detection of image Shannon entropy, image segmentation,
Picture signal gray processing, image binaryzation；Image processing unit includes road surface estimation, feature detection, visual odometry, visually
Figure, pedestrian's identification, pedestrian's tracking, vehicle identification, car tracing, Lane detection, traffic mark identification；After vehicle-mounted vision system
End include vehicle location, structured road building, driving environment state-detection, vehicle-mounted vision system rear end be finally driver/
Pilotless automobile provides driving behavior security warning, control decision foundation, data fusion visual signature, data, decision interface.
The actual functional capability of the vehicle-mounted vision system is to judge camera types；According to camera types, to the phase of camera
OFF signal is handled；And the data that final output can directly be used by vehicle-mounted control decision package, or to driver's row
It sails behavior and carries out security warning；Data, feature, decision interface are provided for information fusion；Simultaneously, to camera treatment process
The related important results and camera initial data of middle generation are stored.
Pass through vehicle-mounted visual perception system:
The first, judge vehicle-mounted vision camera type, covering camera types has: 1) the pure visible light of infrared camera-is taken the photograph
As head；2) monocular vision camera-multi-vision visual camera；3) monocular wide-angle camera-monocular common camera；4) binocular is vertical
Body camera-binocular image splices-two monocular common cameras of camera；5) more mesh full-view cameras-multiple monoculars are common
Camera.
The second, according to vehicle-mounted vision camera type and image data, image preprocessing is completed.It is gone including image
Mist-optimization processing, Shannon entropy detection, gray processing, binaryzation, the image segmentation based on edge detection.
Third, according to vehicle-mounted vision camera type, image data and image preprocessing calculation result, complete at image
Reason.Including road surface estimation, feature detection, visual odometry, vision map, pedestrian's identification, pedestrian's tracking, vehicle identification, vehicle
Tracking, Lane detection, traffic mark identification.
4th, according to vehicle-mounted vision camera type, image data and image procossing calculation result, vehicle-mounted vision is completed
System back-end function is realized, including vehicle location, structured road building, driving environment state-detection.
5th, this vehicle-mounted vision system finally provides driving behavior security warning, control for driver/pilotless automobile
Decision-making foundation, data fusion visual signature, data, decision interface.
Referring to illustrate attached drawing and embodiment to the present invention at large illustrated:
Fig. 1 is structural block diagram of the invention, lists system relevant components unit in detail, comprising: vehicle-mounted camera signal passes
Defeated unit 111, DSP processing unit 112, camera types judging unit 113, vision data storage unit 114, image information are pre-
Processing unit 115, road surface estimation unit 116, visual odometry unit 120, box counting algorithm unit 121, vision map structure
Build unit 122, image identification unit 123, picture charge pattern unit 124, visual signature library storage element 125, structured road structure
Build unit 130, running environment detection unit 131, system data interaction interface unit 132.
Wherein, vehicle-mounted camera signal transmission unit 111 is passed for the conversion of vehicle-mounted hardware sensitive component signal, signal
It is defeated.DSP processing unit 112 includes signal filtering, A/D conversion for completing vehicle-mounted camera Digital Signal Processing.Camera class
Type judging unit 113 is responsible for the type of hardware of the vehicle-mounted a variety of camera adaptation module systems of vision of identification, is vehicle module
System function provides camera types information.Vision data storage unit 114 is for storing vehicle-mounted camera data as driving
Record, the data-interface that storage image preprocessing result data is merged as sensor, storing data includes: original camera number
According to, camera defogging-optimization data, camera gradation data, camera binaryzation data, camera Shannon entropy monitoring data.
Image information pretreatment unit 115 carries out defogging-for pre-processing camera data according to camera types, to image
Optimization processing, Shannon entropy detection, gray processing, binaryzation, the image segmentation based on edge detection.Road surface estimation unit 116 is used for
Image data after being optimized according to defogging estimates vehicle running surface, is classified, and major function includes；Coefficient of road adhesion
Estimation, road surface variation estimation, structuring/unstructured road classification, the classification of road material major class.Visual odometry unit 120
For carrying out relative pose resolving to vehicle driving according to defogging optimization processing data/greyscale image data.Box counting algorithm
Unit 121 is used to carry out defogging optimization processing data, greyscale image data visual signature resolving, and major function includes feature inspection
It surveys, feature description, calculating feature is respectively Haar, HOG, FAST, ORB, BRIEF, LBP, and the practical resolving type of feature is by imaging
Head process demand is specified.Vision map constructing unit 122 is used to establish visual signature map office for BRIEF feature, with vision
Odometer is fused to visual environment SLAM, optimizes processing to vision map, carries out real-time winding detection.Image identification unit
123 for carrying out visual identity to vehicle, pedestrian, traffic mark, lane line according to known-image-features.Picture charge pattern unit
124 for vehicle, pedestrian, traffic mark, lane line, position to carry out characteristic matching, calculating, estimation vehicle in different frame picture
, pedestrian, traffic mark, lane line under world coordinate system at the time of state, position.Visual signature library storage element 125 is used
In the visual signature data that storage box counting algorithm unit 121 calculates, storage mode is according to 3D structural remodeling under timing
Feature database storing mode is the calculating front end of vision map constructing unit 122, provides calculating data-interface for it.Structuring road
Road construction unit 130 is used to resolve using road surface estimation unit 116, vision map constructing unit 122, image identification unit 123
As a result building structure road model, Optimized model result.Running environment detection unit 131 is used for according to road surface estimation unit
116, vision map constructing unit 122, image identification unit 123,124 data calculation of picture charge pattern unit are as a result, detect from vehicle
(L can be 100-200 to L meters of radiation scope, preferably 150) interior to travel environmental change, including road surface, vehicle, pedestrian, traffic mark
Knowledge, lane line, barrier state change information；Wherein, it blocks in range and travels from camera view in two meters of vehicle radiation scope
Environmental change is estimated according to optimal estimating theory EKF method.System data interaction interface unit 132 is used to be man-machine total
It drives, human-computer interaction, vehicle control decision location data is extracted, vehicle control policy setting data are extracted, Data Fusion of Sensor
Decision level data extraction etc., provides data-interface.
Fig. 2 is the structural block diagram of the camera types judging unit of invention, comprising: vehicle-mounted visual signal input module 21,
Camera types judgment module 22 and camera types message output module 23.
Wherein, vehicle-mounted visual signal input module 21 refers to, and 112 processing result signal of DSP processing unit, infrared ray occur
Device signal inputs vehicle-mounted camera type judging unit 113, provides camera types and judges data-interface.
Camera types judgment module 22, comprising: infrared/pure visible image capturing head judgment module 221, the inspection of camera quantity
Survey module, camera wide-angle detection module 223, camera view detection module.Wherein infrared/pure visible image capturing head judges mould
Scheme is judged in block 221 are as follows: a) detects infrared generator signal, b) detection night vision/without visible light environment visual image data
Signal.Camera quantity detection module detects camera number according to camera image data.Camera number includes: singly to take the photograph
As head 222, dual camera 224, multi-cam 226.After being detected as 226 situation of dual camera 224 and multi-cam, to camera
Dual camera visual field detection module 225, multi-cam visual field detection module 227 are carried out, detection mode is image local feature
Match, topography's Shannon entropy compares, partial image pixel matching.Detection module 227 testing result in the multi-cam visual field judges
No is that panoramic mosaic camera, dual camera visual field detection module 225 detect whether as binocular solid camera.Single camera
Whether 222, which continue camera wide-angle detection module 223, exports result are as follows: 1) be wide-angle camera, 2) camera wide-angle number
Value.
Whether camera types message output module 23 exports camera judging result to this system: 1) being infrared photography
Head, 2) camera number, 3) whether be full-view camera, 4) output of full-view camera image mosaic image, 5) whether be binocular
Three-dimensional camera, 6) whether be monocular cam, 7) camera wide-angle numerical value export.
Fig. 3 is the structural block diagram of image information pretreatment unit and image information pretreatment unit of the invention, as schemed institute
Show, vehicle-mounted visual information input unit 311 includes camera types information 311a, camera installation site information 311b.Camera shooting
Head type information 311a exports 23 step results of camera types message output module.Camera installation site information 311b packet
It includes: the vehicle-mounted installation direction setting of vehicle-mounted camera mounting height, single camera, the vehicle-mounted installation direction setting of multi-camera system,
Vehicle-mounted visual information input 311 comprehensive camera types information 311a, camera installation site information 311b, judge camera vehicle
Carry position.
Vehicle-mounted visual signal input unit 312 transmits original camera data.
Image preprocessing 32 is the actual functional capability module of image information pretreatment unit 115, comprising: camera internal reference number mark
Fixed 321, image preprocessing functional module 322.Camera internal reference number is demarcated there are two types of 312 schemes: 1) being manually entered in camera
Parameter value；2) according to vehicle-mounted visual information input 311, vehicle-mounted visual signal input 312, self-calibration is carried out to camera.Image
Preprocessing function module 322 includes: image defogging-optimization 322e, image gray processing 322a, image binaryzation 322b, image perfume
Agriculture entropy detects 322c, image segmentation 322d.322 data storage of image preprocessing functional module, and provided for image processing module
Data-interface.
Vehicle-mounted visual pattern processing function module 33 includes: road surface estimation 331, characteristic processing 332, visual identity 333, view
Feel odometer 334, vision map structuring 335, visual pursuit 336.Characteristic processing 332 includes: image detection 332a, feature description
332b, calculating feature is respectively Haar, HOG, FAST, ORB, BRIEF, LBP, and calculation result is visual identity 333, vision mileage
Meter 334, vision map structuring 335, visual pursuit 336 provide resolved data and resolve feature.Visual identity 333 includes: vehicle
Identification 333a, pedestrian identify 333b, Lane detection 333c, traffic mark identify 333d.Visual pursuit 336 is chased after including vehicle
Track 336a, pedestrian track 336b, lane line tracks 336c.Wherein, road surface estimation 331, characteristic processing 332, visual identity 333,
Visual odometry 334,336 image data of visual pursuit be image defogging-optimization 322e data, image gray processing 322a data with
And image binaryzation 322b data, according to correlation provided by camera internal reference number calibration 312, vehicle-mounted visual information input 311
Data are handled, and specific process flow is as shown in Fig. 4,5,6,7,8.
Vehicle-mounted visual performance module 34 includes: that structured road constructs 341, running environment state-detection 342, vehicle location
343.Vehicle-mounted 34 resolved data of visual pattern processing function module are as follows: estimate from vehicle radiation scope three-dimensional environment coordinate system, road surface
331 data results, 333 data result of visual identity, 335 data result of vision map structuring, 334 data knot of visual odometry
Fruit, 336 data result of visual pursuit.
Vehicle-mounted vision module system includes that information merges 351, driving safety warning 352, control with external interface module 35
Decision 353.Information fusion 351 provides fused data (offer of vision data storage unit 114), fusion spy for external data fusion
(offer of characteristic processing 332), fusion decision (vehicle-mounted visual performance module 34) are provided.Driving safety warns 352, control decision 353
Driver's driving safety is warned according to vehicle-mounted 34 calculation result of visual performance module or for unmanned intelligent vehicle
It is controlled.
Fig. 4 show single vehicle mounted infrared camera image processing thread flow figure, workflow start trigger item in figure
Part is that vehicle-mounted visual information input 311 judges that vehicle-mounted vision system is infrared camera.Single infrared camera data 41 are vehicle
It carries visual signal and inputs 312 data.
This system carries out image storage 421a to infrared picture data, camera internal reference number demarcates 422b.Image storage
421a stores infrared camera initial data and binaryzation data.This system simultaneous multi-threading carries out: image data binaryzation
442e is estimated on 431b, feature calculation 441, Shannon entropy detection 432c, image segmentation 433d, road surface.Image data binaryzation 431b
Binaryzation data are for ORB characterizing part, BRIEF feature calculation part in image storage 421a and feature calculation 441.
It is image preprocessing functional module 322 that Shannon entropy, which detects 432c, image segmentation 433d, may act on panoramic picture detection, active
Visual performance expands interface.Road surface estimation 442e is directly used in structured road building 472q.441 calculation result of feature calculation is used
452g, Lane detection 453h, traffic mark identification 454i, visual odometry 455g, view are identified in vehicle identification 451f, pedestrian
Feel map structuring 456k.Vehicle identification 451f, 441 resolved data of feature calculation and single infrared camera data 41 are used for vehicle
Tracking 461m.Pedestrian identifies that 452g, 441 resolved data of feature calculation and single infrared camera data 41 are chased after for pedestrian
Track 462n.441 resolved data of feature calculation and single infrared camera data 41 are for lane line tracking 463p.Vision mileage
Count 455g, vision map structuring 456k resolved data is used for vehicle location 464.Lane detection 453h, traffic mark identification
454i, lane line tracking 463p, vision map structuring 456k, road surface estimation 442e calculation result are constructed for structured road
472q.Vehicle identification 451f, pedestrian identify that 452g, car tracing 461m, pedestrian track 462n calculation result and be used for running environment
State-detection 471.
Fig. 5 show single vehicle-mounted pure visible image capturing head image procossing thread flow figure, workflow starting touching in figure
Clockwork spring part is that vehicle-mounted visual information input 311 judges that vehicle-mounted vision system is pure visible image capturing head.Single pure visible image capturing
Head data 51 are that vehicle-mounted visual signal inputs 312 data.
This system carries out image storage 521a, camera internal reference number calibration 522 to infrared picture data.Image storage 521a
Store pure visible image capturing head initial data, gray processing data and binaryzation data.It, will be grey after visual pattern gray processing 531b
It spends image data and carries out image storage 521a, and carry out next step.
This system simultaneous multi-threading carries out: image data binaryzation 541c, feature calculation 551, Shannon entropy detection 542d, figure
As 552f is estimated on segmentation 543e, road surface.Image data binaryzation 541c binaryzation data are used for image storage 521a and feature
Calculate ORB characterizing part, BRIEF feature calculation part in 551.It is image that Shannon entropy, which detects 542d, image segmentation 543e,
Preprocessing function module 322 may act on panoramic picture detection, active vision functions expanding interface.Estimate that 552f is direct in road surface
582r is constructed for structured road.551 calculation result of feature calculation identifies 562h, lane for vehicle identification 561g, pedestrian
Line identifies that 563i, traffic mark identify 564j, visual odometry 565k, vision map structuring 566m.Vehicle identification 561g, feature
551 resolved datas and single pure visible image capturing head data 51 are calculated for car tracing 571n.Pedestrian identifies 562h, feature
It calculates 551 resolved datas and single pure visible image capturing head data 51 and tracks 572p for pedestrian.551 solution of feature calculation counts
Pure visible image capturing head data 51 are used for lane line tracking 573q accordingly and individually.Visual odometry 565k, vision map structuring
566m resolved data is used for vehicle location 574.Lane detection 563i, traffic mark identification 564j, lane line tracking 573q, view
Feel that map structuring 566m, road surface estimation 552f calculation result construct 582r for structured road.Vehicle identification 561g, Hang Renshi
Other 562h, car tracing 571n, pedestrian track 572p calculation result and are used for running environment state-detection 581.
Fig. 6 is multi-cam panoramic picture differentiation process flow diagram.Camera is determined as multi-cam panorama 61, vehicle-mounted
Visual information input 311 judges vehicle-mounted vision system for panoramic picture.For pure visible image capturing head then by camera 1 to camera shooting
Head N initial data image, binary image, gray level image, pan feature and structured road calculation result are spliced, complete
621 and structured road building result splicing 622 are spelled at image.It is then that camera 1 is former to camera N for infrared camera
Beginning data image, binary image, pan feature and structured road calculation result are spliced, complete image spell 621 with
And structured road building result splicing 622.
Fig. 7 is binocular stereo vision image difference alienation process flow diagram.Camera is determined as binocular stereo vision 71, vehicle-mounted
Visual information input 311 judges vehicle-mounted vision system for binocular stereo vision.Then, according to left and right camera data computation vision spy
3 d space coordinate 72 is levied, while visual odometry 731, vision map structuring 732, knot are completed according to binocular stereo vision algorithm
Structure road building 733.
Fig. 8 is monocular wide-angle camera image difference alienation process flow diagram.Camera is determined as monocular wide-angle camera 81,
Vehicle-mounted visual information input 311 judges vehicle-mounted vision system for monocular wide-angle camera.Wide-angle is carried out to monocular wide-angle camera
Camera distortion correction 82, and export correction camera signals data 83.
Finally, it is stated that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although referring to compared with
Good embodiment describes the invention in detail, those skilled in the art should understand that, it can be to skill of the invention
Art scheme is modified or replaced equivalently, and without departing from the objective and range of technical solution of the present invention, should all be covered at this
In the scope of the claims of invention.
Claims (4)
1. a kind of vehicle-mounted visual perception system of multi-cam adaptation characterized by comprising
Vehicle-mounted camera signal transmission unit, signal conversion and signal transmission for vehicle-mounted hardware sensitive component；
DSP processing unit is connected with vehicle-mounted camera signal transmission unit, and the signal for vehicle-mounted camera digital signal filters
And A/D conversion；
Camera types judging unit is connected with camera types judging unit, for identification the type of hardware of vehicle-mounted camera
And camera types information is provided for system function；
Vision data storage unit is connected with DSP processing unit, for storing vehicle-mounted camera data as driving recording, storage
Deposit the data-interface that image preprocessing result data is merged as sensor；And
Image information pretreatment unit is connected with camera types judging unit and vision data storage unit, for that will image
Head data are pre-processed according to camera types, carry out defogging-optimization processing, Shannon entropy detection, gray processing, two-value to image
Change and the image segmentation based on edge detection；
The system further includes image processing unit, and described image processing unit is connected with image information pretreatment unit；The figure
As processing unit includes:
Road surface estimation unit estimates vehicle running surface for the image data after being optimized according to defogging, is classified；
Visual odometry unit, for carrying out opposite position to vehicle driving according to defogging optimization processing data/greyscale image data
Appearance resolves；
Box counting algorithm unit is used to carry out visual signature resolving to defogging optimization processing data, greyscale image data, including
Feature detection, feature description, calculating feature is respectively Haar, HOG, FAST, ORB, BRIEF or LBP, the practical resolving kind of feature
Class is specified by camera process demand；
Vision map constructing unit is fused to regard for establishing visual signature map office for BRIEF feature with visual odometry
Feel environment SLAM, processing is optimized to vision map, carries out real-time winding detection；
Image identification unit, for carrying out visual identity to vehicle, pedestrian, traffic mark, lane line according to known-image-features；
Picture charge pattern unit, for position to carry out feature in different frame picture to vehicle, pedestrian, traffic mark, lane line
Match, calculates, estimates state, position at the time of vehicle, pedestrian, traffic mark, lane line are under world coordinate system；And
Visual signature library storage element, the visual signature data calculated for storing box counting algorithm unit, storage mode
To provide calculating data-interface for vision map constructing unit according to the feature database storing mode of 3D structural remodeling under timing；
The system further includes System Back-end, and the System Back-end is connected with image processing unit；The System Back-end includes:
Structured road construction unit, for being resolved using road surface estimation unit, vision map constructing unit, image identification unit
As a result building structure road model, Optimized model result；
Running environment detection unit, for according to road surface estimation unit, vision map constructing unit, image identification unit, image
Tracing unit data calculation is as a result, detection penetrates in range L rice from rung and travels environmental change；And
System data interaction interface unit be used for for it is man-machine drive altogether, human-computer interaction, vehicle control decision location data extract, vehicle
Control decision environmental data extracts and Data Fusion of Sensor decision level data are extracted and provide data-interface.
2. the vehicle-mounted visual perception system of multi-cam according to claim 1 adaptation, which is characterized in that the L is
100-200。
3. the vehicle-mounted visual perception system of multi-cam adaptation according to claim 1, which is characterized in that the camera
Type judging unit includes:
Vehicle-mounted visual signal input module judges data-interface for providing camera types, by DSP processing unit processes result
Signal, infrared generator signal input vehicle-mounted camera type judging unit；
Camera types judgment module, for judging camera types comprising infrared/pure visible image capturing head judgment module,
Camera quantity detection module, camera wide-angle detection module and camera view detection module；And
Camera types message output module, for exporting camera judging result.
4. the vehicle-mounted visual perception system of multi-cam adaptation according to claim 3, which is characterized in that the system is also wrapped
It includes:
Vehicle-mounted visual information input unit, is located between camera types judging unit and image information pretreatment unit, is used for
Transmission camera type information and camera installation site information data；And
Vehicle-mounted visual signal input unit, is located between camera types judging unit and image information pretreatment unit, is used for
Transmit original camera data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710193302.9A CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710193302.9A CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106926800A CN106926800A (en) | 2017-07-07 |
| CN106926800B true CN106926800B (en) | 2019-06-07 |
Family
ID=59426518
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710193302.9A Expired - Fee Related CN106926800B (en) | 2017-03-28 | 2017-03-28 | The vehicle-mounted visual perception system of multi-cam adaptation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106926800B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107600072A (en) * | 2017-08-31 | 2018-01-19 | 上海科世达-华阳汽车电器有限公司 | A kind of acquisition methods and system of the common preference parameter of more passengers |
| CN107613262B (en) * | 2017-09-30 | 2021-04-16 | 驭势科技（北京）有限公司 | Visual information processing system and method |
| CN109754415A (en) * | 2017-11-02 | 2019-05-14 | 郭宇铮 | A kind of vehicle-mounted panoramic solid sensory perceptual system based on multiple groups binocular vision |
| CN109842782A (en) * | 2017-11-29 | 2019-06-04 | 深圳市航盛电子股份有限公司 | A kind of vehicle auxiliary test methods, test equipment and storage medium |
| CN108195378A (en) * | 2017-12-25 | 2018-06-22 | 北京航天晨信科技有限责任公司 | It is a kind of based on the intelligent vision navigation system for looking around camera |
| CN108259764A (en) * | 2018-03-27 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | Video camera, image processing method and device applied to video camera |
| CN108566529A (en) * | 2018-04-08 | 2018-09-21 | 深圳市沃特沃德股份有限公司 | The method and device of the adaptive camera video format of onboard system |
| CN108609001A (en) * | 2018-05-09 | 2018-10-02 | 上海蓥石汽车技术有限公司 | A kind of design method for pedestrian's anticollision of actively braking |
| CN109188932A (en) * | 2018-08-22 | 2019-01-11 | 吉林大学 | A kind of multi-cam assemblage on-orbit test method and system towards intelligent driving |
| CN109522825A (en) * | 2018-10-31 | 2019-03-26 | 蔚来汽车有限公司 | The Performance Test System and its performance test methods of visual perception system |
| CN110136199B (en) * | 2018-11-13 | 2022-09-13 | 北京魔门塔科技有限公司 | Camera-based vehicle positioning and mapping method and device |
| CN109752724A (en) * | 2018-12-26 | 2019-05-14 | 珠海市众创芯慧科技有限公司 | A kind of image laser integral type navigation positioning system |
| CN109801339B (en) * | 2018-12-29 | 2021-07-20 | 百度在线网络技术（北京）有限公司 | Image processing method, apparatus and storage medium |
| CN109752008B (en) * | 2019-03-05 | 2021-04-13 | 长安大学 | Intelligent vehicle multi-mode cooperative positioning system and method and intelligent vehicle |
| CN109857123A (en) * | 2019-03-21 | 2019-06-07 | 郑州大学 | A kind of fusion method of view-based access control model perception and the indoor SLAM map of laser acquisition |
| CN111833627B (en) * | 2019-04-13 | 2022-02-08 | 长沙智能驾驶研究院有限公司 | Vehicle visual range expansion method, device and system and computer equipment |
| CN111860050B (en) * | 2019-04-27 | 2024-07-02 | 北京初速度科技有限公司 | Loop detection method and device based on image frames and vehicle-mounted terminal |
| DE102019132024A1 (en) * | 2019-11-26 | 2021-05-27 | Sick Ag | security system |
| JP7314858B2 (en) * | 2020-04-30 | 2023-07-26 | トヨタ自動車株式会社 | Vehicle control system |
| CN111959400A (en) * | 2020-08-31 | 2020-11-20 | 安徽江淮汽车集团股份有限公司 | Vehicle driving assistance control system and method |
| CN113031594B (en) * | 2021-02-26 | 2022-06-24 | 山东交通学院 | Maintenance operation-based adjoint type active safety warning robot, system and method |
| CN113327428A (en) * | 2021-06-21 | 2021-08-31 | 深圳腾达智能科技有限公司 | Method for rapidly acquiring data of vehicles entering and leaving parking lot |
| CN113525234A (en) * | 2021-07-26 | 2021-10-22 | 北京计算机技术及应用研究所 | Auxiliary driving system device |
| CN115643353A (en) * | 2022-10-18 | 2023-01-24 | 广东笑翠鸟教育科技有限公司 | Based on AI teaching perception equipment |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5682788B2 (en) * | 2011-09-27 | 2015-03-11 | アイシン精機株式会社 | Vehicle periphery monitoring device |
| JP6163718B2 (en) * | 2012-08-30 | 2017-07-19 | トヨタ自動車株式会社 | Vehicle control device |
| KR101860610B1 (en) * | 2015-08-20 | 2018-07-02 | 엘지전자 주식회사 | Display Apparatus and Vehicle Having The Same |
| CN205632380U (en) * | 2016-03-30 | 2016-10-12 | 郑州宇通客车股份有限公司 | Vehicle is with multi -functional integrated form initiative safety coefficient and vehicle |
| CN205620323U (en) * | 2016-04-18 | 2016-10-05 | 华南理工大学 | Highway road surface analytic system based on image processing |
| CN106231284B (en) * | 2016-07-14 | 2019-03-05 | 上海玮舟微电子科技有限公司 | The imaging method and system of 3-D image |
-
2017
- 2017-03-28 CN CN201710193302.9A patent/CN106926800B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN106926800A (en) | 2017-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106926800B (en) | The vehicle-mounted visual perception system of multi-cam adaptation | |
| CN110032949B (en) | Target detection and positioning method based on lightweight convolutional neural network | |
| CN106845547B (en) | A kind of intelligent automobile positioning and road markings identifying system and method based on camera | |
| US8699754B2 (en) | Clear path detection through road modeling | |
| CN105654732A (en) | Road monitoring system and method based on depth image | |
| US20090295917A1 (en) | Pixel-based texture-less clear path detection | |
| CN107600067A (en) | A kind of autonomous parking system and method based on more vision inertial navigation fusions | |
| CN104902261B (en) | Apparatus and method for the road surface identification in low definition video flowing | |
| CN115032651A (en) | Target detection method based on fusion of laser radar and machine vision | |
| CN111753639B (en) | Perception map generation method, device, computer equipment and storage medium | |
| CN110083099B (en) | Automatic driving architecture system meeting automobile function safety standard and working method | |
| CN117111085A (en) | Automatic driving automobile road cloud fusion sensing method | |
| CN112130153A (en) | Method for realizing edge detection of unmanned vehicle based on millimeter wave radar and camera | |
| WO2022115987A1 (en) | Method and system for automatic driving data collection and closed-loop management | |
| CN115410181A (en) | Double-head decoupling alignment full scene target detection method, system, device and medium | |
| CN115810179A (en) | Human-vehicle visual perception information fusion method and system | |
| US20220284623A1 (en) | Framework For 3D Object Detection And Depth Prediction From 2D Images | |
| CN115273005A (en) | Visual navigation vehicle environment perception method based on improved YOLO algorithm | |
| CN110472508A (en) | Lane line distance measuring method based on deep learning and binocular vision | |
| CN114818819A (en) | Road obstacle detection method based on millimeter wave radar and visual signal | |
| CN114620059A (en) | Automatic driving method and system thereof, and computer readable storage medium | |
| CN117334040A (en) | Cross-domain road side perception multi-vehicle association method and system | |
| CN117372991A (en) | Automatic driving method and system based on multi-view multi-mode fusion | |
| CN116729042A (en) | Improved YOLOX-NANO model and automobile electric control suspension control method based on model | |
| CN116343513A (en) | Rural highway beyond-sight-distance risk point safety monitoring and early warning method and system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190607 Termination date: 20200328 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |