KR101976497B1 - System and method for operating unit support of vehicle - Google Patents

Abstract

A vehicular operating system auxiliary system and method thereof are disclosed.
The auxiliary system for a vehicle operating system according to an embodiment of the present invention includes a camera unit for photographing an inside of a vehicle and acquiring an image; An image processing unit for storing a coordinate system inside the vehicle and detecting hand coordinates in the image received by the camera unit; An operating system selection determining unit for determining whether or not the hand of the passenger is in contact with the operating system through the matching between the hand coordinates detected in the image processing unit and the in-vehicle operating system coordinate; And a control unit that enables the operation of the operating system when the operating system is in contact with the hands of the occupant and limits the operation of the operating system when the operating system is not in contact.

Description

Technical Field [0001] The present invention relates to a system and a method for operating a vehicle,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a vehicle operating system auxiliary system and a method thereof.

Generally, a vehicle is equipped with an engine, a transmission, a steering wheel, a pedal, a side brake, a door, an air conditioner, a lamp, an audio visual system, a diagnostic information system (DIS) Sheet changing, etc., and an operating system for the operation of each apparatus is provided. In recent years, there has been an increasing number of cases in which vehicle functions are controlled electronically, thereby increasing the number of operating systems.

Here, the operating system includes a button-type operating system fixed at a predetermined position such as an air conditioning switch, a touch-type operating system displayed on a display screen such as an AV system, and a joystick-type operating system for moving a certain region such as a transmission, a wiper, .

Such an operating system should be operated normally by the driver and the occupant, but it may be operated by a child who has a lot of external shocks or curiosity, or a neglected pet, which may cause a malfunction.

For example, there is a problem that when a vehicle parked on a ramp is released from a parking brake due to an external impact, or when a vehicle is left unattended by a child or a pet, the transmission is arbitrarily operated and the brake is released.

In other words, there is a possibility that a malfunction of an unintended operating system may occur due to an increase in the operating system, and it is inevitable that a precise judgment can be made to prevent a major accident in advance.

Embodiments of the present invention provide a vehicle operating system auxiliary system and method for preventing a malfunction of an operating system operated by an external shock or a child and a pet, not an intended operation of a driver on board a vehicle, thereby preventing a safety accident.

According to an aspect of the present invention, a vehicular operating system auxiliary system includes: a camera unit for photographing an inside of a vehicle to acquire an image; An image processing unit for storing a coordinate system inside the vehicle and detecting hand coordinates in the image received by the camera unit; An operating system selection determining unit for determining whether or not the hand of the passenger is in contact with the operating system through the matching between the hand coordinates detected in the image processing unit and the in-vehicle operating system coordinate; And a control unit that enables the operation of the operating system when the operating system is in contact with the hands of the occupant and limits the operation of the operating system when the operating system is not in contact.

In addition, the image processor recognizes the hand as a feature point of color, shape, and brightness in the received image when the hand coordinate is detected, and detects the end point coordinates of the finger through classification and modeling of the hand have.

In addition, the image processing unit may detect the hand coordinates as three-dimensional coordinates or two-dimensional coordinates according to the performance of the camera unit.

In addition, the operation system selection determination unit may store various in-vehicle control system coordinates based on the coordinate system inside the vehicle.

The operating system selection determination unit determines that the occupant intentionally makes contact with the operating system when the detected hand coordinates of the occupant match the operating system coordinate, and when the hand coordinate does not match the operating system coordinate, It can be judged that it is not selected and not selected.

In addition, the operating system selection determination unit may determine whether or not the operation mode selection unit is selected through a movement locus of at least one of a transmission, a wiper operation switch, and a lamp operation switch and a movement locus matching of the hand coordinates.

The control unit may be configured to process the operation signal only when the operation system is in contact with the operation system and when the operation system inputs an operation signal to the operation system, Can be ignored.

The control unit may adjust the tension of the operating system to be lower than a predetermined reference value when the hand touches the operating system and adjust the tension of the operating system to be higher than a predetermined reference value when the hand does not touch the operating system.

In addition, the image processing unit analyzes the received image to identify the number of passengers, whether children or pets are boarded through object recognition modeling, and detects the seat position of a child, The operation of the operating system can be restricted when the child or the pet is left unattended.

According to an aspect of the present invention, there is provided an auxiliary system for a vehicle operating system for preventing an unintended operation of a vehicle operating system, the method comprising: a) analyzing an image of a vehicle interior acquired through a camera to detect a hand shape; b) detecting hand coordinates according to the position of the hand shape through a coordinate system inside the vehicle; c) determining whether or not the hand of the passenger is in contact with the operating system through the matching between the hand coordinate and the in-vehicle operating system coordinate; And d) enabling the operation of the operating system when the passenger's hand and the operating system are in contact with each other or limiting the operation of the operating system when the operating system is not in contact with the passenger's hand.

In addition, the step (a) may recognize the hand shape as a feature point of color, shape, and brightness in the image, and the step (b) may further include separating a finger and a palm from each other, It is possible to detect the end point coordinates.

The step d) may include switching the operating system to an operable state when the hand and the operating system are in contact with each other, and presenting the status information visually or audibly; And controlling the operation system to be smoothly operated by adjusting the tension of the operation system to be lower than a predetermined reference value.

The step d) may further include the step of controlling the operation of the operating system in a state where the operating system is not in contact with the hand, and presenting the status information visually or audibly; And limiting the physical operation of the operating system by adjusting the tension of the operating system to a value higher than a predetermined reference value.

The step d) may include the step of processing an operation signal inputted from the operation system only when the hand touches the operation system, and ignoring the operation signal inputted when the hand is not in contact .

The method may further include, after the step d), analyzing the image to limit the operation of the operation system when it is recognized that a child or a pet exists in a situation without a parent in the vehicle.

According to the embodiment of the present invention, it is possible to easily operate only when the intention of the passenger is reflected on the operating system by using the camera mounted in the vehicle, thereby preventing malfunction of the operating system against external shocks.

In addition, it is possible to prevent a safety accident by predetermining a child or a pet left unavoidably in a vehicle through a camera, and preventing malfunction of an unintended operation system by a child or a pet.

Fig. 1 schematically shows the configuration of a vehicle-mounted-system auxiliary system according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating an auxiliary method for a vehicle operating system according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, " " module, " and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

Throughout the specification, the terms first or second etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are intended to distinguish one element from another, for example, without departing from the scope of the invention in accordance with the concepts of the present invention, the first element may be termed the second element, The second component may also be referred to as a first component.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A vehicle auxiliary system and method according to an embodiment of the present invention will now be described in detail with reference to the drawings.

Fig. 1 schematically shows the configuration of a vehicle-mounted-system auxiliary system according to an embodiment of the present invention.

1 is a block diagram illustrating a vehicle control system auxiliary system 100 according to an embodiment of the present invention. The vehicle control system 100 includes a camera unit 110, an image processing unit 120, an operation system selection determination unit 130, .

The camera unit 110 is installed in the vehicle, photographs the inside of the vehicle, and transmits the photographed image to the image processing unit 120.

The camera unit 110 is configured as a three-dimensional camera having an infrared function to detect hand coordinates in 3D (X, Y, Z).

The image processing unit 120 stores the in-vehicle coordinate system and recognizes the hand of the passenger in the image received by the camera unit 110 based on the coordinate system.

At this time, when the hand coordinate is detected, the image processing unit 120 recognizes the hand as a minutia such as color, shape, and brightness in the received image, and then detects a hand end point through modeling and modeling of the hand . Here, the hand end point is the end point of the finger, and the coordinate (position) of the hand means the end point of the finger.

In addition, the image processing unit 120 may analyze the received image and determine the number of passengers and whether a child (child), a pet (dog, cat, etc.) is boarded through object recognition modeling.

In addition, the image processing unit 120 stores the three-dimensional coordinate system, and can detect the hand coordinates as 3D (X, Y, Z) with the received three-dimensional image.

The operating system selection deciding unit 130 determines whether or not the occupant touches the operating system through the matching between the hand coordinates detected by the image processing unit 120 and the in-vehicle operating system coordinates.

To this end, the operating system selection deciding unit 130 stores various operating system coordinates (positions) in the vehicle based on the coordinate system.

That is, the operating system selection determination unit 130 can determine that the occupant intentionally touches the operating system when the detected hand position (position) of the occupant matches the coordinates (position) of the specific operating system.

On the other hand, the operating system selection determination unit 130 determines that the operating system is not selected unless the hand coordinates of the occupant match the coordinates of the operating system.

In addition, the operating system selection determination unit 130 determines whether or not the movement locus and the movement coordinates of the hand coordinates of the hand coordinates are identical to each other in consideration of the movable range of the joystick operation system such as the transmission, the wiper operation switch, and the lamp operation switch (headlight, rear lamp, It is possible to judge whether or not to select it.

In the meantime, the embodiment of the present invention described above has been described as constituting the camera unit 110 as a three-dimensional camera. However, the present invention is not limited thereto, and the camera unit 110 may be a 2D camera.

That is, even if the hand coordinate at the time when the occupant operates the operating system is detected as 2D (X, Y), the operating system selection determination unit 130 determines whether or not the hand coordinate system is in contact with the manipulation system have.

The controller 140 controls the operations of the respective units for the vehicle-mounted-system auxiliary function according to the embodiment of the present invention, and stores data and programs for the operations.

The control unit 140 controls the functions of the respective operation systems on the basis of the hand contact information on the operation system in the operation system selection determination unit 130. [

For example, according to one embodiment of the present invention, the control unit 140 confirms whether or not the operation system is in contact with the hand, and processes the operation signal only when the operation signal is input to the operation system.

On the other hand, when an incorrect operation signal is input due to an external impact of the vehicle, the control unit 140 ignores the operation signal to prevent malfunction because no manual contact is made with the operation system.

In addition, according to one embodiment of the present invention, even if a wrong operation signal is inputted due to a pets left in the vehicle, the control unit 140 does not detect hand coordinates by the image processing unit 120, It is possible to prevent malfunction by ignoring the operation signal.

The control unit 140 controls the tension of the operation system to be normal when the hand coordinates are on the operation system, or adjusts the tension lower than the normal value for smooth operation to reduce the force to be applied.

On the contrary, when the hand coordinate is not on the operation system, the control unit 140 removes the physical operation so that the operation can be performed only when the operation force is applied more than when the operation is normal.

For example, it is possible to restrict the physical movement of the joystick operating system such as the transmission, the wheel, and the side brakes to such a degree that the operation can be performed only by applying a force exceeding a normal value to the button operation such as a window or a hot line switch.

In addition, in the case of a touch-type operation system, the control unit 140 can restrict the function operation to be impossible when a non-hand portion is touched.

In addition, the control unit 140 can restrict the function of the window switch by grasping the child's boarding seat position and holding the corresponding window switch, which may be a problem in safety due to misoperation of the child. In addition, the console portion can be depressed by other objects or parts other than the hand, but it can be verified whether the button is operated by the hand position, thereby restricting unintentional erroneous operations.

On the other hand, a vehicle operation system assisting method based on the configuration of the vehicle operation system assisting system 100 described above will be described.

FIG. 2 is a flowchart illustrating an auxiliary method for a vehicle operating system according to an embodiment of the present invention.

Referring to FIG. 2, an auxiliary system 100 for a vehicle control system according to an embodiment of the present invention captures an image of a vehicle interior through a camera 110 and acquires an in-vehicle image (S101).

The vehicular operating system auxiliary system 100 analyzes the acquired image to detect the hand shape (S102).

The vehicle control system auxiliary system 100 extracts a hand end point by separating a finger and a palm from the detected hand shape, and detects the extracted hand end point coordinates (S103).

The vehicular operating system auxiliary system 100 determines whether the coordinates of the operating system and the hand end point coordinates are matched (S104).

At this time, the vehicle-mounted system auxiliary system 100 controls the operating system to be operable (S106) if it is determined that the coordinates of the hand end point and the coordinates of the operating system match and are in contact with the operating system (S105;

At this time, the vehicle-mounted-system auxiliary system 100 can visually and audibly express that the operating system is controlled to be operable (S107).

In addition, the vehicle-mounted-system auxiliary system 100 adjusts the tension of the operating system to a normal value or lower than a predetermined reference value so that the operating system can be smoothly operated (S108).

On the other hand, if it is determined in step S105 that the coordinates of the hand end point and the coordinates of the operating system do not match (S105: No), the vehicular operating system auxiliary system 100 operates in a state of restricting the operation of the operating system (S109) .

At this time, the vehicle-mounted-system auxiliary system 100 can visually and audibly express a state in which the operation of the operating system is limited (S110).

In addition, the vehicle-mounted-system auxiliary system 100 restricts the operation by adjusting the tension of the operating system to be higher than a predetermined reference value (S111). At this time, physical manipulation can be restricted so that the manipulation system can be operated only when a greater amount of manipulation force is applied than during normal operation, or the manipulation signal input can be ignored even if the manipulation of the manipulation system is performed.

As described above, according to the embodiment of the present invention, it is easy to operate only when the intention of the passenger is reflected on the operating system by using the camera attached to the vehicle, thereby preventing the malfunction of the external shock.

In addition, it is possible to prevent a safety accident by preliminarily judging a child or a pet left unavoidably in a vehicle through a camera, and preventing malfunction of an unintended operation system by a child or a pet.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary,

For example, in the embodiment of the present invention shown in FIG. 2, it is possible to recognize the child and the pet by analyzing the in-vehicle image information, and to restrict the operation of the operation system in a state where the child or pet is left in the vehicle.

In other words, in case of a child, when a driver who is a driver gets off the vehicle for a while after parking, the driver recognizes that the child alone is carrying the vehicle through the camera and can restrict the operation of the operation system even if the child recognizes the hand.

The embodiments of the present invention are not limited to the above-described apparatuses and / or methods, but may be implemented through a program for realizing functions corresponding to the configuration of the embodiment of the present invention, a recording medium on which the program is recorded And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

100: Car operating system auxiliary system 110:
120: Image processing unit 130: Operation system selection judgment unit
140:

Claims (15)

A camera unit for photographing the interior of the vehicle to acquire an image;
An image processing unit for storing a coordinate system inside the vehicle and detecting hand coordinates in the image received by the camera unit;
An operating system selection determining unit for determining whether or not the hand of the passenger is in contact with the operating system through the matching between the hand coordinates detected in the image processing unit and the in-vehicle operating system coordinate; And
And a control unit for enabling the operation of the operating system when the passenger's hand and the operating system are in contact with each other and for limiting the operation of the operating system when not in contact,
Wherein the controller controls the tension of the operating system to be lower than a predetermined reference value when the hand touches the operating system and controls the tension of the operating system to be higher than a predetermined reference value when the hand does not touch the operating system. Operating system auxiliary system. The method according to claim 1,
Wherein the image processing unit comprises:
Wherein when the hand coordinate is detected, the hand is recognized as a feature point of color, shape, and brightness in the received image, and the end point coordinates of the finger are detected through classification and modeling of the hand. . 3. The method according to claim 1 or 2,
Wherein the image processing unit comprises:
And detects the hand coordinates as three-dimensional coordinates or two-dimensional coordinates according to the performance of the camera unit. The method according to claim 1,
Wherein the operating-
And stores various control system coordinates in the vehicle based on a coordinate system inside the vehicle. The method according to claim 1,
Wherein the operating-
If the hand coordinate of the detected occupant matches the operating system coordinate, the occupant judges that the driver has intentionally contacted the operating system to make a selection, and if the hand coordinate does not match the operating system coordinate, For a vehicle. 6. The method according to claim 1 or 5,
Wherein the operating-
Wherein the selection unit determines whether the selection is made through the movement locus of at least one of the transmission, the wiper operation switch, and the lamp operation switch through the movement locus matching of the hand coordinates. The method according to claim 1,
Wherein,
Wherein the control unit is configured to process the operation signal only when a hand touches the operation system when an operation signal is input to the operation system and to ignore the operation signal when the hand is not in contact with the operation system, . delete The method according to claim 1,
Wherein the image processing unit comprises:
And analyzing the received image to determine the number of passengers and whether or not the child and the pet are boarded through object recognition modeling and detecting the seat position of the child,
Wherein the control unit limits the operation of the operating system when a child or a pet is left without a parent in the vehicle. A vehicle operating system auxiliary method for preventing unintended operation of a vehicle operating system,
a) analyzing an image of a vehicle interior acquired through a camera to detect a hand shape;
b) detecting hand coordinates according to the position of the hand shape through a coordinate system inside the vehicle;
c) determining whether or not the hand of the passenger is in contact with the operating system through the matching between the hand coordinate and the in-vehicle operating system coordinate; And
d) enabling the operation of the operating system when the passenger's hands and the operating system are in contact with each other or limiting the operation of the operating system when the operating system is not in contact with the passenger's hands,
Wherein the step d) includes the steps of switching the operating system to an operable state when the hand and the operating system are in contact with each other, and visually or audibly presenting the status information, and adjusting the tension of the operating system to a value lower than a predetermined reference value, So as to be smoothly operated. 11. The method of claim 10,
The step (a) may include recognizing the shape of the hand as a feature point of color, shape, and brightness in the image,
Wherein the step b) includes detecting the end point coordinates of the finger by separating the finger and the palm from each other through the type and modeling of the hand. delete 11. The method of claim 10,
The step d)
Controlling the operation of the operating system in a state of restricting operation of the operating system when the operating system is not in contact with the hand, and displaying the state information visually or audibly; And
Limiting the physical operation of the operating system by adjusting the tension of the operating system to be higher than a predetermined reference value
And the vehicle operating system auxiliary method. 11. The method of claim 10,
The step d)
Processing the operation signal inputted from the operating system only when the hand touches the operating system, and ignoring the operating signal inputted when the hand is not in contact. 11. The method of claim 10,
After the step d)
Further comprising the step of analyzing the image to limit the operation of the operating system when it is recognized that a child or a pet exists in a situation where there is no parent in the vehicle.

Images