KR20240092376A - Passenger posture estimation method and apparatus for vehicles - Google Patents

Abstract

The present invention includes the steps of acquiring an image including a passerby in front of a vehicle; Obtaining the passing position of the passerby from the acquired image; Measuring the distance between each body part of the passerby from the acquired passing position of the passerby; and determining the passing posture of the passerby based on the measured distance between each body part of the passerby. Constructing a method of estimating the posture of a passerby for a vehicle including a step.

Description

Passenger posture estimation method and device for vehicles {PASSENGER POSTURE ESTIMATION METHOD AND APPARATUS FOR VEHICLES}

The present invention relates to a method and device for estimating the posture of a passenger for a vehicle, and more specifically, to a method and device for estimating the posture of a vehicle passenger for determining the posture of a passenger and allowing the vehicle's passenger protection system to operate according to the posture of the passenger. It's about.

As vehicle-related technologies continue to develop, today's vehicles are equipped with a variety of functions, including basic driving functions as well as functions related to the safety of passengers. In addition, in order to ensure that these functions are properly applied to prevent malfunctions and provide improved performance, technologies for detecting or classifying pedestrians in front of the vehicle to which each function is applied have been developed.

An example of a technology for detecting or classifying pedestrians or cyclists is a pedestrian collision determination technology. Passenger collision determination technology is a technology that determines the possibility of collision between a passenger and a cyclist using the relative speed/time remaining until collision/vertical relative distance/lateral relative distance provided by the fusion signal of camera and radar. More specifically, when an actual collision occurs, the actual collision is determined using the acceleration sensor and pressure sensor mounted on the vehicle. However, this type of pedestrian collision determination technology can generally only detect the presence of a pedestrian or cyclist, so there is a limitation in that it cannot even estimate the posture of a pedestrian or cyclist when colliding with a vehicle.

Accordingly, there is a need to propose a method to detect pedestrians or cyclists more accurately and classify them in detail.

The matters described as background technology above are only for the purpose of improving understanding of the background of the present invention, and should not be taken as recognition that they correspond to prior art already known to those skilled in the art.

KR 10-1619672 B

The present invention is intended to provide a method and device for estimating a passenger's posture for a vehicle to determine the posture of a passenger and allow a vehicle's passenger protection system to operate with different degrees of protection depending on the posture of the passenger.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

As a method for solving the above technical problem, the present invention includes the steps of acquiring an image including a passerby in front of the vehicle; Obtaining the passing position of the passerby from the acquired image; Measuring the distance between each body part of the passerby from the acquired passing position of the passerby; and determining the passing posture of the passerby based on the measured distance between each body part of the passerby. Constructing a method of estimating a passerby posture for a vehicle including a step.

For example, the image may be an image captured by a camera provided toward the front of the vehicle.

For example, the step of acquiring the passing position of the passerby may include acquiring the passing position of the passerby by inputting the acquired image into a predetermined artificial intelligence learning model.

For example, an artificial intelligence learning model is:

As an artificial intelligence algorithm, it may include at least one of machine learning, neural network, deep learning, classification algorithm, or a combination thereof.

For example, measuring the distance between each body part of the passerby may include detecting at least one preset point included in the body part of the passerby from the acquired passing position of the passerby; and measuring the distance between each body part of the passerby based on the detected points.

For example, measuring the distance between each body part may include determining the width and height between the positions of each body part of the passerby based on the detected points.

For example, the step of determining the passing posture of the passerby may include comparing the width and height between the positions of each determined body part and a preset reference value, and determining the passing posture of the passerby based on the comparison result. You can.

For example, the step of activating a passerby protection operation based on the determined passerby's passerby posture may be further included.

For example, the step of obtaining the driving acceleration of the vehicle and the driving pressure of the vehicle may further include, and the step of activating the passenger protection operation may further include the step of obtaining the driving acceleration of the vehicle and the driving pressure of the vehicle. It may include a step of operating.

For example, acquiring the driving acceleration of the vehicle and the driving pressure of the vehicle may include obtaining the driving acceleration of the vehicle and the driving pressure of the vehicle from a sensor.

As a means to solve the above technical problem, the present invention includes: a camera for photographing the front of a vehicle; And obtain the passing position of the passerby from the image including the passerby in front of the vehicle acquired through the camera, measure the distance between each body part of the passerby from the obtained passerby position, and measure the distance between each body part of the passerby measured. Constructs a passenger posture estimation device for a vehicle including a controller that determines the passing posture of the passerby based on the distance.

For example, the controller may obtain the passage position of the passerby by inputting the acquired image into a predetermined artificial intelligence learning model.

For example, the artificial intelligence learning model is an artificial intelligence algorithm and may include at least one of machine learning, neural network, deep learning, classification algorithm, or a combination thereof.

For example, the controller may detect at least one preset point included in the passerby's body part from the acquired passing position of the passerby, and measure the distance between each body part of the passerby based on the detected point. .

For example, the controller may determine the width and height between the positions of each body part of the passerby based on the detected points.

According to the method and device for estimating the posture of a passenger for a vehicle of the present invention, it is possible to determine the posture of a passenger and enable the vehicle's passenger protection system to operate with different degrees of protection depending on the posture of the passenger.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

Figure 1 is a block diagram showing a vehicle passenger posture estimation device according to an embodiment of the present invention.
Figure 2 is a diagram showing points included in the body parts of a passerby according to an embodiment of the present invention.
FIG. 3 is a flowchart of operating the vehicle passenger posture estimation device of FIG. 1 according to an embodiment of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted.

The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

In describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes. Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

A controller is a communication device that communicates with other controllers or sensors to control the function it is in charge of, a memory that stores the operating system, logic commands, input/output information, etc., and a device that performs the judgments, calculations, and decisions necessary to control the function it is in charge of. It may include more than one processor.

According to one embodiment of the present invention, it is proposed to determine the posture of the passerby and to have the vehicle's passerby protection system operate with different degrees of passerby protection depending on the posture of the passerby.

Figure 1 is a block diagram showing a vehicle passenger posture estimation device according to an embodiment of the present invention.

Referring to FIG. 1, a vehicle passenger posture estimation device 100 according to an embodiment may include a camera 140 and a controller 150.

First, the camera 140 performs the function of photographing the interior of the vehicle and transmitting the photographed image to the controller 150, etc. For the convenience of determining the posture of the passerby, it is appropriate that the camera 140 be installed at the front of the vehicle and take pictures toward the front of the vehicle to better recognize the passerby. Meanwhile, the camera 140 is mounted on the exterior of the vehicle and can check passengers in real time, making it possible to continuously monitor the passengers in front of the vehicle using images provided in real time.

Additionally, the controller 150 may obtain images of pedestrians, including pedestrians and cyclists, in front of the vehicle from the camera 140, and obtain a passage position through these images. In addition, the distance between each body part of the passerby can be measured from the obtained passing position of the passerby, and the passing posture of the passerby can be determined based on the measured distance between each body part of the passerby. Here, the passage posture of the passerby of the controller 150 may be determined through preset points included in the passerby's body parts. A more detailed description of the posture estimation judgment of the passerby by the controller 150 will be described with reference to FIGS. 2 and 3 below.

Meanwhile, the controller 150 may acquire the passerby's passing position by inputting the image obtained from the camera 140 into a predetermined artificial intelligence learning model. Here, the artificial intelligence learning model is an artificial intelligence algorithm and may include at least one of machine learning, neural network, deep learning, classification algorithm, or a combination thereof.

At this time, the Human Pose Estimation of the artificial intelligence learning model can be used to make the decision. Human Pose Estimation detects a plurality of key points corresponding to important human body parts and estimates the target human pose through them. Among the detected key points, those included in the body parts of a pedestrian or cyclist are included in the present invention. It can be used as points to determine the posture of the passerby in the embodiment.

Figure 2 is a diagram showing points included in the body parts of a passerby according to an embodiment of the present invention.

Referring to FIG. 2, the controller 150 may detect at least one preset point included in the passerby's body part from the passerby's passing position obtained through Human Pose Estimation. Specifically, each point may have 2D(x,y) or 3D(x,y,z) coordinates. The controller 150 can measure the distance between each body part of each passerby based on the detected points.

More specifically, the controller 150 may determine the width and height between the positions of each body part of the passerby based on the detected points. The controller 150 can determine the width of each body part by differentiating the x-coordinates at each point expressed in 2D or 3D coordinates, and determine the height of each body part by differentiating the y-coordinates. Additionally, the thickness of each body part can be determined by differentiating the z-coordinates. The straight-line distance between the locations of each body part as well as the width, height, and thickness between the locations of each body part may be calculated. Through this process, the measurement of the distance between each body part of the controller 150 can become more precise and accuracy can be improved.

Additionally, the controller 150 may compare the width and height between the positions of each determined body part and a preset reference value, and determine the passing posture of the passerby based on the comparison result. This will be explained using the passerby's body part as an example.

In Figure 2, numbers 1 and 2 represent the points of the left shoulder and right shoulder, respectively, and number 3 represents the chest point. Additionally, number 4 represents the point in the center of the abdomen, and numbers 5 and 6 represent the left buttock and right hip points, respectively. Additionally, numbers 7 and 8 represent the points of the left knee and right knee, respectively. Lastly, numbers 9 and 10 represent points on the left ankle and right ankle, respectively.

At this time, the passing posture of the passerby can be determined based on the following conditions. For example, the controller 150 determines that the width between numbers 1 and 3 is greater than the standard value, the width between numbers 4 and 5 is greater than the standard value, and the width between numbers 2 and 3 is greater than the standard value. If both sides of the body part (numbers 1, 2, 5 to 10) are recognized under the condition that the width between the numbers is larger than the standard value, it can be assumed that the passerby is looking to the front and back. Additionally, when the width between numbers 1 and 3 is smaller than the standard value or the width between numbers 2 and 3 is smaller than the standard value, it can be assumed that the passerby is looking to the side. In addition, under the condition that the width between numbers 4 and 5 is smaller than the standard value or the width between numbers 4 and 6 is smaller than the standard value, the left side of the body part (numbers 1, 5, 7, and 9) is not recognized. In conditions where the right side of the body part (numbers 2, 6, 8, and 10) is not recognized, it can be assumed that the passerby is looking to the side.

Additionally, under the condition that the height between numbers 3 and 4, the height between numbers 1 and 4, or the height between numbers 2 and 4 are smaller than the reference value, it can be assumed that the passerby is in a bent posture. Conversely, under the condition that the height between numbers 3 and 4, the height between numbers 1 and 4, or the height between numbers 2 and 4 are greater than the reference value, it can be assumed that the passerby is in a standing posture. Here, the reference value is a preset value and may be set differently depending on the body type of the pedestrian or cyclist.

The conditions for estimating the passing posture of a passerby described so far are examples for understanding the present invention, and it is obvious to those skilled in the art that they may be set differently depending on the situation.

Meanwhile, the controller 150 can optimally operate the passerby protection operation based on the determined passerby's passing posture. The passenger protection operation may include a passenger airbag provided in the vehicle. For example, when the passenger is facing the front or rear, the collision area is the lower or upper body of the passenger, so the degree of protection for the passenger can be set to the maximum so that the airbag for the passenger operates to the maximum. Additionally, when the passerby is looking to the side, the collision site may be one leg of the lower body of the pedestrian or cyclist, so the degree of passerby protection can be set to less than the maximum.

Additionally, when it is determined that the passerby is in a leaning position, the crushing prevention system can be activated to prevent the passerby from being crushed under the underside of the vehicle.

The above-described passenger protection operation of the controller 150 may be performed by additionally considering the driving posture of the passenger determined by the controller 150, as well as the driving acceleration of the vehicle and the driving pressure of the vehicle. Here, the driving acceleration of the vehicle and the driving pressure of the vehicle can be obtained through sensors. Specifically, if the vehicle's driving acceleration or the vehicle's driving pressure is high, the degree of passenger protection can be set to the maximum, and if the vehicle's driving acceleration or the vehicle's driving pressure is low, the degree of passenger protection can be set to a low level. .

Based on the above-described vehicle passenger posture estimation device 100, a method for estimating a vehicle passenger posture according to an embodiment will be described with reference to FIG. 3 .

FIG. 3 is a flowchart of operating the vehicle passenger posture estimation device 100 of FIG. 1 according to an embodiment of the present invention.

Referring to FIG. 3, first, the camera 140 may acquire an image including a passenger in front of the vehicle (S310). Afterwards, the controller 150 may obtain the passage position of the passerby from the acquired image (S320). The controller 150 may acquire the passerby's passing position by inputting the image acquired from the camera 140 into a predetermined artificial intelligence learning model.

At this time, the Human Pose Estimation of the artificial intelligence learning model can be used to make the decision. Human Pose Estimation detects a plurality of key points corresponding to important human body parts and estimates the target human pose through them. Among the detected key points, those included in the body parts of a pedestrian or cyclist are included in the present invention. It can be used as points to determine the posture of the passerby in the embodiment.

Afterwards, the controller 150 can measure the distance between each body part of the passerby from the obtained passing position of the passerby (S330). Additionally, the controller 150 may determine the passing posture of the passerby based on the measured distance between each body part of the passerby (S340). Afterwards, the controller 150 may determine whether there is a possibility of collision with a passerby (S350). If there is a possibility of collision (YES in S350), the controller 150 may optimally operate the passerby protection operation based on the determined passing posture of the passerby (S360).

According to the embodiments of the present invention described so far, it is possible to determine the posture of a passenger and allow the vehicle's passenger protection system to operate with different degrees of protection depending on the posture of the passenger.

Although the present invention has been shown and described in relation to specific embodiments, it is known in the art that the present invention can be modified and changed in various ways without departing from the technical spirit of the present invention as provided by the following claims. This will be self-evident to those with ordinary knowledge.

100: Passenger posture estimation device for vehicles
140: camera
150: controller

Claims (15)

Obtaining an image including a passerby in front of the vehicle;
Obtaining the passing position of the passerby from the acquired image;
Measuring the distance between each body part of the passerby from the acquired passing position of the passerby; and
A method of estimating a passenger posture for a vehicle including: determining the passing posture of the passenger based on the measured distance between each body part of the passenger. In claim 1,
The image is,
A method of estimating a passenger posture for a vehicle, characterized in that the image is captured by a camera provided to face the front of the vehicle. In claim 1,
The steps to obtain the passing position of the passerby are:
A method of estimating the posture of a passenger for a vehicle, comprising: acquiring the passing position of the passenger by inputting the acquired image into a predetermined artificial intelligence learning model. In claim 3,
The artificial intelligence learning model is,
An artificial intelligence algorithm, comprising at least one of machine learning, neural network, deep learning, classification algorithm, or a combination thereof. In claim 1,
The step of measuring the distance between each body part of the passerby is:
Detecting at least one preset point included in a body part of the passerby from the acquired passing position of the passerby; and
A method for estimating a passenger posture for a vehicle, comprising: measuring the distance between each body part of the passenger based on the detected points. In claim 5,
The steps to measure the distance between each body part are:
A method for estimating a passenger posture for a vehicle, comprising: determining the width and height between the positions of each body part of the passenger based on the detected points. In claim 6,
The steps to determine the passing posture of the passerby are:
Comparing the width and height between the determined positions of each body part and a preset reference value, and determining the passing posture of the passenger based on the comparison result. A method for estimating a passenger posture for a vehicle, comprising: In claim 1,
A method for estimating a passenger posture for a vehicle, further comprising: activating a passenger protection operation based on the determined passing posture of the passerby. In claim 8,
Further comprising: acquiring the driving acceleration of the vehicle and the driving pressure of the vehicle,
The steps for activating the passerby protection operation are:
A method for estimating a passenger posture for a vehicle, comprising: activating a passenger protection operation by additionally considering the driving acceleration of the vehicle and the driving pressure of the vehicle. In claim 9,
The step of acquiring the driving acceleration of the vehicle and the driving pressure of the vehicle is,
A method for estimating a passenger posture for a vehicle, comprising: acquiring the driving acceleration of the vehicle and the driving pressure of the vehicle from a sensor. A camera that captures the front of the vehicle; and
Obtain the passing position of the passerby from the image including the passerby in front of the vehicle acquired through the camera, measure the distance between each body part of the passerby from the obtained passerby position, and measure the distance between each body part of the passerby measured. A passenger posture estimation device for a vehicle including a controller that determines the passing posture of the passerby based on the distance. In claim 11,
The controller is,
A passenger posture estimation device for a vehicle, characterized in that it obtains the passing position of the passenger by inputting the acquired image into a predetermined artificial intelligence learning model. In claim 12,
The artificial intelligence learning model is,
An artificial intelligence algorithm, comprising at least one of machine learning, neural network, deep learning, classification algorithm, or a combination thereof. In claim 11,
The controller is,
Passenger posture estimation for a vehicle, characterized in that detecting at least one preset point included in the passerby's body part from the obtained passerby's passerby position, and measuring the distance between each body part of the passerby based on the detected point. Device. In claim 14,
The controller is,
A passenger posture estimation device for a vehicle, characterized in that it determines the width and height between the positions of each body part of the passerby based on the detected points.

Images