KR102549797B1 - Electronic device and method for determining a state of a driver - Google Patents

Abstract

An electronic device is disclosed. The electronic device includes a sensing unit for sensing pressure applied to a seat of a mobile body by a driver and a processor for determining a driver's condition based on a frequency characteristic of a change in pressure sensed by the sensing unit.

Description

Electronic device and method for determining its driver's state { ELECTRONIC DEVICE AND METHOD FOR DETERMINING A STATE OF A DRIVER }

The present invention relates to an electronic device and a method for determining a driver's state thereof, and more particularly, to an electronic device for determining a state of a driver driving a moving object and a method for determining a driver's state thereof.

In recent years, due to an increase in accidents caused by drowsy driving, many human casualties have been brought about every year. Accordingly, in order to prevent accidents caused by drowsy driving, various technologies capable of measuring and preventing driver drowsiness have been developed.

For example, there is a method of detecting the driver's drowsiness by photographing the driver using a camera installed in the vehicle. However, in this method, there is a problem in that the driver's drowsiness cannot be accurately determined at night or by external factors such as the user's sunglasses.

Accordingly, a search for a method for more effectively determining whether a driver is drowsy is required.

SUMMARY OF THE INVENTION The present invention has been made in accordance with the above-described needs, and an object of the present invention is to provide an electronic device capable of determining a driver's condition based on a frequency characteristic of a change in pressure applied by the driver and a method for determining the driver's condition. .

An electronic device according to an embodiment of the present invention for achieving the above object is a sensing unit for detecting pressure applied to a seat of a mobile body by a driver and a frequency characteristic for a change in pressure detected by the sensing unit. and a processor for determining the driver's condition based on the driver's condition.

Here, the sensing unit may include a plurality of piezoelectric sensors arranged in a matrix form on the seat.

In addition, the processor may determine the driver's condition by calculating a difference in magnitude of the detected pressure at a predetermined time interval and analyzing a frequency component of the calculated difference values.

Here, the processor may determine that the driver is in a drowsy state in a time interval in which a frequency component of the change in pressure includes a low-frequency component.

In addition, the processor represents the calculated difference values as a signal on the time axis, frequency-converts the signal at each preset time to determine the frequency component of the waveform in each time interval, and determines the frequency component of the waveform at each preset threshold, and the low frequency below the preset threshold. It may be determined that the driver is in a drowsy state in a time section in which only the components are detected.

Meanwhile, the processor may provide feedback corresponding to the driver's condition.

Here, if it is determined that the driver is in a drowsy state, the processor may control the speed of the moving object to be reduced or a preset audio to be output.

Meanwhile, a method for determining a driver's condition of an electronic device according to an embodiment of the present invention includes detecting pressure applied to a seat of a moving object by a driver and determining the driver's condition based on a frequency characteristic of a change in the sensed pressure. It includes the step of determining

Here, in the sensing step, the pressure may be sensed using a plurality of piezoelectric sensors arranged in a matrix form on the seat.

In the determining step, the driver's condition may be determined by calculating a difference in magnitude of the detected pressure at a predetermined time interval and analyzing a frequency component of the calculated difference values.

Here, in the determining step, it may be determined that the driver is in a drowsy state in a time interval in which a frequency component of the change in pressure includes a low-frequency component.

In addition, the determining step represents the calculated difference values as a signal on the time axis, frequency-converts the signal at each preset time to determine the frequency component of the waveform in each time interval, and determines the frequency component of the waveform at each preset threshold value or less. It may be determined that the driver is in a drowsy state in a time interval in which only low-frequency components of are detected.

Meanwhile, the method for determining a driver's condition according to an embodiment may further include providing feedback corresponding to the driver's condition.

Here, in the providing step, if it is determined that the driver is in a drowsy state, the speed of the moving object may be reduced or a preset audio may be output.

According to various embodiments of the present invention as described above, in that the driver's condition is determined based on the frequency characteristic of the change in pressure applied to the seat in which the driver sits, the influence of external factors is relatively small and in real time The driver's condition can be judged.

1 is a diagram for explaining an implementation example of an electronic device according to an embodiment of the present invention;
2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present invention;
3 is a view for explaining a disposition form of a piezoelectric sensor according to an embodiment of the present invention;
4 to 8 are diagrams for explaining a method for detecting a driver's condition according to an embodiment of the present invention;
9 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present invention;
10 to 14 are diagrams for explaining a method for detecting a driver's gaze direction according to an embodiment of the present invention;
15 is a block diagram for explaining a detailed configuration of an electronic device according to an embodiment of the present invention, and
16 is a flowchart illustrating a method for determining a driver's condition of an electronic device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a diagram for explaining an implementation example of an electronic device according to an embodiment of the present invention.

The electronic device 100 may be provided in a vehicle. Here, the mobile body may be a vehicle as shown in FIG. 1 , but this is only an example, and may be a motorcycle, a train, or an airplane that a driver can ride on and drive. In this case, the electronic device 100 may be provided while being fixed to the mobile body. Alternatively, the electronic device 100 may be provided by being detachably attached to the mobile body.

Meanwhile, the electronic device 100 may determine the state of the driver riding the moving object. Specifically, the electronic device 100 may determine whether a driver sitting on a seat provided in the mobile body is driving the mobile body or is sleeping.

To this end, the electronic device 100 may detect the amount of pressure applied to the seat by the driver and analyze a frequency component of the change in the amount of pressure to determine the driver's condition.

In this way, according to an embodiment of the present invention, the electronic device 100 determines the driver's condition by analyzing the frequency component of the change in the amount of pressure applied to the seat by the driver, thereby determining the driver's condition in real time. can be judged by

2 is a block diagram for explaining the configuration of an electronic device according to an embodiment of the present invention.

Referring to FIG. 2 , the electronic device 100 includes a sensing unit 110 and a processor 120 .

The sensing unit 110 senses the pressure applied to the seat of the mobile body by the driver. That is, the sensing unit 110 may detect the amount of pressure applied to the seat of the mobile body by the driver and output pressure data indicating the amount of the sensed pressure.

To this end, the sensing unit 110 may include a plurality of piezoelectric sensors. That is, the plurality of piezoelectric sensors may detect the amount of pressure applied to the seat of the moving object by the driver and output pressure data indicating the amount of the sensed pressure.

Meanwhile, as shown in FIG. 3 , a plurality of piezoelectric sensors may be arranged in a matrix form on a seat.

Referring to FIG. 3 , m×n piezoelectric sensors may be arranged in a matrix form at a portion where the driver's buttocks touch when the driver sits on the seat. In this case, the piezoelectric sensor may be built into the seat or may be separately installed on a cushion or the like and disposed on the seat. In addition, the piezoelectric sensor may be installed in a backrest portion of a seat in addition to a portion where the driver's hip touches.

Meanwhile, when the piezoelectric sensors are arranged in a matrix form, the number of piezoelectric sensors arranged in each column and the number of piezoelectric sensors arranged in each row may be the same or different from each other, and may vary depending on the size and shape of the seat.

In addition, the piezoelectric sensor may be arranged in various shapes, such as a circular shape, in addition to a matrix shape.

Meanwhile, the sensing unit 110 may sense pressure at predetermined time intervals.

Specifically, the preset time interval may be 1/T second (ie, T Hz). In this case, the sensing unit 110 may detect the amount of pressure applied by the driver at intervals of 1/T second through a plurality of piezoelectric sensors, and output pressure data representing the amount of the sensed pressure. Here, T may be 50. That is, the sensing unit 110 may detect pressure at intervals of 1/50 second and output pressure data.

Meanwhile, since pressure is sensed by a plurality of piezoelectric sensors, pressure data sensed by a plurality of piezoelectric sensors at a specific time point may constitute one frame.

That is, the frame may include pressure data sensed at the same time by a plurality of piezoelectric sensors. Further, since the plurality of piezoelectric sensors are arranged in a matrix form, pressure data sensed by each piezoelectric sensor may be included in a matrix form according to positions of the plurality of piezoelectric sensors in the frame.

For example, it is assumed that the sensing unit 110 detects pressure at intervals of 1/T second. In this case, the sensing unit 110 may sense pressure through a plurality of piezoelectric sensors every 1/T second and sequentially output frames including a plurality of sensed pressure data (output T frames per second). .

That is, as shown in FIG. 4 , the sensing unit 110 outputs pressure data (a ₁₁ ⁽¹⁾ , a ₁₂ ⁽¹⁾ ,..., a _mn ⁽¹⁾ detected by a plurality of piezoelectric sensors in 1/T sec. ), the pressure data detected by the plurality of piezoelectric sensors in 2/T seconds (a ₁₁ ⁽²⁾ , _{a 12} ⁽²⁾ ,..., a _mn ⁽²⁾ ) The pressure data (a ₁₁ ⁽³⁾ , a ₁₂ ( ³⁾ ,..., a _mn ⁽³⁾ ) detected by the plurality of piezoelectric sensors in the second frame (F ₂ ) including The third frame (F ₃ ), including ..., can be sequentially output.

In this way, the sensing unit 110 includes pressure data (a ₁₁ ^(M) , a ₁₂ ^(M) ,..., a _mn ^(M) ) sensed by a plurality of piezoelectric sensors at M/T seconds. The Mth frame (F _M ) may be output.

Meanwhile, in FIG. 4 , the subscript of the pressure data indicates the position of the piezoelectric sensor.

That is, in the matrix, the pressure data sensed by the piezoelectric sensor disposed in the first column of the first row is a ₁₁ , and the pressure data sensed by the piezoelectric sensor disposed in the second column of the first row is a ₁₂ . .., the pressure data detected by the piezoelectric sensor disposed in the n-th column of the m-th row is a _mn .

Meanwhile, in the above example, it has been described that the pressure is sensed at T Hz, but this is only an example, and the sensing unit 110 may sense the pressure within a range of 1 to T Hz.

The processor 120 controls overall operations of the electronic device 100 . To this end, the processor 120 performs operations or data related to control of other elements included in the electronic device 100, including a central processing unit (CPU), random access memory (RAM), and read only memory (ROM). processing can be executed.

First, the processor 120 may control the sensing unit 110 to sense the pressure applied to the seat of the mobile body by the driver and receive pressure data output from the sensing unit 110 .

In this case, the processor 120 may control the sensing unit 110 to sense the pressure at predetermined time intervals.

Accordingly, the sensing unit 110 senses the pressure applied to the seat of the moving object by the driver at predetermined time intervals through a plurality of piezoelectric sensors, and sequentially processes the plurality of pressure data sensed at each point in frame units. (120) can be output.

Meanwhile, the processor 120 determines the driver's condition based on frequency characteristics of changes in pressure detected by the sensing unit 110 .

Specifically, the processor 120 may determine the driver's condition by calculating a difference in magnitude of the detected pressure at a predetermined time interval and analyzing a frequency component of the calculated difference values.

Here, the driver's state is a sleepy state (ie, the driver is asleep while driving the moving object), a driving state (ie, moving the foot to step on the brake pedal and the accelerator pedal, or the hand to operate the steering wheel). moving) and a state of moving the body for a purpose other than driving (eg, moving to operate navigation while driving).

To this end, when pressure data is input from the sensing unit 110 on a frame-by-frame basis, the processor 120 may calculate a difference between the pressure data between frames.

Specifically, the processor 120 may calculate a difference value between pressure data included in a frame and a previous frame of the corresponding frame. That is, the processor 120 may calculate a difference value between the pressure data included in the M-th frame F _M and the M−1-th frame F _M .

In this case, the processor 120 may calculate a difference value between the pressure data sensed by the piezoelectric sensor at the same location, and calculate a difference value between the pressure data included in the frame and the previous frame.

For example, as shown in FIG. 5 , the processor 120 subtracts the pressure data included in the first frame F ₁ from the pressure data included in the second frame F ₂ , and obtains the second frame F ₂ . A differential frame F _{1 '} (= F ₂ -F ₁ ) including a difference value between the first frames F 1 may be calculated.

In this case, the processor 120 calculates a difference value between pressure data sensed by piezoelectric sensors at the same position in each frame.

That is, as shown in FIG. 5 , the second frame F ₂ includes pressure data (a ₁₁ ⁽²⁾ , a ₁₂ ⁽²⁾ ,..., a _mn ⁽²⁾ ) detected by m×n piezoelectric sensors. , and the first frame F ₁ includes pressure data (a ₁₁ ⁽¹⁾ , a ₁₂ ⁽¹⁾ ,..., a _mn ⁽¹⁾ ) detected by m×n piezoelectric sensors. , a plurality of elements constituting the difference frame F ₁ ' are (a ₁₁ ⁽²⁾ -a ₁₁ ⁽¹⁾ , a ₁₂ ⁽²⁾ -a ₁₂ ⁽¹⁾ ,..., a _mn ⁽²⁾ -a _mn ⁽¹⁾ Same as ).

In addition, the processor 120 subtracts the pressure data included in the second frame (F ₂ ) from the pressure data included in the third frame (F ₃ ), and calculates the third frame (F ₃ ) and the second frame (F ₂ ). A difference frame F ₂ ′ (=F ₃ -F ₂ ) including a difference value between the frames may be calculated.

In this case, the plurality of elements constituting the difference frame F ₂ ' are (a ₁₁ ⁽³⁾ -a ₁₁ ⁽²⁾ , a ₁₂ ⁽³⁾ -a ₁₂ ⁽²⁾ ,..., a _mn ⁽³⁾ - Same as a _mn ⁽²⁾ ).

In this way, the processor 120 subtracts the pressure data included in the M-1th frame (F _{M -1} ) from the pressure data included in the M-th frame (F M ), and the M-th frame (F _M ) and M- A difference frame F _N '(= F _M -F M- ₁ ) including a difference value between the first frames F _M-1 may be calculated.

In this case, the plurality of elements constituting the difference frame F _N 'are (a ₁₁ ^(M) -a ₁₁ ^(M-1) , a ₁₂ ^(M) -a ₁₂ ^(M-1) ,..., a _mn ^(M) -a _mn ^(M-1) ).

Meanwhile, calculating the difference value between the frames in this way is to remove the noise of the low frequency component and more accurately detect the state according to the driver's movement.

Thereafter, the processor 120 may take absolute values of difference values of pressure data between frames and sum them.

That is, the processor 120 may take an absolute value for each of a plurality of elements constituting the difference frame and sum the plurality of elements of which the absolute values are taken. In this case, the summed value can be regarded as a representative value for the difference frame.

의 복수의 요소를 합산하여, 차이 프레임 F₁'에 대한 대표값 P₁(=

+

+...+

)을 산출할 수 있다. For example, as shown in FIG. 6 , the processor 120 takes an absolute value in the difference frame F ₁ 'and the difference frame in which the absolute value is taken.

By summing a plurality of elements of , the representative value for the difference frame F ₁ 'P ₁ (=

+

+...+

) can be calculated.

의 복수의 요소를 합산하여, 차이 프레임 F₂'에 대한 대표값 P₂(=

+

+...+

)을 산출할 수 있다. Further, the processor 120 takes an absolute value in the difference frame F ₂ ′ and the difference frame in which the absolute value is taken.

By summing a plurality of elements of , the representative value for the difference frame F ₂ 'P ₂ (=

+

+...+

) can be calculated.

의 복수의 요소를 합산하여, 차이 프레임 F_N'에 대한 대표값 P_N(=

+

+...+

)을 산출할 수 있다.As such, processor 120 takes an absolute value in the difference frame F _N 'and the difference frame in which the absolute value is taken.

By summing a plurality of elements of , the representative value P _N for the difference frame F _N '(=

+

+...+

) can be calculated.

That is, the processor 120 may calculate the representative value P _N based on Equation 1 below.

On the other hand, in the above example, it has been described that the absolute value is taken for the difference value of the pressure data between the frames, but this is only an example, and the difference value of the pressure data between the frames is squared or RMS (Root Mean Square) can be applied. may be

Then, the processor 120 may square the representative value P _N for the difference frame. In this case, a value obtained by squaring the representative value P _N can be regarded as power (or energy) for the representative value P _N . Meanwhile, the reason for squaring the representative value P _N in this way is to clarify the difference between signals in order to determine the driver's condition.

Meanwhile, when the representative value P _N is squared and represented as a graph, it can be expressed as a signal (or waveform) as shown in FIG. 7 , for example. In FIG. 7 , the x-axis represents time, and the y-axis represents the magnitude of a value (ie, power) obtained by squaring the representative value P _N .

In this case, the processor 120 may determine the driver's state based on the signal representing the power for the representative value P _N .

On the other hand, in that the representative value is the sum of a plurality of elements of the difference frame, the representative value can be seen as the difference value of the pressure data between the frames, and eventually, the difference in the magnitude of the pressure between the frames, that is, the difference in the pressure of the seat over time It can be seen as corresponding to the difference in the magnitude of the applied pressure. Therefore, analyzing a signal expressed based on the representative value can be regarded as analyzing a signal representing a difference in magnitude of pressure.

First, in that a driver's body movement for a purpose other than driving corresponds to an arbitrary motion, the motion is large and, accordingly, the pressure applied to the seat changes over time.

Accordingly, as shown in ① of FIG. 8, the processor 120, when the signal representing the power for the representative value P _N has a magnitude equal to or greater than the predetermined threshold value (E _th ), the driver is not driving in the time section of the corresponding signal. can be judged as moving the body for other purposes.

On the other hand, in the case of a time interval in which the magnitude of the signal representing the power for the representative value P _N is small (eg, an interval in which the magnitude of the signal representing the power is close to 0), there is little difference in the magnitude of the pressure applied to the seat by the driver Since it is a time interval, it can be regarded that there is no movement of the driver in the corresponding time interval.

Accordingly, as in ② of FIG. 8 , the processor 120 may determine that there is almost no movement of the driver in the corresponding time section.

On the other hand, when the driver moves to drive the moving object (ie, driving motion) and when the driver moves while sleeping (ie, drowsiness motion), there is some movement, but these movements are not large.

Accordingly, when the driver drives or sleeps, the power based on the change in the amount of pressure applied to the seat is smaller than the predetermined threshold value, but has a larger value than when the driver is not moving.

However, in these cases, since the size of the movement is similar, it is not easy to distinguish them only by the change in the size of the pressure over time.

Accordingly, in order to determine the driver's condition, the processor 120 may frequency-convert a signal representing power with respect to the representative value P _N and analyze a frequency component of the signal. In this case, the processor 120 may frequency-convert the signal representing the power of the representative value P _N in real time and analyze the frequency component of the signal.

Specifically, the processor 120 may frequency-convert signals included in each preset time interval to analyze frequency components of signals included in each time interval.

Here, the preset time interval may be the same as the time interval at which the sensing unit 110 senses the pressure.

For example, when the processor 120 senses the pressure at 1/T second intervals through the sensing unit 110, each signal included in the time interval of 1/T second interval, that is, 0 second to 1/T second The signal at , the signal from 1/T sec to 2/T sec, the signal from 2/T sec to 3/T sec, ... is Fourier-transformed, and the signal included in the time interval of 1/T sec interval The frequency components for each can be analyzed.

Also, the processor 120 may determine the driver's condition based on the frequency component of each signal.

Specifically, the driver performs movements such as manipulating a steering wheel with his hands and stepping on a pedal with his feet in order to drive a moving object. In this way, since the driver moves in various ways using various muscles of the body when driving the moving object, the signal based on the change in the magnitude of the pressure applied to the seat while the driver is driving the moving object has various frequency components, that is, low frequency It may consist of high-frequency components in the components.

On the other hand, since the driver performs a simple movement such as tilting his head forward gradually or momentarily when he is asleep, the change in the amount of pressure applied to the seat while the driver is dozing may consist of a frequency component within a specific range. Here, a frequency component within a specific range may be a low frequency component.

Accordingly, as a result of analyzing the frequency components in each time interval, the processor 120 determines that the driver is driving the moving object in the time interval in which the high frequency component and the low frequency component are detected, and the time interval in which only the low frequency component is detected. , it can be determined that the driver is dozing.

For example, in the case of FIG. 8 , a time interval ③ having a frequency component of 1 to 25 Hz is when the driver is driving a moving object, and a time interval ④ having a frequency component of 1 to 5 Hz can be considered that the driver is dozing off.

In this way, the processor 120 determines that the driver is in a drowsy state in a time period in which the frequency component for the difference in pressure is composed of low-frequency components, and the frequency component for the difference in pressure is in a time section composed of high-frequency and low-frequency components. In , it may be determined that the driver is in a state of driving the moving object.

Specifically, the processor 120 represents the difference values calculated at predetermined time intervals as signals on the time axis, frequency-converts the signal in each predetermined time interval, determines the frequency component of the signal in each time interval, It may be determined that the driver is in a drowsy state in a time interval in which only low frequency components below a predetermined threshold are detected. Further, the processor 120 may determine that the driver is in a state of driving the moving object in a time interval in which a high frequency component and a low frequency component equal to or less than a preset threshold are detected.

As such, when the sensing unit 110 detects pressure through the plurality of piezoelectric sensors, the processor 120 calculates a change in magnitude of the detected pressure at a preset time interval in real time, and calculates the frequency of the difference in magnitude of the calculated pressure. In terms of analyzing the components, it is possible to determine the driver's condition in real time.

On the other hand, in the above example, it has been described that the low frequency component is 1 to 5 Hz and the high frequency component is 5 to 25 Hz, but this is only an example.

Specifically, the processor 120 determines that the driver is driving a moving object in a time interval in which a low-frequency component of 0.5 to 5 Hz and a high-frequency component of 5 to 50 Hz are both detected, and only the low-frequency component of 0.5 to 5 Hz is detected. In the section, it may be determined that the driver is dozing.

In addition, when determining the driver's condition, the driver's condition can be determined regardless of the driver's weight in that the difference in the amount of pressure applied to the seat is used.

Meanwhile, as described above, the processor 120 may determine that the driver is in a drowsy state in a time interval in which the frequency component for the pressure difference is a low-frequency component.

Here, the time period composed of low-frequency components may include not only a time period composed of only low-frequency components, but also a time period in which high-frequency components exist to some extent together with low-frequency components, but the size of high-frequency components is very small.

Meanwhile, in the above example, it has been described that the driver's condition is determined using frequency characteristics, but this is only an example.

For example, the processor 120 may determine the driver's condition based on a change in pressure calculated for a specific time period rather than a change in pressure calculated in real time.

That is, the processor 120 may determine the driver's condition based on a signal indicating power for a representative value for a specific time period.

Specifically, the processor 120 may detect a peak in a signal representing power for a representative value during a specific time interval. In this case, a peak detection algorithm may be used for peak detection.

Further, the processor 120 may determine a time period in which the driver does not move and a time period in which the driver moves the body for a purpose other than driving, among a specific time period, based on the magnitude of power for the representative value.

That is, the processor 120 determines that the driver does not move during a specific time interval in which the signal representing the power of the representative value is small, and determines that the driver is not moving in a time interval having a magnitude greater than or equal to the threshold value. It can be judged as moving the body for another purpose.

For example, in the case of FIG. 8 , the processor 120 generates a time interval in which the magnitude of a signal representing power for a representative value is equal to or less than a threshold value P _t , that is, a time interval in which the magnitude of the signal is almost zero. In ②, it is determined that there is no movement of the driver, and in the time period having a magnitude greater than a predetermined threshold value (E _th ), it is determined that the driver is moving the body, that is, the driver is moving the body for a purpose other than driving. can

Meanwhile, the processor 120 may calculate a time interval between detected peaks. In this case, a peak-to-peak detection algorithm may be used. In addition, the processor 120 may calculate the area of the peak signal by integrating the peak signal corresponding to the peak.

Here, the fact that the time interval with other peaks is relatively small and the area of the peak signal is relatively large means that the driver made various movements relatively slowly, the time interval with other peaks is relatively large, and the area of the peak signal is relatively large. The relatively small area means that the driver made a simple movement instantaneously.

Accordingly, the processor 120 determines that the driver is driving the moving object in a time interval in which a peak signal having a small time interval with another peak and a large area of the peak signal exists among a specific time interval, and It may be determined that the driver is drowsy in a time interval in which a peak signal having a large time interval and a small area of the peak signal exists.

In this way, in the above-described method, in order to determine the driver's condition, the interval between peaks is considered, and thus, it is based on a change in pressure calculated for a specific time period rather than real time.

Meanwhile, the processor 120 may provide feedback corresponding to the driver's condition.

Specifically, if it is determined that the driver is in a dozing state, the processor 120 may reduce the speed of the moving object.

For example, if it is determined that the driver is asleep, the processor 120 may gradually reduce the speed of the moving object to a preset speed or stop the moving object.

Also, if it is determined that the driver is in a dozing state, the processor 120 may output a preset audio.

For example, if it is determined that the driver is in a drowsy state, the processor 120 may output a warning sound of a specific sound through a speaker provided in the moving object or output a voice to guide that the driver is currently drowsy.

Also, if it is determined that the driver is in a dozing state, the processor 120 may display specific information on a display (not shown).

For example, if it is determined that the driver is in a drowsy state, the processor 120 may output a warning message to the front window of the moving object or a message to inform that the driver is currently drowsy.

For example, if it is determined that the driver is in a dozing state, the processor 120 may automatically change the driver's driving mode to the autonomous driving mode.

Meanwhile, the processor 120 may determine the direction of the driver's gaze based on the change in the amount of pressure.

To this end, the sensing unit 110 may include a plurality of piezoelectric sensors disposed on the left side and a plurality of piezoelectric sensors disposed on the right side with respect to the center of the matrix. For example, as shown in FIG. 9 , m×n/2 piezoelectric sensors may be respectively disposed on the left and right sides of the center of the matrix.

In this case, the plurality of piezoelectric sensors disposed on the left and right sides detect the pressure applied by the driver at predetermined time intervals, and sequentially output the plurality of pressure data sensed at each point to the processor 120 in frame units can do.

Also, the processor 120 may determine the direction of the driver's gaze based on the change in the magnitude of the pressure sensed by the left piezoelectric sensor and the change in the magnitude of the pressure sensed by the right piezoelectric sensor.

To this end, the processor 120 may calculate a change in magnitude of pressure detected by the left piezoelectric sensor and a change in magnitude of pressure detected by the right piezoelectric sensor.

In detail, when pressure data is input frame by frame from the left piezoelectric sensor, the processor 120 may calculate a difference value between the pressure data included in the frame and the previous frame. Also, when pressure data is input frame by frame from the right piezoelectric sensor, the processor 120 may calculate a difference value between the pressure data included in the frame and the previous frame.

Here, since the method for calculating the pressure data difference value between frames is the same as the method described with reference to FIGS. 4 and 5 with only a difference in the number of pressure sensors, a detailed description thereof will be omitted.

For example, as shown in FIG. 10 , the plurality of piezoelectric sensors on the left transmit pressure data (l ₁₁ ^(M) , l ₁₂ ^(M) ,..., l _{mn /2} ^(M) ) sensed at M/T seconds. It is possible to output the M-th frame (F _L,M ) including, and the processor 120 may output the M-1-th frame (F _{L,M -1 ) from the pressure data included in the M-th frame (F L,M} ). Difference frame F _{L , N} '(=F _{L, M} -F _L,M-1 ) can be calculated.

In this case, the plurality of elements constituting the difference frame F _L,N ′ are (l ₁₁ ^(M) -l ₁₁ ^(M-1) , l ₁₂ ^(M) -l ₁₂ ^(M-1) ,..., Same as l _{mn /2} ^(M) -l _{mn /2} ^(M-1) ).

In addition, as shown in FIG. 11, the plurality of piezoelectric sensors on the right side include pressure data (r ₁₁ ^(M) , r ₁₂ ^(M) ,..., r _{mn /2} ^(M) ) detected at M/T seconds. It is possible to output an M-th frame (F _{R ,M} ) to be performed, and the processor 120 may output the M-th frame (F _{R ,M} -1 ) from the pressure data included in the M-th frame (F _{R ,} M ). Difference _{frame F R , N ' ( = FR , M} -F _{R ,M-1} ) can be calculated.

In this case, the plurality of elements constituting the difference frame F _{R ,N} ' are (r ₁₁ ^(M) -r ₁₁ ^(M-1) , r ₁₂ ^(M) -r ₁₂ ^(M-1) ,..., Same as r _{mn /2} ^(M) -r _{mn /2} ^(M-1) ).

Thereafter, the processor 120 may take absolute values of difference values of pressure data between frames and sum them.

That is, the processor 120 may take an absolute value for each of a plurality of elements constituting the difference frame and sum the plurality of elements of which the absolute values are taken. In this case, the summed value can be regarded as a representative value (or pressure change value) for the difference frame.

의 복수의 요소를 합산하여, 차이 프레임 F_L,N'에 대한 대표값 P_{L ,N}(=

+

+...+

)을 산출할 수 있다.For example, as shown in FIG. 12, the processor 120 takes an absolute value in the difference frame F _L,N 'and the difference frame in which the absolute value is taken.

By summing a plurality of elements of , the representative value for the difference frame F _L,N 'P _{L ,N} (=

+

+...+

) can be calculated.

의 복수의 요소를 합산하여, 차이 프레임 F_{R ,N}'에 대한 대표값 P_R,N(=

+

+...+

)을 산출할 수 있다.In addition, the processor 120 takes an absolute value in the difference frame F _{R ,N} 'and the difference frame in which the absolute value is taken.

By summing a plurality of elements of , the representative value P _R,N for the difference frame F _{R ,N} '(=

+

+...+

) can be calculated.

That is, the processor 120 may calculate representative values P _{L ,N} , and P _R,N based on Equations 2 and 3 below.

Then, the processor 120 may calculate a difference between a representative value for the left difference frame and a representative value for the right difference frame.

Specifically, the processor 120 subtracts the representative value P _{R ,N} of the difference frame on the right from the representative value P L _,N of the difference frame on the left, and subtracts the difference between the representative values (ie, P _{L ,N} - P _R,N ) can be calculated.

On the other hand, if the difference value P _{L ,N} -P _R,N between the representative values is graphed, it may be represented as a signal as shown in FIG. 13, for example. In FIG. 13, the x-axis represents time, and the y-axis represents the magnitude of the difference between the representative values.

In this case, the processor 120 may determine the direction of the driver's gaze based on the signal representing the difference value P _{L ,N} -P _R,N between the representative values.

Specifically, when the driver sits in the seat and looks to the left, relatively greater pressure is applied to the left side of the seat than to the right side of the seat where the driver sits, and conversely, when the driver sits on the seat and looks to the right, A relatively greater pressure is applied to the right side of the seat than to the left side.

Accordingly, the processor 120 determines that the driver's gaze direction is toward the left in a time interval in which the difference between the representative values P _{L ,N} -P _R,N is a positive number, and the difference between the representative values P _{L ,N} In the time section in which -P _R,N is a negative number, it can be determined that the direction of the driver's gaze is directed to the right.

For example, in the case of FIG. 14 , the processor 120 determines that the driver's gaze direction is toward the left in a time period ① in which the difference value P _{L ,N} -P _R,N between the representative values is a positive number, and between the representative values In the time interval ② where the difference value P _{L ,N} -P _R,N of is a negative number, it can be determined that the driver's gaze direction is directed to the right.

In addition, the processor 120 may determine the degree to which the driver's gaze direction is directed to the left or right with respect to the front according to the size of the difference value P _{L ,N} -P _R,N between the representative values.

Specifically, as the driver seated in the seat turns his/her eyes more to the left, more and more pressure is applied to the left area of the seat, and as the driver seated in the seat turns more and more eyes to the right, more and more pressure is applied to the right area of the seat. will be applied to

Accordingly, the processor 120 determines that, when the magnitude of the difference value P _{L ,N} -P _R,N between the representative values is a positive number, the larger the magnitude of the difference value, the more leftward the driver's gaze direction is relative to the front. It can be judged to be directed towards . In addition, the processor 120 determines that, when the magnitude of the difference value P _{L ,N} -P _R,N between the representative values is a negative number, the larger the difference value is, the more right the driver's gaze direction is relative to the front. It can be judged to be directed towards .

Meanwhile, in the above example, it has been described that the representative value P _{R ,N} of the difference frame on the right is subtracted from the representative value P _{L ,N} of the difference frame on the left, but this is only an example.

That is, the processor 120 may subtract the representative value P _{L ,N} of the difference frame on the left from the representative value P _{R ,N} of the difference frame on the right.

In this case, the processor 120 determines that the driver's gaze direction is toward the right in a time interval in which the difference between the representative values P _R,N -P _{L ,N} is a positive number, and the difference between the representative values P _R,N In the time interval in which -P _{L ,N} is a negative number, it can be determined that the driver's gaze direction is toward the left.

Meanwhile, the processor 120 may output information according to the direction of the driver's gaze.

For example, the processor 120 may display information on the left area of the windshield relative to the seat when the driver's gaze direction is to the left, and display information on the left area of the windshield relative to the seat when the driver's gaze direction is to the right. Information can be displayed in the right area of .

In this case, the processor 120 may determine a location where information is output according to the degree of the driver's gaze direction.

For example, the processor 120 may display information in an area to the left as the driver's gaze direction moves to the left, and display information in an area to the right as the driver's gaze direction moves to the right. can

Also, the processor 120 may provide various feedbacks according to the degree of the driver's gaze direction.

For example, when the driver's line of sight is directed to the left or right, the processor 120 outputs a message to the window of the moving object to warn distracted driving or induce forward gaze, or to sound a warning sound to inform the driver through a speaker. can be printed out.

In addition, the processor 120 determines whether an object exists around the moving object through an additional sensor provided on the moving object, such as a camera or a proximity sensor, and in a state where the object exists around the moving object, the driver turns his or her head to look left or right. If you see it, you can provide feedback corresponding to it.

For example, the processor 120 activates a camera or the like for capturing a blind spot when the driver turns his or her head to look left or right while an object exists around the moving object, or activates a blind spot through the activated camera. When it is determined that an object exists in the object, a notification sound for the object may be output through a speaker.

Also, the processor 120 may provide feedback corresponding to the direction of the driver's line of sight and the moving direction of the moving object.

For example, the processor 120 may detect that the moving object moves to the left lane and detect that the driver's line of sight is directed to the right. That is, when the driver's gaze is directed in a direction opposite to the moving direction of the vehicle and it is determined that the possibility of an accident is high, a corresponding notification sound may be output through a speaker or displayed on the window of the moving object.

Meanwhile, in the above-described example, the processor 120 determines the direction of the driver's gaze through the difference between the representative value for the difference frame on the left and the representative value for the difference frame on the right, but this is only an example.

As another example, the processor 120 compares the pressure detected by the left piezoelectric sensor with the pressure detected by the right piezoelectric sensor, determines whether the center of gravity of the driver exists in the left area or the right area, and determines whether the center of gravity is Depending on the existing region, the driver's gaze direction may be determined.

Specifically, the processor 120 may determine that the center of gravity of the driver is present in the left region when the magnitude of the pressure detected by the left piezoelectric sensor is relatively greater than the magnitude of the pressure detected by the right piezoelectric sensor. In this case, the processor 120 may determine that the driver's gaze direction is toward the left.

In addition, the processor 120 may determine that the center of gravity of the driver is present in the right region when the magnitude of the pressure detected by the right piezoelectric sensor is relatively greater than the magnitude of the pressure detected by the left piezoelectric sensor. In this case, the processor 120 may determine that the driver's gaze direction is toward the right.

Meanwhile, the processor 120 may determine whether the center of gravity of the driver is biased to one side while the driver moves with a large motion based on the pressure data, and provide feedback corresponding thereto.

Specifically, when the driver moves in a large motion and the driver's center of gravity is biased to one side at the same time, the change over time in the pressure applied to the seat is large, and the pressure applied to a specific area is relatively higher than the pressure applied to other areas. big. Accordingly, the processor 120 may determine the driver's condition based on the pressure data and provide feedback corresponding thereto.

For example, when a driver tilts his body to the right to open a console box for a purpose other than driving, the pressure applied to the seat varies greatly over time, and the driver's center of gravity exists on the right side. .

In this case, the processor 120 may output a message to warn distracted driving on the window of the moving object or output a warning sound for the message through a speaker.

In addition, the processor 120 determines whether the driver does not move significantly based on the pressure data, but the center of gravity is maintained for a predetermined time without returning to the original state after being skewed to one side, and providing feedback corresponding thereto. can also provide.

Specifically, when the driver does not move much, but the center of gravity is on one side for a predetermined time, the change over time in the pressure applied to the seat is small, and the pressure applied to a specific area during the predetermined time is different from the other area. relative to the applied pressure. Accordingly, the processor 120 may determine the driver's condition based on the pressure data and provide feedback corresponding thereto.

For example, when the driver turns his head and has a conversation with a person sitting in the right seat, or is looking at a navigation system installed on the right side of the driver's seat, the change in pressure applied to the seat is not large, but the driver's center of gravity shifts for a certain period of time. while it exists in the right area.

In this case, the processor 120 may output a message for warning the forward gaze to the window of the moving object or output a warning sound for the message through the speaker.

In addition, the processor 120 may determine whether or not the center of gravity of the driver currently driving is maintained for a predetermined time while moving to one side based on the pressure data, and provide feedback corresponding thereto.

Specifically, when the driver maintains the center of gravity moving to one side for a predetermined time while driving, the frequency of the change in the amount of pressure applied to the seat has a low frequency component and a high frequency component, and in this state, the preset During time, the pressure applied to a specific area is relatively greater than the pressure applied to other areas. Accordingly, the processor 120 may determine the driver's condition based on the pressure data and provide feedback corresponding thereto.

For example, when a driver holds a mobile phone while driving and makes a call, the center of gravity is shifted to one side for a certain period of time or more.

In this case, the processor 120 may output a message to warn of manipulation of the mobile phone to the window of the moving object or output a warning sound for the message through a speaker.

Meanwhile, pressure applied to the seat may be used to determine whether the mobile phone is operated, but in addition to this, the processor 120 additionally determines whether the driver's voice is received through a microphone provided in the moving body, thereby determining whether the driver's voice is received. When the center of gravity is shifted and the driver's voice is received through the microphone, it may be determined that the driver is holding a mobile phone while driving and making a call.

Meanwhile, the processor 120 divides the plurality of piezoelectric sensors arranged in a matrix form into four regions (ie, an upper right region, an upper left region, a lower right region, and a lower left region based on the center), and in each region An area where the center of gravity of the driver is present may be determined among the four areas using the difference value of the detected pressure.

In this case, the processor 120 may determine an area where the magnitude of the detected pressure is relatively greater than other areas among the plurality of areas as the area where the center of gravity of the driver exists.

Accordingly, the processor 120 may provide various feedbacks according to the area where the center of gravity of the driver exists.

For example, when a driver picks up an object that has fallen under his/her feet, the driver's center of gravity exists in the upper region.

Accordingly, when the center of gravity of the driver is present in the upper region, the processor 120 determines that the driver takes an action of picking up an object that has fallen under his feet, and sends a message to warn distracted driving or induce forward gaze to the moving object. It can be output on the window of the window, or a warning sound can be output through the speaker.

As another example, when the driver turns his head to the rear left or right to talk to a person sitting in the rear left or right seat, or picks up an object in the rear seat, the driver's center of gravity is in the lower left or lower right area. exist in the realm.

Accordingly, when the center of gravity of the driver is present in the lower left or lower right region, the processor 120 determines that the driver is talking to a person sitting in the back seat or taking an action of picking up an object in the back seat, and distracted driving. A message for warning or inducing forward gaze may be output on the window of the moving object, or a warning sound may be output therefor.

15 is a block diagram showing a detailed configuration of an electronic device according to an embodiment of the present invention.

Referring to FIG. 15 , the electronic device 100 may further include a camera 130, a display 140, a speaker 150, and a memory 160 in addition to the sensing unit 110 and the processor 120, and these Components may be controlled by processor 120 . Meanwhile, the components shown in FIG. 15 are just one embodiment, and new components may be added or at least one component may be deleted according to implementation examples.

The camera 130 is disposed on the moving body to photograph the driver. That is, the camera 130 may capture the driver's face, upper body, eyes, and the like. To this end, the camera 130 may be disposed on the upper end of the windshield of the mobile body.

In this case, the processor 120 may determine the driver's condition based on the image captured by the camera.

That is, the processor 120 may analyze the image captured by the camera to determine the degree of movement of the driver's face and upper body, whether the driver's eyes are closed, and determine whether the driver is in a drowsy state.

For example, the processor 120 may determine that the driver is in a dozing state when the driver's face and upper body are momentarily or gradually turned downward or the driver's eyes are closed.

In this way, the processor 120 may determine the driver's condition through the camera 130 provided in the moving body.

However, judging the driver's condition through photography may cause errors due to external factors. For example, when the driver is wearing glasses or sunglasses, it is not possible to accurately determine whether the user is in a drowsy state because the user's eyes cannot be accurately photographed.

As such, the processor 120 may determine the driver's condition based on the change in pressure detected by the sensing unit 110 when the driver's condition cannot be clearly determined from the photographed image.

However, this is only an example, and the processor 120 determines the user's condition based on the image of the driver, and separately determines the user's condition based on the magnitude of the pressure sensed by the sensing unit 110. may be Also, the processor 120 may determine the user's state using only one of these methods.

The display 140 may display driving information on the windshield of the mobile body. In this case, the display 140 may be a head-up display. The head-up display is a display capable of providing sea of cloud information or other information to the front of the driver while driving a vehicle or aircraft, that is, an area (eg, a windshield) that does not deviate from the driver's main line of sight.

In this case, the processor 120 may display various information through the display 140 .

For example, the processor 120 may display information (eg, navigation, speed, fuel amount, etc.) necessary for a driver to drive a moving object.

In particular, the processor 120 may control the display 140 to display information related to the user's status. That is, if it is determined that the user is dozing off, the processor 120 may output a warning message or a message for guiding that the driver is currently dozing off.

In this case, the processor 120 may display information in consideration of the direction of the user's gaze.

Specifically, the processor 120 may display information on the left area of the windshield with respect to the seat when the driver's gaze direction is to the left, and may display information on the left area of the windshield with respect to the seat when the driver's gaze direction is to the right. Information can be displayed in the right area.

The speaker 150 is provided in the moving body and outputs various audios. In particular, when it is determined that the driver is in a drowsy state, the processor 120 may output a warning sound of a specific sound through the speaker 150 or output a voice to guide that the driver is currently drowsy.

Meanwhile, the processor 120 may receive a command from other elements through a configuration such as a bus (not shown), decode the received command, and execute an operation or data processing according to the decoded command.

Also, the processor 120 may include a main processor and a sub-processor, and the sub-processor may be configured as a low-power processor. In this case, the main processor and the sub-processor may be implemented in the form of one chip or may be implemented as separate chips. In addition, the subprocessor may include a memory in the form of a buffer or a stack therein.

Meanwhile, the processor 120 may be implemented as at least one of a graphic processing unit (GPU), a central processing unit (CPU), and an application processor (AP), and may also be implemented as a single chip.

The memory 160 is received from the processor 120 or other components (eg, the sensing unit 110, the camera 130, the display 140, the speaker 150, etc.) or the processor 120 or other components. Can store instructions or data generated by Also, the memory 160 may include, for example, programming modules such as a kernel, middleware, an application programming interface (API), or an application. Each of the programming modules described above may be composed of software, firmware, hardware, or a combination of at least two of these.

Also, the memory 150 may store various information related to the operation of the electronic device 100 .

Meanwhile, the memory 150 may be implemented with various memories. For example, the memory may be implemented as an embedded memory. The built-in memory may include, for example, volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), etc.) or nonvolatile memory (eg, OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, NAND flash memory, NOR flash memory, etc.) there is. According to one embodiment, the embedded memory may take the form of a Solid State Drive (SSD). Also, the memory 150 may be implemented as an external memory. External memory is a flash drive, such as CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), or MemoryStick. can include

Meanwhile, the electronic device 100 may perform the above-described operation through interworking with a terminal device (not shown) such as a mobile phone or a tablet.

To this end, the electronic device 100 may further include a communication chip (not shown) for communication with a terminal device (not shown) such as Bluetooth or Wi-Fi.

For example, the electronic device 100 may transmit sensing data sensed through a piezoelectric sensor to a terminal device (not shown). In this case, the terminal device (not shown) may determine the driver's condition using the sensing data and transmit information about the driver's condition to the electronic device 100 . Accordingly, the electronic device 100 may perform an operation corresponding to the driver's condition.

16 is a flowchart illustrating a method for determining a driver's condition of an electronic device according to an embodiment of the present invention.

First, the pressure applied to the seat of the mobile body by the driver is sensed (S1610). In this case, the pressure may be sensed using a plurality of piezoelectric sensors arranged in a matrix form on the seat.

Then, the driver's condition is determined based on the frequency characteristic of the magnitude change of the sensed pressure (S1620).

Specifically, the driver's condition may be determined by calculating a difference in magnitude of the detected pressure at a predetermined time interval and analyzing a frequency component of the calculated difference values.

In this case, it may be determined that the driver is in a drowsy state in a time interval in which the frequency component for the pressure difference is a low-frequency component.

Specifically, the calculated difference values are represented as signals on the time axis, the signal at each predetermined time is Fourier transformed to determine the frequency component of the signal in each time interval, and the driver in the time interval in which only low-frequency components are detected Joe can be judged to be in a state.

Meanwhile, feedback corresponding to the driver's condition may be provided. In this case, if it is determined that the driver is in a drowsy state, the speed of the moving object may be reduced or a preset audio may be output.

Meanwhile, the embodiments of the invention and all functional operations described in this specification may be performed within a digital electronic circuit or within computer software, firmware, or hardware including the structures disclosed in this specification and equivalent structures thereof, or one of them. Any combination of the above can be practiced.

Meanwhile, a non-transitory computer readable medium may be provided in which a program for sequentially executing a method for determining a driver's condition of an electronic device according to the present invention is stored.

A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above may be stored and provided in non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although a bus is not shown in the above-described block diagram of an electronic device, communication between components in an electronic device may be performed through a bus. Also, the electronic device may further include a processor such as a CPU or a microprocessor that performs the various steps described above.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: electronic device 110: sensing unit
120: processor

Claims (14)

In electronic devices,
a sensing unit for sensing the pressure applied to the seat of the mobile body by the driver; and
A processor for determining the driver's condition based on the frequency characteristic of the change in pressure sensed by the sensing unit;
the processor,
Calculate the magnitude difference of the sensed pressure at a predetermined time interval, express the calculated difference values as a signal on the time axis, and frequency-convert the signal at each predetermined time to obtain a frequency component of the waveform at each predetermined time interval An electronic device that determines that the driver is in a drowsy state in a time interval in which only low frequency components below a predetermined threshold are detected. According to claim 1,
The sensing unit,
An electronic device comprising a plurality of piezoelectric sensors arranged in a matrix form on the seat. delete delete delete According to claim 1,
the processor,
An electronic device characterized in that for providing feedback corresponding to the driver's condition. According to claim 6,
the processor,
If it is determined that the driver is in a drowsy state, the electronic device is controlled to reduce the speed of the moving object or to output a preset audio. A method for determining a driver's condition of an electronic device,
sensing the pressure applied to the seat of the mobile body by the driver; and
Determining the driver's condition based on the frequency characteristic of the change in the sensed pressure;
The step of judging is
Calculate the magnitude difference of the sensed pressure at a predetermined time interval, express the calculated difference values as a signal on the time axis, and frequency-convert the signal at each predetermined time to obtain a frequency component of the waveform at each predetermined time interval and determining that the driver is in a drowsy state in a time interval in which only low-frequency components below a predetermined threshold are detected. According to claim 8,
The detecting step is
The method for determining the driver's condition, characterized in that the pressure is sensed using a plurality of piezoelectric sensors arranged in a matrix form on the seat. delete delete delete According to claim 8,
The method for determining a driver's state, further comprising providing feedback corresponding to the driver's state. According to claim 13,
The step of providing,
and if it is determined that the driver is in a drowsy state, the speed of the moving object is reduced or a preset audio is output.

Images