KR101316316B1 - Apparatus and method for extracting the pupil using streo camera - Google Patents

Abstract

The present invention relates to an apparatus for extracting eyes using a stereo camera and a method thereof, and to obtain a first face image and a second face image through a stereo camera having an infrared light on one side, and to obtain the first and second face images. After extracting the driver's pupils through noise reduction process after calculating the difference image (coordinate difference image) by coordinate matching, the pupil extraction apparatus using a stereo camera that can extract the driver's pupils quickly and accurately in a small calculation process and It is intended to provide a method.
To this end, the present invention provides a device for extracting pupils using a stereo camera, comprising: a stereo camera for capturing a first face image having white eyes and a second face image having black eyes at the same exposure time, targeting the same driver; Coordinate matching means for performing coordinate matching such that the first face image and the second face image photographed by the stereo camera have the same parallax; And difference image calculating means for extracting white eyes by calculating a difference image between the first face image and the second face image matched to have the same parallax in the coordinate matching means.

Description

Eye Extraction Apparatus and Method Using Stereo Camera {APPARATUS AND METHOD FOR EXTRACTING THE PUPIL USING STREO CAMERA}

The present invention relates to an apparatus for extracting eyes using a stereo camera and a method thereof, and more particularly, to obtain a first image and a second image through a stereo camera having an infrared light located on one side, and to obtain the first image and the second image. The present invention relates to an apparatus and a method for extracting the pupil of a driver using a second image.

The system for recognizing the driver's gaze direction and promoting the driver's safety and convenience will be generalized.

In order to accurately recognize the driver's gaze direction, it is necessary to detect four factors such as the three-dimensional position and orientation of the driver's head and the three-dimensional position and orientation of the driver's pupil in the vehicle space.

A device that can detect these four elements simultaneously is a stereo camera. Monocular cameras can detect head and eye orientation, but cannot know the three-dimensional position of the head and pupil.ToF (Time of Flight) cameras can detect the three-dimensional position and orientation of the head, but the pupil's three-dimensional position. And direction cannot be detected.

Therefore, there is a need for a method that can extract the driver's eyes quickly and accurately with a small calculation process using a stereo camera.

In order to solve the above problems of the prior art, the present invention obtains a first face image and a second face image through a stereo camera having an infrared light on one side, and obtains the obtained first image and the second image. An apparatus and method for extracting pupils using a stereo camera that can extract a driver's pupils quickly and in a small amount of computation by calculating a difference image (coordinate difference image) by coordinate matching and then removing the driver's pupils through a noise removing process The purpose is to provide.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

The apparatus of the present invention for achieving the above object, in the pupil extraction apparatus using a stereo camera, the same face, the first face image having a white eye and the second face image having a black eye with the same exposure time for the same driver; Stereo camera to shoot each; Coordinate matching means for performing coordinate matching such that the first face image and the second face image photographed by the stereo camera have the same parallax; And difference image calculating means for extracting white eyes by calculating a difference image between the first face image and the second face image matched to have the same parallax in the coordinate matching means.

In addition, the method of the present invention for achieving the above object, in the pupil extraction method using a stereo camera, the stereo camera targets the same driver, the first face image having a white eye and the second face image having a black eye Photographing each with the same exposure time; Performing coordinate matching by coordinate matching means such that the first face image and the second face image have the same parallax; And extracting white eyes by calculating a difference image between the matched first face image and the second face image to have the same parallax.

As described above, the present invention obtains a first face image and a second face image through a stereo camera having an infrared light located on one side, and coordinates the acquired first image and the second image to obtain a difference image (differential image). ), And after extracting the driver's pupils through the noise removal process, it is possible to extract the driver's pupils quickly and accurately in a small calculation process.

In addition, the present invention has the effect of improving the driver's eye detection performance by extracting the driver's eyes quickly and accurately in a small calculation process.

1 is a configuration diagram of an embodiment of a pupil extraction apparatus using a stereo camera according to the present invention;
2 is a structural diagram of an embodiment of a stereo camera according to the present invention;
3 is a structural diagram of another embodiment of a stereo camera according to the present invention;
4 is an exemplary view showing a face image of a driver photographed through a stereo camera and white eyes extracted through the same according to the present invention;
5 is a diagram for explaining a coordinate matching method of a coordinate matching unit according to the present invention;
6 is a flowchart illustrating a method of extracting pupils using a stereo camera according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It can be easily carried out. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an embodiment of a pupil extraction apparatus using a stereo camera according to the present invention.

As shown in FIG. 1, the apparatus for extracting pupils using the stereo camera according to the present invention includes a stereo camera 10, a coordinate matching unit 20, a difference image calculating unit 30, a noise removing unit 40, and a pupil. And a position detector 50.

Referring to each of the above components, first, the stereo camera 10 is a camera for photographing the driver's face image through the lens located on one side and the lens located on the other side, respectively, at the same time point (same exposure time) Take a picture of each driver's face. At this time, the face image photographed through one lens and the face image photographed through the other lens are images captured at different positions, and minute parallax occurs.

The stereo camera 10 targets the same driver to photograph a face image having white eyes through one lens 11, and captures a face image with black eyes through the other lens 13. Position the infrared light 12 around the one lens as shown in FIG. 2. In this case, the larger the distance X between the other lens 13 and the infrared light 12 is greater than the distance Y between the infrared light 12 and the one lens 11, the brighter the one lens 11. The white eye is photographed on the other lens 13 with a darker black eye image, and as a result, the two images can be differentiated to obtain a brighter and clearer white eye image.

In addition, the infrared light 12 may be implemented in various forms as shown in FIG. 3 to obtain more brightness or to reduce cost.

In particular, as shown in (c) of FIG. 3, a sharper and brighter (white) pupil may be obtained when the white / black pupil image difference is maximized by maximizing the distance between the other lens 13 and the illumination.

The coordinate matching unit 20 performs coordinate matching so that the first face image and the second face image photographed by the stereo camera 10 have the same parallax. That is, as shown in FIG. 4, the coordinate matching unit 20 determines one of the first face image (a) having white eyes and the second face image (b) having black eyes, based on one of the other faces. The image is moved left or right by a predetermined pixel (eg 10px) to match the coordinates.

The difference image calculator 30 calculates a difference image between the first face image and the second face image matched by the coordinate matching unit 20 to have the same parallax. That is, the difference image calculating unit 30 subtracts the second face image (b) from the first face image (a) or subtracts the first face image (a) from the second face image (b) on a pixel-by-pixel basis. Extract.

The noise removing unit 40 removes noise except white eyes through generally known binarization and morphology operations. The white pupil from which the noise is removed is as shown in FIG.

The pupil position detection unit 50 performs stereo image processing, which is widely known, to detect the position of the pupil. That is, the pupil position detector 50 detects the pupil position using a disparity map.

Hereinafter, various coordinate matching methods of the coordinate matching unit 20 will be described in detail with reference to FIG. 5.

FIG. 5A illustrates stereoscopic camera operations (rectification, matching, and the like) to find an amount of change in the position of an object at a distance between a camera mounted within a two image and a normal distance due to the space structure in a vehicle, and the face portion images of the two images. This is a method of matching by moving so as to overlap.

5 (b) shows the two images by detecting a position of a face (or a unique feature part in the face such as an eye) in one image and then performing 'template matching' in the other image with the detected face. The difference between the positions of the faces is obtained and the difference is calculated based on the differences.

At this time, in order to calculate only the difference in translation (x, y displacement) between the two images (reduced amount of calculation), one lens and the other lens are mounted horizontally so that there is no relative rotation. In this case, since the baseline of the two lenses is not far, the same image exists only in the displacement difference, and if the correct alignment of the two lenses is impossible, image rectification may be performed in advance.

FIG. 5C illustrates a method of matching a face position in two images by assuming a specific plane in a typical driver's face position by converting the plane as a difference between a mounting position (and a rotation angle) between two cameras. to be. This can be done with a small amount of computation, but has a disadvantage that the matching error increases as the face position moves away from the plane assumed as the face position from above.

6 is a flowchart illustrating a method of extracting pupils using a stereo camera according to the present invention.

First, the stereo camera 10 photographs the first face image having white eyes and the second face image having black eyes at the same exposure time, targeting the same driver (601).

Thereafter, the coordinate matching unit 20 performs coordinate matching so that the first face image and the second face image have the same parallax (602).

Thereafter, the difference image calculating unit 30 extracts white eyes by calculating a difference image between the matched first face image and the second face image to have the same parallax (603).

In addition, the noise removing unit 40 removes noise from the calculated difference image.

In addition, the pupil position detector 50 performs stereo image processing to detect the pupil position.

Through this process, it is possible to extract the driver's eyes quickly and accurately with a small calculation process, and also improve the driver's gaze detection performance.

Meanwhile, the method of the present invention as described above can be written in a computer program. And the code and code segments constituting the program can be easily deduced by a computer programmer in the field. In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings.

10: stereo camera 20: coordinate matching unit
30: difference image calculation unit 40: noise removal unit
50: pupil position detection unit

Claims (7)

A stereo camera having infrared illumination around one side of the lens, for capturing a first face image having white pupils and a second face image having black pupils at the same exposure time for the same driver;
Coordinate matching means for performing coordinate matching such that the first face image and the second face image photographed by the stereo camera have the same parallax; And
Difference image calculating means for extracting white eyes by calculating a difference image between the first face image and the second face image matched to have the same parallax in the coordinate matching means
Eye extraction device using a stereo camera comprising a.
The method of claim 1,
Noise removing means for removing noise from the difference image calculated by the difference image calculating means
Eye extracting device using a stereo camera further comprising.
The method of claim 1,
Eye position detection means for detecting the position of the pupil by performing stereo image processing
Eye extracting device using a stereo camera further comprising.
delete A stereo camera having infrared illumination around one lens, photographing a first face image having white eyes and a second face image having black eyes at the same exposure time, targeting the same driver;
Performing coordinate matching by coordinate matching means such that the first face image and the second face image have the same parallax; And
Extracting white eyes by calculating a difference image between the first face image and the second face image matched to have the same parallax;
Eye extraction method using a stereo camera comprising a.
The method of claim 5, wherein
Noise removing means for removing noise from the calculated difference image
Eye extraction method using a stereo camera further comprising.
The method of claim 5, wherein
Eye position detecting means for performing stereo image processing to detect eye position
Eye extraction method using a stereo camera further comprising.

Images