KR101573289B1 - Apparatus and method for recongnizing position using camera - Google Patents

Abstract

The present invention relates to a method of recognizing a feature included in a real image photographed in real time through a camera and estimating a position of the portable device by using the recognized positional information of the feature and the relative directional information between the feature and the portable device, And a portable device using the same.

Position measurement, image recognition, direction angle,

Description

TECHNICAL FIELD [0001] The present invention relates to a position measuring method and a portable terminal using the same. [0002]

The present invention relates to a method for estimating a position information of a portable device having a digital camera and a direction sensor, and a portable device using the same. More particularly, the present invention relates to a method and apparatus for recognizing a feature included in a real- And a method for estimating a position of a portable device using positional information, relative direction information between the feature and the portable device.

As a method for measuring the position of a portable device, a radio wave radar positioning method for acquiring position information by mounting a GPS receiver or obtaining a position using a radio wave of a mobile communication base station or a wireless LAN (WLAN) is generally known. In addition, when the portable device has a camera, the portable device may measure the position using a camera.

When the position is obtained by using the camera image recognition, the portable device recognizes the image of the feature or facility photographed by the built-in camera, and recognizes the position of the user as the position coordinate of the recognized background object or facility, As shown in FIG.

In the conventional method of recognizing a location of a camera captured by a camera image and determining the location of the portable device, the distance between the feature and the portable device is assumed to be the focal distance of the camera, . In this method, the position of the portable device is roughly obtained, and a method of obtaining the position precisely is required for the location-based service in the future.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a digital camera and a directional sensor that recognize a topographic object existing in an image by receiving image information of a real world, estimate the position of the portable device using the positional information of the recognized topographic object, A method for providing a location using the present invention is presented.

Another object of the present invention is to provide a method for providing accurate position information by correcting a position error of a GPS receiver and other position measuring apparatus mounted on a portable device using a camera image recognition and a direction sensor, and a portable device apparatus using the method do.

It is another object of the present invention to provide an improved navigation system and a service using a camera image by providing geographical information on an accurate topographical object using a camera, thereby providing a user with a benefit.

According to a first aspect of the present invention, there is provided a portable device having a function of providing location information, the device including: a camera for photographing a real image; A feature image DB for image recognition; A feature image recognition system that receives feature information captured from a camera and recognizes a feature existing in the feature through image recognition; A feature information DB storing feature location location information; A direction measurement sensor for acquiring a shot azimuth; A position calculation unit for calculating a position of the portable device; And a position determining unit for determining that the accuracy of the position calculation is finally sufficient.

As the direction measurement sensor, an electronic compass using geomagnetism may be used. The feature information database may be stored in a memory, located in a portable device, stored in an online server, and transmitted to a portable device via a wireless communication network It is possible.

The feature of the detailed configuration of the portable device having the position estimation function according to the present invention is that the real image information is a still image or a continuous image by a preview function.

Another feature of the detailed structure of the portable device having the position estimation function according to the present invention is that the memory is detachable.

Another feature of the portable device having the position estimation function according to the present invention is that the display device displays the information of the feature item by a vector method or a bitmap method.

As described above, by inputting the image information, recognizing the feature object existing in the image, estimating the position of the portable device by using the position information of the recognized feature object and the direction sensor, recognizing the landmark image, It can be said that it is a more precise method than the method of roughly determining, and it is possible to correct the position error by the existing position measuring methods.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

Hereinafter, the present invention will describe a technique for estimating a position of a terrain object and a position using a direction sensor in a portable device.

1 is a block diagram schematically illustrating an internal configuration of a portable terminal apparatus according to a preferred embodiment of the present invention.

1, the portable terminal device includes a camera 110 for capturing an image, an image database (DB) 111 for image recognition, and a direction sensor such as an electronic compass (e-compass) An orientation sensor unit 112 for obtaining absolute azimuth information directed by the camera 110 and an input device such as the camera 110, the image DB 111, and the direction sensor unit 112 are controlled and the positioning is completed A control unit 120 for controlling the entire system up to the time when the image is captured by the camera 110 using the image recognition technique based on the real image information captured by the camera 110 and the image DB 111, An image recognition unit 130 for recognizing the included features and a feature database 131 for obtaining position information from the DB of the main feature based on the image recognition result and the position coordinates of the recognized feature and the camera azimuth angle With built-in camera A position calculating unit 140 for obtaining the position of the mobile terminal and a position determining unit 150 for determining the position of the portable terminal based on the accuracy of the position obtained by the position calculating unit. Here, the GPS receiver 210 receiving the current position information in real time may be optionally included.

For example, the camera section 110 drives a solid-state image pickup device to obtain an image signal, processes it into a digital image signal, and temporarily stores the digital image signal in a memory. Then, the image recognition unit 130 recognizes the feature included in the real image using the image recognition technique, and searches the facility location information stored in the image DB 111 to extract the location information of the matching feature.

Here, as the image recognition technique, a variety of recognition techniques such as feature extraction comparison and contour extraction comparison can be applied. In this method, data consisting of feature points and contour points necessary for comparison are extracted from images of various types of features An image DB 111 is required. As a matter of fact, in the case of the feature extraction comparison method, the feature point data of each feature item in the stored feature image DB 111 is compared with the feature point data extracted from the image signal of the feature item currently illuminated in the camera, The recognition is performed by finding the highest matching rate among the registered feature data. Generally, since the comparison process requires a large amount of calculation, the speed and accuracy of the image recognition are increased by using the position information obtained from the GPS receiver in the portable device and the direction information obtained from the electronic compass sensor.

Specifically, if the position of the screen captured by the camera is limited to a certain range, the number of recognition candidate data in the feature database to be compared can be greatly reduced, thereby greatly increasing the image recognition speed and accuracy.

The process of estimating the location information of the current user based on the location information of the feature recognized by the image recognition is as follows.

FIG. 2 shows a screen on which an image photographed from a camera is displayed on a display according to an embodiment of the present invention.

2, two buildings A and B are located on the screen. If the building is recognized by performing recognition on the screen, the location information of each building can be obtained from the feature database. When the true north direction is defined as 0 degrees, the building A is positioned at -10 degrees with respect to the front direction of the portable device, and the building B is positioned at +10 degrees with respect to the front direction of the portable device. This orientation information can be obtained through an electronic compass.

FIG. 3 illustrates a method of determining a location of a current portable device using location information and direction information of two buildings according to an embodiment of the present invention.

FIG. 3 shows the principle of determining the location of the portable device using the location information and the direction information of each building. Looking at the angle when looking at a mobile device in each building, the mobile device is located in the direction of +170 degrees from the building A and in the direction of +190 degrees from the building B. If a straight line between building A and mobile device is a, and a straight line connecting building B and mobile device is b, the mobile device must be located on the intersection of a and b. The location can be obtained.

4 shows a method for determining the position of a portable device from four buildings.

When the location and direction information of a plurality of buildings is obtained as shown in FIG. 4, the location of the current portable device can be obtained by obtaining an intersection point of each line passing through the current position of the portable device at the position of each building. Assuming that buildings are well recognized, location accuracy increases as the number of buildings increases.

FIG. 5 illustrates a case where only one building is photographed according to another embodiment of the present invention.

In the case where only one building is included in the camera screen as in the case of FIG. 5, first, the building in the current screen is recognized and one straight line is obtained. Then, the portable device is rotated, Find a new straight line and find the intersection point where two or more straight lines intersect to find the current position of the mobile device. However, at this time, if a new straight line is obtained while rotating the portable device 180 degrees with respect to the first direction, the two straight lines are the same and the intersection point can not be obtained.

FIG. 7 is a flowchart illustrating a process of providing a geographic information according to the present invention.

First, the terminal captures images of features and facilities to be recognized using a built-in camera. At this time, the image may be a still image or a continuous image using a preview function of the terminal. The terminal recognizes the feature or facility included in the real image using the real image obtained using the image recognition unit (701). At this time, the recognized feature or facility is referred to as a recognizer A. At this time, the terminal retrieves the feature or facility information recognized by the image recognition unit from the database storing the location information of the main feature and facility, and obtains the corresponding location information (703). Meanwhile, the terminal acquires the azimuth angle information that the camera is aiming using a direction sensor such as an electronic compass (705).

The terminal obtains a straight line a extending from the recognizer A to the camera as shown in FIG. 3 using the obtained position coordinates and azimuth (707). Then, the terminal performs position calculation (709) in such a manner that an intersection of straight lines obtained up to the present stage is obtained. At this time, the terminal determines whether additional image recognition is necessary to obtain two or more straight lines required for positioning (711), repeats the above process for another feature or facility existing in the camera image, (707). Repetition of such a process is adjusted by the number of recognitions, the recognition threshold, or the number of features or facilities existing in the screen. On the other hand, if there is one feature or facility within the camera image, the terminal rotates until another feature or facility appears, and then performs the recognition process and the straight line calculation process. The position of the portable device equipped with the camera is finally determined using the straight lines obtained through the above process.

FIG. 8 is a flowchart showing an addition of a portion for using GPS position information in image recognition in a portable device having a built-in GPS receiver.

First, the terminal acquires position information of the GPS and transmits it to the image recognition process (801).

The terminal captures an image of a feature and a facility to be recognized using a built-in camera. At this time, the image may be a still image or a continuous image using a preview function of the terminal. In the image recognition process, the image DB of the feature or facility to be referred to is limited to the DB of the feature or facility existing around the position coordinates obtained from the GPS.

The terminal recognizes the feature or facility included in the real image using the real image obtained using the image recognition unit (803). At this time, the recognized feature or facility is referred to as a recognizer A. At this time, the terminal retrieves the feature or facility information recognized by the image recognition unit from the database storing the location information of the main feature and facility, and obtains the corresponding location information (805). On the other hand, the terminal acquires azimuth angle information that the camera is facing using a direction sensor such as an electronic compass (807).

The terminal obtains a straight line a extending from the recognizer A to the camera as shown in FIG. 3 using the obtained position coordinates and azimuth (809). Then, the terminal performs position calculation in a manner of obtaining an intersection point of the straight lines obtained up to the present stage (811). At this time, the terminal determines whether additional image recognition is required to obtain two or more straight lines required for positioning (813), repeats the above process for another feature or facility existing in the camera image, (809). Repetition of this process is adjusted by the number of recognition or recognition time preset by the user, or by the number of features or facilities existing in the screen. On the other hand, if there is one feature or facility within the camera image, the terminal rotates until another feature or facility appears, and then performs the recognition process and the straight line calculation process. The position of the portable device equipped with the camera is finally determined using the straight lines obtained through the above process.

Such a process can increase the recognition speed and accuracy as described in the description of the operation of the invention, and ultimately improve the position estimation performance through image recognition and can be applied as a function of correcting the position of the GPS.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Therefore, the scope of the present invention should not be limited by the illustrated embodiments, but should be determined by the scope of the appended claims and equivalents thereof.

1 is a block diagram schematically showing an internal configuration of a portable terminal apparatus according to a preferred embodiment of the present invention,

FIG. 2 is a diagram illustrating a screen on which an image photographed from a camera is displayed on a display according to an embodiment of the present invention,

3 is a diagram illustrating a method of determining a location of a current portable device using location information and direction information of two buildings according to an embodiment of the present invention;

4 shows a method of determining the position of a portable device from four buildings,

5 is a view illustrating a case where only one building is photographed according to another embodiment of the present invention.

6 is a diagram illustrating a method for determining the position of a portable device,

FIG. 7 is a flowchart illustrating a process of a geographic information providing method according to the present invention, and FIG.

FIG. 8 is a flowchart showing the addition of a portion for using GPS position information for image recognition in a portable device having a built-in GPS receiver.

Claims (15)

In a portable device, A camera for photographing the feature image; A direction measurement sensor for acquiring a photographing azimuth angle of the camera; An image processing unit for recognizing at least two pieces of feature information from the image photographed by the camera and for identifying the position information of the at least two pieces of feature information based on the feature database defining the position information of at least one piece of pre- A recognition unit; And And a position calculation unit for calculating a position of the portable device based on the taken azimuth angle and the positional information. The method according to claim 1, The photographing azimuth angle, Wherein the portable device is acquired using an electronic compass using geomagnetism. The method according to claim 1, The portable device includes: Further comprising a memory, And the memory includes the feature database. The method of claim 3, The memory comprising: Wherein the portable device is detachable. The method according to claim 1, The feature database includes: And is stored in an external on-line server and transmitted to a wireless communication network. The method according to claim 1, The image captured by the camera is, A still image, and a continuous image by a preview function. The method according to claim 1, Wherein the image recognizing unit comprises: And recognizes at least two pieces of feature information within the same image. The method according to claim 1, Wherein the image recognizing unit comprises: And recognizes at least one feature item in each of the plurality of images. The method according to claim 1, The position calculation unit calculates, Wherein the position of the portable device is obtained using an intersection of at least two straight lines connecting the recognized at least two features and the portable device. A method of operating a portable device, Obtaining an image; Measuring a captured azimuth angle with respect to the acquired image; Recognizing at least two features in the acquired image; Identifying location information of at least two recognized features based on a feature database defining location information for at least one feature stored in advance; And And calculating the position of the portable device based on the taken azimuth and the positional information. 11. The method of claim 10, The photographing azimuth angle, Characterized in that it is obtained using an electronic compass using geomagnetism. 11. The method of claim 10, The feature information database includes: And stored in an external on-line server and transmitted to a wireless communication network. 11. The method of claim 10, Wherein the image includes: A still image, and a continuous image by a preview function. 11. The method of claim 10, The operation of recognizing the feature may include: And recognizing two or more features in the same image or in a plurality of images. 11. The method of claim 10, Wherein the calculating of the position of the portable device based on the photographing azimuth and the position information comprises: And obtaining the position of the portable device using an intersection of at least two straight lines connecting the recognized at least two features and the portable device.

Images