KR100633231B1 - Portable terminal with gesture recognition based information retrieval function and its implementation method - Google Patents

Abstract

Disclosed are a portable terminal having a gesture recognition based information retrieval function and an implementation method thereof. A portable terminal having a gesture recognition-based information retrieval function according to the present invention monitors image data photographed through a camera unit, detects whether a user gesture is input, analyzes a plurality of image data in which a gesture is detected, and retrieves an information search target. A gesture recognition unit for detecting and obtaining position information of an information search target; an image editing unit for generating one image data having improved image quality by editing a plurality of image data; and a recognition object from image data having improved image quality using position information. An image recognition unit having a character recognizer and an object recognizer to extract and recognize, an information database unit storing dictionary data and object data, and a camera unit in response to selection of an information retrieval mode, and a character recognizer according to the selected information retrieval mode. Or optionally running an object recognizer Use emitter base portion to search information about the object recognized by the object recognizer or word composed of the characters recognized by the character recognizer to comprise a central processing unit for outputting to the outside through the display or speaker.

Description

Portable terminal with a information search function based on gesture recognition and implement method

1 is a block diagram of a portable terminal having a gesture recognition based information retrieval function according to an embodiment of the present invention;

FIG. 2 is a diagram showing an example of image data corrected by the image correcting unit shown in FIG. 1;

3A and 3B are flowcharts illustrating a method of implementing an information retrieval function using a portable terminal according to an embodiment of the present invention;

4 is a block diagram of a portable terminal having a gesture recognition based information retrieval function according to another embodiment of the present invention;

5 is a flowchart illustrating a method of implementing an information retrieval function using a portable terminal according to another embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

105, 405: camera unit 110, 410: image processing unit

115, 415: display unit 120, 420: gesture recognition unit

125, 425: video editing unit 130, 430: video correction unit

135: image recognition unit 136: character recognizer

137: object recognizer 140: information DB unit

170, 470: central processing unit 435: interface unit

The present invention relates to a portable terminal having a gesture recognition based information retrieval function and a method of implementing the same. The present invention relates to a portable terminal having a gesture recognition-based information retrieval function for retrieving and providing information on a detected information retrieval object and an implementation method thereof.

Conventionally, if there is a word that does not know the meaning, it was possible to know the meaning by searching the word using a dictionary of books. However, a book dictionary has a disadvantage in that it takes a lot of time to search for words and is inconvenient to carry.

The electronic dictionary was developed to solve this problem. Electronic dictionaries have the advantages of being easier to carry and convenient to use than dictionaries of existing books. However, they are expensive and require users to carry electronic dictionaries separately. .

Therefore, recently, an electronic dictionary function is added to a portable terminal such as a mobile phone, a PDA, etc., which is carried on a daily basis, thereby eliminating the inconvenience of carrying a separate electronic dictionary.

However, in the case of an electronic dictionary or a portable terminal having an electronic dictionary function, the user may input information to search for a desired word only by inputting a character to be searched by operating a character key provided in the key panel, which may cause inconvenience in use. .

Accordingly, an object of the present invention is to provide a gesture recognition-based information retrieval function that can provide user convenience by analyzing a document photographed through a camera of a portable terminal and a gesture of a user and automatically detecting an information retrieval object. It is to provide a portable terminal and an implementation method thereof.

In order to solve the technical problem as described above, the portable terminal having a gesture recognition-based information retrieval function according to the present invention, by monitoring the image data taken through the camera unit to detect whether the user gesture input, the gesture is detected A gesture recognition unit analyzing a plurality of image data to detect an information search target and obtaining position information of the information search target; An image editing unit configured to edit the plurality of image data to generate one image data having improved image quality; An image recognition unit including a character recognizer and an object recognizer for extracting and recognizing a recognition object from the image data having improved quality by using the location information; An information database unit storing dictionary data and object data; And driving the camera unit in response to selection of an information search mode, and selectively driving the character recognizer or the object recognizer according to the selected information search mode, and using the information database unit as a character recognized by the character recognizer. And a central processing unit for searching for information about the constructed word or the object recognized by the object recognizer and outputting the information to the outside through a display unit or a speaker.

The user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool.

The image correction unit may further include correcting the image data having improved image quality, converting the image data into a form recognizable by the image recognition unit, and correcting the position information by using the corrected image data. The recognition unit may extract and recognize the recognition object from the corrected image data by using the corrected position information.

The information retrieval mode includes a dictionary information retrieval mode or an object information retrieval mode, and the central processing unit drives the character recognizer when the dictionary information retrieval mode is selected as the information retrieval mode, and the information retrieval mode. When the object information search mode is selected, characterized in that for driving the object recognizer.

On the other hand, in order to solve the above technical problem, the method of implementing the information search function using the portable terminal according to the present invention, whether to input the user gesture by monitoring the image data taken through the camera unit in response to the selection of the prior information search mode Sensing; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Extracting a character recognition target region from image quality-enhanced image data generated by editing the plurality of image data using the position information, and recognizing a character included in the extracted region; And retrieving information of a word composed of the recognized characters by using dictionary data stored in an information database unit and outputting the information to the outside through a display unit or a speaker.

Preferably, the method further includes correcting the image data having improved image quality, converting the image data into a form recognizable in the recognizing step, and correcting the position information using the corrected image data. The character recognition target area may be extracted from the corrected image data by using the corrected position information.

The user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool.

In addition, in order to solve the above technical problem, the method of implementing the information retrieval function using the portable terminal according to the present invention, whether the user inputs the gesture by monitoring the image data captured by the camera unit in response to the object information search mode selection Detecting; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Extracting an object recognition region from image quality-enhanced image data generated by editing the plurality of image data using the position information, and recognizing an object included in the extracted region; And searching for information on the recognized object by using object data stored in an information database unit and outputting the information about the recognized object to the outside through a display unit or a speaker.

Preferably, the method further includes correcting the image data having improved image quality, converting the image data into a form recognizable in the recognizing step, and correcting the position information using the corrected image data. The object recognition target area may be extracted from the corrected image data by using the corrected position information.

Furthermore, in order to solve the technical problem as described above, the portable terminal having a gesture recognition-based information retrieval function according to the present invention, by monitoring the image data taken through the camera unit to detect whether the user gesture input, A gesture recognition unit analyzing the plurality of detected image data to detect an information search target and obtaining position information of the information search target; An image editing unit configured to edit the plurality of image data to generate one image data having improved image quality; An interface unit for performing data communication with the information providing server; And controlling the interface unit to drive the camera unit in response to selection of an information retrieval mode, and transmit the information on the selected information retrieval mode, the image data with improved image quality, and the location information to the information providing server. And a central processing unit receiving the information on the information search target from the server and outputting the information to the outside through a display unit or a speaker.

The user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool.

Preferably, the image correction unit for correcting the image quality and the position information of the improved image quality further includes, wherein the central processing unit transmits the image data and position information corrected in the image correction unit to the information providing server Characterized in that.

And the information retrieval mode is a dictionary information retrieval mode or an object information retrieval mode.

On the other hand, in order to solve the above technical problem, the method of implementing the information retrieval function using the portable terminal according to the present invention, by detecting the user gesture input by monitoring the image data taken through the camera unit in response to the information search mode selection Doing; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Transmitting information on the selected information search mode, image data with improved image quality generated by editing the plurality of image data, and the location information to an information providing server connected through an interface unit; And receiving the information on the information search target from the information providing server and outputting the information to the outside through a display unit or a speaker.

The user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings. However, in describing the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a block diagram of a portable terminal having a gesture recognition based information retrieval function according to an embodiment of the present invention.

1, a portable terminal according to an embodiment of the present invention includes a camera unit 105, an image processing unit 110, a display unit 115, a gesture recognition unit 120, an image editing unit 125, and an image correction unit. 130, an image recognition unit 135, an information database (hereinafter referred to as "DB") unit 140, an RF unit 145, a data processing unit 150, an audio processing unit 155, a key input unit And a central processing unit (CPU) 170.

The camera unit 105 captures an image of a subject photographed through a lens under the control of the CPU 170. The camera unit 105 according to the present exemplary embodiment requires a resolution of at least 100 DPI (Dot Per Inch) or more for character recognition, and a resolution of 200 DPI or more, that is, about 300,000 pixels is required for an optimal recognition environment. The image photographed by the camera unit 105 is applied to the image processing unit 110.

The image processor 110 converts an image signal output from the camera unit 105 into data that can be displayed on the display unit 115 and outputs the data. The image processor 110 processes the image signal output from the camera unit 105 in units of frames, and adjusts and outputs the frame image data according to the resolution and screen size supported by the display unit 115.

The display unit 115 displays image data output from the image processing unit 110 and user data output from the CPU 170.

The gesture recognition unit 120 monitors image data processed and output by the image processing unit 110 and detects whether a user gesture is input. When the user gesture is detected, the gesture recognition unit 120 captures a plurality of image data in which the user gesture is detected. do. In this case, image data of about 5 to 10 frames is required for user gesture recognition. In the present embodiment, the user gesture is a gesture by a user's hand gesture, and refers to an operation in which the user refers to a specific part (information search target) on the document by using a tool such as a finger or a pen. Here, examples of operations referring to the information search target may include tapping an information search target using a finger or a pen, underlining the information search target, and drawing a circle or a square on the information search target. .

The gesture recognition unit 120 transmits the captured image data to the image editing unit 125. The gesture recognition unit 120 analyzes a plurality of captured image data to detect an information search target and obtains position information (coordinate value information) of the detected information search target. The location information of the information search target detected by the gesture recognition unit 120 is provided to the CPU 170.

The image editing unit 125 edits a plurality of image data output from the gesture recognition unit 120 to generate one image data having improved image quality. The image quality improved image data generated by the image editing unit 125 is provided to the CPU 170.

The image corrector 130 corrects and outputs the image data having improved image quality and the position information of the information search target provided from the CPU 170. That is, as illustrated in FIG. 2, the image corrector 130 corrects (eg, rotates, resizes, shears, or improves) the image data having improved image quality output from the image editing unit 125 to thereby recognize the image recognizer 135. To a recognizable form. In FIG. 2, (a) shows image data before image correction, and (b) shows image data converted into a form recognizable by the image recognition unit 135.

In addition, the image corrector 130 corrects the position information of the information search target by using the corrected image data. This is because the position of the information search target can be changed by correcting the image data. The image data and the position information corrected by the image corrector 130 are provided to the CPU 170.

The image recognizer 135 includes a text recognizer 136 and an object recognizer 137.

The character recognizer 136 is controlled by the CPU 170, extracts a character recognition target region from the corrected image data by using the position information of the information search target corrected by the image corrector 130, and executes a character DB (not shown). The character included in the extracted area is recognized by using the character information stored in the C).

The object recognizer 137 is controlled by the CPU 170, extracts an object recognition target region from the corrected image data by using the position information of the information search target corrected by the image corrector 130, and stores an object DB (not shown). Recognize the object included in the extracted area by using the object information stored in The character recognized by the character recognizer 136 or the object recognized by the object recognizer 137 is provided to the CPU 170.

The information DB unit 140 stores dictionary information such as English-Korean dictionary, Korean-English dictionary, Korean dictionary, Japanese-Korean dictionary, Korean-Japanese dictionary, and the like, and object information about physical objects such as objects, animals, and the like. The information DB unit 140 according to the present embodiment may be stored in an internal memory of the portable terminal, or alternatively, may be stored in an external memory card and installed in the portable terminal.

The RF unit 145 receives a signal sent by an external portable terminal connected through a mobile communication network through an antenna, and transmits a signal to be transmitted to the external portable terminal through an antenna.

The data processor 150 may include a modem for encoding and modulating a signal to be transmitted to an external portable terminal through the RF unit 145 and a codec for demodulating and decoding a signal received through the RF unit 145. It is configured to include.

The audio processor 155 outputs an audio signal output from the data processor 150 to the outside through a speaker or transmits an audio signal acquired through a microphone to the data processor 150.

The key input unit 160 includes numeric keys, character keys, and various function keys. The function key required in this embodiment is a dictionary information search mode key or an object information search mode key. The key input unit 160 generates a key signal corresponding to a key selected by the user and applies it to the CPU 170.

The CPU 170 controls the overall operation of the portable terminal. That is, the CPU 170 switches the operation mode of the portable terminal to the pre-information search mode or the object information search mode according to the key signal input from the key input unit 160, and controls the operation of the portable terminal device according to the set mode. . For example, the CPU 170 automatically turns on the camera unit 105 in response to the information search mode selection, and selectively selects the character recognizer 136 or the object recognizer 137 according to the set information search mode. Drive it. That is, the CPU 170 drives the character recognizer 136 when the dictionary information search mode is set as the information search mode, and drives the object recognizer 137 when the object information search mode is set as the information search mode.

In addition, the CPU 170 may search for information about a word composed of characters recognized by the character recognizer 136 or an object recognized by the object recognizer 137 using the information DB unit 140, and display the retrieved information in the display unit ( 115) Or output through the speaker to the outside.

3A and 3B are flowcharts illustrating a method of implementing an information retrieval function using a portable terminal according to an embodiment of the present invention.

As shown in FIG. 3A, when the pre-information search mode key signal is received through the key input unit 160 (S200), the CPU 170 switches the operation mode of the portable terminal to the pre-information search mode, and the camera unit 105. (S210). An image of a document photographed by the camera unit 105 is displayed on the display unit 115 through the image processing unit 110.

When the operation mode of the portable terminal is switched to the dictionary information search mode, the gesture recognition unit 120 monitors the image data output from the image processing unit 110 and detects whether the user gesture is input (S220). When the user gesture is detected while monitoring the image data, the gesture recognition unit 120 captures a plurality of image data from which the gesture is detected, transmits the image data to the image editing unit 125, and simultaneously analyzes the captured image data. The information search target character is detected, and position information of the detected character is obtained (S230).

The CPU 170 controls the image corrector 130 to correct image quality improved image data output from the image editor 125 in a form recognizable by the character recognizer 136 (S240). The image corrector 130 corrects the image data with improved image quality output from the image editor 125 under the control of the CPU 170, converts the image data into a form recognizable by the character recognizer 136, and uses the corrected image data. By correcting the position information obtained from the gesture recognition unit 120. When the correction of the image data and the position information is completed, the CPU 170 provides the character recognizer 136 with the position information of the image data and the information search target character corrected by the image corrector 130.

The character recognizer 136 extracts a character recognition target region from the corrected image data by using the position information of the corrected information search target character, and recognizes a character included in the extracted region using the character DB (S250).

The CPU 170 combines the characters output from the character recognizer 136 to form words, and retrieves information on the combined words using dictionary data stored in the information DB unit 140 (S260).

Finally, the CPU 170 outputs information on the searched word to the outside through the display unit 115 or the speaker (S270). Here, the information on the searched words differs according to the selected dictionary. For example, when the Korean-English dictionary is selected in response to the dictionary information search mode setting, the character recognizer 136 provides English information about the recognized Korean and the English-Korean dictionary is selected in response to the dictionary information search mode setting. Korean information about the English recognized at 136 is provided.

Meanwhile, as shown in FIG. 3B, when an object information search mode key signal is received through the key input unit 160 (S300), the CPU 170 switches the operation mode of the portable terminal to the object information search mode and the camera. The unit 105 is driven (S310).

When the operation mode of the portable terminal is switched to the object information search mode, the gesture recognition unit 120 monitors the image data output from the image processing unit 110 to detect whether the user gesture is input (S320). When the user gesture is detected while monitoring the image data, the gesture recognition unit 120 captures a plurality of image data from which the gesture is detected, transmits the image data to the image editing unit 125, and simultaneously analyzes the captured image data. The location information of the information search target object is obtained (S330).

The CPU 170 controls the image corrector 130 to correct image quality improved image data output from the image editor 125 in a form recognizable by the object recognizer 137 (S340). The image corrector 130 corrects the image data with improved image quality output from the image editor 125 under the control of the CPU 170, converts the image data into a form recognizable by the character recognizer 136, and uses the corrected image data. By correcting the position information obtained from the gesture recognition unit 120.

The object recognizer 137 extracts an object recognition target region from the corrected image data by using the position information of the corrected information search target object, and recognizes an object included in the extracted region using the object DB (S350). .

The CPU 170 retrieves information on the object recognized by the object recognizer 137 using the object data stored in the information DB unit 140 (S360). Finally, the CPU 170 outputs information on the searched object to the outside through the display unit 115 or the speaker (S370). For example, when the information search target object is recognized as an 'apple', information (Korean and English information) about the selected object is provided through the display unit 115 or the speaker.

As described above, according to the information retrieval function according to the present invention, it is possible not only to search for information on a word on a document based on a user gesture, but also to search for information on objects included in an image book such as a picture book or a story book. can do.

Meanwhile, FIG. 4 is a block diagram of a portable terminal having a gesture recognition based information retrieval function according to another embodiment of the present invention.

Referring to FIG. 4, a portable terminal according to another embodiment of the present invention includes a camera unit 405, an image processor 410, a display unit 415, a gesture recognition unit 420, an image editing unit 425, and an image correction unit. 430, an interface unit 435, an RF unit 440, a data processor 445, an audio processor 450, a key input unit 460, and a CPU 470. Here, since the functions of the blocks except for the interface unit 435 and the CPU 470 are substantially the same as the functions of the blocks of the portable terminal illustrated in FIG. 1, a detailed description thereof will be omitted.

The interface unit 435 supports wired / wireless data communication with the information providing server under the control of the CPU 470. To this end, the interface unit 435 includes a wired interface unit (not shown) that supports wired data communication with the information providing server and a wireless interface unit (not shown) that supports wireless communication with the information providing server.

When the information retrieval mode is set, the CPU 470 according to the present embodiment provides information on the information retrieval mode set according to whether the user gesture is detected, image data corrected by the image correction unit 430, and location information of the information retrieval target. The interface unit 435 is controlled to transmit to the providing server. In addition, the CPU 470 receives information on an information search target from the information providing server through the interface unit 435, and outputs the received information to the outside through the display unit 415 or the speaker.

In the present embodiment, the image correction unit 430 is provided in the portable terminal to transmit the image data corrected by the image correction unit 430 and the position information of the information search target to the information providing server. It is not. That is, of course, it can be implemented to perform the image correction function in the information providing server having a character and object recognition function.

5 is a flowchart illustrating a method of implementing an information retrieval function using a portable terminal according to another embodiment of the present invention.

As shown in FIG. 5, when a pre-information search mode key signal or an object information search mode key signal is received through the key input unit 460 (S500), the CPU 470 selects an operation mode of the portable terminal in the pre-information search mode. Alternatively, the camera enters the object information search mode and drives the camera unit 405 (S510).

When the operation mode of the portable terminal is switched to the information search mode, the gesture recognition unit 420 monitors the image data output from the image processing unit 410 to detect whether the user gesture is input (S520).

When a user gesture is detected while monitoring the image data, the gesture recognition unit 420 captures a plurality of image data from which the gesture is detected, transmits the image data to the image editing unit 425, and simultaneously analyzes the captured image data. The location information on the information search target is obtained (S530). The image editing unit 425 edits a plurality of image data provided from the gesture recognition unit 420 to generate one image data having improved image quality.

After the step 530, the CPU 470 may determine information about the information search mode for the set information search mode, image data with improved image quality output from the image editing unit 425, and information search target output from the gesture recognition unit 420. The interface unit 435 is controlled to transmit the location information to the information providing server (S540).

After the step 540, the CPU 470 detects whether or not to receive information on the information search target from the information providing server (S550). When the information on the information search target is received from the information providing server, the CPU 470 outputs the received information to the outside through the display unit 415 or the speaker (S560).

On the other hand, the present invention has been described in detail through a representative embodiment, but those skilled in the art to which the present invention pertains various modifications within the scope of the present invention without departing from the scope of the present invention. I will understand what is possible. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims below and equivalents thereof.

As described above, according to the present invention, by detecting the user's gesture to automatically detect the information search target word, search for information on the detected word and provide it to the user, the user inputs the information search target word There is no need to make it possible to improve user convenience.

In addition, according to the present invention, by having a dictionary DB that database the object data, it is possible to search the information on the physical objects included in the image book as well as the word on the general document, thereby improving the utilization of the portable terminal Can be.

Claims (16)

A gesture recognition unit configured to monitor image data photographed through a camera unit to detect whether a user gesture is input, analyze a plurality of image data in which the gesture is detected, detect an information search target, and obtain position information of the information search target; An image editing unit configured to edit the plurality of image data to generate one image data having improved image quality; An image recognition unit including a character recognizer and an object recognizer for extracting and recognizing a recognition object from the image data having improved quality by using the location information; An information database unit storing dictionary data and object data; And Driving the camera unit in response to selection of an information search mode, selectively driving the character recognizer or the object recognizer according to the selected information search mode, and a word composed of characters recognized by the character recognizer using the information database unit Or a central processing unit searching for information on the object recognized by the object recognizer and outputting the information to the outside through a display unit or a speaker. The method of claim 1, The user gesture is a gesture of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool. The method of claim 1, And an image corrector configured to correct the image data having improved image quality, convert the image data into a form recognizable by the image recognizer, and correct the position information by using the corrected image data. And the image recognition unit extracts and recognizes the recognition object from the corrected image data by using the corrected position information. The method of claim 1, The information search mode includes a dictionary information search mode or an object information search mode. The CPU may drive the character recognizer when the dictionary information search mode is selected as the information search mode, and drive the object recognizer when the object information search mode is selected as the information search mode. A portable terminal having a gesture recognition based information retrieval function. Detecting user gesture input by monitoring image data photographed through a camera unit in response to selection of a pre-information search mode; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Extracting a character recognition target region from image quality-enhanced image data generated by editing the plurality of image data using the position information, and recognizing a character included in the extracted region; And And retrieving information of a word composed of the recognized characters by using dictionary data stored in an information database unit and outputting the information to the outside through a display unit or a speaker. The method of claim 5, And correcting the image data having improved image quality, converting the image data into a form recognizable in the recognition step, and correcting the position information by using the corrected image data. And wherein the recognizing step extracts the text recognition target area from the corrected image data by using the corrected position information. The method of claim 5, And the user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool. Detecting whether a user inputs a gesture by monitoring image data photographed through a camera unit in response to selection of an object information search mode; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Extracting an object recognition region from image quality-enhanced image data generated by editing the plurality of image data using the position information, and recognizing an object included in the extracted region; And And searching for information on the recognized object by using object data stored in an information database unit and outputting the information about the recognized object to the outside through a display unit or a speaker. The method of claim 8, And correcting the image data having improved image quality, converting the image data into a form recognizable in the recognition step, and correcting the position information by using the corrected image data. And wherein the recognizing step extracts the object recognition target area from the corrected image data by using the corrected position information. The method of claim 8, And the user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool. A gesture recognition unit configured to monitor image data photographed through a camera unit to detect whether a user gesture is input, analyze a plurality of image data in which the gesture is detected, detect an information search target, and obtain position information of the information search target; An image editing unit configured to edit the plurality of image data to generate one image data having improved image quality; An interface unit for performing data communication with the information providing server; And In response to selection of an information retrieval mode, controlling the interface unit to transmit the information on the selected information retrieval mode, the image data with improved image quality, and the location information to the information providing server, and the information providing server. And a central processing unit for receiving the information on the information search target from the display unit and outputting the information to the outside through a display unit or a speaker. The method of claim 11, The user gesture is a portable terminal having a gesture recognition based information retrieval function, characterized in that the user refers to a specific portion of the document photographed through the camera unit using a finger or a tool. The method of claim 11, And an image corrector configured to correct the image data and the position information having improved image quality. The CPU is a portable terminal having a gesture recognition-based information providing function, characterized in that for transmitting the image data and position information corrected by the image correction unit to the information providing server. The method of claim 11, And the information retrieval mode is a dictionary information retrieval mode or an object information retrieval mode. Detecting user gesture input by monitoring image data photographed through a camera unit in response to selection of an information search mode; Capturing a plurality of image data from which the gesture is detected, transmitting the image data to an image editing unit, analyzing the plurality of image data, and detecting an information search target to obtain location information of the information search target; Transmitting information on the selected information search mode, image data with improved image quality generated by editing the plurality of image data, and the location information to an information providing server connected through an interface unit; And And receiving the information on the information search target from the information providing server and outputting the information to the outside through a display unit or a speaker. The method of claim 15, And the user gesture is an operation of referring to a specific part of a document photographed through the camera unit by a user using a finger or a tool.

Images