JP2008160852A - Information processing apparatus, information processing method, and software - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile phone with a means capable of reducing a data size of translation data for translating a foreign language and displaying both summarized and detailed translations. <P>SOLUTION: The information processing apparatus is provided with: a communication unit for communicating with another apparatus; an imaging unit for outputting image information; a recognition unit for recognizing identification information contained in the image information outputted by the imaging unit; an associated information acquisition instruction unit for instructing the acquisition of associated information associated with the identification information recognized by the recognition unit; and a control unit which controls to acquire and display the associated information via the communication unit in a case where the acquisition of the associated information is instructed by the associated information acquisition instruction unit. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing apparatus such as a mobile phone, PHS (Personal Handy-phone System), PDA (Personal Digital Assistant), and PC (Personal Computer).

  Japanese Patent Laid-Open No. 2002-252691 discloses that characters included in captured image information are converted into character information by an OCR (Optical Character Recognition) function, and an address, name, telephone number, URL (Uniform Resource) posted on a printed matter is disclosed. Mobile phones that can input Locator) have been proposed. Japanese Patent Application Laid-Open No. 5-274346 discloses an image of a character string printed on paper, uses the OCR function to recognize the character string from the image, converts it into character information, and refers to a translation dictionary. An electronic dictionary that displays a translation corresponding to the character string on a display unit has been proposed.

JP 2002-252691 A JP-A-5-274346

  However, although the above-mentioned patent document 1 discloses a mobile phone equipped with an OCR function that can input information such as an address posted on a printed material, it is easy to use such input information. The examination of was insufficient. In Patent Document 2, for example, when displaying a translation of a word input by the OCR function, only a simple translation is displayed. If you want to know more detailed translation and how to use the word, a separate English-Japanese dictionary is available. There was a problem that it was difficult to use.

  SUMMARY An advantage of some aspects of the invention is that it provides an information processing apparatus, an information processing method, and software with improved usability.

  In order to solve the above problem, for example, as described in claim 1, a communication unit that communicates with another device, an imaging unit that outputs image information, and identification information included in the image information output by the imaging unit A recognition unit that recognizes the information, a related information acquisition instruction unit that instructs acquisition of related information related to the identification information recognized by the recognition unit, and an instruction to acquire the related information by the related information acquisition instruction unit And a control unit that controls to acquire and display the related information via the communication unit.

  According to the present invention, it is possible to provide an information processing apparatus, an information processing method, and software with improved usability.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following drawings, the same symbols are used for the same components.

  FIG. 1 is a configuration block diagram of a mobile phone according to the present embodiment (first embodiment). In this embodiment, it is assumed that a Japanese translation of English words posted on paper or the like is output using an image recognition function (OCR function) installed in a mobile phone.

  The cellular phone 100 is configured to include an input unit 101, a CPU 102, an imaging unit 103, a storage unit 104, an image recognition unit 105, dictionary data 106, a display unit 107, a communication unit 108, a search unit 109, and the like.

  The input unit 101 is an input unit for inputting a telephone number and other information. The input unit 101 is used when a user gives an input to the mobile phone, such as when turning on / off the power of the mobile phone or capturing an image of a subject. is there. The CPU 102 operates based on a program stored in the storage unit 104, and controls each unit according to an input from the input unit 101, for example.

  The imaging unit 103 is configured by, for example, a camera, and captures a human face, a scene, characters, and the like as a subject, converts the image into image information that can be handled by the CPU 102, and the converted image information is necessary. In response, the data is output to the display unit 107 or the like. The user selects an image to be captured while monitoring the image information output to the display unit 107 and gives an instruction for imaging using a shutter or the like. When the shutter is pressed, the captured image information is stored in the storage unit 104. The storage unit 104 also stores data and software handled by the mobile phone 100.

  The image recognition unit 105 recognizes identification information such as characters and symbols included in the image information. Here, the identification information is information represented by characters, symbols, etc., and is represented by characters, symbols, etc., such as homepage addresses, e-mail addresses, addresses, telephone numbers, map information, or English words. Any information other than the above example may be used as long as the information can identify the thing.

  Here, recognition refers to extracting identification information included in the image information captured by the imaging unit 103, dividing the identification information into predetermined areas, and converting the divided areas into predetermined parameter values. In addition, it means determining what kind of information the divided area is based on the parameter value.

  More specifically, for example, considering the case of “recognizing” the character “abc” in the image information, first, the position of “abc” in the image information is specified, and then Are divided into “a”, “b”, and “c”, and “a” is divided into predetermined parameter values (for example, the composition of a line segment when the character is viewed vertically, horizontally, and diagonally, and the white portion of the character 0 ”, the black part is converted to a parameterized value as“ 1 ”). Then, the character associated with the parameter value most similar to the converted parameter value is selected from the character pattern data table. The character pattern data table is a table in which parameter values and characters such as alphabets corresponding to the parameter values are described and stored in advance. Note that the image recognition unit 105 has an OCR function, and a specific configuration for realizing the OCR function is described in Patent Document 1, for example.

  In the dictionary data 106 stored in the storage unit 104, for example, identification information such as words in a foreign language, etc., a Japanese translation of the identification information, various Japanese translations of the identification information, usage / examples, etc. are written in detail. URL of homepage etc. included. The translation language is not limited to the Japanese translation, and may be English or French. Furthermore, data such as pictures and sounds may be used as long as it is understood that the information is identification information.

  The display unit 107 displays the image captured by the imaging unit 103 and the identification information recognized by the image recognition unit 105. In addition, the display unit 107 displays, for example, power status, radio wave intensity, remaining battery level, server connection status, operating status such as unread mail, input phone number, mail destination, mail transmission document, video, still image, incoming call, etc. Display the received data such as the telephone number of the calling party, the incoming mail document, the connected internet screen, and the screen necessary for using the functions of the mobile phone. The communication unit 108 performs communication processing for the mobile phone 100 to access the Internet or other information processing apparatus. The search unit 109 searches the identification data that matches the identification information obtained by the image recognition unit 105 with reference to the dictionary data 106.

  In addition, the CPU 102 uses the mode determination flag, and the recognition mode is a normal imaging mode in which the imaging mode is a normal still image or moving image, or a mode in which the identification information is recognized using the image recognition unit 105. It is determined whether it is. Here, the mode determination flag is, for example, a variable that exists in the software program stored in the storage unit 104 and has a different value in the recognition mode and the normal imaging mode. The recognition mode and the normal imaging mode are switched by, for example, an operation by the input unit 101. Hereinafter, in the embodiment, a case where there are two types of “recognition mode” and “normal imaging mode” is considered, but the present invention is not limited to this, and there may be other modes.

  FIG. 2 shows an example of the dictionary data 106. A column 201 is a column in which words in a foreign language are written, and a column 202 is a column in which a summary of translation corresponding to the words in 201 is written. A column 203 is a column in which related information corresponding to the words in the column 201, for example, a URL of a home page on which details of translation are posted, is written. Here, although English words are described in the column 201 as an example, it goes without saying that any language such as French or German may be used. Here, the entire URL is described in the column 203, but only a part of the URL may be described.

  FIG. 3 is an example of the dictionary data 106 when only a part of the URL is described. In the case of FIG. 2, since “http://www.xxx.com” is common, this common part is described in the URL common part 204, such as “a” or “abacus”. Only unique parts are described in the column 206 as URL-specific parts. The URL creation method 205 describes a method for creating a URL of a home page on which detailed translation is posted by combining the URL common part 204 and the URL specific part 206. In this way, the common part of the URL is described as a representative location in the dictionary data 106, and only the different part is described corresponding to each English word. There is an advantage that the data size can be reduced and the amount of data to be held by the mobile phone can be reduced.

Further, as shown in FIG. 3, when the information described in the column 206 is the same as the word described in the column 201, the column 206 describing the URL specific part is changed to the dictionary data 1.
It may be omitted from 06. As a result, the data size of the dictionary data can be further reduced.

  Next, the processing procedure of the mobile phone in the present embodiment will be described using the flowchart of FIG. First, after the “recognition mode” is selected by the user via the input unit 101, a switch (hereinafter referred to as a shutter button) that instructs the imaging unit 103 of the input unit 101 to capture an image is pressed. Then, the CPU 102 instructs the image capturing unit 103 to capture an image (step S401). Then, the image information of the object that is the subject imaged (converted) by the imaging unit 103 is stored in the storage unit 104 via the CPU 102 (step S402).

  Next, the image recognition unit 105 reads the image information stored in the storage unit 104, reads the identification information included in the image information, for example, the word “recognition” in the sentence of the foreign book, and stores the recognition result in the storage unit 104. (Step S403). The search unit 109 reads this result, and searches for a match from the words in the column 201 of the dictionary data 106 stored in the storage unit 104 (step S404).

  As a search method, it is preferable to first search for a completely matching identification information, and search for a word that is different by one character but does not match another character when there is no completely matching information. By adopting this method, even when the image recognition unit 105 mistakenly recognizes a character on the paper, it is possible to find the word closest to the character string, and it is time-consuming to repeat the character recognition by taking an image many times. Is easy to use.

  If there is no word that differs only by one character, you can search by increasing the number of different characters such as two different characters, three different characters, etc. In this case, even if the recognition rate is low, find the corresponding word be able to.

  If a corresponding word is found in step S405, the process proceeds to step S406. If the corresponding word is not found, the process proceeds to step S410, a screen for notifying the user that the corresponding word is not found is displayed on the display unit 107, and the process ends.

  In step S406, the search unit 109 searches in the dictionary data in step S404, and the CPU 102 reads the column 202 (translation outline) corresponding to the searched word and displays it on the display unit 107 (step S406).

  Thereafter, the CPU 102 determines whether or not a more detailed request for translation has been instructed via the input unit 101 or the display unit 107 (step S407). If the request is instructed, the process proceeds to step S408.

  In step S408, the CPU 102 reads the URL corresponding to the word searched by the search unit 109 in step S404, and the CPU 102 outputs an instruction signal to the communication unit 108 so as to connect to the URL.

  In step S409, the CPU 102 reads the content written on the home page via the communication unit 108 and stores it in the storage unit 104. Thereafter, the CPU 102 displays the contents on the display unit 107.

  As described above, when you want to refer to detailed translations as well as simple translations, you can display a homepage that contains detailed translations, so you can search for the word without having to redraw the English-Japanese dictionary. You can know the detailed meaning and it is easy to use.

  In addition, since a large-capacity memory cannot be installed in a mobile terminal such as a mobile phone, the minimum necessary information (simple translation, representative translation, etc.) is stored in the terminal as in this embodiment. When detailed information is desired, it is very effective to acquire and refer to information stored on another information processing apparatus such as the Internet.

  FIG. 5 is an example of a display screen of the present embodiment, where a character string written on a sheet or the like is imaged, the image is converted into character information by character recognition, and a summary translation held in the dictionary data 106 is displayed. If there is an input requesting the display of the detailed translation after being displayed on the screen, the display screen when the content stored in the URL is displayed on the display unit 107 is connected to the URL held in the dictionary data 106 An example is shown. A button corresponding to a function such as “menu” displayed at the bottom of the display unit 107 is, for example, a button arranged at the top of the input unit 101, and the button is pressed by the user. Then, a display screen corresponding to a function such as “menu” is displayed on the display unit 107. Similarly, each time the display screen is changed, the button or the like is assigned as a button for executing a function displayed at the bottom of each display screen. Hereinafter, for convenience, a button for executing a function displayed at the bottom of the display screen is referred to in association with the displayed function name. For example, a button corresponding to the “menu” function is referred to as a “menu button”, or a button corresponding to the “return” function is referred to as a “return button”. Note that the execution button described above may be assigned to each button according to the position where each function such as lower right or lower left is displayed, or may be realized by one button.

  A screen 501 is a screen example in which “recognition mode” is selected as an imaging mode, and for example, the word “Recognition” in English is identified and tried to be recognized. Here, as a means for specifying the English word “Recognition”, for example, if the “+” mark displayed at the center of the display unit 107 is brought on the identification information “Recognition”, “ The word “Recognition” may be automatically specified with the portion from the space information to the space information in the vicinity from the position of the “+” mark as a single word, or the user may use the input unit 104 to, for example, The recognition part such as the mail address information may be designated with a cursor or the like. If there are a plurality of methods for determining the recognition target, the recognition target may be selected. In the case of a moving image, the moving image may be set so that it can be frame-fed as a still image, and the recognition target portion may be selected on the frame-advancing screen.

  Also, here, the English word “Recognition” that is the recognition target is clarified by surrounding it with a dotted line etc., but the image editing unit creates an image that enlarges or reduces the “Recognition” that is the recognition target, and displays the image May be. In order to enlarge, for example, what is displayed with 1 × 1 pixel is displayed with 2 × 2 pixels. In addition, the image editing unit may convert the image part of the character to be recognized to black and white, convert it to sepia, or create a reversed black and white image and display the image. However, the display is not limited to the above example as long as it is clearly displayed which is the recognition target.

  In order to change from the standby state to the state of the screen 501, for example, by pressing the “recognition mode” button or proceeding from the standby state to the menu screen and performing a button operation along the menu screen, You may move to. When a “recognition” button at the bottom of the screen 501 is pressed, the screen transitions to the screen 502.

  A screen 502 is a screen indicating that the character recognition unit 103 detects a recognizable portion such as a URL in a captured image and is performing a recognition process. When the recognition process ends, the screen transitions to a screen 503.

  A screen 503 is a screen on which the result recognized by the image recognition unit 105 and the outline translation 202 corresponding to the character string are displayed on the display unit 107. For example, in order to clarify what the recognition target is, the image of the part other than the character to be recognized is the same color as the color of the part where the captured image or icon is not displayed. It is painted. When the user wants to know the meaning and usage of the word in more detail, when the user presses a button corresponding to the detailed display displayed in the lower left portion of the screen 503, the screen transitions to the screen 504.

  A screen 504 is a screen indicating that the communication unit 108 is executing processing for connecting to the column 203. When connection to the column 203 is possible, the screen transitions to the screen 505.

  A screen 505 represents a screen displaying the contents of the home page in the column 203.

  By using the display screen as described above, after the user knows a simple translation, the user can know the detailed meaning of the word and its usage / examples with a simple operation, which is convenient.

In the above embodiment, it is assumed that the URL of one type of homepage is used to refer to the meaning and usage of a detailed word. However, the present invention is not limited to this. You may make it memorize | store in the memory | storage part 104, and when selecting a detailed translation etc., you may enable it to select favorite URL from the some URL. Thereby, the user can select a detailed translation related to the word from abundant information sources, which is easy to use. Of course, it is not necessary to limit to URL information as information for referring to detailed information regarding words.
Furthermore, in the above example, as the detailed information regarding the translation of the English word, the URL of the home page described in detail regarding the translation of the word is assumed. However, the present invention is not limited to this. Anything can be used as long as it can be referred to.

  In the above example, the homepage is used to obtain detailed information. However, the translated detailed information may be stored in the storage unit 104 of the mobile phone. In this case, the simple translation and the detailed translation of the word are distinguished and stored, the normal recognition result is displayed in a simple translation, and if the user requests detailed information, the memory is stored. Detailed information stored in the unit 104 is displayed. Thereby, since detailed information of the translation is stored in the mobile phone, it is possible to quickly display the detailed information without connecting to the homepage, so that wasted time can be saved, Detailed information can be referred to even outside the communication range.

  Further, detailed information may be acquired using a device such as a PC (Personal computer) in which detailed information is stored.

  In this case, information necessary for reading detailed information such as translation stored in the PC is written in the column 206 of the mobile phone and dictionary data 106. Here, information necessary for referring to detailed translation information (hereinafter referred to as reference required information) is, for example, information (identifier such as an IP address) for identifying a PC that holds detailed translation information, A path name and file name of a file in which details of translation are described, or an address indicating a portion in which detailed translation is described in the storage unit of the PC.

  For example, when recognizing the word “abalone” and wanting more detailed translation information, the user uses a display screen example as shown in FIG. Upon receiving the request instruction signal, the CPU 102 transmits necessary reference information corresponding to “abalone” of the dictionary data stored in the storage unit 104 to the PC via the communication unit 108. When the PC receives the reference necessary information, it transmits detailed information such as word translations stored in the storage unit of the external information processing apparatus to the mobile phone based on the reference necessary information. The CPU 102 of the mobile phone that has received the detailed information such as the word translation displays it on the display unit 107. Note that the means for connecting the mobile phone and the PC may be wired or wireless.

  According to this, since detailed information such as word translation can be referred to in the mobile phone using the storage unit of the PC, the storage capacity in the storage unit 104 can be reduced. In addition, detailed information can be referred to even outside the communication range, which is convenient. Similarly, reference may be made via a storage medium in which detailed information is stored. In this case, the mobile phone is provided with a recording medium reading unit for reading information on the recording medium. The processing may be the same as described above.

  Further, detailed information itself such as English words may be created and edited by the user himself / herself. In this case, a setting screen for the user to create or edit (addition or deletion) detailed information is displayed on the display unit 107. Information input via such a setting screen is stored in the dictionary data column 203. When referring to the detailed information of the recognized word, the CPU 102 that has received the detailed information request instruction displays detailed information on the display unit 107 based on the dictionary data stored in the storage unit 104.

  This allows you to create and edit your own original dictionary, and because you can refer to the detailed information of the word in the original dictionary that has been created, it is not described in the preset detailed information Information can be added or deleted, which further improves usability.

  Furthermore, in the above, the information stored in the mobile phone is used to obtain simple information, and the PC or storage medium is used to obtain detailed information. When referring, information stored in a PC or a storage medium may be used.

  Next, a second embodiment will be described. The first embodiment is an example in which an English word is recognized and a Japanese translation is displayed. However, there is a problem that other languages such as French and German cannot be translated. The present embodiment has been made with this point in mind.

  The difference from the first embodiment is that the dictionary data stored in the storage unit 104 is increased to two dictionary data a and dictionary data b, and a dictionary flag is included.

  For example, the dictionary data a is an English-Japanese dictionary and the dictionary data b is a German-Japanese dictionary. Here, two types of languages (English and German) are assumed as the dictionary data, but dictionary data corresponding to each of them may be stored assuming three or more types of languages.

  The dictionary flag is a flag used by the CPU 102 to select which dictionary data is used. The dictionary flag is, for example, a variable that exists in a software program stored in the storage unit 104, and an identification number assigned to the dictionary data is substituted. The CPU 102 stores dictionary data to be used based on the variable. select. The dictionary flag value can be changed by the user's operation via the input unit 101, so that the user can obtain translations for multilingual words and is easy to use.

  FIG. 6 shows a screen display example of the mobile phone 100 when the user selects a dictionary to be used. A screen 601 is a screen representing a state in which the mobile phone can perform imaging by the imaging unit 103. In order to change from the standby state to this state, for example, a switch on the side of the mobile phone is pressed, or an operation such as going from the standby state to the menu screen and pressing the button several times is performed. When a button corresponding to the “menu” at the lower right of the screen is pressed on the screen 601, the screen transitions to the screen 602.

  A screen 602 is a screen for selecting a menu related to imaging and character recognition. In this state, when “(1) dictionary selection” is selected via the input unit 101, the screen transitions to the screen 603.

  A screen 603 is a screen for selecting a dictionary to be used. In this state, when “(3) ΔΔΔ Japanese and English” is selected via the input unit 101, the screen transitions to the screen 604.

  A screen 604 is a screen that informs the user that the dictionary to be used is a ΔΔΔ Japanese-English dictionary. Here, when the button corresponding to “confirm” is pressed, the screen transitions to a screen 605, and the mobile phone is again in a state in which imaging by the imaging unit 103 can be performed.

  Thus, by having a plurality of dictionary data, it is possible to have an electronic dictionary function corresponding to multiple languages, which is easy to use.

  For example, when the dictionary data to be used is set in an English-Japanese dictionary, when a German word is recognized, the corresponding word cannot be found. In such a case, if you have a German-Japanese dictionary in addition to the English-Japanese dictionary, you can display translations for German words by searching all the dictionary data held by the mobile phone. Is possible. As a method of searching all dictionary data, an array including all dictionary data identification numbers is prepared, and after the CPU 102 searches the dictionary data set in the dictionary flag, the first element of the array is set as the first element. The dictionary data corresponding to the stored identification number is searched. If the dictionary data is not found, the dictionary data is changed and searched for the second and third. By doing so, it is possible to realize a user-friendly dictionary function that saves the user from having to manually select dictionary data.

  The second embodiment is an example in which multilingual translation can be output by holding a plurality of dictionary data, but this is limited to languages that the image recognition unit 105 can recognize. For example, the image recognition unit 105 that can recognize only English cannot recognize Thai or Arabic. Therefore, there is a problem that even if Thai or Arabic dictionary data is held, the translation cannot be output because it cannot be recognized.

  In order to solve these problems, it is only necessary to connect to the Internet via the communication unit 108, download multilingual character pattern data, and add it to the character pattern data table. Processing in this case will be described below.

  A processing procedure when the mobile phone according to this embodiment downloads character pattern data from the Internet will be described with reference to the flowchart of FIG.

  First, the CPU 102 having received the instruction to download the character pattern data table connects to the site on the Internet where the character pattern data is written via the communication unit 108 (step S1001). With this site displayed, the user presses a button on the input unit 101 to instruct the downloading of the character pattern (S1002). In response to this, the CPU 102 downloads character pattern data from the site via the communication unit 108 and stores it in the storage unit 104 (step S1003).

  Thereafter, the CPU 102 adds the character pattern data to the character pattern data table (step S1004).

  FIG. 11 shows an example of a screen on which the mobile phone band of this embodiment connects to a site where character pattern data can be downloaded and downloads Arabic character pattern data.

  A screen 1101 displays a site on the Internet where character patterns can be downloaded. Here, when “Arabic” is selected, the screen changes to a screen 1102. A screen 1102 is a screen for confirming with the user whether or not to download Arabic character pattern data. If “yes” is selected here, the screen changes to a screen 1103. A screen 1103 is a screen indicating that an Arabic character pattern is being downloaded. In this state, it waits for a while, and when the download is completed, the screen transitions to a screen 1104. A screen 1104 is a screen indicating that Arabic character pattern data has been downloaded. Similarly, dictionary data corresponding to the character pattern of each language is downloaded.

  As described above, since the character pattern data and dictionaries of various languages can be downloaded, character recognition and translation reference can be performed for multiple languages, so that usability is improved.

  In addition, when recognizing characters other than the character pattern data table stored in the storage unit 104 in advance, the user is notified that recognition is impossible, and in that case, the user is prompted to download the character pattern data table. The screen may be displayed on the display unit 107. As a result, even if the character to be recognized cannot be recognized, it can be quickly recognized and the usability is improved.

  Furthermore, for example, when there is only dictionary data related to English words, when trying to recognize words other than English words, the user is notified that translation is impossible, and in that case, dictionary data is downloaded. A screen for prompting may be displayed on the display unit 107.

  The difference from the first embodiment is that the dictionary data stored in the storage unit 104 is increased to two dictionary data a and dictionary data b, and a dictionary flag is included.

  For example, the dictionary data a is an English-Japanese dictionary and the dictionary data b is a German-Japanese dictionary. Here, two types of languages (English and German) are assumed as the dictionary data, but dictionary data corresponding to each of them may be stored assuming three or more types of languages.

  In addition, there are a plurality of homepages such as English-Japanese dictionaries on the Internet, and there are homepages that can be browsed by anyone and homepages that can be browsed only by registered users. A homepage that can be browsed only by registered users can be browsed by the user entering a user ID and password. When a detailed translation request is instructed, a site that can be browsed only by registered users may be displayed. However, in this case, every time a detailed translation is displayed, the user must input a user ID and a password, which is inconvenient. Therefore, a contrivance is provided in which the user ID and password are described in a predetermined part of the dictionary data. FIG. 12 shows a specific example in the case where a plurality of dictionary data is held and a detailed translation homepage included in the dictionary data requires user authentication. The portable terminal can acquire dictionary data from the dictionary provider. The dictionary data is accompanied by an ID and a password. The ID and password are used for user authentication when accessing a home page on which a detailed translation written in the dictionary data is written.

  FIG. 13 shows a flowchart for displaying a homepage on which detailed translation is described using the dictionary data of this embodiment.

  In step S1301, the mobile phone acquires dictionary data. As the method, a dictionary data file existing on the Internet may be downloaded via the communication unit 108, or dictionary data stored in a storage medium may be acquired by connecting to a mobile phone.

  Character recognition is performed with the dictionary data held in this manner (step S1302), and summary translation is displayed (step S1303).

When the CPU 102 receives a signal requesting to display the detailed translation (step S1304), the CPU 102 acquires a user ID and a password necessary for accessing the server from the dictionary data. As a method of obtaining, it is conceivable that a place where the user ID and password in the dictionary data are described in advance is determined and the CPU 102 reads the place. Further, the location where the user ID and password are described is described in the header portion of the dictionary data, and the CPU 102 reads the header portion to acquire the location where the user ID and password are described, and reads the user ID and password. In step S1306, the CPU 102 transmits an access signal to the server via the communication unit 108. At this time, the user ID and password are also transmitted together. The server that receives the access signal, the user ID, and the password performs user authentication, and transmits the detailed translation data to the terminal when the authentication is successful.

  In step S1307, the CPU 102 acquires detailed translation data via the communication unit 108 and stores it in the storage unit. In step S1308, the CPU 102 reads the detailed translation data stored in the storage unit and displays it on the display unit.

  In this way, when accessing a home page that requires a user ID and password, the user is authenticated without performing an input operation, which is convenient.

  Next, a third embodiment will be described. There is a limit to human memory capacity, and it is often possible to forget the meaning of English words that have been examined before. In such a case, re-recognition is troublesome and troublesome.

  This embodiment has been studied for such a problem, and is characterized in that history information such as translation of recognized words is left. Hereinafter, the process of creating such history information and the process of using history information will be described separately.

  First, processing for creating history information will be described. FIG. 7 shows an example of the case where the search result is stored as history information in the storage unit 104. For example, as shown in FIG. 7, the word searched by the search unit 109, the time when the search was performed, etc. The history is written.

  A column 701 is a word whose character is recognized in the column 201 of the dictionary data. Column 702 represents the number of times each word has been recognized. A column 703 represents the latest time among the times recognized so far. A column 704 indicates the number of times the meaning of an English word or the like has been confirmed using history information, and a column 705 represents the last confirmed time.

  FIG. 8 is a diagram showing a flowchart when the mobile phone according to the third embodiment of the present invention creates a history table as shown in FIG. 7, for example, after character recognition. Step S406 and step S407 represent step S406 and step S407 in FIG. 4, respectively, and the flow of steps S801 to S804 is performed during that time.

  Step S406 is a state in which the outline translation of the dictionary data 106, that is, a representative translation or a simple translation is displayed. Here, it is determined whether or not the word as a result of search by the search unit 109 exists in the history information stored in the storage unit 104 (step S801), and if it exists, the process proceeds to step S802. In step S802, the CPU 102 performs an update process for incrementing the history information recognition count 702 by one. In step S803, the time acquisition unit 902 acquires the time when the processing performed by the CPU 102 reaches step S803, and describes the time in the final recognition time 703 of the history information.

  If the word does not exist in the history information in step S801, the process proceeds to step S804, and the word and the last recognition time are additionally described in the history information. As the final recognition time, the time at which the process reaches step S804 is described in the storage unit 104. Through the above processing, the history information of new recognition results is updated in the history table. In this case, the history table data stored in the storage unit 104 and the dictionary data are stored in association with each other when considering the meaning of the word again using the history, as will be described later. Is desirable. As a method of storing the information in association with each other, for example, a method of providing a space where history information can be added in the dictionary data and writing the history information in the space one after another can be considered.

  Next, processing when using the created history information will be described. First, when an instruction to refer to history information stored in the storage unit 104 is made through an operation by the user input unit 101, a history reference instruction signal is input to the CPU. The CPU 102 to which the history reference instruction signal is input processes to display the meaning of English words on the display unit 107 based on the history table data and dictionary data stored in the storage unit 104.

  FIG. 9 shows an example of a screen on which the mobile phone of the present embodiment displays word translations without performing character recognition using history information.

  When a button corresponding to “Menu” in the lower right of the screen is pressed on the screen 901, the screen transitions to the screen 902. A screen 902 is a screen for selecting a menu related to imaging and character recognition. In this state, when “(5) History table” is selected, the screen transitions to a screen 903. A screen 903 is a screen that displays a word, the number of times of recognition for the word, the number of times of confirmation for the word, etc. in the history information. If “recognition” is selected in this state, the screen transitions to the screen 904.

  A screen 904 is a screen displaying the translation of “recognition”. In this state, when a button corresponding to “confirm” is pressed, the screen transitions to a screen 905.

  A screen 905 displays a word in the history table 901, the number of times of recognition for the word, the number of confirmations for the word, and the like, similar to the screen 903. While the number of confirmations for “recognition” is 0 on the screen 903, the number of confirmations for “recognition” is increased to 1 on the screen 905 because the translation of “recognition” is confirmed on the screen 904.

  As described above, the user can check anytime, anywhere and any number of times without performing an operation for re-recognizing the meaning of the word once recognized, thereby improving usability.

  Further, by displaying the display screen as shown in FIG. 9, the user can easily notify the number of times of recognition and the number of times of confirmation. Furthermore, since the words that the user has looked at many times are obvious, if they are displayed from the top in the order of the number of recognitions and confirmations, the words that are difficult to remember can be easily understood, so they can be remembered consciously. Yes, it's even easier to use.

  In addition, as in the first embodiment, when it is configured to be able to refer to detailed information such as translations and examples (for example, the address of a home page where detailed information is described) by further operations, it is easy. When the user confirms the meaning of an English word or the like using the history information, it stores “simple translation” and “detailed”. If one of “translation” can be selected, the usability is further improved. Further, here, the history information is not limited to the information listed in FIG. 7, and any information regarding the history may be used.

  Furthermore, when the identification information is recognized by the image recognition unit 105 among the image information output to the display unit 107 by the imaging unit 103, it may be configured to notify the user whether there is history information. . For example, when it is configured to perform the main determination when the provisional determination for deciding the target for the time being and the provisionally determined target are sufficient, it is stored in the storage unit 104 at the temporary determination stage. If the presence / absence of history information is displayed on the display unit 107, the meaning of the identification information and the like can be confirmed with reference to the history information without recognizing it again. Alternatively, the user may be notified with a simple display such as “with history” by simply moving the cursor over the recognition target. When the identification information such as English words to be recognized is information that has been recognized so far, not only the display using characters, icons, marks, etc., but also a voice output unit ( Needless to say, the notification may be made by sound via a not shown).

  Furthermore, a position information acquisition unit may be provided in a mobile phone such as a GPS, and the position information acquired by the position acquisition unit may be stored in the storage unit 104 as history information. According to this, it is possible to easily confirm where the word is recognized.

  Further, when the user revisits a place stored as history information, the user may be notified of the fact, the word recognized at that time, and its meaning.

  Next, a fourth embodiment will be described. FIG. 14 is a diagram illustrating the appearance of a mobile phone according to the fourth embodiment. As shown in FIG. 14, in the mobile phone of this embodiment, a housing 200 having a display unit 107 and a housing 300 having an input unit 101 can be folded via a hinge unit 1403, and rotate to the hinge unit. Consider a mobile phone with a possible imaging unit.

  The mobile phone of this embodiment includes a fold detection unit that detects whether the mobile phone is open as shown in FIG. 14 (1) or closed as shown in FIG. 14 (2). . The fold detection unit includes, for example, a magnet in the housing 200 and a Hall element in the housing 300, and detects the open / closed state based on the Hi level or Low level of the voltage value of the Hall element to which a magnetic field is applied by the magnet. . Note that the fold detection unit may use a sensor or a concavo-convex engagement as long as the open / closed state can be detected. The display unit 107 is a display unit (main display unit) that is mainly used in an open state, but the mobile phone of this embodiment is folded (closed) in addition to the main display unit. A sub-display unit 1407 that is visible even in a state is provided.

  In order to facilitate selection of the imaging mode, a normal imaging button 1401 and a recognition imaging button 1402 are provided, for example, on the side surface of the housing.

  When the normal imaging button 1401 is pressed, the imaging unit 104 captures an imaging target, stores the captured image in the storage unit 104, and displays the captured image as necessary. When the recognition imaging button 1402 is pressed, the imaging unit 104 captures an imaging target, and the image recognition unit 105 performs character recognition, and then the CPU 102 performs processing for displaying the recognition result on the display unit 107 or the sub display unit 1407. Do.

  As described above, by providing dedicated buttons for the recognition mode and the normal imaging mode, the user can easily select each mode, which is easy to use. Also, as shown in FIG. 14 (1), if it is arranged on the side surface of the housing 300, the operation is the same whether the cellular phone is opened or folded, and the operation can be changed when opening and closing. Because there is not, usability improves. Further, as shown in FIG. 14, when the operation key is arranged at a position where the mobile phone can be viewed in the closed state such as the side of the housing, the recognition mode can be executed even in the closed state, so the mobile phone is purposely opened. Character recognition and the like can be performed without any problem, and it is easy to use.

  The normal imaging button 1401 and the recognition imaging button 1402 are preferably different buttons, but the same button may be used. However, in this case, when the buttons are the same, it is necessary to distinguish between a button operation for imaging and a button operation for recognition. In this case, for example, the imaging mode and the recognition mode may be switched according to the time when the button is pressed or the number of times the button is pressed (double click or the like). In addition, by making the buttons the same in this way, it is possible to reduce the size of the mobile phone without taking up space. The normal imaging button 1401 and the recognition imaging button 1402 may be any sensor that can be handled by the user among the sensors attached to the mobile phone 100.

  In the above description, the case where the normal imaging button 1401 and the recognition imaging button 1402 are provided on the side surface of the housing has been described. However, the present invention is not limited to this, and the button may be a button that can be operated in a closed state. For example, you may take the structure arrange | positioned on the back surface of a mobile telephone, for example. Here, the back surface of the mobile phone is a surface facing the display unit 107 that can be viewed in a folded (closed) state as shown in 1407 of FIG. 14 (2). Yes, it is a surface different from the surface mainly constituted by the battery.

  Next, a fifth embodiment will be described. As for the appearance of the mobile phone of this embodiment, consider a mobile phone that can be folded as shown in FIG. 14 as in the fourth embodiment. Hereinafter, an example will be described in which summary translation or the like for the recognition result is displayed on the main display unit 107, and when the mobile phone is opened, detailed translation or the like for the recognition result is displayed on the sub display unit 1407.

  The processing procedure of the mobile phone in the present embodiment will be described using the flowchart of FIG.

  First, step S1501 is a state in which the mobile phone is waiting for a key input from the input unit 101. In this state, when the CPU 102 detects a signal for starting the camera function of the mobile phone (step S1502), the fold detection unit detects whether the mobile phone is in a folded state.

  If the mobile phone is in a folded state, a signal indicating that the mobile phone is in a folded state is input to the CPU 102 by the fold detection unit (step S1503). The information is displayed on the display unit 1407 (step S1504). Next, when the shutter button is pressed by the user or the like, a signal for instructing imaging is input to the CPU 102 (step S1505), and the image information of the subject imaged (converted) by the imaging unit 103 is stored via the CPU 102. It is stored in 104 (step S1506).

  Next, the image recognition unit 105 reads the image information stored in the storage unit 104, reads the identification information included in the image information, for example, the word “recognition” in the sentence of the foreign book, and stores the recognition result in the storage unit 104. (Step S1507). The search unit 109 reads this result, and searches for a matching word from the words in the column 201 of the dictionary data 106 stored in the storage unit 104. Again, the folding detection unit detects whether or not the mobile phone is in a folded state, and inputs information on the open / close state to the CPU 102 as a signal. The CPU 102 refers to the opening / closing state information (step S1508), and if it is determined in step S1509 that the mobile phone is in a closed state, the process proceeds to step S1510. In step S1510, the CPU 102 reads the column 202 (translation summary) corresponding to the searched word from the dictionary data and displays it on the sub display unit 1407.

  If it is determined in step S1509 that the mobile phone is open, the CPU 102 reads the URL corresponding to the searched word from the dictionary data (step S1511), and the CPU 102 An instruction signal is issued to connect to the URL. In step S 1512, the CPU 102 reads the content written on the homepage of the URL via the communication unit 108 and stores it in the storage unit 104. Thereafter, the CPU 102 displays the contents on the display unit 107.

  FIG. 16 is an example of a display screen of the present embodiment, and the contents of the homepage of the URL describing the detailed translation held in the dictionary data 106 when the mobile phone is opened from the closed state by the user. The example of the screen which displays (description information) on the display part 107 is shown. In screens 1601 to 1603, the display such as “menu” displayed at the lower left of the screen is associated with button 1408a, and the display such as “recognition” displayed at the lower right of the screen is associated with button 1408b. When the user presses the associated button, the screen changes. For example, when “menu” is displayed at the lower left of the screen, the button 1408a is referred to as a “menu” button.

  The screen 1601 is displayed when the mobile phone is folded, when “recognition mode” is selected as the imaging mode, for example, when the word “Recognition” in English is identified and attempted to be recognized. It is an example of a screen displayed on the display unit 1407. When the “recognition” button is pressed in this state, the screen transitions to a screen 1602.

  A screen 1602 indicates that “Recognition” has been selected as a recognition target. In this screen, image portions other than “Recognition” on the screen are displayed in black, and “Recognition” is emphasized. Also, sneak shots can be prevented by painting the parts other than letters black. When the “OK” button is pressed in this state, the screen changes to a screen 1603.

  A screen 1603 represents a screen on which a summary translation of “Recognition” as a recognition target is displayed on the sub display unit 1407. When the mobile phone is opened in this state, a screen 1604 is displayed on the display unit 107.

  A screen 1604 is a screen example in which detailed translation is displayed on the display unit 107 in a state where the mobile phone is opened.

  In this way, since character recognition can be performed even when the mobile phone is folded, it is possible to easily perform character recognition without having to open the mobile phone one by one. Furthermore, when the user wants to know detailed translation, the detailed translation can be displayed on the display unit 107 simply by opening the mobile phone in a state where the summary translation is displayed, which is very convenient.

Next, a sixth embodiment will be described. The mobile phone according to the present embodiment displays on the display unit 107 the result of the image recognition unit 105 recognizing the image captured by the image capturing unit 103, the identification information such as characters included in the image, the summary translation for the recognition result, and the like. It is assumed that when displayed, a recognition result editing means for converting each character of the recognition result into a different character is provided.
FIG. 17 illustrates a case where the mobile phone of the present embodiment captures a character string “Recognition” written on a paper surface and the image recognition unit 105 recognizes the result “Reconifiom”. The example of a screen which edits a result is represented.

  On the screen 1701, the imaging unit 103 captures an object that is a subject, and the character string “Recognition” included in the image is erroneously recognized as “Reconifiom”, for example, by “Reconifiom”. When the search unit 109 searches for the closest word from the dictionary data 106 and the search result is “reconfirm”, the recognition result “Reconifiom”, the search result “reconfirm”, and the outline translation “ [Verb] Reconfirmation is an example of a screen displayed on the display unit 107. If the “Menu” button is pressed here, the screen changes to a screen 1702.

  A screen 1702 is a screen that displays a menu related to the imaging function and the recognition function. Here, when “(1) Edit result” is selected and the “OK” button is pressed, the screen transitions to a screen where the recognition result can be edited.

  A screen 1703 represents a screen in which “g” is added between “Reco” and “nifiom” of “Reconifiom” on a screen where the recognition result can be edited. On the screen on which the recognition result can be edited, a cursor 1706 is displayed in an area 1705 where characters of the recognition result are displayed. Characters can be input / deleted by selecting a position to be edited using the cursor 1706 and pressing a button related to character input in the input unit 101 several times. The character of the recognition result edited in this way is stored in the storage unit 104 by the CPU 102. When the “Search” button is pressed in this state, an input signal for instructing the CPU 102 to perform a search is input by the input unit 101. In response to this, the CPU 102 reads the recognition result character edited with reference to the storage unit 104 and passes it to the search unit 109. In response to this, the search unit 109 searches the dictionary data 106 for a matching word and displays it in the search result display area.

  An example of the screen is a screen 1704. A screen 1704 is an example of a screen that is displayed when the “search” button is pressed after editing the character of the recognition result to “Recognitiom” on the screen where the recognition result can be edited. “Recognition” is displayed as a search result.

  In this way, when the recognition result is misrecognition and a character different from the English word that the user wanted to recognize is displayed as the search result, it is not necessary to re-image and recognize it, which is convenient. .

  In the above description, when the “search” button is pressed, the search unit 109 searches the dictionary data 106 for the word closest to the recognition result. However, in the screen where the recognition result can be edited, After editing the character, the recognition result editing means may input an input signal for instructing the CPU 102 to perform a search, and the search unit 109 may perform the search. In this case, the user does not have to press the “search” button every time the user wants to display the search result, which is convenient.

  Next, a seventh embodiment will be described. This embodiment assumes a case where a result recognized by the image recognition unit 105 using a mail function or the like provided in advance in a mobile phone or the like, that is, simple information or detailed information is transmitted.

  In this case, for example, a predetermined operation is performed by the input unit 101, and the CPU 102 that has received the input signal performs processing to shift to the recognition mode. If the user performs an operation for activating the mail function after recognizing characters or the like to be recognized in the recognition mode via the input unit 101 (for example, pressing the mail activation key), the CPU 102 that has received the input signal. Starts mail software stored in the storage unit 104 in advance on the display unit 107 and displays a mail creation screen so that mail can be transmitted. At that time, when the CPU 102 recognizes and displays, for example, a typical translation of the English word “abalone” such as “abalone”, the CPU 102 automatically inserts it into the text field of the mail composition such as “abalone → abalone”. To process. In addition, as in the first embodiment, not only simple information but also detailed information such as URL information in which detailed information such as an example or similar information is described can be acquired. Correct information is automatically inserted into the text field.

  The address information (name, telephone number, or mail address information) stored in the storage unit 104 may be selected by the user and inserted into the destination field of the transmission partner.

  As described above, the recognition result obtained by the image recognition unit 105 can be transmitted to a partner to be transmitted by a simple operation, and the usability is improved. Furthermore, if a subject such as “OCR result” is automatically inserted and described in the subject column, it is possible to save further input and improve usability.

  Also, when creating a mail using the recognition result, if a mail address is recognized, a method of automatically setting the mail address as a destination in the destination column is also conceivable. However, with this method, for example, even when an e-mail address is recognized and the e-mail address is to be written in the mail text field, the recognition result cannot be used because the e-mail address is set.

  In order to solve these problems, for example, in the case of creating a mail, it is possible to select which item to insert the recognition result out of items such as a mail destination, a subject, and a text field. This function will be described with reference to FIG.

  The screen (1) is a screen that displays a recognized image and a recognition result when the recognition target “yamazaki@xxx.jp” is recognized. When the “menu” button is pressed in this state, the screen transitions to the screen (2). Screen (2) is a screen for selecting which function to use using “yamazaki@xxx.jp” as the recognition result. When creating a mail, if “(2) Create mail” is selected, the screen changes to screen (3). The screen (3) is a screen for selecting the item into which the recognition result is inserted among the items for creating the mail. When “(3) Body” is selected on this screen, a transition is made to a mail creation screen in which “yamazaki@xxx.jp” is substituted for the body of the mail (screen (4)).

  In this way, if the user can select which item in the email the recognition result is inserted into, the recognition result can be displayed in the body of the email just by performing a few input operations after displaying the recognition result. You can move to the creation screen of the inserted mail, improving usability.

  Furthermore, when the menu of the screen (2) is displayed, the position of the cursor can be determined by the type of character string that is the recognition result. For example, if the recognition result includes “@” in the recognition result, the CPU 102 determines that it is a mail address, and if “http” is included, the CPU 102 determines that it is a mail address. When the screen (2) is displayed, the cursor position is displayed on the display unit 107 in accordance with “(2) Create Mail”, and when it is determined that the URL is a URL, the cursor position is aligned with “(1) Internet”, etc. Just do it. In this way, since the operation of selecting the menu on the screen (2) can be omitted, the usability is further improved.

  Further, when the recognition result is an e-mail address, if it is desired to register the e-mail address in the address book, “(4) Address book registration” may be selected on the screen (2). In this case as well, the same effect as described above can be obtained by making it possible to select in which item of the address book the mail address of the recognition result is to be inserted.

  In the above description, the “address”, “subject”, and “text” fields are considered as the fields of the mail software creation screen. However, the present invention is not limited to this. In addition to this, even if a column is provided, if the related information is automatically inserted, it is possible to save the input and to improve the usability.

  In the above, the configuration is such that the recognition result (eg, URL information) is automatically inserted in the mail creation screen column. However, the user himself / herself inputs a comment or the like via the input unit 101. You can make it possible. As a result, when the OCR function is performed and the result is transmitted to the call partner, the user can not only simply transmit the result but also add a comment to the result. At that time, if a fixed sentence or pictograph stored in advance in the storage unit 104 is used, the usability is further improved.

  In the above embodiment, a usage scene in which the user wants to know the meaning and examples in detail is assumed for words such as English, but the present invention is not limited to this. For example, sports, movies, history, restaurants, Information regarding words of various genres such as science, dinosaurs, animals and plants, and entertainers may be referred to. As a result, the categories that can be referred to increase and the usability of the user improves. In this way, when there is a lot of information that can be referred to, for example, when a user is looking at an article such as a magazine or a newspaper, or when casually walking in the city, a word of interest is noticeable, and its details Even in situations where you want to refer to other information, you can quickly refer to the meaning of the word and so on, which is very convenient.

  In addition, for example, when the user wants to refer to the detailed information of the dinosaur such as “gamelan”, the user's satisfaction is low when the detailed information of the “gamelan” of the plant is acquired. Therefore, when requesting detailed information, for example, a notification such as “Which genre do you want detailed information?” Is displayed on the display unit 107 so that the user can select various genres via the display screen or the like. If it is, it will be even easier to use.

  Next, an eighth embodiment will be described. In the present embodiment, the co-occurrence relationship of words is used in order to notify the exact meaning of the recognized English word. Co-occurrence means that a plurality of words appear in the same document.

  In the first place, the same word often has a plurality of different meanings even if it is the same word, and if the Japanese translation of the dictionary prepared in advance is displayed as it is, what meaning is the recognized English word used? Must be considered by the user reading the context. At that time, if it is caught in the wrong meaning, there is a problem that the sentence itself including the English word is caught in the wrong meaning. The present embodiment has been made with this point in mind.

  FIG. 19 is a diagram showing a configuration example according to the eighth embodiment of the present invention. This embodiment includes a mobile phone 100 and a server 1900.

  The server 1900 includes an SV-CPU 1902, a co-occurrence relation database 1903, an SV storage unit 1904, an SV image recognition unit 1905, an SV dictionary data 1906, an SV communication unit 1908, an SV search unit 1909, a user registration database 1910, and the like. Composed.

  The SV communication unit 1908 performs communication processing for the server 1900 to access the Internet or other information processing apparatuses.

  The SV-CPU 1902 operates based on a program stored in the SV storage unit 1904, and controls each unit according to a signal from the SV communication unit 1908, for example.

  The co-occurrence relation database is data relating to the co-occurrence relation. For example, as shown in FIG. 20, a word, a translation of the word, a co-occurrence word (hereinafter referred to as a co-occurrence word), and a co-occurrence word It is configured to include a ratio that corresponds to FIG. 20 is an example in which the word is “character” and “character” and “character” are assumed as the translation of the “character”.

  The co-occurrence relation database 1903 holds data for a plurality of words, and holds a translation of each word and a probability value used as the translation.

  When there is a recognized word and a word that co-occurs with the word, the most appropriate translation for the recognized word is obtained by comparing the value a and the value b for the word using the co-occurrence relation database 1903. Can do. For example, the Japanese translation of “character” includes “character” and “character”, but if the word “recognition” co-occurs, “character” is appropriate as the Japanese translation of “character” Assume that the ratio is 0.95 and that the ratio of “characters” is 0.05. In this case, as a Japanese translation of “character”, a “character” having a large value indicating an appropriate ratio may be selected.

  The SV storage unit 1904 stores data handled by the server 1900 such as data received from the communication unit. The SV image recognition unit 1905 recognizes identification information such as characters and symbols included in the image information.

  In the SV dictionary data 1906 stored in the storage unit 104, for example, identification information such as words in a foreign language, a Japanese translation of the identification information, various Japanese translations of the identification information, usage / examples, and the like are described in detail. URL of the homepage that has been posted, and images and sounds related to the identification information.

  The SV search unit 1909 selects an appropriate Japanese translation of the word 2001 with reference to the co-occurrence relation database 1903 using the word and the word co-occurring with the word.

  The user registration database 1910 is a database that records a user ID and password for proving that the user can connect to the server 1900.

  Next, the processing procedure of the mobile phone in the present embodiment will be described using the flowchart of FIG.

  By performing the same processing as in steps S1501 to S1507 in FIG. 15, the result of capturing the image and recognizing the characters from the monitor display state (step S2101) is stored in the storage unit 104 ( Step S1502 to Step S1504). In step S2105, recognition position data and recognition word data are extracted from the stored recognition results. Here, the recognition position data is data related to the position on the image of the word to be recognized included in the image captured by the image capturing unit 103, and the recognized word data is the search unit in step S2105. This is a word found as a result of the search by 109.

  Next, when an access signal that instructs to connect to the server 1900 is input to the CPU 102 via the input unit 101 or the like, the CPU 102 transmits the access signal to a predetermined server via the communication unit 108 (step S2107).

  Next, the server 1900 that received the access signal transmits a user ID and a password input request signal corresponding to the user ID. If the mobile phone 100 receives the request signal (step S2108), the CPU 102 displays a screen for inputting a user ID and password on the display unit 107. If the user ID and password are input via the input screen or the like, they are transmitted to the server 1900 (step S2109). Here, although the form which authenticates a user who inputs a user ID and a password was considered, if authentication of a user can be performed, it will not be limited to this. For example, authentication may be performed using a voiceprint, fingerprint, iris, or the like.

  The CPU 102 determines whether the signal received from the server is authentication success data or authentication failure data (step S2110). If the signal is authentication failure data, the input screen is displayed again on the display unit 107. If the signal is authentication success data, the image captured by the imaging unit 103, recognition position data, recognition word data, and the like are transmitted to the server 1900 (step S2111).

  Thereafter, upon receiving optimum translation data indicating a translation for the recognized word data from the server 1900 (step S2112), the CPU 102 displays the optimum translation data on the display unit 107 (step S2113).

  Next, the processing procedure of the server 1900 in the present embodiment will be described using the flowchart of FIG.

  When the server 1900 receives an access signal from a terminal such as the mobile phone 100 (step S2201), the server 1900 requests the mobile phone 100 to input a user ID and password (step S2201). Is received (step S2202), the SV-CPU 1902 refers to the user registration data 1910. If a match with the user ID and password registered in the user registration data 1910 is found, the SV-CPU 1902 notifies the mobile phone 100 that the user authentication has been confirmed via the SV communication unit 1908 (step S2204). ), The image, the recognition position data, and the recognition word data are received from the mobile phone 100 (step S2205).

  Next, the SV image recognition unit 1905 recognizes characters included in the received image. FIG. 23 shows an example of a received image. When performing the recognition process, the SV image recognition unit 1905 first extracts all the words included in the image and obtains recognition position data thereof. Among these, a word whose one side of the extraction region is at the edge of the image is regarded as a word not to be recognized.

  In the image example of FIG. 23, a word surrounded by a shaded square such as the word 2301 is regarded as a word that is not a recognition target. In the example of the image of FIG. 23, other words such as the word 2303 “The” are recognized and the extracted word does not reach the end of the image. The SV search unit 1909 searches from 1906 (step S2206).

  Here, a word that the user wants to recognize is defined as a “recognition target word”, and a target word that is actually recognized by the mobile phone is defined as a “recognition word”. Usually, the recognition target word matches the recognition word, but the two may be different due to low recognition accuracy or the like. For example, when a user desires a translation of “abalone”, it may be mistakenly recognized instead of “abalone”, and the recognition target word may be “abalone” and the recognition word may be “abalome”. In that case, it is searched whether or not “abalome” recognized in error is in the dictionary data, and if not, the most similar word (for example, abalone) among the words included in the dictionary data is regarded as the recognized word. Processing is performed.

  Next, the number n of recognition target words (hereinafter referred to as co-occurrence words) other than words (hereinafter referred to as translation target words) determined based on the recognition position data and recognition word data received from the mobile phone is measured (step S2207). ). In step S2208, if the number n of co-occurrence words is one or more, it is determined whether or not there is a translation target word in the co-occurrence relation database 1903 (step S2209). It is determined whether there is a word that matches the co-occurrence word in 2004 (step S2210).

  If a word that coincides with the co-occurrence word is found in the co-occurrence relation database, the most appropriate translation of the translation target word translations is determined (step S2211).

  As a determination method, the probability is calculated based on the respective translations for the co-occurrence words in the co-occurrence relation database 1903. For example, consider a case where the translation target word is “character” and the co-occurrence words are “string” and “recognition”. The ratio that the translation is “character” when “string” exists in the co-occurrence word is 0.90, and the ratio that the translation is “character” when “recognition” exists in the co-occurrence word is 0.95, these values The sum is 1.85. If the same calculation is performed on the translated “character”, the total value is 0.08. The translation that maximizes these values can be treated as the most appropriate translation. In the above description, the optimal translation is determined by adding the ratio values. However, the present invention is not limited to this, and may be determined by combining multiplication, division, division, or the like. Finally, the step of transmitting the optimum translation data to the terminal (S2212).

  If the number n of co-occurrence words is 0 in step S2208, the process proceeds to step S2213, and a representative translation is searched from the SV dictionary data 1906. In step S2214, the translation is transmitted to the terminal.

  If there is no translation target word in the co-occurrence relation database 1903 in step S2209 and there is no word matching the co-occurrence word in the co-occurrence relation database 1903 in step S2210, the process similarly proceeds to step S2213. Send the translation to the terminal.

  In the above flow, the optimal translation data is determined in step S2211, but a plurality of translation candidates may be determined. For example, six total values of ratios corresponding to the co-occurrence words are obtained in descending order. This data is called translation display order data. FIG. 25 (1) shows a state before the translation of “character” is rearranged, and FIG. 25 (2) shows a state after the order is sorted in an appropriate order as the meaning of the translation.

  A translation is transmitted to the terminal based on the translation display order data. The CPU 102 of the mobile phone that has received the translation display order data controls to rearrange the translations based on the translation display order data and display them on the display unit 107.

As a result, the user can know the meaning of the recognition target word and can easily know what the recognition target word is likely to have in context, which is easy to use. It is assumed that the optimal translation selected by the user is wrong, but even in such a case, the user himself / herself can select not only the optimal translation as shown in FIG. It can be a great help when judging from above.

  Next, a processing example related to the exchange between the mobile phone and the server will be described with reference to FIG. Here, it is assumed that a word that is a cellular phone is recognized and an optimal translation is received from the server.

  First, when a certain word (hereinafter referred to as a recognition target word) is recognized by the imaging unit 103 and the image recognition unit 105 (step S2401), the CPU 102 recognizes the captured image data and a recognition word obtained by converting the recognition target word included in the image data into data. Recognition position data indicating the position of the data and the recognition target word is extracted (step S2402). Next, when the user gives an instruction to refer to the detailed translation of the recognition target word via the input unit 101 or the like, the CPU 102 transmits an access signal for connecting to the server (step S2403).

  The server that has received the access signal transmits an ID / password request signal for requesting the user ID and password to the mobile phone (steps S2404 and S2405), and the CPU 102 of the mobile phone that has received the ID / password request signal After the user inputs the ID and password via the input screen or the like, the ID and password are transmitted to the server (step S2406).

  Next, the server authenticates whether or not the user is a regular registrant based on the received user ID and password. As a result of authentication, the server 1900 transmits authentication success data when it is recognized as a regular registrant (valid), and transmits authentication failure data when it is not recognized as a regular registrant (invalid). (Steps S2407 to S2409). Here, consider a case where authentication is valid.

  The mobile phone that has received the authentication success data transmits the captured image data, the recognition word data obtained by converting the recognition target word included in the image data into data, and the recognition position data indicating the position of the recognition target word to the server (step S2410). Step S2411).

  As described above, the server that has received the image data, the recognition word data, and the recognition position data specifies the recognition target word included in the image data from the recognition word data (step S2413), and the recognition word included in the image data. A word other than data and a word (co-occurrence word) that is cut off in the middle of the word at the top, bottom, left, and right edges of the image data and protrudes from the frame of the image data are extracted, and recognition processing is performed (steps S2412 to S2414). ). Thereafter, referring to the co-occurrence relation database owned by the server, the optimum translation data indicating the optimum translation of the recognition target word is transmitted to the mobile phone (steps S2415 and S2416).

  The mobile phone that has received the optimal translation data displays the optimal translation data on the display unit 107 (steps S2417 and S2418).

  As described above, the user can refer to an optimal translation in accordance with the context of translation of a certain word, and is convenient. In addition, the processing load and the storage capacity of the mobile phone are borne by causing the server, which has a processing capacity and a storage capacity, to be superior to that of the mobile phone, to store the co-occurrence database and to determine the optimal translation from the co-occurrence database This contributes to a reduction in the amount of data and a system that is balanced in terms of processing capacity and storage capacity.

  Further, from the standpoint of the server, if a service such as the “optimal translation service” is assumed using the server that performs the information processing as described above, for example, a fixed amount of income can be obtained every month. But it is effective.

  In addition, instead of providing the user for free or at a low price, the user may contract with an advertising company to obtain income as an advertising fee from the advertising company. In this case, the server adds the data related to the advertisement to the optimum translation data and transmits it to the mobile phone, and the mobile phone that has received the advertisement data and the optimum translation data displays the advertisement data together with the optimum translation data. That's fine. At that time, if the server also sends location data for displaying both so that the advertisement data does not overlap with the optimal translation data, the mobile phone that received the location data will display based on the location data The user can view the optimal translation and advertisement without mixing.

  The various functions described above may be realized by software, and the user can realize the various functions by receiving the software from, for example, a recording medium or a server. This makes it easy to install only necessary functions and to add, delete, or update various functions according to preference.

  Note that the image recognition unit and the search unit included in the mobile phone and the server may be configured by a CPU. Furthermore, it goes without saying that the embodiments described above can be combined.

  Further, the present invention is not limited to the above-described embodiments, and the principles and new characteristics disclosed herein include a wide technical scope.

1 is a configuration block diagram of a mobile phone according to a first embodiment. It is the figure which showed the example of the content of dictionary data. It is the figure which showed the example of the content of dictionary data. It is a flowchart which shows the process sequence in a 1st Example. It is an example of the display screen in a 1st Example. It is an example of the display screen in a 2nd Example. It is an example of the history table which concerns on a 3rd Example. It is a flowchart which shows the process sequence in a 3rd Example. It is an example of the display screen in a 3rd Example. It is a processing flow figure of the Example which downloads character pattern data. It is a table | surface screen example of the Example which downloads character pattern data. It is an image figure of the Example by which ID and a password are linked | related with dictionary data. It is a processing flow figure of the Example by which ID and a password are linked | related with dictionary data. It is a general-view figure of the mobile phone in the 4th or 5th Example. It is a flowchart which shows the process sequence in a 5th Example. It is an example of a display screen of a mobile phone in the fifth embodiment. It is an example of the display screen of the mobile phone in the sixth embodiment. It is an example of the display screen of the mobile phone in the seventh embodiment. It is a whole system figure in an 8th Example. It is an example of a co-occurrence relation database. It is a flowchart which shows the process sequence of the mobile telephone of an 8th Example. It is a flowchart which shows the process sequence of the server of an 8th Example. It is a figure explaining the positional relationship with a captured image, the recognition object word, and a co-occurrence word in the 8th Example. It is a processing sequence figure of the mobile telephone and server in an 8th Example. It is an example of a display screen of a mobile phone in the eighth embodiment.

Explanation of symbols

100 mobile phone 101 input unit 102 CPU
DESCRIPTION OF SYMBOLS 103 Image pick-up part 104 Memory | storage part 105 Image recognition part 106 Dictionary data 107 Display part 108 Communication part 109 Search part 110 Dictionary flag 200 Case 300 which has a main display part Case 201 which has an input part Foreign language 202 Summary translation 203 Detailed translation URL
204 URL common part 205 URL creation method 701 Word 702 Recognition frequency 703 Final recognition time 704 Confirmation frequency 705 Final confirmation time 1401 Normal imaging button 1402 Recognition imaging button 1403 Hinge unit 1407 Sub display unit 1408a Input unit 1408b Input unit 1900 Server 1902 SV- CPU
1903 Co-occurrence relation database 1904 SV storage unit 1905 SV image recognition unit 1906 SV dictionary data 1908 SV communication unit 1909 SV search unit 1910 User registration database 2301 Non-recognition word 2303 Recognition target word

Claims (36)

A communication unit that communicates with other devices;
An imaging unit that outputs image information;
A recognition unit for recognizing identification information included in the image information output by the imaging unit;
A related information acquisition instruction unit for instructing acquisition of related information related to the identification information recognized by the recognition unit;
An information processing apparatus comprising: a control unit that controls to acquire and display the related information via the communication unit when the related information acquisition instruction unit is instructed to acquire the related information. The information processing apparatus according to claim 1,
A receiving unit for receiving an ID and a password for acquiring the related information;
The information processing apparatus according to claim 1, wherein the control unit controls to transmit the ID and password received by the receiving unit when the related information acquisition instructing unit instructs acquisition of the related information. The information processing apparatus according to claim 2,
The information processing apparatus according to claim 1, wherein the related information is a homepage address where translation in a foreign language is described. The information processing apparatus according to claim 1,
The information processing apparatus, wherein the identification information is a foreign language word, and the related information is information related to the foreign language word. The information processing apparatus according to claim 4.
The information processing apparatus, wherein the related information includes an example or translation of the foreign language. The information processing apparatus according to claim 4.
An information processing apparatus comprising: a dictionary selection unit for selecting the dictionary information, wherein the related information includes a plurality of foreign language dictionary information. The information processing apparatus according to claim 4.
A receiving unit for receiving a character pattern for recognizing a plurality of languages;
A language pattern storage unit for storing the character pattern received by the reception unit,
The information processing apparatus, wherein the recognition unit recognizes the foreign language word based on the character pattern stored in the language pattern storage unit. An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A storage unit that stores word information and information related to the word information in association with each other;
A recognition unit for recognizing word information included in the image information displayed by the display unit;
A detailed information acquisition instruction unit for instructing acquisition of information related to word information recognized by the recognition unit;
A control unit that controls to display information on the word information stored in the storage unit on the display unit when acquisition of information on the word information is instructed by the detailed information acquisition instruction unit. A characteristic information processing apparatus. The information processing apparatus according to claim 8.
An information processing apparatus comprising: an editing unit that creates or edits information related to the word information stored in the storage unit. A communication unit for communicating with other devices;
An imaging unit that outputs image information;
A recognition unit for recognizing word information included in the image information output by the imaging unit;
A storage unit for storing information on the address of a home page on which detailed information on word information is described;
A detailed information display instruction unit for instructing to display detailed information on the word information recognized by the recognition unit;
When an instruction to display the detailed information is given by the detailed information display instruction unit,
An information processing apparatus comprising: a control unit that controls to access the home page based on the address of the home page stored in the storage unit via the communication unit and to display the content of the home page . The information processing apparatus according to claim 10.
The information processing apparatus, wherein the storage unit distinguishes and stores a common part in the address of the home page and a unique part for each word information. A communication unit for communicating with other devices;
An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A storage unit that stores a word, simple information related to the word, and address information of a homepage on which detailed information related to the word is described;
A recognition unit for recognizing a word included in the image information displayed by the display unit;
A detailed information display instruction unit for instructing display of detailed information regarding the word recognized by the recognition unit;
When the recognition unit recognizes the word, the recognition result and simple information about the word are displayed on the display unit, and the detailed information display instruction unit instructs the display of the detailed information. And a control unit that controls to access the homepage stored in the storage unit via the communication unit and display the contents of the homepage on the display unit. . An imaging unit that outputs image information;
A recognition unit for recognizing the image information output by the imaging unit;
A storage unit that stores the identification information and related information related to the identification information in association with each other;
A history storage unit that stores identification information recognized by the recognition unit as history information;
A history acquisition instruction unit for instructing acquisition of history information stored by the history storage unit;
A history selection unit that selects at least one of the history information instructed by the history acquisition instruction unit;
An information processing apparatus comprising: a control unit that controls to display the related information corresponding to the history information when the history information is selected by the history selection unit. The information processing apparatus according to claim 13,
The related information stored by the storage unit is distinguished and stored as simple information of the identification information and detailed information of the identification information. The information processing apparatus according to claim 13,
When the history information is selected by the history selection unit, the history selection unit includes a related information selection unit that selects whether to display simple information of the history information or detailed information of the history information on the display unit. Information processing apparatus. The information processing apparatus according to claim 13,
The information processing apparatus, wherein the related information includes at least address information of a homepage described in detail with respect to the identification information. The information processing apparatus according to claim 13,
The information processing apparatus according to claim 1, wherein the control unit performs control so as to notify when the identification information recognized by the recognition unit is stored in the storage unit. An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A recognition unit for recognizing a word included in the image information displayed by the display unit;
A history storage unit that stores words recognized by the recognition unit as history information;
A history display instruction unit for instructing display of the word stored by the history storage unit;
When a display instruction is given by the history display instruction unit, a display screen for selecting the word is displayed on the display unit, and related information of the selected word is controlled to be displayed on the display unit. An information processing apparatus comprising a control unit. The information processing device according to claim 13 or 18, wherein the history storage unit stores the identification information recognized by the recognition unit or the number of times the word is recognized. A first casing having a first display section and a second casing having a first operation section are provided, and the first display section and the first operation section are opposed to each other via a hinge section. In the information processing apparatus that can be folded as
An imaging unit that outputs image information;
A second display unit that is arranged at a position that can be referred to when the information processing apparatus is folded, and that displays image information output by the imaging unit;
A second operation unit disposed at a position where the information processing apparatus can be pressed in a folded state;
A recognition unit for recognizing identification information included in the image information displayed on the second display unit;
When an instruction to recognize the identification information displayed on the second display unit is given via the second operation unit, the identification information displayed on the second display unit is displayed by the recognition unit. An information processing apparatus comprising: a control unit that performs control so as to be recognized. The information processing apparatus according to claim 20,
A fold detection unit for detecting whether or not the information processing apparatus is in a folded state;
A storage unit that stores related information related to the identification information;
The control unit recognizes the identification information displayed on the second display unit via the second operation unit, and then detects that the folding detection unit is open. An information processing apparatus that controls to display the related information on the first display unit. The information processing apparatus according to claim 21, wherein
A communication unit that communicates with other devices,
The related information includes a homepage address where detailed information of the identification information is described,
The control unit controls to display the contents of the home page on the main display unit by accessing the home page via the communication unit when the folding detection unit detects the open state. An information processing apparatus characterized by that. It has a first casing having a main display section and a second casing having a main operation section, and can be folded so that the main display section and the main operation section face each other via a hinge section. In the information processing apparatus in which the imaging unit that outputs image information is provided in the hinge unit and is rotatable,
A sub-display unit that is arranged at a position that can be referred to when the information processing apparatus is folded, and that displays image information output by the imaging unit;
A sub operation unit disposed at a position where the information processing apparatus can be pressed in a folded state;
A recognition unit for recognizing a word included in the image information displayed on the sub display unit;
Control is performed so that the recognition unit recognizes the word displayed on the sub display unit when an instruction to recognize the word displayed on the second display unit is issued via the sub operation unit. An information processing apparatus comprising a control unit. A communication unit that communicates with other devices;
A mail software storage unit for storing software for sending and receiving mail via the communication unit;
An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A recognition unit for recognizing identification information included in the image information displayed by the display unit;
A related information acquisition instruction unit for instructing acquisition of related information related to the identification information recognized by the recognition unit;
Software activation selection unit for selecting whether to activate the software stored in the mail software storage unit after the acquisition of the related information is instructed by the related information acquisition instruction unit and the related information is displayed on the display unit When,
When activation of the software is selected by the mail software activation selection unit after the identification information is recognized by the recognition unit, a mail creation display screen is displayed with at least a destination column for inputting a transmission destination and a character to be transmitted. A mail creation screen composed of a text field for input is displayed on the display unit, and at least one of the identification information recognized by the recognition unit and the related information is inserted in the text field An information processing apparatus comprising: a control unit that performs control so as to perform. A communication unit that communicates with other devices;
An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A recognition unit for recognizing word information included in the image information displayed by the display unit;
An email address determination unit that determines whether the word information recognized by the recognition unit is an email address;
A mail software storage unit for storing software for sending and receiving mail via the communication unit;
A mail software activation selection unit for selecting whether to activate the software stored in the email software storage unit;
When activation of the software is selected by the mail software activation selection unit after the word information is recognized by the recognition unit, at least a destination field for inputting a destination of a transmission partner and a text for inputting characters to be transmitted A mail composing screen composed of a field and a word information recognized by the recognizing unit when the word information recognized by the recognizing unit is an e-mail address. Control for inserting information, and on the other hand, if the word information recognized by the recognizing unit is determined not to be an e-mail address by the e-mail address determining unit, control is performed so that the word information is inserted in a column other than the destination column An information processing apparatus comprising a unit. A communication unit that communicates with other devices;
An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A recognition unit for recognizing word information included in the image information displayed by the display unit;
A mail software storage unit for storing software for sending and receiving mail via the communication unit;
A mail software activation selection unit for selecting whether to activate the software stored in the email software storage unit;
When activation of the software is selected by the mail software activation selection unit after the word information is recognized by the recognition unit, at least a destination field for inputting a destination of a transmission partner and a text for inputting characters to be transmitted A mail composing screen composed of a field and a display section. When the word information recognized by the recognition section is the mail address, the word information is inserted into the destination field, and the recognition section An information processing apparatus comprising: a control unit that controls to insert the word information into a field other than the destination field when the recognized word information is not an email address. A communication unit that communicates with other devices;
An imaging unit that outputs image information;
A display unit for displaying the image information output by the imaging unit;
A recognition unit for recognizing identification information included in the image information displayed by the display unit;
A software storage unit that stores at least software for sending and receiving mail via the communication unit;
A mail software activation selection unit for selecting whether to activate the software stored in the software storage unit;
When the activation of the software is selected by the mail software activation selection unit after the identification information is recognized by the recognition unit, at least a destination column for inputting a destination of a transmission destination and a character to be transmitted A control unit that controls to display on the display unit a display screen on which the identification information recognized by the recognition unit can be inserted into which of the mail composition screens composed of a body field. An information processing apparatus characterized by that. An imaging unit that outputs image information;
A recognition unit for recognizing identification information included in the image information output by the imaging unit;
A recognition position extraction unit that extracts position information of the identification information recognized by the recognition unit;
An information processing apparatus comprising: a transmission unit that transmits position information extracted by the recognition position extraction unit, identification information recognized by the recognition unit, and image information output by the imaging unit. The information processing apparatus according to claim 28, further comprising: a receiving unit that receives related information related to the identification information after transmitting the position information, the identification information, and the image information by the transmission unit. Processing equipment. 30. The information processing apparatus according to claim 29, wherein the identification information is a foreign language word and the related information is a translation of the foreign language. Receiving image information including at least one word and word specifying information for specifying a predetermined word included in the image information;
Specifying the predetermined word included in the received image information based on the image information and the word specifying information;
Extracting words other than the identified word from the words included in the image information;
Determining related information of the identified word with reference to a database for determining related information of the first word according to a combination of the first word and the second word stored in advance; An information processing method characterized by transmitting the determined related information. The information processing method according to claim 31,
The information processing method, wherein the word specifying information includes position information of the predetermined word. The information processing method according to claim 31 or 32,
An information processing method, wherein the word is a foreign language word and the related information is a translation of the foreign language word. The information processing method according to claim 31,
When extracting words other than the identified word among the words included in the image information,
Specify an area of a word other than the identified word among the words included in the image information,
An information processing method comprising: extracting the word as a word included in the image information when the designated area does not overlap an edge of the image information. Receive translation request data requesting translation of English words,
The meaning of the word information from the word to be translated and the word other than the translation target included in the received translation request data is stored in advance according to the combination of the first word and the second word. Extracting from a database containing statistical data for determining the meaning of the first word;
An information processing method comprising rearranging the extracted meanings based on the database and transmitting translation display order data for displaying the rearranged meanings. Software for executing an information processing apparatus having a communication unit that communicates with another device, an imaging unit that outputs image information, and a recognition unit that recognizes identification information included in the image information output by the imaging unit ,
When the acquisition of related information related to the identification information recognized by the recognition unit is instructed, the information processing apparatus is executed to acquire and display the related information via the communication unit software.

Images