JP2007028077A - Portable terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a portable terminal for setting a communication destination, based on face image data of a person without error and performing processing such as a telephone call and the transmission of an e-mail. <P>SOLUTION: A feature acquiring means 21 acquires feature data from the face image data of the person photographed with a camera 5. When the feature data coincides with high precision with the feature data which is previously stored in a feature data storage part 32, a communication destination selecting means 15, which is included in the portable terminal, selects a telephone number or a mail address, as the communication destination, which is related to the coincident feature data by association data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a portable terminal that realizes transmission of information to a designated communication destination via a communication line by designating one of a plurality of address data stored in an address storage unit as a communication destination. The present invention relates to a technique for improving a communication function in a telephone or a PDA having a communication function.

  As a technique related to the communication terminal configured as described above, there is one described in Patent Document 1. In other words, a person's face image and the person's telephone number (communication destination) are stored in association (stored) on the mobile phone, and when making a call, the call destination is determined from the face image displayed on the liquid crystal display. If the telephone number and the face image are already stored at the time of incoming call, the communication partner can be confirmed by displaying the face image on the liquid crystal display unit.

JP 2000-49913 A (paragraph numbers [0018], [0020] to [0022] FIGS. 1 and 6)

  As a situation when a telephone call is made, there is a case where it is desired to make a call with a person photographed in a photograph. In such a case, even if neither the name nor the phone number can be recalled, in the telephone having the function described in Patent Document 1, if the face image and the phone number are registered in association with each other, the face image is displayed on the liquid crystal display unit. Is displayed in sequence, and a call is realized by searching for a face image of a person photographed in the photograph from the display. However, when the number of registered face images is large, the selection is troublesome and time-consuming. In particular, if the face image of the person photographed in the photo appears to be slightly different from the face image registered on the phone, it may take time to specify the call destination from the registered face image, It is possible to make a wrong choice.

  In recent cellular phones, there are many opportunities to send e-mails to individuals via the Internet. Even when this e-mail is transmitted, it may be possible to save the trouble of inputting the e-mail address by registering the face image and the e-mail address in association with each other using the function described in Patent Document 1. However, when an incorrect selection is made as described above, an unnecessary electronic mail may be sent by mistake. Unlike telephone calls between people, all information such as documents is sent to e-mails. Therefore, improvement in confidentiality is desired.

  An object of the present invention is to configure a portable terminal that can perform communication by specifying a communication destination without error when a person is photographed or when a photograph of a person is at hand.

A feature of the present invention is that a portable terminal that realizes transmission of information to a designated communication destination via a communication line by designating one of a plurality of address data stored in the address storage unit as a communication destination.
From the acquired face image data of the person, a feature acquisition means for acquiring feature data of the face is provided,
A person who includes association means for associating and storing the feature data with at least one of the plurality of address data, and for comparing the feature data acquired by the feature acquisition means with the feature data already associated And a communication destination selection means for selecting, as a communication destination, the address data associated with the feature data when it is determined that the character data match with high accuracy as a result of the comparison by the person determination means. There is in point.

  According to this configuration, when the person determination unit compares the feature data acquired by the feature acquisition unit from the face image data and the feature data associated with the address data by the association unit, the communication destination selection is performed. The means selects the address data as a communication destination. In other words, when the feature data acquired from the acquired face image data of the person and the feature data already stored match with high accuracy, the address data associated with the feature data already stored is communicated. The operator automatically selects the destination, and the operator can transmit information to the set communication destination. As a result, when a photograph of a person is at hand, a portable terminal that can perform communication by designating a communication destination without error for those in which feature data of the face image of the person is stored in association with address data Was configured. In particular, in the communication for sending an e-mail, the inconvenience of sending an e-mail to an incorrect e-mail address is avoided, so that leakage of secrets can be prevented.

  The present invention includes a display that displays acquired face image data of a person in the portable terminal, and the association means displays the face image data and the input address data in a series on the display, and displays the display on the operator. After the confirmation, the feature data acquired by the feature acquisition unit from the face image data may be associated with the address data confirmed by the operator.

  With this configuration, for example, a person's face image data obtained by means such as directly photographing a person with a camera provided in a portable terminal or photographing a person with a camera is displayed on the display, and the person has already been displayed. Since stored address data and arbitrarily set address data can be displayed on the display, the operator can confirm the correspondence from this display, and an error can be avoided by associating after the confirmation.

  In the present invention, the feature acquisition means transmits the face image data to a processing server via the communication line, and the feature data acquired from the face image data by processing in the processing server is transmitted to the communication line. You may get through.

  With this configuration, it is not necessary to perform the process of extracting the feature data from the face image data inside the mobile terminal, and it is possible to avoid the inconvenience of increasing the weight of the process inside the mobile terminal, and at the same time, the advanced processing result in the processing server It is also possible to acquire.

  According to the present invention, the person determination unit quantifies the degree of matching between the feature data of the face image data and the feature data associated with the address data, and those whose numerical values exceed a threshold value are matched with high accuracy. Then, a determination result is given to the communication destination selection means, and those that do not exceed the threshold may be those with a high degree of matching.

  With this configuration, when the degree of matching between the feature data associated with the address data and the feature data of the human face image data photographed by the camera exceeds the threshold with high accuracy, the feature data is associated. For example, when setting the communication destination from the face data of a person with a poorly photographed image, even if the numerically matched degree does not exceed the threshold, the degree of matching is set. Since a higher candidate is used as a communication destination candidate, the operator can check address data set as a candidate and set it as a communication destination.

  In the present invention, the address storage unit may store a telephone number or a mail address using the Internet as the address data, and the communication destination selection unit may set the telephone number or the mail address as a communication destination. .

  With this configuration, a specified telephone number or a specified mail address can be set as a communication destination.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Basic configuration]
As shown in FIG. 1, a main case CA having a plurality of keys 1 used for inputting telephone numbers and characters, and a microphone 2 for capturing sound, a liquid crystal display 3 for displaying information, a receiver 4 for outputting sound, and an image A mobile phone as an example of a mobile terminal is configured by connecting a camera 5 that captures an image and a sub case CB including an antenna 6 that transmits and receives radio signals via a hinge 7 so as to be foldable.

  An example of a communication path by this cellular phone is shown in FIG. When the mobile phone X of the present invention is used as the communication path, the radio wave from the mobile phone X is received by the base station 40. The base station 40 is connected to a wired public telephone line PN and a wired internet IN, and the public telephone line PN can communicate via a fixed telephone 41 and the base station 40. Mobile phone 42 and personal personal computer 43 using this telephone line PN for the Internet can be connected. A personal computer 43 is connected to the Internet IN, and a Web server 44 and a processing server 45 are connected to the Internet IN. The telephone line PN and the Internet IN described above correspond to the communication line in the present invention.

  The mobile phone X uses a function (call mode) for making a call with the fixed telephone 41 or another mobile phone 42 via the telephone line PN, and uses the Internet IN or the telephone line PN as the Internet. The function of transmitting an e-mail to a mail address set in another mobile phone 42 or the personal computer 43 and receiving the e-mail from these (mail mode), and logging in to the Web server via the Internet IN And a function of browsing information transmitted from the server (Web mode).

  When the face of the person is photographed by the camera 5, the mobile phone X acquires characteristic data that can identify an individual from the face image data of the person and associates it with address data such as a telephone number or an e-mail address. Can be stored. When a person's face is photographed by the camera 5 after this association is performed, or when a photograph or a printed face is photographed, feature data is acquired from the photographed face image data. When the feature data acquired in this way matches the already stored feature data with high accuracy, the address data associated with the feature data is set as a communication destination.

(Control configuration)
A control system that realizes the communication function described above and the function of setting the communication destination based on the facial feature data is provided for the main case CA and the sub case CB. This control system can be shown as shown in FIG.

  This control system includes a communication processor 10 that mainly performs processing related to communication, and a multimedia processor 20 that mainly performs image processing. The communication processor 10 accesses a signal between the transmission / reception unit 11 that transmits and receives radio waves by the antenna 6, the key unit 12 that acquires operation information of the plurality of keys 1, the microphone 2, and the receiver 4. And a system for accessing signals among the communication processing unit 13, the address storage unit 14, and the communication destination selection unit 15.

  The communication processing unit 13 uses a call mode that enables a normal call using a telephone line, and a mail that enables transmission / reception of an e-mail using the Internet IN (including those that use the telephone line PN for the Internet). The communication is realized in either the mode or the Web mode in which the user logs in to the Web server 44 or the like using the Internet IN. That is, in the call mode, voice signal transmission / reception for the telephone number specified by the operation of the key 1 is realized, and in the mail mode, the mailer is activated, and the contents of the mailer are transmitted to the specified mail address and received. A process in which the mailer displays the e-mail on the liquid crystal display 3 is realized. In the Web mode, a browser is activated, and a process in which the content of the designated URL is displayed on the liquid crystal display 3 is realized.

  Further, the address storage unit 14 includes a telephone number as a communication destination in the call mode, a mail address as a communication destination (transmission destination) of the e-mail in the mail mode, and an address of a login destination in the Web mode ( Communication destination: URL (Uniform Resource Locater)) is stored. The address storage unit 14 stores the telephone number, the personal name having the mail address, the company name, etc. corresponding to the telephone number and the mail address, and when the URL is a personal homepage, The home page name and the name of the individual to be operated are stored, and if the company is a company, the company name is stored.

  Further, the communication destination selection means 15 performs a process of selecting address data set based on a communication destination selection process described later as a communication destination.

  The multimedia processor 20 forms a system for accessing signals among the liquid crystal display 3, the camera 5, the feature acquisition unit 21, the association unit 22, and the person determination unit 23. Further, an EEPROM type semiconductor memory 30 to which information from the communication processor 10 and the multimedia processor 20 is accessed is provided.

  The feature acquisition unit 21 has a function of setting a plurality of feature points in the face image data of a person photographed by the camera 5 and acquiring the positions of the feature points as feature amounts. As a processing mode for acquiring the feature amount in this way, generally, a specific position (for example, left and right eyes) (for example, with the face outline extracted from the face image data and grasping the shape of the entire face) , The center position of the iris), the specific position of the nose, the specific position of the mouth, and the like, and a process of quantifying the lines connecting these positions and the coordinates of each position is performed. The feature quantity digitized in this way is called feature data.

  Although not shown in the drawing, the camera 5 includes a photographing lens and a photoelectric conversion means such as a CCD. When photographing with the camera 5, a photographing mode is operated by operating the plurality of keys 1. By operating the key 1 set in advance at an appropriate timing while monitoring the image from the camera 5 on the liquid crystal display 3, image data is acquired from the photoelectric conversion means. The acquired image data is basically compressed into a JPEG format (a format established by the Joint Photographic Experts Group).

  The associating unit 22 performs processing for associating the feature data acquired by the feature acquiring unit 21 with the telephone number, the mail address, or the URL stored in the address storage unit 14. These telephone numbers, e-mail addresses, and URLs are collectively referred to as address data.

  The person determination unit 23 acquires the feature acquisition unit 21 from the feature data associated with the address data stored in the address storage unit 14 by the association unit 22 and the face image data captured by the camera 5. A process of comparing the feature data and determining the degree of matching of the feature data is performed.

  These feature acquisition means 21 and person determination means 23 are assumed to be a combination of hardware and software with dedicated logic, but may be configured only by hardware or only by software. Also good. Further, although the associating means 22 is assumed to be configured by software, it may be configured by hardware, or may be configured by a combination of software and hardware.

  The semiconductor memory 30 is composed of a nonvolatile memory such as an EEPROM. The semiconductor memory 30 functions as a communication history storage unit 31, a feature data storage unit 32, a thumbnail data storage unit 33, an association data storage unit 34, and an image data storage unit 35. The address storage unit 14 may be configured using the semiconductor memory 30.

  The communication history storage unit 31 is an area for storing a communication history including address data of a communication destination and communication date and time. The feature data storage unit 32 is an area for storing feature data associated with address data after being acquired by the feature acquisition unit 21. The thumbnail data storage unit 33 reduces the data amount so that the number of pixels suitable for display on the liquid crystal display 3 is reduced by the process of subtracting the number of pixels from the face image data photographed by the camera 5, thereby converting the face converted into the thumbnail format. This is an area for storing image data.

  The association data storage unit 34 is an area for storing association data indicating association between address data and the feature data. As shown in FIG. 4 as the structure of this association data, the feature data index (for example, file name), address data / name, and thumbnail data index (for example, file name) are converted into CSV (Comma Separated Values). A file format described in a simple text data structure is assumed.

  The image data storage unit 35 is an area for storing image data taken by the camera 5. The image data storage unit 35 is assumed to have a processing form for storing image data in the above-described JPEG format (a format established by the Joint Photographic Experts Group). However, a bitmap structure or a TIFF (Tag Image File Format) format is assumed. The image data may be stored.

[Association]
In this cellular phone X, it is possible to show the association process when the face image data and the address data are associated and stored as shown in the flowchart of FIG.

  That is, when face image data is obtained by photographing a person's face with the camera 5 (the face image data may be obtained by taking a photograph or a printed material), the obtained face image data is displayed on the liquid crystal display 3. Then, feature data is acquired from the face image data (steps # 101 to # 103).

  As described above, when associating the feature data acquired from the face image data with the address data, it is necessary to select a mode for associating. The timing of selecting this mode may be before or after shooting. There may be. In step # 102, the face image data is displayed on the liquid crystal display 3 as shown in FIG. 8, and at the same time, a message such as "Do you want to register this person's address?" Display, for example, confirm the display of Yes / No by operating the cursor 51, and further, by operating the specific key 1 for confirmation, the process of acquiring the feature data from the face image data is executed Is done.

  In addition, when acquiring feature data from face image data, it is assumed that the feature data is acquired by processing in the feature acquisition unit 21. However, the face image data is transferred to the processing server 45 via the Internet IN. The feature acquisition unit 21 may be configured to transmit the feature data from the face image data in the processing server 45 and transmit the feature data to the mobile phone. As described above, when the feature data is acquired using the processing server 45, it is possible to perform highly accurate processing in a shorter time than the processing performed in the mobile phone X.

  As described above, when the feature data is acquired, the address data is displayed on the liquid crystal display 3 and confirmation is obtained (steps # 104 and # 105).

  In this process, as shown in FIG. 9, the face image data is displayed on the liquid crystal display 3 and a message such as “Please enter an address” is displayed. The address book selection target is displayed. An input mode is set by selecting these selection targets by operating the cursor 52 and further operating a specific key 1 for confirmation.

  When manual is selected as the selection target, a plurality of keys 1 are operated to input address data. When a history is selected, it is stored in the communication history storage unit 31. One of the address data of the history is selected, and when the address book is selected, one of the address data already stored in the address storage unit 14 is selected.

  Next, after the operator confirms the face image data and the address data, thumbnail data is generated from the face image data, and the association data (FIG. 4) so as to associate the feature data, the thumbnail data, and the address data. Generated). Then, feature data is stored in the feature data storage unit 32, thumbnail data is stored in the thumbnail data storage unit 33, association data is stored in the association data storage unit 34, and the process is terminated (# 106, #). 107 steps).

[Communication destination selection processing]
After the feature data and the address data are stored in association with each other as described above, communication with the preset address data is realized from the face image data photographed by the camera 5. The communication destination selection process for selecting the communication destination in this way can be shown as in the flowchart of FIG. In this flowchart, the communication destination is limited to a telephone number and a mail address.

  That is, when image data is obtained by photographing a human face with the camera 5 (photographs or printed materials may be obtained to obtain human image data), the captured face image data is stored in the liquid crystal display 3. Then, feature data is acquired from the face image data (steps # 201 to # 203).

  As described above, when setting the address data from the feature data acquired from the face image data, it is necessary to select a mode for selecting a communication destination from the face image data. It may be before or after shooting. In step # 202, the face image data is displayed on the liquid crystal display 3 as shown in FIG. 10, and at the same time, a message such as “Call this person / mail” is also displayed as shown in FIG. For example, a process for acquiring feature data from face image data is performed by selecting a communication mode by operating the cursor 53 for the display of telephone / mail and operating a specific key 1 for confirmation. The

  In addition, when acquiring feature data from face image data, it is assumed that the feature data is acquired by the processing in the feature acquisition means 21, but the face image data is transferred to the Internet IN in the same manner as the processing in # 103 described above. The feature acquisition means 21 may be configured to transmit the face image data to the processing server 45, acquire the feature data from the face image data in the processing server 45, and transmit the feature data to the mobile phone. As described above, in the case of acquiring the feature data using the processing server 45, it is possible to perform highly accurate processing in a shorter time than the processing performed in the mobile phone.

  After acquiring the feature data, the acquired feature data is compared with the already stored feature data, the degree of matching is quantified (for example, as a percentage), and matches with a high accuracy exceeding a preset threshold. In this case, the address data associated with the feature data is selected as the communication destination, and at the same time, the address of the communication destination and the associated thumbnail data are displayed on the liquid crystal display 3 as shown in FIG. 204- # 206 steps).

  By the way, when performing the above-mentioned comparison, the acquired feature data is compared with all the feature data that has already been stored. The index is identified, and the association data having the index of the feature data is extracted from the association data storage unit 34. Then, the address data included in the association data is selected as a communication destination, and the thumbnail data indicated by the index of the thumbnail data included in the association data is extracted from the thumbnail data storage unit 33 and displayed on the liquid crystal display 3. The

  In the figure, a telephone number is displayed as address data, and a message for final confirmation for communication is displayed.

  If the communication mode is the call mode after the operator confirms that the specific key 1 is operated based on this display, a connection is made to the telephone number selected by the communication destination selection means 15. Processing is performed (steps # 207 and # 208). When the communication mode is the mail mode, the mailer is activated and the mail editing screen is displayed with the communication destination selecting means 15 selecting the mail address to be the communication destination. (Not shown) is displayed on the liquid crystal display 3. After confirming the contents of the mail to be transmitted by the operator on this edit screen, the necessary contents are transmitted to the mail address selected as described above (# 207, # 209, Step # 210).

  In this process, as shown in FIG. 10, the process of selecting the communication mode is performed at the timing when the face image data is acquired. Of the feature data acquired from the face image data and the stored feature data, The selection order may be set so that the communication mode is selected after selecting the matching with high accuracy.

  When the feature data acquired from the face image data and the stored feature data do not match with high accuracy (when the degree of matching does not exceed the threshold value), a manual setting routine (step # 300) is executed. (See FIG. 7). This manual setting routine (step # 300) assumes a case where, for example, face image data of a photographed face with a blurred print or a photograph of a small face is acquired, as shown in the flowchart of FIG. Even if the data does not match with high accuracy, a plurality of feature data having a high degree of matching is extracted, and address data and personal names (names, etc.) associated with the extracted feature data are listed and displayed on the liquid crystal display 3. Display processing is performed (the display form is not shown), and when the number of displays is large, all displays are performed by scrolling (steps # 301 and # 302).

  Next, when the operator selects the address data, the corresponding thumbnail data is displayed based on the association data, and the address data is selected and displayed for the confirmed data (steps # 303 and # 304). ). When the communication destination is set in this way, the communication destination selection means 15 selects a call destination as in steps # 207 to # 210 described above, and communication with the communication destination is realized.

  If there is no communication destination to be selected from the displayed address data and personal name, the setting is stopped (step # 306).

  As described above, according to the present invention, a person's face is photographed by the camera 5 provided in the mobile phone, the process of associating the person's face with the telephone number and the mail address is performed, and after the association, You can directly shoot the face of the camera with the camera 5, or shoot the face of the photo or album with the camera 5 and select the address. By comparing with the feature data acquired from, the phone number and mail address can be set. As a result, even when the person's name in the photograph cannot be remembered or when it takes time and effort to input, it is possible to perform communication by setting an error-free telephone number and mail address.

[Another embodiment]
The present invention may be configured as follows in addition to the embodiment described above.

(A) The present invention is applied to a terminal having a communication function such as a PDA as a communication terminal. In particular, when a PDA is used, there are many opportunities to connect to a specific site via the Internet. For example, facial feature data and URL as address data may be associated with each other.

(B) As the process of acquiring feature data, the process of acquiring feature data from the face image data immediately after the person's face is photographed by the camera 5 is shown. The face image from which the feature data is to be extracted. The data may be data stored in the image data storage unit 35 as previously taken. In this case, after selecting the photographed face image data, an operation for acquiring the feature data is performed.

  It can be applied to a mobile phone and a camera separated. In this case, communication means for acquiring facial image data captured by the camera with a mobile phone or processing for acquiring facial image data from the semiconductor media used by the camera is performed. The present invention can also be applied to a mobile phone that is not equipped.

Perspective view showing mobile phone from different directions Schematic diagram showing the communication path Block circuit diagram showing control system of cellular phone Diagram showing the structure of association data Association processing flowchart Flow chart of communication destination selection process Manual setting routine flowchart The figure which shows the screen displayed when registering an address based on face image data Figure showing the screen displayed when entering an address The figure which shows the screen displayed when communicating based on face image data Diagram showing the screen displayed at the start of communication

Explanation of symbols

3 Display: Liquid crystal display 14 Address storage unit 15 Communication destination selection means 21 Feature acquisition means 22 Association means 23 Person determination means 45 Processing server IN Internet

Claims (5)

A portable terminal that realizes transmission of information to a designated communication destination via a communication line by designating one of a plurality of address data stored in an address storage unit as a communication destination,
From the acquired face image data of the person, a feature acquisition means for acquiring feature data of the face is provided,
A person who includes association means for associating and storing the feature data with at least one of the plurality of address data, and for comparing the feature data acquired by the feature acquisition means with the feature data already associated And a communication destination selection means for selecting, as a communication destination, the address data associated with the feature data when it is determined that the character data match with high accuracy as a result of the comparison by the person determination means. Mobile terminal.   The portable terminal is provided with a display that displays the acquired face image data of the person, and the association means displays the face image data and the input address data in series on the display, and after the operator confirms this display 2. The portable terminal according to claim 1, wherein the feature data acquired by the feature acquisition unit from the face image data is associated with the address data confirmed by the operator.   The feature acquisition means transmits the face image data to a processing server via the communication line, and acquires the feature data acquired from the face image data through the processing on the processing server via the communication line. The portable terminal according to claim 1 or 2.   The person determination means quantifies the degree of matching between the feature data of the face image data and the feature data associated with the address data, and determines that the numerical value exceeding the threshold is determined to match with high accuracy. The portable terminal according to any one of claims 1 to 3, wherein a result is given to the communication destination selection means, and those that do not exceed a threshold value have a high degree of matching as a communication destination candidate.   The address storage unit stores a telephone number or a mail address using the Internet as the address data, and the communication destination selection unit sets the telephone number or the mail address as a communication destination. The portable terminal of Claim 1.

Images