JP2019159647A - Program, network system, and image determination method - Google Patents

Abstract

To provide a program and the like capable of preventing an image of a face different from the face of a person from being set as a profile image.SOLUTION: Provided is a program for a user terminal comprising a photographing section for photographing the face image of a user. The program causes a computer to function as: a selection section for selecting an image specified by the user from images stored in the user terminal as a candidate image; a determination section for determining whether or not the similarity between the face image included in the candidate image and the face image photographed by the photographing section in a period relating to the selection of the candidate image satisfies a prescribed reference; a communication control section for sending to a server device, the candidate image determined that the similarity satisfies the prescribed reference as the profile image of the user; and a display control section for displaying a profile image of another user sent from the server device onto a display section.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a program, a network system, and an image determination method.

  SNS (Social Networking Service) allows users to communicate with users who have never met directly by registering their profile with the site (server) for each user and sharing the profile with other users. It has a special mechanism. However, since most of these SNSs log in to the site by entering a password, there is a possibility that SNS can be used by anyone other than the person registered on the site as long as the password is known. As a technique for solving such a problem, there is known a technique for permitting login to a site when a face image registered in the site matches a face image taken by a user terminal (Patent Document). 1). According to this method, the use of a site other than the registered person can be prevented.

JP 2017-073062 A

  However, even when the person registered on the site uses the site, there are the following SNS-specific problems. In the conventional SNS, the user can freely set and set the profile image, so even if the person registered on the site uses the site, the face image of another person is set as his / her profile image It was possible to do. In this case, a face image of a person different from the user registered on the site is shared between the users as a profile image, so that other users who have never met the user can identify the person in the profile image as the user. I sometimes misunderstood it.

  The present invention has been made in view of the problems as described above, and an object thereof is a program capable of preventing a face image different from the person's face from being set as a profile image. Another object is to provide a network system and an image determination method.

  (1) The present invention is a program for a user terminal connected via a network to a server device that stores a profile image for each user, and the user terminal includes a photographing unit that photographs a user's face image. A selection unit that selects an image designated by the user from among images stored in the user terminal as a candidate image, a face image included in the candidate image, and a period related to selection of the candidate image. A determination unit that determines whether the similarity with the face image captured by the imaging unit satisfies a predetermined criterion; and the candidate image that is determined that the similarity satisfies the predetermined criterion. A communication control unit that transmits to the server device as a profile image, and a display control unit that displays a profile image of another user transmitted from the server device on the display unit; A program for causing a computer to function Te. The present invention also relates to a computer-readable information storage medium that stores the program.

In addition, the present invention is a network system including a server device including a storage unit that stores a profile image for each user, and a user terminal connected to the server device via a network. An image capturing unit that captures the face image, a selection unit that selects an image designated by the user from among images stored in the user terminal as a candidate image, a face image included in the candidate image, and the candidate image A determination unit that determines whether or not a similarity with the face image captured by the imaging unit in a period related to selection satisfies a predetermined criterion, and a communication control unit that transmits the candidate image to the server device The server device includes, as the profile image of the user, the candidate image determined by the determining unit to determine that the similarity satisfies the predetermined criterion. A storage control unit to be stored in the storage unit Te, the profile image of the user, a network system which comprises a communication control unit that transmits to the user terminal of another user.

  Further, the present invention provides a selection step of selecting an image designated by the user from among images stored in the storage unit as a candidate image, and a shooting step of photographing the user's face image in a period related to the selection of the candidate image. A determination step for determining whether the similarity between the face image included in the candidate image and the photographed face image satisfies a predetermined criterion, and the similarity is determined to satisfy the predetermined criterion And a display control step of displaying the candidate image as a profile image of the user on a display unit.

  According to the present invention, it is determined that the similarity between the face image included in the candidate image selected by the user and the face image photographed by the photographing unit in a period related to the selection of the candidate image satisfies a predetermined criterion. By using the candidate image as the profile image of the user, it is possible to prevent a candidate image including a face image of a person different from the person from being set as a profile image.

  (2) In the program and the information storage medium according to the present invention, the determination unit determines whether or not a human face image is included in the candidate image, and when the human face image is included. Determining whether the similarity between the face image included in the candidate image and the face image photographed by the photographing unit satisfies the predetermined criterion, and the communication control unit includes a human face image. The candidate image determined not to be transmitted and the candidate image determined to have a similarity satisfying the predetermined criterion may be transmitted to the server device as the user profile image.

  According to the present invention, a candidate image including a face image of a person different from the person is prevented from being set as a profile image, and a candidate image that does not include a person face image and is not likely to be misidentified is prevented. It can be set as a profile image.

  (3) In the program and the information storage medium according to the present invention, the computer may further function as a photographing control unit that activates the photographing unit at a timing when the candidate image is selected.

  According to the present invention, a face image of a user who has selected a candidate image can be taken at the timing when the candidate image is selected.

  (4) In the program and the information storage medium according to the present invention, the determination unit includes a human face image and a plurality of human face images in the image taken by the photographing unit. If not, it may be determined whether or not the degree of similarity between the face image included in the candidate image and the face image photographed by the photographing unit satisfies the predetermined criterion.

  According to the present invention, it is possible to accurately determine the degree of similarity between a captured face image and a face image included in a candidate image.

(5) In the program and the information storage medium according to the present invention, the display control unit may display a shooting guide on the display unit at the time of shooting by the shooting unit.

  According to the present invention, it is possible to accurately determine the degree of similarity between a captured face image and a face image included in a candidate image.

  (6) Further, in the program and the information storage medium according to the present invention, the determination unit includes a face image included in the candidate image and the photographing unit when an inclination of the photographing unit during photographing satisfies a predetermined condition. It may be determined whether the degree of similarity with the photographed face image satisfies the predetermined criterion.

  According to the present invention, it is possible to accurately determine the degree of similarity between a captured face image and a face image included in a candidate image.

  (7) In the program and the information storage medium according to the present invention, the determination unit determines whether the similarity between the face image included in the candidate image and the face image captured by the imaging unit satisfies a first criterion. When the first determination is made to determine whether or not the similarity satisfies the first criterion, the similarity between the face image included in the candidate image and the face image captured by the imaging unit is the second A second determination is performed to determine whether or not the criterion is satisfied. When the similarity satisfies the second criterion, it is determined that the similarity satisfies the predetermined criterion, and the first determination and the second determination are performed. The feature amount when determining the similarity may be changed.

  According to the present invention, for example, it is determined whether or not the face image is the person's face image in the first determination, and further is determined whether or not the image is an image processed or the like in the second determination. Not only prevents candidate images that include human face images from being set as profile images, but also prevents face images that are likely to be misidentified but are set as profile images. Can do.

  (8) In the program and the information storage medium according to the present invention, the display control unit is a candidate image selected by the other user when matching between the user and the other user is established. The candidate image determined that the similarity does not satisfy the predetermined criterion may be displayed on the display unit.

  According to the present invention, it is possible to display candidate images of other users that are less likely to be misidentified because matching with the user is established.

The figure which shows the network system of this embodiment. The figure which shows an example of the functional block diagram of the server of this embodiment. The figure which shows an example of the functional block diagram of the terminal of this embodiment. The figure which shows an example of the display screen displayed on the display part of a terminal. The figure which shows an example of the display screen displayed on the display part of a terminal. The figure which shows an example of the table information which stores a user's profile information It is a flowchart which shows an example of the process of the terminal of this embodiment. It is a flowchart which shows the other example of a process of the terminal of this embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. Moreover, not all of the configurations described in the present embodiment are necessary constituent requirements of the present invention.

1. Configuration FIG. 1 shows a network system (communication system) of this embodiment. In this embodiment, it is comprised by the some terminal 10 (user terminal) and the server 20 (server apparatus). That is, as shown in FIG. 1, the network system of the present embodiment is configured such that the server 20 that provides the service and the terminals 10 (10A, 10B, 10C,...) Can be connected to the network.

  The server 20 is an information processing apparatus that provides a service capable of providing communication among a plurality of users. In this embodiment, the server 20 is a server that provides a community-type service called SNS (social networking service). is there. In other words, the server 20 transmits user information (user name, profile, profile image, etc.) not only to the logged-in user (person himself) but also to other users so as to communicate with each other. The server also provides the terminal 10 with a chat service that can send and receive messages between users. The server 20 may provide a service only to the user who has registered as a member. The server 20 can be configured by one or a plurality of servers (authentication server, SNS server, communication server, billing server, database server, etc.).

  The terminal 10 is an information processing apparatus such as a mobile terminal (smart phone, tablet terminal, mobile phone, etc.), personal computer (PC), etc., and an apparatus that can be connected to the server 20 via a network such as the Internet (WAN) or LAN. It is. The communication line between the terminal 10 and the server 20 may be wired or wireless.

  FIG. 2 shows an example of a functional block diagram of the terminal 10 of the present embodiment. Note that the terminal of this embodiment may have a configuration in which some of the components (each unit) in FIG. 2 are omitted.

  The input unit 150 is a device for inputting (detecting) input information from the user, and outputs the user input information (operation input) to the processing unit 100. The function of the input unit 150 can be realized by an input device such as a touch panel, a touch pad, a mouse, direction keys and buttons, and a keyboard.

  The photographing unit 160 is a digital camera including an image sensor such as a CCD sensor or a CMOS sensor and an optical lens, and a position where the user's face image can be photographed in the housing of the terminal 10 (for example, display on the display unit 190). Near the surface).

  The storage unit 170 stores programs and various data for causing the computer to function as each unit of the processing unit 100, and also functions as a work area of the processing unit 100. The function can be realized by a hard disk, a RAM, or the like.

  The display unit 190 outputs the game image generated by the processing unit 100, and its function can be realized by a display such as a touch panel, LCD, or CRT that also functions as the input unit 150.

  The sound output unit 192 outputs the sound generated by the processing unit 100, and its function can be realized by a speaker, headphones, or the like.

  The communication unit 196 performs various controls for communicating with the server 20, and the function can be realized by hardware such as various processors or a communication ASIC, a program, or the like.

  In addition, the information storage medium which the server 20 has, the program and various data for functioning a computer as each part of the process part 100 memorize | stored in the memory | storage part are received via a network, and the received program and data are memory | storage parts. 170 may be stored. The case of receiving the program and various data and causing the terminal to function is also included in the scope of the present invention.

  The processing unit 100 (processor) performs various processes based on input information (operation input) from the input unit 150, a program, data received via the communication unit 196, and the like. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs. The processing unit 100 includes a selection unit 110, an imaging control unit 112, a determination unit 114, a communication control unit 116, a display control unit 118, an image generation unit 120, and a sound generation unit 130.

  The selection unit 110 selects, as candidate images, images designated (selected) by the user from among the images described in the storage unit 170 based on user operation input.

  The imaging control unit 112 performs control for starting and stopping the imaging unit 160. The imaging control unit 112 may perform control to activate the imaging unit 160 at a timing when a candidate image is selected (or timing when a display screen for selecting a candidate image is displayed). Further, the imaging control unit 112 may cause the imaging unit 160 to perform imaging when an operation input instructing execution of imaging is performed by the user, or automatically at the timing when a candidate image is selected. 160 may be used for shooting.

  The determination unit 114 determines whether the similarity between the face image included in the candidate image and the face image captured by the image capturing unit 160 during a period related to the selection of the candidate image satisfies a predetermined criterion. To do. The period related to the selection of the candidate image is a predetermined period including the time point when the candidate image is selected. In addition, the determination unit 114 determines whether or not the candidate image includes a human (human) face image. When the human face image is included, the determination unit 114 determines whether the candidate image includes a face image and a photographing unit. It may be determined whether the similarity with the face image photographed at 160 satisfies the predetermined criterion. The determination unit 114 may determine the similarity when the candidate image includes a face image of a person and does not include a plurality of person face images. When a candidate image including a plurality of human face images is allowed (similarity determination is performed), the determination unit 114 includes the face image captured by the image capturing unit 160 and the plurality of faces included in the candidate image. Find the similarity to each image. In addition, the determination unit 114 determines whether or not the image captured by the imaging unit 160 includes a person's face image, and the image captured by the imaging unit 160 includes a person's face image and a plurality of images. When a face image of a person is not included, it may be determined whether the similarity between the face image included in the candidate image and the face image captured by the capturing unit 160 satisfies the predetermined criterion. . Further, the determination unit 114 determines that the degree of similarity between the face image included in the candidate image and the face image photographed by the photographing unit 160 is the predetermined value when the inclination at the time of photographing of the photographing unit 160 satisfies a predetermined condition. It may be determined whether or not the criterion is satisfied. The determination unit 114 performs a first determination to determine whether the similarity between the face image included in the candidate image and the face image captured by the capturing unit 160 satisfies the first criterion, and the similarity A second determination that determines whether or not the similarity between the face image included in the candidate image and the face image captured by the imaging unit 160 satisfies the second criterion when the first criterion is satisfied. If the similarity satisfies the second criterion, it may be determined that the similarity satisfies the predetermined criterion. In this case, the feature amount when determining (calculating) the similarity may be changed between the first determination and the second determination.

The communication control unit 116 transmits the candidate image determined to have the similarity satisfying the predetermined criterion to the server 20 as the profile image of the user, and is determined that the similarity does not satisfy the predetermined criterion. The candidate image is not transmitted to the server 20. In addition, the communication control unit 116 determines that the candidate image that has been determined not to include a person's face image (a candidate image in which a person's face could not be detected, for example, an image whose face is hidden by a hand or the like) (Including images that have been subjected to mosaic processing, images of the back of a person, and images in which a person's face is too small to be identified) may be transmitted to the server 20 as the user's profile image. In addition, when the determination unit 114 obtains the similarity between the face image captured by the image capturing unit 160 and each of the plurality of face images included in the candidate image, the determination unit 114 calculates the similarity between any of the face images included in the candidate image. When it is determined that the degree of similarity satisfies a predetermined criterion, the communication control unit 116 transmits the candidate image to the server 20 as a user profile image. However, among face images included in the candidate images, face images whose similarity does not satisfy a predetermined standard are subjected to a process of making them invisible (for example, mosaic processing, blurring processing, painting processing, trimming processing, etc.). In addition, it is preferable to obtain a profile image. Note that if it is determined that none of the similarities with the plurality of face images included in the candidate image satisfies a predetermined criterion, the communication control unit 116 does not transmit the candidate image to the server 20. Further, the communication control unit 116 may transmit the selected candidate image to the server 20 regardless of the similarity determination result. In this case, the candidate image determined to satisfy the predetermined criterion and the candidate image determined not to include a human face image are effective as profile images. Information is added and transmitted to the server 20, and information indicating that the similarity is invalid as a profile image is added to the candidate image determined that the degree of similarity does not satisfy the predetermined standard, and transmitted to the server 20. May be. In addition, the communication control unit 116 transmits request information for requesting profile information (profile image) of another user to the server 20.

  The display control unit 118 controls the display unit 190 to display other user's profile images transmitted from the server 20 according to the request information. In addition, the display control unit 118, when matching between the user and the other user is established, the candidate selected by the other user (another user that satisfies a predetermined matching condition with the user) Candidate images that are images and whose similarity is determined not to satisfy the predetermined criterion may be displayed on the display unit 190. In addition, the display control unit 118 may display a shooting guide on the display unit 190 when shooting by the shooting unit 160.

  The image generation unit 120 performs drawing processing based on the results of various processes performed by the processing unit 100, thereby generating an image and outputting the image to the display unit 190.

  The sound generation unit 130 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  FIG. 3 shows an example of a functional block diagram of the server 20 of the present embodiment. Note that the server of this embodiment may have a configuration in which some of the components (each unit) in FIG. 3 are omitted.

  The storage unit 270 stores programs and various data for causing the computer to function as each unit of the processing unit 200, and also functions as a work area of the processing unit 200. The function can be realized by a hard disk, a RAM, or the like. The storage unit 270 includes a storage unit 272 (for example, a database).

  The storage unit 272 stores user information of each of a plurality of users participating in the SNS provided by the network system according to the present embodiment. For example, the storage unit 272 stores a user name, a password, destination information (IP address, etc.) of the terminal 10 as user information in association with user identification information (user ID) of each of a plurality of users. In addition, the storage unit 272 stores information for specifying other users that have been matched with the user as user information. The storage unit 272 stores profile information including the user's profile image in association with the user's identification information.

  The communication unit 296 performs various controls for communicating with the terminal 10 and other servers, and the function can be realized by hardware such as various processors or a communication ASIC, a program, or the like.

  The processing unit 200 (processor) performs various processes such as user information and profile information management, login / logout processing, and communication control processing based on data, programs, and the like transmitted from the terminal 10 and received via the communication unit 296. I do. The processing unit 200 performs various processes using the storage unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs. The processing unit 200 includes a storage control unit 210 and a communication control unit 212.

  The storage control unit 210 stores a profile image (candidate image whose similarity is determined to satisfy the predetermined criterion, candidate image determined not to include a human face image) transmitted from the terminal 10. Control to be stored in 270 (storage unit 272) is performed. Further, when a candidate image to which information indicating whether or not the profile image is valid is transmitted from the terminal 10, the storage control unit 210 adds information indicating that the profile image is valid. The stored candidate image is stored in the storage unit 270 as a user profile image (valid profile image), and the candidate image to which information indicating that the profile image is invalid is added as the invalid profile image. May be stored.

  The communication control unit 212 performs control to transmit other user's profile images (valid profile images) to the user's terminal 10 in accordance with the profile information request information transmitted from the user's terminal 10. In addition, when a predetermined matching condition is satisfied between the user and another user, the communication control unit 212, in addition to the valid profile image of the other user, the invalid profile image of the other user May be transmitted to the terminal 10 of the user.

2. Next, the method of this embodiment will be described with reference to the drawings.

  FIG. 4 is a diagram illustrating an example of a display screen displayed on the display unit 190 (touch panel) of the terminal 10. The display screen DS shown in FIG. 4 is a screen that displays a list of profiles of a plurality of users (other users), and the profile image PI and profile PF of each user are displayed on the display screen DS. Here, as the profile PF, a user name (user initials), age, and place of birth are displayed.

  When the user performs an operation of selecting any of the user's profile images PI on the display screen DS illustrated in FIG. 4, the profile of the selected user (here, the user “YF”) is displayed as illustrated in FIG. 5. The display screen DS (profile screen) is displayed. On the display screen DS shown in FIG. 5, a user profile image PI, a detailed profile PF, and a button BT for transmitting approval information (“Like”) to the user are displayed. For example, when the user performs an operation of selecting the button BT on the profile screen of the user “YF”, the user “YF” is notified that the approval information has been transmitted from the user. Accordingly, when the user “YF” selects the button BT on the profile screen of the user and transmits the approval information to the user, matching between the user and the user “YF” is established. When matching between users is established, messages can be exchanged by chat between the users.

FIG. 6 is a diagram illustrating an example of table information for storing user profile information. The table information 300 is stored in the storage unit 272 of the server 20. The table information 300 is associated with the user ID 310 of the user, and the user's profile image 320 (information for specifying the profile image) and the user's profile 330 (user name, age, birthplace, height, annual income, etc.) ) And. The server 20 transmits the profile information of each user stored in the table information 300 to the terminal 10, and the terminal 10 displays the display screen as shown in FIGS. 4 and 5 based on the profile information transmitted from the server 20. The DS is displayed on the display unit 190. That is, the profile image 320 of each user stored in the table information 300 is displayed as the profile image PI of each user on the display screen DS, and the profile 330 of each user stored in the table information 300 is displayed on the display screen DS. It is displayed as the user profile PF.

  In the present embodiment, the user uses the terminal 10 to select from a plurality of images stored in the storage unit 170 (images captured by the imaging unit 160 or another digital camera and stored in the storage unit 170). Select an image (candidate image) to be a candidate for its own profile image. The user may select his / her face image as a candidate image of the profile image, or select an image other than the face image (for example, an image of a pet or a landscape, an image shot while hiding his / her face). Also good. When the candidate image selected by the user includes a human face image, the image capturing unit 160 is activated, and the face image included in the candidate image and the face image captured by the image capturing unit 160 (the user who selected the candidate image) It is determined whether or not the degree of similarity with the face image satisfies a predetermined criterion (the degree of similarity is sufficiently high). The candidate images determined to satisfy the predetermined criteria and the candidate images determined not to include the face image are transmitted to the server 20 as the profile image of the user, and the server 20 It is stored in the table information 300 as a profile image 320.

  FIG. 7 is a flowchart illustrating an example of processing of the terminal 10 according to the present embodiment. First, based on the input information from the input unit 150, the selection unit 110 selects an image designated by the user from among the images stored in the storage unit 170 as a candidate image (step S10). Next, the determination unit 114 recognizes (detects) a human face image from the selected candidate image by image recognition processing, and determines whether the candidate image includes a human face image (step S11). .

  If the candidate image includes a human face image (Y in step S11), the imaging control unit 112 activates the imaging unit 160 (step S12), and the display control unit 118 captures the image with the imaging unit 160. The preview image being displayed is displayed on the display unit 190, and the shooting guide is superimposed on the preview image (step S13). As the shooting guide, a guide for shooting the entire face of the user with a predetermined size (for example, a frame for guiding the size of the face) may be displayed, or the user's face is shot from the front. For example, a line for guiding the position of the eyes and the position of the nose may be displayed. Also, a guide for photographing the face of the user in the same orientation as the detected orientation (for example, an arrow for guiding the orientation of the face) is detected for photographing. It may be displayed as a guide. Further, the candidate image may be displayed superimposed on the preview image. In addition, a message that prompts the user to capture a face image may be further displayed. Next, the imaging control unit 112 determines whether or not an imaging operation by the user has been performed based on the input information from the input unit 150 (step S14). When a shooting operation is performed (Y in step S14), shooting is performed by the shooting unit 160 to acquire a shot image (step S15).

Next, the determination unit 114 recognizes a human face image from the captured image by image recognition processing, and determines whether the captured image includes a face image (step S16). If the photographed image does not include a face image (N in step S16), the process proceeds to step S14 (recapture is performed). When the captured image includes a face image (Y in step S16), the determination unit 114 determines whether the captured image includes face images of a plurality of persons (step S17). If the image includes a plurality of face images (Y in step S17), the process proceeds to step S14. When the photographed image does not include a plurality of face images (step S
In N of 17, the determination unit 114 determines whether or not the inclination of the photographing unit 160 is incorrect (does not satisfy a predetermined condition) (step S <b> 18), and the inclination is incorrect. In (Y of step S18), it transfers to step S14. For example, the determination unit 114 may determine that the tilt is incorrect when the face image included in the captured image does not follow the shooting guide, or the inclination of the shooting unit 160 may be determined by a sensor (acceleration sensor). Or an angular velocity sensor) or image processing on a captured image, and when the detected inclination is not perpendicular (vertical) to the ground, it may be determined that the inclination is incorrect.

  When the inclination is not incorrect (N in step S18), the determination unit 114 extracts (calculates) the feature amount of the face image included in each of the candidate image and the captured image, and based on the extracted feature amount. The similarity between the face image included in the candidate image and the face image included in the photographed image is calculated, and it is determined whether or not the calculated similarity satisfies a first criterion (a criterion for distinguishing between the person and another person). A first determination is made (step S19). In the first determination, a feature amount for determining an individual feature (a feature amount used for general face authentication such as a distance between eyes and a positional relationship between eyes and nose) is extracted to calculate a similarity. Thus, it is determined whether or not the face image included in the candidate image is the face image of the user himself / herself.

  When the calculated similarity satisfies the first criterion (Y in step S20), the determination unit 114 determines the feature amount of the face image included in each of the candidate image and the captured image (the feature amount extracted in the first determination). Is extracted at least in part, and the similarity between the face image included in the candidate image and the face image included in the photographed image is calculated based on the extracted feature amount. A second determination is made to determine whether or not a second criterion (a criterion for distinguishing whether or not an image has been processed) is satisfied (step S21). In the second determination, the face image is processed by image processing (for example, the eyes are enlarged, the face outline is narrowed to make a small face, the whiskers are scribbled on the face image), and the feature amount changes depending on the shooting angle. By extracting the similarity (for example, the ratio of the eye size to the face, the ratio of the face contour to the eye or nose size), the face image included in the candidate image is processed by image processing It is determined whether the image is a captured image or an image captured at an extreme angle. If the similarity does not satisfy the first criterion (N in step S20), and if the similarity does not satisfy the second criterion (N in step S22), the process ends without transmitting the candidate image to the server 20. .

  When the candidate image does not include a face image (N in step S11) and the calculated similarity satisfies the second criterion (Y in step S22), the communication control unit 116 selects the candidate image. It transmits to the server 20 as a user's profile image (step S23). The second determination (steps S21 and S22) for determining whether or not the image is a processed image is omitted, and only the first determination (steps S19 and S20) for determining whether or not the user is the person is performed. If the result of 1 determination satisfies the first criterion (Y in step S20), the process may proceed to step S23. Conversely, the first determination may be omitted, and only the second determination may be performed. If the second criterion is satisfied as a result of the second determination (Y in step S22), the process may proceed to step S23. In addition, the threshold value (reference value) is lowered when determining whether or not the person is the person without determining the feature amount used in the determination process, and the threshold value is determined when determining whether the image is a processed image or the like. May be made higher.

  The storage control unit 210 of the server 20 stores the profile image received from the terminal 10 in the table information 300 as the profile image 320 of the user of the terminal 10. In addition, when the communication control unit 212 receives profile information request information from the user terminal 10, the communication control unit 212 sends profile information (profile image 320, profile 330) of a user other than the user to the user terminal 10. Send.

As described above, according to the present embodiment, when a candidate image is selected by the user, the photographing unit 160 is activated to photograph the user's face image, and the face image included in the candidate image is photographed. By calculating the similarity with the face image included in the image, and transmitting the candidate image determined that the calculated similarity satisfies the predetermined criterion to the server 20 as the user's profile image, It is possible to prevent a face image from being set as a profile image. In addition, by determining the similarity based on the two-stage criteria (the first criterion and the second criterion), not only the face image of a person different from the person but also the face image of the person may be misidentified. It is possible to prevent a certain face image (a face image obtained by processing an image, a face image taken at an extreme angle) from being set as a profile image. If the candidate image does not include a face image, the candidate image is transmitted to the server 20 as a user profile image without determining the similarity, thereby including a human face image. Candidate images that are not likely to be misidentified and can be set as profile images. In addition, a shooting guide is displayed on the display unit 190 at the time of shooting, and when the photographed image does not include a person's face image or includes a plurality of person's face images, the photographing unit 160 is inclined. If the image is invalid, the image is re-captured to obtain a captured image suitable for determining the similarity with the face image included in the candidate image, and the face image and the candidate image included in the captured image are acquired. The degree of similarity with the included face image can be accurately determined.

  In the above example, the case where the photographing unit 160 is activated at a timing when a candidate image is selected has been described. However, the photographing unit 160 may be activated at a timing when a transition is made to a selection screen for selecting candidate images. In the above example, the case is described in which shooting is performed when a shooting operation is performed after the shooting unit 160 is started. However, after the shooting unit 160 is started, automatically (without waiting for the shooting operation). ) Shooting may be executed. Further, with reference to the shooting date and time (time stamp) of the shot image selected by the user among the shot images stored in the storage unit 170, the selected shot image includes a predetermined period including the time point when the candidate image is selected. If the selected photographed image is photographed within the predetermined period, the similarity between the photographed image and the candidate image is determined, If the selected photographed image is not photographed within the predetermined period, the similarity may not be determined. If the selected photographed image is not photographed within a predetermined period, the photographed image may be selected again, a notification prompting the user to photograph by the photographing unit 160 may be given, or photographing may be performed. The unit 160 may be activated to perform shooting. In this case, the selected candidate image may not be set as the profile image (not transmitted to the server 20), or the selection of the candidate image may be canceled.

  In the above example, a case has been described in which a candidate image whose similarity is determined to satisfy a predetermined criterion is transmitted to the server 20 as a user profile image. However, the selected candidate image is related to the similarity determination result. It may be transmitted to the server 20 first.

  FIG. 8 is a flowchart illustrating a process flow of the terminal 10 when a candidate image is transmitted to the server 20 regardless of the similarity determination result. Note that steps S30 to S42 in FIG. 8 are the same as steps S10 to S22 in FIG. As shown in FIG. 8, when the candidate image does not include a face image (N in step S31) and when the calculated similarity satisfies the second criterion (Y in step S42), communication control is performed. The unit 116 adds information (valid information) indicating that it is valid as a profile image to the candidate image and transmits it to the server 20 (step S43). On the other hand, when the similarity does not satisfy the first criterion (N in step S40), and when the similarity does not satisfy the second criterion (N in step S42), the communication control unit 116 adds the candidate image to the candidate image. Information indicating that the profile image is invalid (invalid information) is added and transmitted to the server 20 (step S44).

When the storage control unit 210 of the server 20 receives the candidate image to which the valid information is added from the terminal 10, the storage control unit 210 stores the candidate image in the table information 300 as the profile image 320 of the user of the terminal 10, and invalid information Is received from the terminal 10, the candidate image is stored in the table information 300 as a candidate image of the user of the terminal 10 (not stored as the profile image 320).

  When the communication control unit 212 receives profile information request information from the user terminal 10, the communication control unit 212 transmits profile information (profile image 320, profile 330) of a user other than the user to the user terminal 10. . At this time, when there is another user who has been matched with the user, along with the other user's profile image 320 (candidate image to which valid information is added) for the user's terminal 10, The other user's candidate image (candidate image with invalid information added) is transmitted. However, when the profile image 320 of the other user is not registered, only the candidate image is transmitted, and when the candidate image of the other user is not registered, only the profile image 320 is transmitted. In addition to (or instead of) determining that matching between users who have transmitted approval information has been established, the server 20 receives position information from each terminal 10, and the position information of the terminal 10 is predetermined. It may be determined that matching between users within the distance range (estimated to have met the implementation) has been established. In this way, when matching between the user and another user is established, it is determined that there is little possibility of misidentification even if the candidate image is added with invalid information, and the other user candidate is determined. The image is transmitted to the terminal 10 of the user. In addition, when matching with the said user and another user is materialized, until the matching is cancelled | released, in addition to the said other user's candidate image memorize | stored in the server 20 at the time of matching establishment, matching The other user candidate images (candidate images to which valid information has been added and candidate images to which invalid information has been added) transmitted to the server 20 after being established may also be transmitted to the terminal 10 of the user. .

  Note that the communication control unit 212 transmits other user candidate images (candidate images to which valid information is added, candidate images to which invalid information is added) to the user terminal 10, and the display control unit of the terminal 10. 118 displays a candidate image to which valid information is added among other user candidate images received from the server 20 as a profile image PI of the other user on the display screen DS, and a candidate to which invalid information is added. The image may not be displayed on the display screen DS. In addition, the candidate image to which invalid information is added may be displayed on the display screen DS after being subjected to a process of making it invisible (mosaic process or the like). And when matching with the said other user is materialized, you may make it display the candidate image to which invalid information was added among the candidate images of the said other user on the display screen DS.

  Further, at a predetermined timing after the registration of the profile image, the user may take a face image, and the similarity between the face image included in the registered profile image and the taken face image may be determined. For example, every time a user logs in, the terminal 10 captures the user's face image to acquire the captured image, and the server 20 or the terminal 10 includes the face image and the captured image included in the user's profile image. It is determined whether or not the similarity with the face image satisfies a predetermined criterion, and when it is determined that the similarity does not satisfy the predetermined criterion, the profile image is deleted from the table information 300, or the profile image By adding invalid information to, the profile image may not be displayed on the terminal 10 of another user.

The server device according to the present invention may be a user terminal. That is, the user terminal may function as the server device according to the present invention. Moreover, you may implement | achieve this invention only with a user terminal, without using a server apparatus. For example, at each user's terminal, it is determined whether or not the selected candidate image satisfies a predetermined criterion, the candidate image determined to satisfy the predetermined criterion is stored, and the candidate image is stored as a profile image. You may make it transmit to another user's terminal. More specifically, the present invention is a program for a user terminal, and the user terminal includes a photographing unit that photographs a user's face image, and is designated by the user among images stored in the user terminal. The similarity between the selection unit that selects the image as a candidate image, the face image included in the candidate image, and the face image captured by the imaging unit in a period related to the selection of the candidate image is a predetermined value. A determination unit that determines whether or not a criterion is satisfied, and a communication control unit that transmits the candidate image determined to satisfy the predetermined criterion to a user terminal of another user as the user's profile image It may be a program characterized by causing a computer to function. In addition, the user terminal transmits the selected candidate image and the captured image to another user terminal, and determines whether the candidate image satisfies a predetermined criterion in the other user terminal, Candidate images determined to satisfy the predetermined criteria are displayed on the display unit as profile images, and candidate images determined not to satisfy the predetermined criteria are not displayed on the display unit (and the candidate images are deleted from the storage unit). You may do it. More specifically, the present invention is a network system including a plurality of user terminals (first and second user terminals) connected to each other via a network, wherein the first user terminal is a first user. A photographing unit that captures the face image, a selection unit that selects, as candidate images, an image designated by the first user among images stored in the first user terminal, the candidate images, and the candidates A communication control unit that transmits a face image photographed by the photographing unit to a second user terminal of a second user during a period related to image selection, wherein the second user terminal A determination unit that determines whether a similarity between a face image included in the candidate image transmitted from one user terminal and the face image transmitted from the first user terminal satisfies a predetermined criterion And the similarity is before The candidate image determined to satisfy the predetermined criterion may be a network system which comprises a said display control unit for displaying on the display unit as the first user profile picture.

  Further, the candidate image including the face image is not limited to the captured image (face photo). For example, the candidate image including the face image may be a portrait image or an avatar character image (2D image, 3D image) imitating a person. You may make it determine the similarity with the image | photographed face image (face photograph). In this case, the threshold value (reference value) for determining the similarity is set to a value that allows the difference between the deformed face image and the face photograph, or the feature amount used for determining the similarity is used. Or, it may be different from what is compared between face photos. For example, in the determination of similarity, it may be determined only whether the avatar and the face photograph have the same gender, or both eyes, nose shape, face shape, and hair color are similar. It may be determined. In addition, when a plurality of face parts of avatars are prepared in advance for each part (for example, there are plural types of parts for each eye, contour, nose, mouth), each part in the face photograph taken by the photographing unit It may be determined whether or not the feature is included in the face part of the avatar in the candidate image for each part. In this case, for each type of face part of the avatar, a range of feature amounts extracted from the corresponding human face photograph is associated in advance, and the feature amount extracted from the face photograph taken by the photographing unit is The similarity may be determined by determining whether or not it is included in the range of the feature amount associated with the part.

  The present invention is not limited to that described in the above embodiment, and various modifications can be made. For example, terms cited as broad or synonymous terms in the description in the specification or drawings can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

DESCRIPTION OF SYMBOLS 10 ... Terminal, 20 ... Server, 100 ... Processing unit, 110 ... Selection unit, 112 ... Shooting control unit, 114 ... Determination unit, 116 ... Communication control unit, 118 ... Display control unit, 120 ... Image generation unit, 130 ... Sound Generation unit 150 ... Input unit 160 ... Shooting unit 170 ... Storage unit 190 ... Display unit 192 ... Sound output unit 196 ... Communication unit 200 ... Processing unit 210 ... Storage control unit 212 ... Communication control unit 270 ... Storage unit, 272 ... Storage unit, 296 ... Communication unit

Claims (10)

A program for a user terminal connected via a network with a server device that stores a profile image for each user,
The user terminal includes a photographing unit that photographs a user's face image,
A selection unit that selects, as candidate images, images designated by the user among images stored in the user terminal;
A determination unit that determines whether a similarity between a face image included in the candidate image and a face image photographed by the photographing unit in a period related to selection of the candidate image satisfies a predetermined criterion;
A communication control unit that transmits the candidate image determined to satisfy the predetermined criterion to the server device as the user profile image;
A program that causes a computer to function as a display control unit that displays a profile image of another user transmitted from the server device on a display unit. In claim 1,
The determination unit
It is determined whether or not a human face image is included in the candidate image, and when the human face image is included, the face image included in the candidate image and the face image taken by the shooting unit, Whether the degree of similarity satisfies the predetermined criterion,
The communication control unit
Transmitting the candidate image determined not to include a human face image and the candidate image determined to satisfy the predetermined criterion to the server device as the user profile image A program characterized by In claim 1 or 2,
A program that further causes a computer to function as an imaging control unit that activates the imaging unit at a timing when the candidate image is selected. In any one of Claims 1 thru | or 3,
The determination unit
When the image captured by the image capturing unit includes a face image of a person and does not include a plurality of person face images, the image captured by the image capturing unit and the face image included in the candidate image A program for determining whether or not a similarity with a face image satisfies the predetermined criterion. In any one of Claims 1 thru | or 4,
The display control unit
A program for displaying a photographing guide on the display unit at the time of photographing by the photographing unit. In any one of Claims 1 thru | or 5,
The determination unit
Whether or not the degree of similarity between the face image included in the candidate image and the face image photographed by the photographing unit satisfies the predetermined criterion when the tilt at the time of photographing of the photographing unit satisfies a predetermined condition A program characterized by judging. In any one of Claims 1 thru | or 6,
The determination unit
A first determination is made to determine whether or not the similarity between the face image included in the candidate image and the face image captured by the imaging unit satisfies a first criterion, and the similarity satisfies the first criterion. In the case of satisfying, a second determination is made to determine whether or not the similarity between the face image included in the candidate image and the face image captured by the imaging unit satisfies a second criterion, and the similarity is the first A program characterized in that, when the second criterion is satisfied, it is determined that the degree of similarity satisfies the predetermined criterion, and a feature amount for determining the degree of similarity is changed between the first determination and the second determination. In any one of Claims 1 thru | or 7,
The display control unit
If matching between the user and the other user is established, a candidate image selected by the other user and having a similarity determined not to satisfy the predetermined criterion is displayed on the display unit. A program characterized by being displayed on the screen. A network system including a server device including a storage unit that stores a profile image for each user, and a user terminal connected to the server device via a network,
The user terminal is
A photographing unit for photographing a user's face image;
A selection unit that selects, as candidate images, images designated by the user among images stored in the user terminal;
A determination unit that determines whether a similarity between a face image included in the candidate image and a face image photographed by the photographing unit in a period related to selection of the candidate image satisfies a predetermined criterion;
A communication control unit that transmits the candidate image to the server device,
The server device
A storage control unit that causes the storage unit to store, as the user's profile image, the candidate image that is determined by the determination unit to have a similarity that satisfies the predetermined criterion;
A network system, comprising: a communication control unit that transmits a profile image of the user to a user terminal of another user. A selection step of selecting an image designated by the user from among images stored in the storage unit as a candidate image;
A shooting step of shooting the face image of the user in a period related to selection of the candidate image;
A determination step of determining whether a similarity between the face image included in the candidate image and the captured face image satisfies a predetermined criterion;
A display control step of causing the display unit to display the candidate image determined to have a degree of similarity satisfying the predetermined criterion as a profile image of the user.

Images