JP6839324B1 - Input system, input program and input method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a novel input technique capable of avoiding or reducing contact with a large number of users in various input operations. SOLUTION: This is an input system for recognizing a user registered in advance and making a payment, and is a reading means for reading product specific information for specifying a purchased product and a photographing means for photographing a user who makes a payment. When the payment operation by the user is recognized from the user image taken by the photographing means, the operation recognition means for ending the reading by the reading means and the authentication for authenticating the registered user based on the user image. A means and a payment means for executing a payment process for the price of the product based on the registered user information when the payment operation is recognized and the authentication is successful. [Selection diagram] Fig. 1

Description

The present invention relates to input systems, input programs and input methods for avoiding or reducing user contact.

Due to the worldwide epidemic of infectious diseases, infectious disease countermeasures are drawing attention again. For example, it is effective to reduce the chances of people interacting with each other using IT (Information Technology), such as replacing payments at stores with self-checkouts operated by customers themselves from manned cashiers operated by clerk. It is one of the measures against infectious diseases.

Conventionally, automation technology for cash registers and counter operations using IT has been known from the viewpoint of reducing necessary personnel and improving efficiency. For example, Patent Document 1 describes a method of automatic payment using biometric authentication using a face image. Patent Document 1 describes that a user clearly instructs the execution of face recognition by a predetermined gesture.

Further, Patent Document 2 describes that in face recognition when purchasing a product using a vending machine, it is excluded that face recognition is performed by information other than a person by requesting a customer to perform a predetermined operation. Has been done.

JP-A-2019-67075 Japanese Unexamined Patent Publication No. 2020-87093

On the other hand, contact infections caused by the spread of bacteria and viruses through objects touched by an unspecified number of people are also regarded as a problem. In the above-mentioned conventional technology, it is assumed that the user himself / herself inputs, and many people touch the same part at the time of input, which is insufficient from the viewpoint of preventing contact infection. ..

Therefore, an object of the present invention is to provide a novel input technique capable of avoiding or reducing contact with a large number of users in various input operations.

In order to solve the above problems, the present invention is an input system for recognizing a user registered in advance and making a payment, and a reading means for reading the product specific information for specifying the purchased product and payment. Registration based on the user image, the photographing means for photographing the user who performs the operation, the operation recognizing means for terminating the reading by the reading means when the payment operation by the user is recognized from the user image photographed by the photographing means, and the user image. An authentication means that authenticates a completed user, and a payment means that executes a payment process for the price of the product based on the registered user information when the payment operation is recognized and the authentication is successful. , Equipped with.

With such a configuration, all input related to payment can be completed without the user touching the device itself. Therefore, the user can purchase by touching only the product, the risk of infection due to contact with an unspecified number of people can be reduced, and the dirt and wear of the device can be reduced. Furthermore, not only can payments be made easily even in situations where gloves need to be worn, such as at ski resorts and construction sites, but also payments are made based on user images, so it is necessary to carry a wallet, smartphone, etc. It disappears and the convenience of the user is improved.

In a preferred embodiment of the present invention, the motion recognizing means recognizes a pre-registered gesture as the settlement motion from the user image.
With such a configuration, the effect of reducing erroneous recognition can be expected.

In a preferred embodiment of the present invention, the user information includes group-specific information that identifies a group to which the user belongs, and the payment means executes the payment process based on the group-specific information.
With such a configuration, for example, by applying the present invention to payments for shopping at an in-house shop, etc., and making payments by specifying the group to which the user belongs, deductions from salary and welfare expenses can be made. It is possible to realize payment using.

In a preferred embodiment of the present invention, the payment means identifies group information registered in advance based on the group identification information, and executes the payment process according to the payment method specified in the group information.
With such a configuration, it is possible to make a payment by a payment method designated for each group.

In a preferred embodiment of the present invention, after the reading means starts the reading, if the reading is not continuously performed for a predetermined time, a means for canceling the reading is further provided.
With such a configuration, when the user stops inputting in the middle and leaves as it is, it is possible to treat it as a cancellation and accept the next input.

In a preferred embodiment of the present invention, when the operation recognizing means fails to recognize the payment operation after the reading means starts the reading, the reading is terminated and an input for requesting the payment process is performed. A display unit for displaying payment is further provided.
With such a configuration, even if the recognition of the payment operation from the user image does not work well for some reason, the user can request the payment process by another method.

In a preferred embodiment of the present invention, the display unit recognizes the payment operation when the operation recognition means does not recognize the payment operation for a predetermined time after the reading means starts the reading. Consider it a failure.

In a preferred embodiment of the present invention, the payment display requires the user to touch a portion different from the previously displayed portion.
With such a configuration, it is possible to prevent many users from touching the same part even when the recognition of the payment operation fails.

In order to solve the above problems, the present invention is an input program for recognizing a pre-registered user and making a payment, and a reading means for reading the product specific information that identifies the purchased product and payment. Registration based on the user image, the photographing means for photographing the user who performs the operation, the operation recognizing means for terminating the reading by the reading means when the payment operation by the user is recognized from the user image photographed by the photographing means, and the user image. An authentication means that authenticates a completed user, and a payment means that executes a payment process for the price of the product based on the registered user information when the payment operation is recognized and the authentication is successful. Make your computer work as.

In order to solve the above problems, the present invention is an input method for recognizing a pre-registered user and making a payment, and performs a step of reading product specific information that identifies a purchased product and payment. A step of photographing a user to perform, a step of terminating the reading when the payment operation by the user is recognized from the photographed user image, a step of authenticating a registered user based on the user image, and the above-mentioned step. When the payment operation is recognized and the authentication is successful, the computer is made to execute the step of executing the payment process of the price of the product based on the registered user information.

According to the present invention, it is possible to provide a novel input technique capable of avoiding or reducing contact with a large number of users.

It is a functional block diagram of the input system in embodiment of this invention. It is a figure which shows an example of the information registered in the database DB in embodiment of this invention. It is a figure which shows an example of the information stored as a transaction history table in embodiment of this invention. It is a processing flowchart concerning the settlement for the user of the input system in embodiment of this invention to purchase a product. It is a figure which shows an example of the display screen of the input system in embodiment of this invention. It is a figure which shows an example of the display screen of the input system in embodiment of this invention. It is a figure which shows an example of the display screen of the input system in embodiment of this invention. It is a figure which shows an example of the display screen of the input system in embodiment of this invention. It is an example of the recording of the contact position of the input system in the embodiment of the present invention, and is the explanatory view of the coordinates used for expressing the position on the display part. It is a flowchart which concerns on the determination of the display position of the button of the input system in embodiment of this invention. It is a figure which shows the example concerning the determination of the display position of the button of the input system in embodiment of this invention.

Hereinafter, the input system of the present invention will be described with reference to the drawings. The embodiments shown below are examples of the present invention, and the present invention is not limited to the following embodiments, and various configurations can be adopted.

For example, in the present embodiment, the configuration, operation, and the like of the input system will be described, but a method, an apparatus, a computer program, and the like having the same configuration can also exert the same effects. Further, the program may be stored in a recording medium. Using this recording medium, for example, a program can be installed on a computer. Here, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM.

Further, in the present embodiment, on the premise of payment using face authentication, a technique for avoiding or reducing user contact in instructions for payment processing and various other inputs is exemplified, but the present invention is not limited to this. .. For example, by executing the display position determination of the button described later in any other input, many users touch the same part in a system that does not use image authentication or a wide range of systems that assume user contact. It may be used to prevent.

In this embodiment, it is assumed that the embodiment is used by a user belonging to a specific group, such as an in-house shop. Therefore, in the present embodiment, a usage group is determined for each store in which the input system is installed, and a person registered as a user belonging to the group uses the input system of the present invention in the store.

(1) Payment Input Using User Image First, the entire configuration of an input system for making a payment using a user image of a user at the time of payment will be described. FIG. 1 is a diagram showing a functional configuration of the input system of the present embodiment.

The input system of the present embodiment includes a reading means 11, a photographing means 12, an motion recognition means 13, an authentication means 14, a payment means 15, a display unit 16, a contact position recording means 17, and a button display means 18. And the contact position resetting means 19, and are configured to be able to communicate with each other with the database DB.

The input system includes a computing device such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a main storage device such as a RAM (Random Access Memory), an HDD (Hard Disk Drive), an SSD (Solid State), and a SSD (Solid State). It is realized by a general computer device provided with an auxiliary storage device such as, a means for connecting to a network, a reading device for reading product information, a photographing device, various input / output devices including a display, and the like. In the present embodiment, a reading device such as a barcode reader, a camera, a display (preferably a touch panel display), or the like is connected to an arbitrary computer device connected to a network for accessing a database DB or the like. The input system is configured.

The reading means 11 reads the product identification information that identifies the purchased product. Specifically, for example, a barcode reader for reading a barcode, a camera for recognizing a product image or character by image recognition and identifying a product, a tag reader for receiving product identification information by short-range wireless communication, etc. However, it functions as the reading means 11 of the present invention.

The photographing means 12 photographs a user who makes a payment and generates a user image. In the present embodiment, the photographing device installed around the display unit 16 functions as the photographing means 12. The photographing means 12 photographs the face of the user who makes the payment, and delivers the user image to the motion recognition means 13, the authentication means 14, and the display unit 16.

When the motion recognizing means 13 recognizes the gesture registered in advance from the user image as a settlement motion, the motion recognizing means 13 ends the reading by the reading means 11. Further, the motion recognition means 13 of the present embodiment also recognizes the cancel motion in addition to the settlement motion. When the cancel operation is recognized, the product identification information read up to that point is reset and the settlement is canceled. In the present embodiment, a peace sign gesture is registered in advance as a settlement operation, and a gesture in which the palm is turned to the front as a cancel operation is registered in advance. The settlement operation and the cancellation operation are not limited to this, and a specific facial expression such as a smile may be a settlement operation or a cancellation operation. Further, the settlement operation or the cancellation operation may be recognized on condition of both the gesture and the facial expression.

The settlement operation and the cancellation operation may be recognized by any image recognition method. For example, a plurality of sets of images showing each motion and a set of information indicating the type of the motion are input as teacher data, a classification model of each motion is created by using a generally known machine learning method, and the classification is performed. A method of detecting a payment operation or a cancellation operation by inputting a user image into the model can be mentioned. The operations to be classified may be a settlement operation, a cancellation operation, and the like. Here, in the present embodiment, when a settlement operation or a cancel operation is continuously detected for a certain period of time, the operation recognition means 13 recognizes the operation and performs processing.

The authentication means 14 authenticates the registered user based on the user image. In the present embodiment, an image of the user's face taken in advance is registered in association with the user, and the registered user is authenticated by comparing the pre-registered image with the user image. In addition to this, the registered user may be authenticated by any method such as iris authentication or vein authentication, but it is preferable that the authentication is performed by a method in which the user does not touch the device.

The user image analysis process may be performed in the input system, but in the present embodiment, the authentication means 14 requests the external server that manages the registered image for user authentication to analyze the user image, and the analysis result. Is received and collated with the user registered in the database DB, so that the registered user is authenticated. In user authentication, a general face authentication technique that determines a match by extracting the feature amount of the image can be used.

When the payment operation is recognized by the operation recognition means 13 and the authentication of the registered user is successful by the authentication means 14, the payment means 15 setstles the price of the product based on the information of the registered user. To execute. Further, in the present embodiment, the user information is associated with the group specific information that identifies the group to which the user belongs, and the payment means 15 executes the payment process based on the group specific information. Further, based on the group specific information, the settlement process may be executed according to the settlement method specified in the group information registered in advance.

In the present embodiment, as a payment process, the payment means 15 records a transaction history including information identifying a user and a payment amount in a database DB. As a result, it is possible to charge the cost to the individual user or the group to which the user belongs. In addition to this, for example, even if the payment information of a credit card or a bank account is registered as user information and the information for receiving the payment of the product price from the card company or the bank is transmitted to an external server or the like. Good. Further, a payment method may be defined for each group in the group information, and a payment request may be transmitted to an external server or the like according to the payment method.

The display unit 16 displays various information for inputting. After the reading means 11 starts reading, the display unit 16 displays the payment for inputting to end the reading and request the payment processing when the operation recognition means 13 does not recognize the payment operation for a predetermined time. .. In the present embodiment, a touch panel display is used as the display unit 16, and a payment button is displayed on the display unit 16 as a payment display. Then, instead of recognizing the payment gesture by the motion recognition means 13, an instruction by the user is received by input via the payment button, and the reading by the reading means 11 is completed.

On the other hand, if reading is not performed for a predetermined time after the reading means 11 starts reading, the input system resets the product identification information read so far and executes a cancel process for canceling the settlement. The cancel process at this time is the same as the process when the cancel operation is recognized by the motion recognition means 13.

The contact position recording means 17 identifies the contact position touched by the user on the display unit 16 and records it in the database DB together with the time of the contact. Note that the time of contact does not have to be exactly the same as the time of contact, for example, the time of input by the contact or the execution of payment by the contact. Includes time before and after, such as time.

The contact position recording means 17 may specify the contact position by, for example, a contact sensor provided in the display unit 16, or in addition, the contact position is specified by analyzing an image or the like obtained by photographing the display unit 16. You may. In the present embodiment, a button for receiving an input from the user is displayed on the display unit 16, and when the input is performed via the button, the contact position recording means 17 sets the center of the display position of the button. Record as a contact position. In addition, the contact position recording means 17 may identify and record all contact positions by a sensor or the like regardless of the display content on the display unit 16.

Based on the contact position, the button display means 18 determines the position with the least contact after the reference time as the button display position. Here, in the present embodiment, the button display means 18 determines the display position of the payment button displayed as the payment display.

The contact position reset means 19 receives the reset instruction and resets the record of the contact position. When the reset is performed, the button display means 18 determines the position with less contact as the display position based on the contact position after the reset. In the present embodiment, the time when the reset is performed is set as the reference time, so that the display position is determined based on the contact position after the reset. For example, the contact position before that is recorded by the reset instruction. The reset flag may be set to, or the contact position record itself may be erased.

2 and 3 are diagrams showing an example of information registered in the database DB. As shown in FIG. 2, the database DB has a user master, a group master, and a group user master, and the information of each user, the information of the group, and the information of the users belonging to each group are based on these information. And are managed.

The user master is user information about the user, a user ID that is an identifier of the user and is used for login, etc., a system UID that uniquely identifies the user on the system side, a user's name, and image analysis for face authentication. It has a face authentication ID for uniquely identifying the face authentication information in the external server to be executed. By using an immutable system UID that can uniquely identify the user separately from the user ID in this way, the user can be easily changed even if the user freely changes the user ID, for example, when an e-mail address is used as the user ID. Can be identified. Further, various management authorities and the like may be stored as a user master.

Further, when payment is made by withdrawal from a credit card or bank account, the information of the credit card or bank account is further stored as payment information. Here, the information that specifies the payment method may be stored in the user information, and the payment means 15 may execute the payment process based on the designation in the user information. In addition, information such as a login password is managed as needed, and the user logs in to the system based on the user information.

The group master is group information about a group, and includes a group ID that uniquely identifies the group, a group code that is an identifier of the group, a group name that represents the name of the group, and a start date when the group starts using the system. , Has a usage end date, and which the group has terminated or plans to terminate the use of the system. The group code can be arbitrarily changed by the group administrator or the like, but the group ID is an immutable character string that uniquely identifies the group.

The group user master maintains the association between groups and users. The group ID, the system UID of the user belonging to the group, the number of times the user purchased in the group, and the total amount purchased by the user in the group are stored as the group user master. Here, in the present embodiment, "purchased within a group" means a purchase at a store designated as a user group by the group.

In addition, the group master or the group user master may store information that specifies a payment method in the group. Further, in each record of the information, the creator, the creation date and time, the updater, and the update date and time of the record may be further recorded.

FIG. 3 is a diagram showing an example of information stored as a transaction history table. A record in the transaction history table is created for each purchased product. For example, if two types of products are purchased in one transaction, two records are created. The record in the transaction history table identifies the group ID, the transaction date and time, the detail number, the device ID of the device on which the transaction was made, the floor ID of the sales floor where the device is installed, and the user who made the transaction. It includes the system UID, the JAN code of the product, the product name, the product name Kana, the quantity, the unit price, the purchase price, the tax rate, and the consumption tax amount. By recording the transaction history in this way, the purchase amount of each user can be specified for each group, and the user pays the price by a method suitable for each group, such as deduction from salary or use as welfare expenses. It can be performed. As the purchase amount and the consumption tax amount, the amount of the target purchased product is recorded. That is, even if another product is purchased at the same time as the product illustrated in FIG. 3, the transaction history record has the contents of FIG.

Here, in the present embodiment, a time stamp in milliseconds is recorded on the server side as the transaction date and time, and the record of the transaction history at the same time is treated as one transaction. As a result, the input system of the present embodiment can collectively output the information of one transaction in the form of a receipt. In addition, the user can check the transaction history in which his / her system UID is recorded by logging in to the system, and this transaction history can be used as a receipt.

FIG. 4 is a processing flowchart relating to payment for the user to purchase a product. When the input for purchasing the product is started, first, in step S101, the reading means 11 starts reading, the shooting means 12 starts shooting the user, reading the product specific information, or performing the payment operation or the cancel operation. Wait for detection.

FIG. 5 is a diagram showing an example of an input screen displayed on the display unit 16 at this time. The input screen W1 includes a purchase waiting product display unit W11 for displaying the purchase waiting product specified by the reading means 11, and a user image display unit W12 for displaying the user image captured by the photographing means 12.

The user image display unit W12 displays the captured image in real time, and the user can confirm his / her own image. Here, a left-right inverted user image may be displayed on the user image display unit W12 so that the user can easily understand it. Before reading the product identification information, as shown in FIG. 5A, a display requesting reading of the product identification information is performed on the purchase waiting product display unit W11, and when there is one or more purchase waiting products, FIG. 5 ( As shown in b), a display requesting a settlement operation or a cancellation operation is performed at the top and bottom of the screen.

In step S101, the reading means 11 reads the product specific information. In the present embodiment, the code of the product is read by the barcode reader to identify the product. When the product identification information is read, the process proceeds to step S102, the product specified by the reading means 11 is added as a waiting-for-purchase product, and the price of the product is added to the total price. The added information is reflected in the purchase waiting product display unit W11. Then, the process returns to step S101 again, and the input system waits for the next product reading and operation detection.

When the motion recognition means 13 detects an motion in step S101, the process proceeds to step S104 in the case of a settlement operation (“payment operation” in step S103). In step S104, it is determined whether or not the settlement operation is continuously detected for a certain period of time.

If the payment operation does not continue for a certain period of time in step S104, it is further determined in step S105 whether or not a certain time has elapsed without recognizing the payment operation. Then, if a certain time has not elapsed without recognizing the payment operation, the process returns to step S101, and if the payment operation is not recognized for a certain period of time, the process proceeds to step S106 to display the payment button.

As described above, in the present embodiment, if the payment operation is not recognized for a predetermined time, the recognition of the payment operation may have failed for some reason, so that the payment button can be input by the button. Is displayed. In particular, in the present embodiment, when a short-time payment operation less than a preset duration is detected and the payment operation is not recognized for a predetermined time or longer, a payment button is displayed. At this time, a further condition may be that a human face is detected in the image taken by the photographing means 12. Similarly, when the product specific information is not read for a predetermined time, or when a short stop operation less than a preset duration is detected, the display unit 16 displays a stop button. You may.

FIG. 6 is a diagram showing an example of the input screen W1 at the time of recognizing the payment operation. When the payment operation is detected, the recognition display W13 is further displayed on the input screen W1 as shown in FIG. 6A. This allows the user to know that the action has been detected. Here, the recognition display W13 preferably indicates the time until the payment operation is recognized. In the present embodiment, the recognition display W13 changes according to the time when the payment operation is continuously detected, and when the payment operation is continuously detected for a certain period of time, up to the upper part of the screen as shown in FIG. 6A. The recognition display W13 is extended. As a result, the user can be requested to continue the payment operation. Further, during the detection of the payment operation, the countdown until the payment operation is recognized may be displayed as the recognition display W13 together with a message such as "Please do not move", and the user may be requested to continue the payment operation.

When displaying the payment button or the cancel button, since the user touches the button, it is preferable to request that the user touch a part different from the one displayed last time in order to avoid contact of many users with the same part. For example, it is assumed that multiple candidates for the button display position are prepared, the position is changed each time the button is displayed, the contact position by the user is recorded, and the button is displayed at the position where the button is not touched. To. Details of the latter method will be described later.

When the motion recognizing means 13 continuously detects the settlement operation for a certain period of time in step S104, it recognizes the settlement operation and proceeds to step S107. In this way, by asking the user to continue the payment operation for a certain period of time, for example, 3 seconds, if the user accidentally takes a pose similar to the payment operation temporarily, the payment is mistakenly performed. Can be prevented. In step S107, the authentication means 14 performs an authentication process based on the user image. In the present embodiment, the authentication means 14 requests the external server that manages the face recognition information of the user registered in advance to analyze the user image.

The external server analyzes the user image based on the user's face authentication information registered in advance, and identifies the registered user corresponding to the user image. Then, the face authentication ID for uniquely identifying the face authentication information of the user is returned to the input system, and the authentication means 14 receives this. Then, by specifying the user information having the face authentication ID from the database DB, the user who intends to make a payment is specified.

Then, in step S108, the settlement means 15 records the transaction history including the information such as the system UID of the specified user and the purchased product in the database DB, so that the settlement process is performed and the process is completed.

Here, in the present embodiment, the user information is associated with the group identification information, and the group to which the user belongs is specified. This makes it possible to make payments based on group information. For example, for a user of a group for which payment by deduction from salary is specified in the group information, payment processing at the time of purchase is not performed, and billing information for the group is separately generated based on the transaction history. On the other hand, for a user of a group whose personal credit card payment is specified in the group information, a payment processing request is made to a server such as a card company in order to make the payment by the credit card at the time of purchase. ..

If the registered user corresponding to the user image is not specified on the external server that analyzes the user image, or if the user information having the face authentication ID received by the authentication means 14 is not specified, the payment process is performed. It is preferable to display the fact on the display unit 16 instead.

FIG. 7 is a diagram showing an example of a screen displayed on the display unit 16 in steps S107 and S108. While performing face recognition in step S107 or performing payment processing in step S108 after authentication, it is possible to display that the processing for payment is being performed as shown in FIG. 7A. preferable. Further, after the settlement is completed, it is preferable to display that the processing is completed as shown in FIG. 7 (b).

If the motion recognizing means 13 detects the canceling motion in step S101 (“stopping motion” in step S103), the process proceeds to step S109 to see if the canceling motion is continuously detected for a certain period of time as in step S104. Judge whether or not. If the stop operation does not continue for a certain period of time in step S109, the process returns to step S101, and the input system waits again for the next product reading or operation detection. At this time, if the stop operation is not recognized for a certain period of time, the display unit 16 may display the stop button.

FIG. 8 is a diagram showing a display example of a screen when a stop operation is detected. When the cancel operation is detected, the recognition display W13 is displayed on the input screen W1 as in the case of recognizing the payment operation, as shown in FIG. 8A. The display mode of the recognition display W13 may be different between the payment operation and the cancel operation.

When the motion recognition means 13 continuously detects the stop motion for a certain period of time in step S109, the stop motion is recognized and the process proceeds to step S110. In step S110, the input system executes a cancel process for resetting the waiting-for-purchase product and the amount of money, and ends the process. At this time, as shown in FIG. 8B, it is preferable to indicate that the transaction has been canceled.

Here, after the user reads the product specific information, it is assumed that the user leaves the place without performing the stop operation. In order to be able to handle the cancellation in such a case, the input system of the present embodiment proceeds to step S111 if there is no reading of the product in step S101 after the reading means 11 starts reading, and there is no reading. It is determined whether or not a certain period of time (for example, 30 seconds) has passed in the state.

If the reading is not performed continuously for a certain period of time, the process proceeds to step S112 and the cancellation process is performed. At this time, a further condition may be that a human face is not detected in the image taken by the photographing means 12. On the other hand, if a certain period of time has not elapsed, the process returns to step S101 and continues to wait for product reading and operation detection.

The processing procedure shown in FIG. 4 is an example, and the processing order and the like may be arbitrarily changed. For example, the motion recognition means 13 may recognize the settlement operation or the cancel operation, and the authentication means 14 may authenticate the registered user in parallel, or the order may be changed.

As described above, according to the input system of the present embodiment, since the face authentication and the recognition of the payment operation are performed using the user image, the payment can be completed without the user touching the device. This is expected to have the effect of preventing contact infection and preventing wear of the device due to contact. Further, as in the present embodiment, by associating the user information with the group information and performing the payment processing based on the group information, it is possible to perform the payment according to the request of each group.

(2) Determining the display position of the button Next, for example, when displaying the above-mentioned payment button or cancel button, a process relating to determining the display position of the button in order to reduce contact with the same part by a large number of users is shown in FIG. A detailed description will be given with reference to FIG. As described above, the present embodiment is only an example, and the present invention can be used in any input device using a touch panel display.

In the present embodiment, the input system includes a contact position recording means 17 and a button display means 18, and the button display means 18 determines and displays a position on the display unit where there has been little contact in the past as a button display position. A button is displayed at the display position in the unit 16. As a result, it is possible to avoid or reduce a large number of users from touching the same portion on the display unit 16 when the user operates the button, and it is expected to have an effect of reducing the risk of contact infection.

In this embodiment, the database DB further stores the contact position. FIG. 9 shows an example of recording the contact position and an explanatory diagram of the coordinates used for expressing the position on the display unit 16. The contact position recording means 17 of the present embodiment identifies a position touched by the user by a sensor provided in the display unit 16 and records the position in the database DB. In the present embodiment, as the contact position, for example, a region designated by coordinates as shown in the lower part of FIG. 9 and a contact time are recorded.

Here, in order to reduce the processing load, the contact position recording means 17 of the present embodiment does not always specify the contact with the entire display unit 16, and when an input is made via the button, the contact position recording means 17 of the button is used. It is considered that there was contact with the display position. The contact position with respect to the entire display unit 16 may be constantly monitored, and the contact position recording means 17 may specify and record all the contact positions regardless of the display of the button or the like.

The expression of the position on the display unit 16 may be arbitrarily set. Further, when there is contact at a position spanning a plurality of regions, a plurality of regions may be recorded as the contact position.

FIG. 10 is a flowchart relating to the determination of the display position of the button by the button display means 18. First, in step S21, the record of the contact position after the reference time is extracted from the record of the contact position recorded in the database DB. In this way, by determining the display position while avoiding the contact position for the contact after the reference time, for example, by disinfecting the device at a fixed time, contact infection due to contact with the device can be prevented. Can be operated.

Subsequently, in the processes of steps S22 to S25, a position with less contact is determined as a button display position based on the target past contact position. The processing procedure is an example, and may be arbitrarily changed as long as the position where there is little contact can be specified.

First, in step S22, it is confirmed whether or not the contact position is recorded with respect to one of the button display position candidates. If there is no contact record, the process proceeds to step S23, the position is determined as the button display position, and the process ends. On the other hand, if there is a record of the contact position with respect to that position, the process proceeds to step S24 to further confirm the presence or absence of other display position candidates.

If there is no other display position candidate, in step S25, in the recording of the contact position after the reference time, the position with the oldest last contact time is determined as the display position, and the process ends. If there is no position where contact is not recorded after the reference time in this way, multiple users will touch the same part, so notify the terminal device of the store manager who uses the input system to that effect. You may also encourage disinfection of the device.

On the other hand, if there is another display position candidate, it is changed to the next candidate in step S26, the process returns to step S22, and the process is repeated until the display position is determined. As a result, if there is a position where there is no contact after the reference time, that position is determined as the display position, and if there is no position where there is no contact after the reference time, the last contact time after the reference time is set. The oldest position is determined as the display position.

Further, in the present embodiment, the input system includes the contact position reset means 19, and the record of the contact position can be reset. As a result, for example, when a store employee or the like inputs a reset instruction when the device is disinfected, the button can be displayed at a position where there is no contact for the contact after disinfection.

A specific example will be described with reference to FIG. For example, as shown in FIG. 11A, it is assumed that the button is displayed and the user touches the position indicated as the contact position. Then, as the contact position, the coordinates of W and 12 which are the centers of the display positions of the buttons and the contact time are recorded.

Then, when the button is displayed next, the display position is determined while avoiding the contact position as shown in FIG. 11B. Here, as the display position, the center coordinates of the button may be shown, and in the example of FIG. 11B, V and 12 are the display positions.

Here, when operating a round button displayed on the screen, it is known that many users touch the vicinity of the center. Therefore, in FIG. 11B, the edge of the display area overlaps with the contact position, but the center is V, 12, and this coordinate is not recorded as the contact position. Therefore, in this embodiment, such a case occurs. Is also allowed. The display position may be changed on the condition that the display area of the button does not include the past contact position.

The above-mentioned processing procedure is an example, and the design of the processing order and the like can be arbitrarily changed. For example, in the present embodiment, the contact position after the reference time is targeted, and the position with the least contact after the preset reference time is determined as the display position, but the time retroactive from the current time by a predetermined time is treated as the reference time. Can be changed. Further, the position with the least contact may be determined as the button display position based on all the past contact positions without setting a fixed reference time.

11: Reading means 12: Imaging means 13: Motion recognition means 14: Authentication means 15: Payment means 16: Display unit 17: Contact position recording means 18: Button display means 19: Contact position reset means W1: Input screen W11: Purchase standby Product display unit W12: User image display unit W13: Recognition display W14: Payment button

Claims (10)

It is an input system for recognizing pre-registered users and making payments.
A reading means for reading product-specific information that identifies the purchased product, and
Shooting means to shoot the user who makes the payment,
When the payment operation by the user is recognized from the user image captured by the photographing means, the operation recognition means for ending the reading by the reading means and the operation recognition means.
An authentication means that authenticates a registered user based on the user image,
When the payment operation is recognized and the authentication is successful, a payment means that executes a payment process for the purchased product based on the registered user information, and a payment means.
Input system with. The input system according to claim 1, wherein the motion recognition means recognizes a pre-registered gesture as the settlement motion from the user image. The user information is associated with group identification information that identifies the group to which the user belongs.
The input system according to claim 1 or 2, wherein the settlement means executes the settlement process based on the group specific information. The input system according to claim 3, wherein the payment means identifies group information registered in advance based on the group identification information, and executes the payment process according to the payment method specified in the group information. The input according to any one of claims 1 to 4, further comprising a means for canceling the reading when the reading means does not perform the reading continuously for a predetermined time after the reading means starts the reading. system. After the reading means starts the reading, when the motion recognizing means fails to recognize the payment operation, a display unit that displays the payment for finishing the reading and inputting the payment processing is provided. The input system according to any one of claims 1 to 5, further comprising. The display unit considers that the recognition of the payment operation has failed when the operation recognition means does not recognize the payment operation for a predetermined time after the reading means starts the reading. The input system according to 6. The input system according to claim 6 or 7, wherein the payment display requires the user to touch a portion different from the one displayed last time. It is an input program for recognizing pre-registered users and making payments.
A reading means for reading product-specific information that identifies the purchased product, and
Shooting means to shoot the user who makes the payment,
When the payment operation by the user is recognized from the user image captured by the photographing means, the operation recognition means for ending the reading by the reading means and the operation recognition means.
An authentication means that authenticates a registered user based on the user image,
When the payment operation is recognized and the authentication is successful, a payment means that executes a payment process for the purchased product based on the registered user information, and a payment means.
An input program that makes your computer work as. It is an input method for recognizing a pre-registered user and making a payment.
Steps to read product specific information that identifies the purchased product,
The steps to take a picture of the user making the payment,
When the payment operation by the user is recognized from the captured user image, the step of ending the reading and the step of ending the reading.
A step of authenticating a registered user based on the user image, and
When the payment operation is recognized and the authentication is successful, the step of executing the payment process of the price of the purchased product based on the registered user information, and
Input method that causes the computer to execute.