JP7562831B2 - Information processing device - Google Patents

Description

One aspect of the present invention relates to an information processing device.

Conventionally, there is known a mechanism for recognizing a gesture (physical action) made by a user and executing a process according to the recognized gesture. For example, Patent Document 1 discloses a mechanism for determining whether or not a gesture detected from an image captured by a camera includes a preregistered judgment gesture, and treating the operation gesture as valid only if a preregistered operation gesture is recognized during a period in which a judgment gesture is recognized.

JP 2012-185833 A

In the mechanism described in the above-mentioned Patent Document 1, a multi-stage gesture authentication is used to prevent an unintended action of the user from being erroneously recognized as a valid gesture. However, in the above mechanism, an unintended action of the user may coincidentally match an authentication gesture (a judgment gesture or an operation gesture in the mechanism described in Patent Document 1). In addition, in order to avoid such coincidences, it is necessary to make the gesture used for authentication as complicated as possible, but in that case, there is a problem that the convenience for the user is impaired.

Therefore, one aspect of the present invention aims to provide an information processing device that can effectively prevent the occurrence of erroneous operations in a system that performs processing in response to a user's physical movements.

An information processing device according to one aspect of the present invention includes a detection unit that detects a user's physical movements, an extraction unit that extracts specific movements corresponding to the user's habits based on the physical movements detected by the detection unit, a determination unit that determines a movement pattern that does not match the specific movement extracted by the extraction unit as a trigger for a specified process, and a control unit that executes the specified process when a physical movement that matches the movement pattern is detected by the detection unit.

In an information processing device according to one aspect of the present invention, a predetermined process is executed when a physical movement of the user that matches the movement pattern determined by the determination unit is detected as a trigger. That is, an intuitive user interface using physical movements is provided to the user. Furthermore, since the movement pattern is determined so as not to match a specific movement that corresponds to the user's habits, it is possible to accurately prevent the predetermined process from being executed in a situation that the user does not intend. Furthermore, since the movement pattern is determined in this manner, it is not necessary to complicate the movement pattern in order to prevent the occurrence of erroneous operations. Therefore, according to the information processing device, it is possible to preferably prevent the occurrence of erroneous operations in a system that executes processes according to the physical movements of the user.

According to one aspect of the present invention, an information processing device can be provided that can effectively prevent the occurrence of erroneous operations in a system that performs processing in response to a user's physical movements.

1 is a diagram showing an overall configuration of a glasses-type device according to a first embodiment. 1 is a block diagram showing a functional configuration of a glasses-type device according to a first embodiment. FIG. 4 is a diagram illustrating an example of table information stored in a storage unit. 10 is a flowchart showing an example of a procedure of a motion pattern generation process performed by the glasses-type device of the first embodiment. 6 is a flowchart showing an example of a procedure of authentication processing by the glasses-type device of the first embodiment. FIG. 11 is a block diagram showing the functional configuration of a glasses-type device according to a second embodiment. FIG. 11 is a diagram illustrating an example of table information in the second embodiment. 13 is a flowchart showing an example of a procedure of a motion pattern generation process performed by the glasses-type device of the second embodiment. FIG. 13 is a block diagram showing the functional configuration of a glasses-type device according to a third embodiment. FIG. 13 is a diagram illustrating an example of table information in the third embodiment. 13 is a flowchart showing an example of a procedure of a motion pattern generation process performed by a glasses-type device according to a third embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of the eyeglass-type device.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same or corresponding elements are given the same reference numerals, and duplicate descriptions will be omitted.

[First embodiment]
FIG. 1 is a diagram showing the overall configuration of a glasses-type device 10 (information processing device) according to the first embodiment. The glasses-type device 10 is a glasses-type device worn on the head of a user. The glasses-type device 10 is a type of so-called wearable device, and is generally referred to as smart glasses. Examples of the glasses-type device 10 include VR goggles, AR glasses, MR glasses, and the like. As an example, the glasses-type device 10 has a glasses-type main body 21 in which a computer having functional components (see FIG. 2) described later is built in, and a pair of lenses 22 attached to the main body 21 so as to correspond to the positions of both eyes of a user (hereinafter simply referred to as "user") who wears the glasses-type device 10. The lenses 22 are made of a translucent material, similar to the lenses of normal glasses. In addition, an image can be displayed on the inner surface (user-side surface) of the lens 22. The above images include still images and moving images. The user can view the outside world through the lens 22 and can view an image displayed on the inner surface of the lens 22.

The inner surface of the lens 22 may display, for example, a screen of a predetermined application such as a web browser, a menu screen for performing various settings of the eyeglass-type device 10, etc. Here, for example, when logging in to a specific website, web service, etc., or when purchasing content at a shopping site that sells content, user authentication (predetermined processing) may be performed to confirm the above processing (in this example, login processing, purchase processing, etc.). In this embodiment, the eyeglass-type device 10 is configured to perform the above user authentication when a user's eye movement that matches a previously registered operation pattern as a trigger is detected. In other words, the user is able to perform user authentication hands-free by performing a predetermined specific eye movement.

2 is a block diagram showing the functional configuration of the eyeglass-type device 10. As shown in FIG. 2, the eyeglass-type device 10 has an imaging unit 11, a detection unit 12, an extraction unit 13, a memory unit 14, a determination unit 15, and a control unit 16.

The imaging unit 11 captures an image of the user's eyes. The imaging unit 11 may be configured, for example, by a camera provided inside (on the user's side) of the bridge (the portion between a pair of lenses 22) of the main body unit 21. For example, the imaging unit 11 captures an image of the user's eyes (for example, a video including both of the user's eyes as subjects) while the user is wearing the eyeglass-type device 10. However, the configuration of the imaging unit 11 is not limited to the above configuration. For example, the imaging unit 11 may be configured by two cameras arranged for each user's eye (for example, two cameras arranged inside each lens 22).

The detection unit 12 detects the user's physical movements. In this embodiment, the detection unit 12 detects the user's eye movements as the above-mentioned physical movements based on the image of the user's eyes captured by the imaging unit 11. For example, the detection unit 12 pre-stores information on patterns of eye movements such as a movement of moving the gaze in the left-right direction (hereinafter referred to as "gaze movement (left-right)"), a movement of moving the gaze in the up-down direction (hereinafter referred to as "gaze movement (up-down)"), a movement of rotating the gaze in a right-handed direction (clockwise as seen by the user) (hereinafter referred to as "gaze movement (right-handed)"), a movement of rotating the gaze in a left-handed direction (counterclockwise as seen by the user) (hereinafter referred to as "gaze movement (left-handed)"), blinking, winking, etc. Then, the detection unit 12 detects the user's eye movements that match or are similar to the patterns by comparing the user's eye movements included in the image (video) captured by the imaging unit 11 with the pre-stored eye movement patterns. That is, in this embodiment, when the user performs a predetermined type of motion, such as the above-mentioned eye movement, blinking, winking, etc., the detection unit 12 detects these motions.

The extraction unit 13 extracts a specific action corresponding to the user's habit based on the user's eye action detected by the detection unit 12. More specifically, the extraction unit 13 extracts the specific action based on the user's eye action detected by the detection unit 12 during a period other than when user authentication is performed. In other words, the extraction unit 13 learns the user's eye action habit based on the detection result by the detection unit 12. Here, the "specific action corresponding to the user's habit" is, for example, an action habitually performed by the user and an action that may be performed in the user's daily life. In other words, the specific action is an action that may be performed even in a situation other than the situation where the user intentionally performs the above-mentioned user authentication (predetermined processing) during the period when the user is wearing the eyeglass-type device 10. In other words, the specific action includes an action that the user may perform unconsciously in daily life. For example, eye movement (left-handed) and eye movement (right-handed) are not generally performed daily, but there may be users who habitually perform the above-mentioned eye movement to refresh themselves to relieve eye fatigue. In addition, while there are users whose blink frequency (number of times per unit time) is lower than the average, there are also users whose blink frequency is higher than the average. It is also considered that there are individual differences between users in the movement range of the line of sight (left and right) or the line of sight (up and down). The extraction unit 13 extracts such movements corresponding to the habits unique to the user as the specific movements.

For example, the extraction unit 13 creates table information shown in Fig. 3 as information indicating the specific actions of the user. In this example, the table information stores the number of times per unit time (e.g., one minute) and the width of the action for each predetermined type of action.

Figure 3(a) shows the table information before learning. As shown in Figure 3(a), before learning, each item of the table information is empty (no information).

(b) of FIG. 3 shows an example of table information created (updated) by the extraction unit 13 based on the detection results of the detection unit 12 in a predetermined period (e.g., one hour, etc.). In this example, the table information indicates that, per unit time, there were three gaze movements (left and right), two gaze movements (up and down), two gaze movements (leftward), 20 blinks, and no gaze movements (rightward) or winks. As an example, the number of gaze movements (left and right) or gaze movements (up and down) is counted as one movement from a state facing forward to a state facing forward again after a gaze movement of a certain angle or more in either the up, down, left, or right direction. The table information also indicates that the movement range of the gaze movement (left and right) was "30 degrees to 150 degrees," and the movement range of the gaze movement (up and down) was "60 degrees to 120 degrees." Here, the range of gaze movement (left and right) indicates the range of gaze movement of the user when the left direction as seen by the user is 0 degrees and the right direction as seen by the user is 180 degrees. The range of gaze movement (up and down) indicates the range of gaze movement of the user when the downward direction as seen by the user is 0 degrees and the upward direction as seen by the user is 180 degrees.

The above-mentioned table information can be used as information indicating a specific action of a user. For example, a rule may be predefined such that "for gaze movement, an action that is performed a predetermined number of times (e.g., twice) or more per unit time is used as a specific action." In this case, gaze movement (left/right), gaze movement (up/down), and gaze movement (counterclockwise) can be identified as a specific action of a user from the table information shown in FIG. 3(b). In addition, from the table information shown in FIG. 3(b), "the action of blinking at a rate of 20 times per unit time" can also be identified as a specific action of a user.

However, the specific actions extracted by the extraction unit 13 are not limited to the above example (i.e., actions identified from table information). For example, the specific action may be a series of actions consisting of multiple actions. For example, if the detection unit 12 detects "an action of moving one's gaze (left) followed by another movement of one's gaze (right)" a predetermined threshold number of times (e.g., twice) or more during a predetermined period (e.g., one hour), the extraction unit 13 may extract such a series of actions as a specific action.

Information indicating the specific action extracted by the extraction unit 13 (in this embodiment, the table information shown in FIG. 3) is stored in the storage unit 14. The table information can be appropriately updated by continuously performing the processes by the imaging unit 11, the detection unit 12, and the extraction unit 13 described above.

The determination unit 15 determines (generates) a behavior pattern to be used as a trigger for user authentication. The determination unit 15 determines a behavior pattern that does not match the specific behavior extracted by the extraction unit 13 as a trigger for user authentication.

The determination unit 15 may determine a first movement pattern having a certain level of complexity as a trigger for user authentication before a specific movement is extracted by the extraction unit 13 (i.e., before the user's habits are learned and the table information is empty as shown in (a) of Figure 3).

The determination unit 15 may determine, for example, as the first operation pattern, a pattern in which "blinking five or more times in three seconds," "eye movement (counterclockwise)," and "eye movement (up and down) (e.g., eye movement from top to bottom)" are performed in this order. In this way, before learning the user's habits, the determination unit 15 determines a relatively complex first operation pattern that combines multiple operations (here, three operations as an example) as a trigger for user authentication. According to the above process, it is possible to reduce the possibility that the user will accidentally perform an operation corresponding to the first operation pattern. As a result, it is possible to preferably prevent the occurrence of erroneous operations (i.e., user authentication being performed without the user's intention).

The method of generating the first movement pattern is not limited to a specific method, but the first movement pattern may be generated, for example, by randomly extracting one or more movements from a plurality of movements registered in advance. Here, each movement may be associated with a score (complexity) indicating the complexity input (registered) in advance by an operator or the like. In this case, the determination unit 15 can calculate the complexity of the first movement pattern by adding up the scores of each movement constituting the first movement pattern. The determination unit 15 may determine the first movement pattern such that such complexity is equal to or greater than a preset threshold value d1.

On the other hand, after a specific movement is extracted by the extraction unit 13 (i.e., after the user's habits have been learned and table information such as that shown in (b) of Figure 3 has been obtained), the determination unit 15 may determine a second movement pattern that does not match the specific movement and is simpler than the first movement pattern as a trigger for user authentication.

For example, in the example of (b) in FIG. 3, eye movement (clockwise) and winking do not correspond to specific actions. In other words, eye movement (clockwise) and winking are unlikely to be accidentally performed by the user except in a situation where the user intends to perform them (i.e., a situation where user authentication is performed). In this case, the determination unit 15 can determine, for example, eye movement (clockwise) or winking as the second action pattern. In this way, after learning the user's habits, by determining the second action pattern, which does not match the specific action and is simpler than the first action pattern, as the trigger for user authentication, it is possible to prevent erroneous operations due to accidental matching and to improve user convenience.

The method of generating the second movement pattern is not limited to a specific method, but the second movement pattern may be generated, for example, by randomly extracting one or more movements other than the specific movement from among a plurality of movements registered in advance. For example, the determination unit 15 may determine the second movement pattern such that the complexity of the second movement pattern is equal to or less than a preset threshold d2 (d2<d1). For example, the determination unit 15 may determine a single movement other than the specific movement as the second movement pattern. For example, the determination unit 15 may determine, among movements that do not match the specific movement, a movement with the minimum complexity described above as the second movement pattern.

The control unit 16 executes user authentication when the detection unit 12 detects an eye movement of the user that matches the above-mentioned operation pattern (i.e., the operation pattern determined by the determination unit 15 as a trigger for user authentication). For example, consider a scene in which a content purchase screen of a shopping site or the like is displayed on the inner surface of the lens 22, and a pop-up window is displayed requesting user authentication for content purchase. For example, the pop-up window displays a message such as "When purchasing content, please perform user authentication by moving your gaze (clockwise)." In this case, the control unit 16 executes user authentication in response to the detection unit 12 detecting the above-mentioned operation pattern (here, as an example, moving your gaze (clockwise)) after the pop-up window is displayed and the state in which user authentication operation can be accepted is reached. For example, iris authentication is used as the user authentication. After the user authentication is completed, the purchase of the content is confirmed. According to the above configuration, user authentication is executed when the user performs an operation corresponding to the operation pattern determined by the determination unit 15, so that it is possible to appropriately prevent user authentication from being executed at a timing unintended by the user.

With reference to FIG. 4, an example of the procedure of the motion pattern generation process by the determination unit 15 will be described. Before the extraction unit 13 extracts a specific motion, the determination unit 15 determines (generates) a first motion pattern having a certain level of complexity as a trigger for user authentication (step S11). Then, during a predetermined period, the imaging unit 11, the detection unit 12, and the extraction unit 13 learn the user's habits. That is, the imaging unit 11 and the detection unit 12 detect the user's eye movements (step S12), and the extraction unit 13 extracts a specific motion corresponding to the user's habit (step S13). After the user's specific motion is extracted in this way, the determination unit 15 determines (generates) a second motion pattern that does not match the specific motion and is simpler than the first motion pattern as a trigger for user authentication (step S14).

After the second movement pattern is determined, further learning of the user's habits may occur, and a specific movement that had not been discovered until then may be newly extracted. The newly extracted specific movement may then match the movement pattern used as a trigger for user authentication (the movement pattern determined by the determination unit 15). In such a case, the determination unit 15 may appropriately change the movement pattern used as a trigger for user authentication based on the latest information (e.g., the latest table information (see (a) of Figure 3)).

An example of the user authentication process will be described with reference to FIG. 5. First, in a state in which a user authentication operation can be accepted, the image capturing unit 11 and the detection unit 12 detect the user's eye movement (step S21). Next, the control unit 16 determines whether the detected user's eye movement matches the movement pattern determined by the determination unit 15 as a trigger for user authentication (step S22). If it is determined that there is a match (step S22: YES), the control unit 16 executes user authentication (step S23). On the other hand, if it is not determined that there is a match (step S22: NO), the control unit 16 does not execute user authentication.

In the above-described eyeglass-type device 10, the detection of the user's eye movement (physical movement) that matches the movement pattern determined by the determination unit 15 triggers user authentication (predetermined processing). That is, an intuitive user interface using physical movements (gestures) is provided to the user. Furthermore, since the above movement pattern is determined so as not to match a specific movement corresponding to the user's habit, it is possible to accurately prevent user authentication from being performed in a situation unintended by the user. In addition, since the movement pattern is determined in this manner, it is not necessary to complicate the movement pattern in order to prevent the occurrence of erroneous operations. Therefore, according to the eyeglass-type device 10, the occurrence of erroneous operations can be suitably prevented in a system that executes processing according to the user's physical movement.

Furthermore, the eyeglass-type device 10 is a device worn on the user's head, and the detection unit 12 detects the user's eye movements as physical movements based on the image of the user's eyes captured by the imaging unit 11. According to the above configuration, a hands-free user interface can be provided to the user in a situation where the user is wearing the eyeglass-type device 10 and has difficulty viewing his or her hands, making it difficult to input characters (enter a password) into a device such as a smartphone.

In addition, before the specific motion is extracted by the extraction unit 13, the determination unit 15 determines a first motion pattern having a certain level of complexity as a trigger for user authentication, and after the specific motion is extracted by the extraction unit 13 (i.e., after the table information as shown in FIG. 3B is obtained), the determination unit 15 determines a second motion pattern that does not match the specific motion and is simpler than the first motion pattern as a trigger for user authentication. According to the above configuration, by using a relatively complex first motion pattern before learning the user's habits, it is possible to avoid coincidence (i.e., the motion corresponding to the user's habit matches the motion pattern adopted as the trigger for user authentication), and to accurately prevent user authentication from being performed in a situation unintended by the user (misoperation). Furthermore, after learning the user's habits, it is possible to improve the user's convenience while preventing the occurrence of erroneous operations by using a simplified second motion pattern after avoiding the specific motion.

The determination unit 15 may also determine, as the second motion pattern, the motion with the smallest degree of complexity from among one or more motions that do not match the specific motion among a plurality of predetermined motions each associated with a complexity score (complexity). In this case, it is possible to effectively improve user convenience by using the most simplified motion as possible as a trigger for user authentication while preventing the occurrence of erroneous operations.

[Second embodiment]
A glasses-type device 10A according to a second embodiment will be described with reference to Figures 6 to 8. As shown in Figure 6, the glasses-type device 10A is different from the glasses-type device 10 mainly in that it further includes a measurement unit 17 and has a determination unit 15A instead of the determination unit 15. The other configurations of the glasses-type device 10A are the same as those of the glasses-type device 10.

The measurement unit 17 instructs the user to perform a predetermined type of physical movement, and measures whether the user can perform the predetermined type of physical movement based on the physical movement performed by the user in response to the instruction. In this embodiment, the physical movement is the user's eye movement. The predetermined type of physical movement is a type of movement that has been set and registered in advance. In this embodiment, as an example, the predetermined type of physical movement is gaze movement (left and right), gaze movement (up and down), gaze movement (leftward), gaze movement (rightward), blinking, and winking (right eye wink and left eye wink).

The measurement unit 17 instructs the user to perform the body movement of each of the predetermined types, for example. The method of instructing the user is not limited to a specific method. For example, the instruction to the user may be performed by outputting a sound indicating the instruction from a speaker (not shown) provided in the eyeglass-type device 10A, or by displaying and outputting information indicating the instruction (e.g., text, etc.) on the inner surface of the lens 22. The measurement unit 17 determines the accuracy of the movement performed by the user in response to the instruction. For example, the measurement unit 17 may determine the accuracy based on the eye movement of the user detected by the imaging unit 11 and the detection unit 12 described above. Then, the measurement unit 17 adds information indicating whether the user can perform the body movement of each of the predetermined types (i.e., whether the user can perform the movement accurately) to the table information.

Figure 7 is a diagram showing an example of table information generated by the processing of the measurement unit 17 described above. The table information shown in Figure 7 is table information generated by adding the measurement results by the measurement unit 17 (the "Yes or No" column) to the table information shown in Figure 3 (b). In this example, information indicating that the user was able to accurately perform actions other than gaze movement (clockwise) and winking with the right eye, but was unable to accurately perform gaze movement (clockwise) and winking with the right eye, is added to the table information by the measurement of the measurement unit 17. In addition, information indicating that the user is able to perform gaze movement (left and right) within a range of 20 degrees to 160 degrees is added to the table information. In this way, the measurement unit 17 may measure not only the execution possibility of each action, but also the limit value of the movement of each action (such as the maximum movement width of the eyeball movement).

The determination unit 15A determines, as a trigger for user authentication, a movement pattern that does not match the specific movement extracted by the extraction unit 13 and does not include a movement that the user is unable to perform as determined by the measurement unit 17. In this way, the determination unit 15A differs from the determination unit 15 in that the determination unit 15A determines a movement pattern further based on the measurement results of the measurement unit 17.

With reference to FIG. 8, an example of the processing procedure of the motion pattern generation processing in the eyeglass-type device 10A will be described. The flowchart in FIG. 8 shows only the processing procedure executed in addition to the determination unit 15. That is, it is assumed that the extraction unit 13 has already completed the extraction of the specific motion. First, the measurement unit 17 measures whether or not the user can perform each of the predetermined types of eye motions (six types of motions shown in FIG. 7 as an example in this embodiment) (step S101). This processing results in table information with the "possible" column added as shown in FIG. 7. Next, the determination unit 15A determines, as a trigger for user authentication, a motion pattern that does not match the specific motion extracted by the extraction unit 13 and does not include a motion that the user cannot perform as determined by the measurement unit 17 (i.e., a motion with a value of "x" in the "possible" column in the table information shown in FIG. 7). In the example of FIG. 7, neither the gaze movement (clockwise) nor the wink corresponds to the specific motion. However, the user cannot accurately perform the gaze movement (clockwise) or the wink of the right eye. Therefore, the determination unit 15A determines, for example, a motion pattern that does not include eye movement (clockwise) and winking with the right eye (for example, winking with the left eye) as a trigger for user authentication.

According to the eyeglass-type device 10A, as with the eyeglass-type device 10, by using an action pattern that does not match a specific action as a trigger for user authentication, it is possible to accurately prevent user authentication from being performed in a situation that the user does not intend (erroneous operation). In addition, by excluding actions that the user cannot perform accurately from the above action patterns, it is possible to improve user convenience. In other words, by excluding actions that the user is not good at from the candidates for the above action patterns, it is possible to reduce the burden on the user.

[Third embodiment]
A glasses-type device 10B according to a third embodiment will be described with reference to Figures 9 to 11. As shown in Figure 9, the glasses-type device 10B is mainly different from the glasses-type device 10 in that it further includes a stimulation unit 18 and has an extraction unit 13B instead of the extraction unit 13. The other configurations of the glasses-type device 10B are similar to those of the glasses-type device 10.

The stimulation unit 18 provides a stimulation to the user. The stimulation to the user can take various forms. For example, examples of the stimulation include a sudden sound output from a speaker (not shown), a sudden screen display on the inner surface of the lens 22, a sound (vibration) of the eyeglass-type device 10B, etc. The stimulation unit 18 provides the stimulation to the user as described above, thereby momentarily putting the user into a state of tension.

The extraction unit 13B executes the same process as the extraction unit 13 described above, and extracts a specific movement based on the eye movement of the user detected by the detection unit 12 when the user is given the above-mentioned stimulus. According to the above process, the extraction unit 13B can extract, as a specific movement, a movement that the user involuntarily (unconsciously) performs when the user is in a tense state due to the above-mentioned stimulus being given. For example, assume that the user moves their eyes (up and down) when the above-mentioned stimulus is given to the user. In this case, the extraction unit 13B extracts the eye movement (up and down) as a specific movement that corresponds to the habit of the user when in a tense state.

Figure 10 is a diagram showing an example of table information to which information indicating a specific action in a tense state extracted by the stimulation unit 18 and the extraction unit 13B described above is added. The table information shown in Figure 10 is table information generated by adding a column of "occurrence in a tense state" to the table information shown in (b) of Figure 3. In the "occurrence in a tense state" column, "Yes" indicates that it occurred in a tense state, and "No" indicates that it did not occur in a tense state. In this example, the table information indicates that a gaze movement (up and down) was performed when the stimulation unit 18 was applied to the user. In other words, the table information indicates that the gaze movement (up and down) was extracted as a specific action in a tense state. In this example, the gaze movement (up and down) is extracted as a specific action even in normal times (when no stimulation is applied), but if the gaze movement (up and down) is not extracted as a specific action in normal times, it is possible to extract the gaze movement (up and down) as a specific action by the above processing of the stimulation unit 18 and the extraction unit 13B.

With reference to FIG. 11, an example of the processing procedure of the motion pattern generation process in the eyeglass-type device 10B will be described. The flowchart in FIG. 11 shows only the procedure of the specific processing part executed by the stimulation unit 18 and the extraction unit 13B. That is, it is assumed that the extraction unit 13B has already completed the extraction of the specific motion in normal times (when no stimulation is applied). First, the stimulation unit 18 applies the above-mentioned stimulation to the user (step S201). Next, the imaging unit 11 and the detection unit 12 detect the user's eye motion (step S202), and the extraction unit 13B extracts the specific motion corresponding to the user's habit when the stimulation is applied (in a tense state) (step S203). Next, the determination unit 15 determines the motion pattern that does not match the specific motion extracted in normal times (when no stimulation is applied) and the specific motion extracted when the stimulation is applied as a trigger for user authentication (step S204). In addition, in step S204, if extraction of specific movements during normal times has not been completed, the determination unit 15 may determine a movement pattern that does not match the specific movement extracted when a stimulus is applied as a trigger for user authentication.

For example, in a situation where user authentication is required, a pop-up window or a voice requesting user authentication may be output to the user. At this time, the user may be surprised and momentarily tense. As a result, the user may reflexively perform a specific action (in the above example, moving one's eyes (up and down)). With the eyeglass-type device 10B, the action that a user reflexively performs when in a tense state can be extracted as a specific action, so that it is possible to more reliably prevent user authentication from being performed in a situation where the user does not intend it (misoperation).

[Modification]
In the above embodiment, as an example of the "predetermined process", user authentication performed at the time of login, charging, etc. is given, but the predetermined process may be a process other than user authentication. For example, the predetermined process may be a process that can be executed at any time, such as a process of starting up a setting screen of the eyeglass-type device 10. The predetermined process may also be a process of selecting one or more items from a plurality of items. For example, when there are two items A and B, each of "process of selecting A", "process of selecting B", and "process of selecting A and B" may be prepared as a predetermined process, and an operation pattern that serves as a trigger for execution may be determined for each process. For example, when gaze movement (right-handed), gaze movement (left-handed), and wink of the left eye do not correspond to a specific user action, the determination unit 15 may determine gaze movement (right-handed) as a trigger for "process of selecting A", gaze movement (left-handed) as a trigger for "process of selecting B", and wink of the left eye as a trigger for "process of selecting A and B". In this way, the mechanism of the above embodiment is applicable to any process that requires confirmation of the user's intention.

In addition, in the above embodiment, a glasses-type device is given as an example of an "information processing device", but the form of the information processing device is not limited to this. For example, the information processing device may be a smartwatch or the like worn on the user's arm. Furthermore, the "physical movement" may be a movement of a body part other than the movement of the user's eyes. For example, if the information processing device is a smartwatch, the physical movement may be a movement of the user's arm. In this case, the detection unit may detect the movement of the user's arm, for example, by a sensor (for example, an acceleration sensor, a gyro sensor, etc.) provided in the smartwatch.

Note that the block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or may be realized using two or more devices that are physically or logically separated and directly or indirectly connected (e.g., using wires, wirelessly, etc.) and these multiple devices. The functional blocks may be realized by combining the one device or the multiple devices with software.

Functions include, but are not limited to, judging, determining, assessing, calculating, processing, deriving, investigating, searching, verifying, receiving, sending, outputting, accessing, resolving, selecting, choosing, establishing, comparing, anticipating, expecting, regarding, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping and assigning.

For example, the eyeglass-type device 10 in one embodiment of the present disclosure may function as a computer that performs the information processing method of the present disclosure. FIG. 12 is a diagram showing an example of the hardware configuration of the eyeglass-type device 10 according to one embodiment of the present disclosure. Note that the hardware configuration of the above-mentioned eyeglass-type devices 10A and 10B is similar to the hardware configuration of the eyeglass-type device 10. Each of the eyeglass-type devices 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

In the following description, the term "apparatus" may be interpreted as a circuit, device, unit, etc. The hardware configuration of the eyeglass-type device 10 may be configured to include one or more of the devices shown in FIG. 12, or may be configured to exclude some of the devices.

Each function of the eyeglass-type device 10 is realized by loading a specific software (program) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communication via the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc.

The processor 1001 also reads out programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes the computer to execute at least a part of the operations described in the above-mentioned embodiment is used. For example, the determination unit 15, etc. may be realized by a control program stored in the memory 1002 and operating in the processor 1001, and may be similarly realized for other functional blocks. Although the above-mentioned various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may be transmitted from a network via a telecommunications line.

The memory 1002 is a computer-readable recording medium, and may be composed of at least one of, for example, a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing an information processing method according to one embodiment of the present disclosure.

Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc.

The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that the input device 1005 and the output device 1006 may be integrated into one configuration (e.g., a touch panel).

In addition, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

The eyeglass-type device 10 may also be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. The present embodiment can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is intended as an illustrative example and does not have any restrictive meaning with respect to the present embodiment.

The processing steps, sequences, flow charts, etc. of each aspect/embodiment described in this disclosure may be reordered unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order and are not limited to the particular order presented.

The input and output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The input and output information, etc. may be overwritten, updated, or added to. The output information, etc. may be deleted. The input information, etc. may be transmitted to another device.

The determination may be based on a value represented by a single bit (0 or 1), a Boolean (true or false) value, or a numerical comparison (e.g., with a predetermined value).

Each aspect/embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to being done explicitly, but may be done implicitly (e.g., not notifying the specific information).

Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

Additionally, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and/or wireless technologies (such as infrared, microwave), then these wired and/or wireless technologies are included within the definition of a transmission medium.

The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

In addition, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using other corresponding information.

The names used for the parameters described above are not intended to be limiting in any way. Moreover, the formulas etc. using these parameters may differ from those expressly disclosed in this disclosure. The various information elements may be identified by any suitable names, and the various names assigned to these various information elements are not intended to be limiting in any way.

As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Any reference to elements using designations such as "first," "second," etc., used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must precede the second element in some way.

When used in this disclosure, the terms "include," "including," and variations thereof are intended to be inclusive, as is the term "comprising." Additionally, the term "or," as used in this disclosure, is not intended to be an exclusive or.

In this disclosure, where articles have been added by translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are in the plural form.

In this disclosure, the term "A and B are different" may mean "A and B are different from each other." In addition, the term may mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

10, 10A, 10B... glasses-type device (information processing device), 11... imaging unit, 12... detection unit, 13, 13B... extraction unit, 15, 15A... determination unit, 16... control unit, 17... measurement unit, 18... stimulation unit.

Claims (5)

A detection unit that detects a physical movement of a user;
an extraction unit that extracts a specific motion corresponding to a habit of the user based on the physical motion detected by the detection unit;
a determination unit that determines a motion pattern that does not match the specific motion extracted by the extraction unit as a trigger for a predetermined process;
a control unit that executes the predetermined process when the physical movement that matches the movement pattern is detected by the detection unit;
Equipped with
The determination unit is
Before the specific movement is extracted by the extraction unit, a first movement pattern having a certain level of complexity or more is determined as a trigger for the predetermined process;
After the specific action is extracted by the extraction unit, the information processing device determines a second action pattern that does not match the specific action and is simpler than the first action pattern as a trigger for the specified processing . the information processing device is a device worn on the user's head,
An imaging unit that images the user's eyes,
The information processing device according to claim 1 , wherein the detection unit detects an eye movement of the user as the physical movement based on an image of the user's eyes captured by the imaging unit. 3. The information processing device according to claim 1, wherein the determination unit determines, as the second movement pattern, the movement with the smallest score from among one or more movements that do not match the specific movement among a plurality of predetermined movements each associated with a complexity score. a measurement unit that instructs the user to perform a predetermined type of physical movement and measures whether the user can perform the predetermined type of physical movement based on the physical movement performed by the user in response to the instruction;
The information processing device according to any one of claims 1 to 3, wherein the determination unit determines, as a trigger for the predetermined processing, a movement pattern that does not match the specific movement extracted by the extraction unit and does not include a movement that the user is unable to perform that is determined by the measurement unit. A stimulation unit that stimulates the user is further provided.
The information processing device according to claim 1 , wherein the extraction unit extracts the specific movement based on the physical movement detected by the detection unit when the stimulus is given to the user.

Images