JP2020185110A - Rehabilitation support system - Google Patents

Abstract

To provide a rehabilitation support system capable of appropriately carrying out rehabilitation training of a motion using tools required in a daily life.SOLUTION: A rehabilitation support system 1 includes: a simulation article Q grasped by an object person; instruction means 11, 12, and 14 for giving the object person instructions on a predetermined motion using the simulation article Q; determination means 10 for determining whether or not the motion carried out by the object person using the simulation article Q satisfies a determination criterion preset on the simulation article Q or a hand of the object person grasping the simulation article Q; and notification means 11, 12, and 14 for notifying whether or not the object person is executing the predetermined motion correctly based on a result of the determination by the determination means 10.SELECTED DRAWING: Figure 1

Description

The present invention relates to a rehabilitation support system.

A rehabilitation support system that displays an image that supports rehabilitation from an image projection device such as a projector has been considered (see, for example, Patent Document 1). This rehabilitation support system displays the target position on the projected image and moves the limbs according to the target person to promote improvement of physical function and functional training.

JP-A-2018-161529

However, conventional techniques use tools that are commonly used in everyday life, such as "putting a cup", "wiping a desk with a table wipe", "unlocking with a key", and "squeezing a towel". Rehabilitation training is not always optimized.

The present invention has been made in view of such a problem, and provides a rehabilitation support system capable of appropriately performing rehabilitation training of movements using tools required in daily life.

According to the first aspect of the present invention, the rehabilitation support system includes a simulated product held by a target person, an instruction means for instructing the target person to perform a predetermined operation using the simulated product, and the target. The determination means for determining whether or not the operation performed by the person using the simulated product satisfies the determination criteria set for the simulated product or the hand of the target person holding the simulated product, and the determination. A notification means for notifying whether or not the target person is correctly performing the predetermined operation based on the determination result by the means is provided.

According to the second aspect of the present invention, in the rehabilitation support system according to the first aspect, the notification means is a phenomenon corresponding to an operation performed by using the simulated product based on the determination result by the determination means. Visually or audibly.

According to the third aspect of the present invention, in the rehabilitation support system according to the first or second aspect, the simulated product is a towel, the pressure sensor is provided inside, and the indicating means is the subject. On the other hand, the towel is instructed to be squeezed, and the determination means determines whether or not the pressure detection result by the pressure sensor is equal to or higher than a predetermined threshold value.

According to a fourth aspect of the present invention, in the rehabilitation support system according to any one of the first to third aspects, the simulated product is a key, and the instruction means inserts the key into a predetermined position. The key is set to a predetermined position in the direction, and the key is instructed to be turned. The determination means determines whether or not the hand holding the key has rotated by a predetermined amount or more in the insertion direction.

According to a fifth aspect of the present invention, in the rehabilitation support system according to any one of the first to fourth aspects, the simulated product is a cup, and the instruction means puts the cup in a predetermined position. It is instructed to arrange the object, and the determination means uses the detection of the presence of an object at the predetermined position for a predetermined time as a determination criterion.

According to a sixth aspect of the present invention, in the rehabilitation support system according to any one of the first to fifth aspects, the simulated product is a table wipe, and the instruction means is a wipe that causes a wipe operation. The operating range is displayed as an image, and the determination means uses the detection of the presence of the target person's hand at a plurality of locations in the wipe operating range as a determination criterion.

According to the rehabilitation support system according to the present invention, it is possible to appropriately perform rehabilitation training for movements using tools required in daily life.

It is a figure which shows the whole structure of the rehabilitation support system which concerns on one Embodiment of this invention. It is a block diagram which shows the functional structure of the control device which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the control apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the desk-wiping application which concerns on one Embodiment of this invention. It is a figure for demonstrating the function of the desk-wiping application which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the key turning application which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the cup application which concerns on one Embodiment of this invention. It is a figure for demonstrating the function of the cup application which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the towel application which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the processing of the character input application which concerns on one Embodiment of this invention. It is a figure for demonstrating the function of the character input application which concerns on one Embodiment of this invention.

Hereinafter, the rehabilitation support system 1 according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 11.

(Overall configuration of rehabilitation support system)
FIG. 1 is a diagram showing an overall configuration of a rehabilitation support system according to an embodiment of the present invention.
As shown in FIG. 1, the rehabilitation support system 1 includes a control device 10, a projection device 11, a monitor 12, a motion reading device 13, a robot 14, a pressure sensor 15, and an inter-device communication device 16.

The control device 10 is built in the projection device 11 and controls the entire operation of the rehabilitation support system 1. The functions of the control device 10 will be described later. The control device 10 is one aspect of the determination means in the present embodiment.

The projection device 11 includes a projection unit 11a that projects the projected image P onto the work surface T, and a detection sensor 11b that detects the position and movement of an object (a target person's hand, finger, simulated article, etc.). The projection device 11 is an aspect of the instruction means and the notification means in the present embodiment.

The monitor 12 displays operation instructions and various production images related to training. The monitor 12 is an aspect of the instruction means and the notification means in the present embodiment.

The motion reading device 13 detects the shape, position, and posture of the hand located above the main body, and arranges and visualizes a model reflecting the detection result (hand shape, position, posture) in the virtual space. ..

The robot 14 gives an explanation of the training and an effect (speaking, swinging motion, light emitting effect, etc.) according to the result of the training to the target person. The robot 14 is an aspect of the instruction means and the notification means in the present embodiment.

The pressure sensor 15 is provided inside the towel Q4, which is one of the simulated products Q. When the subject squeezes the towel Q4, the degree of the squeezing strength can be detected by an electric signal.

The inter-device communication device 16 is a communication interface that enables mutual communication between the control device 10 (projection device 11), the monitor 12, the motion reading device 13, the robot 14, and the pressure sensor 15 described above. Note that FIG. 1 shows an example in which the device-to-device communication device 16 realizes wired communication between devices via a cable, but the present invention is not limited to this. In another embodiment, the device-to-device communication device 16 may be in a mode of realizing wireless communication between devices by utilizing wireless communication technology.

The simulated supplies Q are a cup Q1, a key Q2, a table wipe Q3, and a towel Q4, respectively. Each is used for rehabilitation training of the subject.

(Functional configuration of control device)
FIG. 2 is a block diagram showing a functional configuration of a control device according to an embodiment of the present invention.
As shown in FIG. 2, the control device 10 includes a CPU 100, a connection interface 101, a memory 102, and a recording medium 103.

The CPU 100 exhibits various functions as a main application 1001, a desk wiping application 1002, a key turning application 1003, a cup application 1004, a towel application 1005, and a character input application 1006 by operating according to a predetermined program.

The connection interface 101 is for exchanging information with other devices via the inter-device communication device 16 (display request to the monitor 12, finger position detection request by the motion reading device 13, production request to the robot 14, etc.). Interface.

The memory 102 is a so-called main storage device, and instructions and data for the CPU 100 to operate based on a program are expanded.

The recording medium 103 is a so-called auxiliary storage device, and may be, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. For example, user information (training history information of the target person) and the like are recorded in the recording medium 103.

The main application 1001 is an application that controls the entire game for training. The main application 1001 selects and executes each training application described later based on the input information by the user (target person).

The CPU 100 has a desk cleaning application 1002, a key turning application 1003, a cup application 1004, a towel application 1005, and a character input application 1006 as training applications.

The desk wiping application 1002 is an application for performing wipe operation training performed by using the table wiping Q3.

The key turning application 1003 is an application for performing key turning operation training performed using the key Q2.

The cup application 1004 is an application for performing cup placement operation training performed using the cup Q1.

Towel application 1005 is an application for performing towel squeezing operation training performed using towel Q4.

The character input application 1006 is an application for performing type training on the keyboard displayed on the projected image P.

(Processing flow of control device)
FIG. 3 is a flowchart showing an example of processing of the control device according to the embodiment of the present invention.
Hereinafter, an example of the processing of the control device 10 will be described in detail with reference to FIG.

When the power of each device (control device 10, projection device 11, monitor 12, motion reading device 13, robot 14, and inter-device communication device 16) of the rehabilitation support system 1 is turned on, the control device 10 is shown in FIG. The main application 1001 is started as shown in (Step S10).

Next, the main application 1001 causes the projection device 11 to display the setting screen (step S11). Then, the projection unit 11a of the projection device 11 projects the projection image P including the setting screen onto the work surface T.
The setting screen is a screen for accepting input of user information and training contents. The user information is identification information (user number, name, etc.) that can identify the user (target person) who carries out the rehabilitation training. The training contents are "wipe motion training", "key turning motion training", "cup placement motion training", "towel squeezing motion training", and "character input (type) training". Includes information such as order, number of times (number of sets), level (difficulty), and hand to be trained (right or left hand).
At this time, the main application 1001 may request the monitor 12 to display explanatory information instructing the contents to be input to the setting screen, the input method, and the like. Further, the main application 1001 may request the robot 14 to output the same explanatory information by voice.

Next, the main application 1001 accepts a user's input operation on the setting screen (step S12). At this time, the detection sensor 11b of the projection device 11 detects the position of the user's hand (finger) and transmits the detection signal to the control device 10. Then, the main application 1001 acquires the user information and the training content input by the user based on the detection signal, and records the information on the recording medium 103 (step S13).

Next, the main application 1001 trains the training application (one of the desk wiping application 1002, the key turning application 1003, the cup application 1004, the towel application 1005, and the character input application 1006) based on the acquired training content. (Step S14). The details of the processing in each training application will be described later.

When the processing of the training application is completed, the main application 1001 determines whether or not all the training specified in the training content is completed (step S15).
If there is training that has not been completed yet (step S15: NO), the main application 1001 returns to step S14 and starts the training application related to the next training specified in the training content.
On the other hand, the main application 1001 ends the user training when all the training specified in the training content is completed (step S15: YES). If there is another user to be trained, the main application 1001 may return to step S11 so that the training of the other user can be accepted.

(Processing flow of desk cleaning application)
FIG. 4 is a flowchart showing an example of processing of a desk wiping application according to an embodiment of the present invention.
FIG. 5 is a diagram for explaining the function of the desk wiping application according to the embodiment of the present invention.
When the next training is specified as "wipe motion training" in the training content, the main application 1001 executes a series of processes related to the training in the desk wiping application 1002 in step S14 of FIG. Let me. Then, when the series of processes of the desk cleaning application 1002 is completed, the main application 1001 proceeds to step S15 of FIG. Here, a series of processing flows of the desk wiping application 1002 in step S14 (FIG. 3) will be described in detail with reference to FIGS. 4 to 5.

As shown in FIG. 4, the main application 1001 activates the desk cleaning application 1002 (step S100).

Next, the desk wiping application 1002 causes the monitor 12 and the robot 14 to output instruction information instructing the simulated product Q used by the user in the “desk wiping (wipe) operation training”, the operation to be performed, the time limit, and the like (step). S101).
Then, the monitor 12 displays instruction information (images, characters, etc.) instructing the user to perform a wipe operation of wiping a predetermined range using the table wipe Q3 within a predetermined time limit. Further, the robot 14 outputs the same instruction information by voice.

When the output of the instruction information is completed, the desk cleaning application 1002 starts training (step S102). At this time, the desk wiping application 1002 starts counting the remaining time of the time limit. The remaining time may be displayed on the monitor 12.

Next, the desk wiping application 1002 causes the projection device 11 to display the wipe operation range M1 and turns on all the “dirt flags” (step S103).
Then, the projection unit 11a of the projection device 11 displays the projected image P including the wipe operation range M1 that causes the wipe operation on the work surface T as shown in FIG. The wipe operation range M1 is expressed as, for example, an image (dirt image) representing "dirt". The wipe operation range M1 is variable according to the level specified in the training content, and the higher the level (difficulty level), the larger the area of the wipe operation range M1.
The dirt flag indicates whether or not the user has performed an operation of wiping dirt by a wipe operation within the wipe operation range M1. For example, as shown in FIG. 5, the wipe operation range M1 includes a plurality of determination areas R1. A dirt flag is set for each of the determination areas R1, and the desk wiping application 1002 turns off the dirt flag of the determination area R1 when it is detected that the user has performed a wipe operation in the determination area R1.
The determination area R1 is not displayed on the projected image P. Further, the number of determination regions R1 is variable according to the area of the wipe operation range M1.

Next, the desk wiping application 1002 detects whether an OnTouch event exists in the projected image P (step S104). The OnTouch event is a signal indicating that the position on the projected image P where the user's hand or the table wipe Q3 is touched is detected. The detection sensor 11b of the projection device 11 transmits a detection signal indicating the presence or absence of an OnTouch event to the control device 10 (desk wiping application 1002). The desk wiping application 1002 detects the presence of an OnTouch event based on the detection signal received from the projection device 11.

Next, the desk wiping application 1002 determines whether or not there is remaining time (step S105).

When the remaining time becomes "0" (step S105: NO), the desk wiping application 1002 causes the projection device 11, the monitor 12, and the robot 14 to output the effect information indicating that the time has expired (step S105: NO). Step S106).
Then, the projection device 11 and the monitor 12 perform an effect expression (display of an image, characters, etc.) indicating that the wipe operation is not completed within the time limit. Further, the robot 14 outputs the same information by voice. After this, the desk wiping application 1002 proceeds to step S113.

On the other hand, when there is remaining time (step S105: YES), the desk wiping application 1002 determines whether or not an OnTouch event exists in any one of the determination areas R1 (FIG. 5) (step S107).

If the OnTouch event does not exist in all the determination areas R1, the desk wiping application 1002 returns to step S104 and waits for the next OnTouch event.

Further, when the OnTouch event exists in any of the determination areas R1, the desk wiping application 1002 wipes the user's hand or the table from the determination area R1 (previous detection area) where the previous OnTouch event was detected (step S107: YES). It is determined whether or not Q3 has moved (step S108).

When the determination area R1 in which the OnTouch event is detected is the same determination area R1 as the previous detection area, the desk wiping application 1002 determines that the user's hand or table wipe Q3 has not moved (step S108: NO). In this case, the desk wiping application 1002 returns to step S104 and waits for the next OnTouch event.

When the determination area R1 in which the OnTouch event is detected is a determination area different from the previous detection area, the desk wiping application 1002 determines that the user's hand or table wipe Q3 has moved to a new determination area R1 (step S108: YES). In this case, the desk wiping application 1002 determines whether or not the dirt flag of the determination area R1 after movement (the determination area R1 in which the OnTouch event is detected this time) is ON (step S109).

When the dirt flag of the determination area R1 after the movement is OFF (step S109: NO), the desk wiping application 1002 determines that the wipe operation has been executed in the determination area R1 and returns to step S104.

On the other hand, when the dirt flag of the determination area R1 after movement is ON (step S109: YES), the desk wiping application 1002 determines that the wipe operation has been executed for the first time in this determination area R1. In this case, the desk wiping application 1002 outputs the effect information (video, sound effect, etc.) for erasing the "dirty" image around the determination area R1 in the wipe operation range M1 to the projection device 11 and outputs the effect information (video, sound effect, etc.) to the determination area R1. Turn off the dirt flag of (step S110). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. As a result, it is possible to make the user recognize that the operation of wiping the dirt is properly executed in the area where the user's hand or the table wipe Q3 is located.

Next, the desk wiping application 1002 determines whether or not the dirt flags of all the determination areas R1 have been turned off (step S111).

When the dirt flag of one or more determination areas R1 is ON (step S111: NO), the desk wiping application 1002 determines that there is an area in which the wipe operation has not been executed yet, and returns to step S104.

On the other hand, when the dirt flags of all the determination areas R1 are OFF (step S111: YES), the desk wiping application 1002 determines that the wipe operation is completed in all the areas of the wipe operation range M1 and performs the wipe operation training. The projection device 11 is made to output the effect information (video, sound effect, etc.) indicating that the above is successful (step S112). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. Further, the desk wiping application 1002 may cause the monitor 12 and the robot 14 to output effect information (text, image, voice, etc.) indicating the success of the wipe operation training.

Next, the desk wiping application 1002 determines whether or not the training for the number of sets (number of times) specified in the training content has been completed (step S113). If all sets have not been completed, for example, if only 1 to 2 sets out of 3 sets have been completed (step S113: NO), the desk wiping application 1002 returns to step S102 and the wipe operation training is performed to the user again. Let me. When the entire set is completed (step S113: YES), the desk wiping application 1002 ends the process.

(Processing flow of key turning application)
FIG. 6 is a flowchart showing an example of processing of the key turning application according to the embodiment of the present invention.
When the next training is specified as "key turning motion training" in the training content, the main application 1001 causes the key turning application 1003 to execute the process related to the training in step S14 of FIG. Then, when the series of processes of the key turning application 1003 is completed, the main application 1001 proceeds to step S15 of FIG. Here, a series of processing flows of the key turning application 1003 in step S14 (FIG. 3) will be described in detail with reference to FIG.

As shown in FIG. 6, the main application 1001 activates the key turning application 1003 (step S200).

Next, the key turning application 1003 causes the motion reading device 13 to register the model of the user's hand (step S201). For example, the key turning application 1003 causes the monitor 12 and the robot 14 to output instruction information instructing the user to hold his / her hand over the motion reading device 13. When the user holds his / her hand over the motion reading device 13, the motion reading device 13 recognizes the shape, position, and posture of the user's hand and creates a model of the user's hand.

Next, the key turning application 1003 causes the monitor 12 and the robot 14 to output instruction information instructing the simulated product Q used by the user in the “key turning motion training”, the motion to be performed, the time limit, and the like (step S202).
Then, the monitor 12 aligns the key Q2 with a predetermined position (a keyhole in the 3D space described later) in a predetermined insertion direction, and instructs the user to perform the operation of turning the key Q2 within a predetermined time limit. Display information (images, characters, etc.). Further, the robot 14 outputs the same instruction information by voice.

When the registration of the hand model and the output of the instruction information are completed, the key turning application 1003 displays the image of the 3D space in which the keyhole is arranged on the monitor 12 and starts the training (step S203). At this time, the key turning application 1003 starts counting the remaining time of the time limit. The remaining time may be displayed on the monitor 12.

Next, the key turning application 1003 detects the shape, position, and posture of the user's hand in the 3D space via the motion reading device 13 (step S204). The motion reading device 13 recognizes the shape, position, and posture of the user's hand, converts the shape, position (coordinates), and posture (angle) of the hand model in 3D space, and converts the control device 10 (key rotation application 1003). ) Is sequentially transmitted as a detection signal. The key turning application 1003 detects the position of the user's hand and the key Q2 in the 3D space based on the detection signal received from the motion reading device 13.
Further, the motion reading device 13 superimposes the shape, position, and posture of the hand model on the image in the 3D space and displays it on the monitor 12. This allows the user to recognize where his or her hand is in 3D space.

Next, the key turning application 1003 determines whether or not there is remaining time (step S205).

When the remaining time becomes "0" (step S205: NO), the key turning application 1003 causes the projection device 11, the monitor 12, and the robot 14 to output effect information indicating that the time has expired (step S205: NO). Step S206).
Then, the projection device 11 and the monitor 12 perform an effect expression (display of an image, characters, etc.) indicating that the key turning operation is not completed within the time limit. Further, the robot 14 outputs the same information by voice. After this, the key turning application 1003 proceeds to step S214.

On the other hand, when there is remaining time (step S205: YES), the key turning application 1003 determines whether or not the position of the hand model in the 3D space matches the position of the keyhole and the key flag is OFF. (Step S207).
The key flag is information indicating whether or not the user has inserted the key into the keyhole in the 3D space. When it is ON, it indicates that the key is inserted into the keyhole, and when it is OFF, the key is inserted into the keyhole. Indicates a non-existent state.

When the position of the hand model in 3D space matches the position of the keyhole and the key flag is OFF (step S207: YES), the key turning application 1003 states that the user has inserted the key into the keyhole in 3D space. to decide. In this case, the key turning application 1003 causes the monitor 12 to output effect information (video, sound effect, etc.) indicating that the key has been inserted into the keyhole, and turns on the key flag (step S208).

When the position of the hand model in the 3D space matches the position of the keyhole, but the key flag is ON (step S207: NO and step S209: YES), the key turning application 1003 is already keyed by the user. Is already inserted in the keyhole, and the process proceeds to step S211.

If the position of the hand model in 3D space does not match the position of the keyhole (step S207: NO and step S209: NO), the key turning application 1003 will place the hand in a position different from the position of the keyhole in 3D space. Judge that the model (key) of is located. In this case, the key turning application 1003 causes the monitor 12 and the robot 14 to output instruction information instructing the hand (key Q2) to move to the position of the keyhole, turns off the key flag (step S210), and steps S204. Return to.

Next, the key turning application 1003 detects the rotation of the user's hand (wrist) (step S211). For example, the key turning application 1003 detects the rotation direction and the angle of the portion corresponding to the wrist of the hand model from the detection signal received from the motion reading device 13.

Next, the key turning application 1003 determines whether or not the user has turned the wrist by a predetermined amount or more in the insertion direction while holding the key Q2 (step S212).
Specifically, the key turning application 1003 determines whether or not the user has performed an operation of turning the wrist a predetermined number of times by a set angle or more.
The set angle is variable according to the level specified in the training content, and the higher the level (difficulty), the larger the angle is set. For example, level 1 (difficulty: low) is set to 20 degrees, level 2 (difficulty: medium) is set to 60 degrees, and level 3 (difficulty: high) is set to 90 degrees. The number of times the wrist is turned is set to, for example, three times. The number of times can be changed arbitrarily, and the main application 1001 may specify the number of times on the setting screen and include it in the training content.

When it is determined that the user has not turned the wrist more than a predetermined amount (step S212: NO), the key turning application 1003 returns to step S204.

On the other hand, when it is determined that the user has turned the wrist more than a predetermined amount (step S212: YES), the key turning application 1003 determines that the user has properly performed the key turning operation, and the key turning operation training is successful. The effect information (text, image, sound effect, etc.) indicating that is output to the monitor 12 (step S213). Further, the key turning application 1003 may cause the projection device 11 and the robot 14 to output effect information (video, audio, etc.) indicating the success of the wipe motion training.

Next, the key turning application 1003 determines whether or not the training for the number of sets (number of times) specified in the training content has been completed (step S214). If the entire set has not been completed (step S214: NO), the key turning application 1003 returns to step S203 and causes the user to perform the key turning operation training again. When the entire set is completed (step S214: YES), the key turning application 1003 ends the process.

(Processing flow of cup application)
FIG. 7 is a flowchart showing an example of processing of the cup application according to the embodiment of the present invention.
FIG. 8 is a diagram for explaining the function of the cup application according to the embodiment of the present invention.
When the training content specifies that the next training is "cup placement motion training", the main application 1001 causes the cup application 1004 to execute a series of processes related to the training in step S14 of FIG. Then, when the series of processes of the cup application 1004 is completed, the main application 1001 proceeds to step S15 of FIG. Here, a series of processing flows of the cup application 1004 in step S14 (FIG. 3) will be described in detail with reference to FIGS. 7 to 8.

As shown in FIG. 7, the main application 1001 activates the cup application 1004 (step S300).

Next, the cup application 1004 causes the monitor 12 and the robot 14 to output instruction information instructing the simulated product Q used by the user in the “cup placement operation training”, the operation to be performed, the time limit, and the like (step S301).
Then, the monitor 12 displays instruction information (image, character, etc.) instructing the user to perform the operation of arranging the cup Q1 at the predetermined position within the predetermined time limit. Further, the robot 14 outputs the same instruction information by voice.

When the output of the instruction information is completed, the cup application 1004 starts training (step S302). At this time, the cup application 1004 starts counting the remaining time of the time limit. The remaining time may be displayed on the monitor 12.

Next, the cup application 1004 causes the projection device 11 to display the cup arrangement range M2 (FIG. 8) indicating the position where the cups are arranged (step S303). Then, the projection unit 11a of the projection device 11 displays the projection image P including the cup arrangement range M2 indicating the position where the cup Q1 is arranged on the work surface T as shown in FIG. The cup arrangement range M2 is variable according to the level specified in the training content, and the higher the level (difficulty level), the farther the cup arrangement range M2 is from the user. Further, when performing a plurality of sets of cup arrangement operation training, the cup application 1004 may change the position of the cup arrangement range M2 for each set.

Next, the cup application 1004 detects whether an OnTouch event exists in the projected image P (step S304). The OnTouch event is a signal indicating that the position on the projected image P where the cup Q1 is placed is detected. The detection sensor 11b of the projection device 11 transmits a detection signal indicating the presence or absence of an OnTouch event to the control device 10 (cup application 1004). The cup application 1004 detects the presence of an OnTouch event based on the detection signal received from the projection device 11.

Next, the cup application 1004 determines whether or not there is remaining time (step S305).

When the remaining time becomes "0" (step S305: NO), the cup application 1004 causes the projection device 11, the monitor 12, and the robot 14 to output the effect information indicating that the time has expired (step). S306).
Then, the projection device 11 and the monitor 12 perform an effect expression (display of an image, characters, etc.) indicating that the cup arrangement operation is not completed within the time limit. Further, the robot 14 outputs the same information by voice. After this, the cup application 1004 proceeds to step S310.

On the other hand, when there is remaining time (step S305: YES), the cup application 1004 determines whether or not the OnTouch event continues to exist in all the determination areas R2 (FIG. 9) of the cup arrangement range M2 for a predetermined time or longer. Is determined (step S307).
As shown in FIG. 9, a plurality of (for example, three) determination regions R2 are set on the projection device 11 side on the outer edge of the cup arrangement range M2. In the cup application 1004, when the OnTouch event indicating that the cup Q1 has been placed in all the judgment areas R2 continues to exist for a predetermined time or longer (for example, 3 seconds or longer), the cup Q1 is within the cup placement range M2. Judge that it was placed in.

When the OnTouch event does not exist in one or more of the plurality of determination areas R2 (step S307: NO), the cup application 1004 determines that the cup Q1 is not correctly arranged in the cup arrangement range M2. In this case, the cup application 1004 causes the monitor 12 and the robot 14 to output instruction information instructing the cup Q1 to move within the cup arrangement range M2 (step S308), and returns to step S304.

On the other hand, when the OnTouch event exists in all of the plurality of judgment areas R2 (step S307: YES), the cup application 1004 determines that the cup placement operation has been completed, and determines that the cup placement operation training has succeeded. The effect information (video, sound effect, etc.) to be shown is output to the projection device 11 (step S309). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. Further, the desk wiping application 1002 may cause the monitor 12 and the robot 14 to output effect information (text, image, voice, etc.) indicating the success of the wipe operation training.

Next, the cup application 1004 determines whether or not the training for the number of sets (number of times) specified in the training content has been completed (step S310). If the entire set has not been completed (step S310: NO), the desk wiping application 1002 returns to step S302 and causes the user to perform the cup placement operation training again. When the entire set is completed (step S310: YES), the cup application 1004 ends the process.

(Towel application processing flow)
FIG. 9 is a flowchart showing an example of processing of the towel application according to the embodiment of the present invention.
When the training content specifies that the next training is "towel squeezing motion training", the main application 1001 causes the towel application 1005 to perform a series of processes related to the training in step S14 of FIG. Then, when the series of processes of the towel application 1005 is completed, the main application 1001 proceeds to step S15 of FIG. Here, a series of processing flows of the towel application 1005 in step S14 (FIG. 3) will be described in detail with reference to FIG.

As shown in FIG. 9, the main application 1001 activates the towel application 1005 (step S400).

Next, the towel application 1005 causes the monitor 12 and the robot 14 to output instruction information instructing the simulated product Q used by the user in the “towel squeezing operation training”, the operation to be performed, the time limit, and the like (step S401).
Then, the monitor 12 displays instruction information (image, character, etc.) instructing the user to perform the operation of squeezing the towel Q4 within a predetermined time limit. Further, the robot 14 outputs the same instruction information by voice.

When the output of the instruction information is completed, the towel application 1005 starts training (step S402). At this time, the towel application 1005 starts counting the remaining time of the time limit. The remaining time may be displayed on the monitor 12.

Next, the towel application 1005 receives a detection signal from the pressure sensor 15 provided inside the towel Q4 (step S403). For example, the towel application 1005 receives a detection signal every second.

Next, the towel application 1005 determines whether or not there is remaining time (step S405).

When the remaining time becomes "0" (step S405: NO), the towel application 1005 causes the projection device 11, the monitor 12, and the robot 14 to output the effect information indicating that the time has expired (step). S406).
Then, the projection device 11 and the monitor 12 perform an effect expression (display of an image, characters, etc.) indicating that the towel squeezing operation is not completed within the time limit. Further, the robot 14 outputs the same information by voice. After this, the towel application 1005 proceeds to step S413.

On the other hand, when there is remaining time (step S405: YES), the towel application 1005 receives the detection signal for detecting the pressure equal to or higher than the threshold value twice in succession, and determines whether or not the loosening flag is OFF. (Step S407).
The threshold value is variable according to the level specified in the training content, and the higher the level (difficulty level), the larger the threshold value is set.
The loosening flag indicates whether the towel Q4 is squeezed (ON) or loosened (OFF).

When the detection signal equal to or higher than the threshold value is received twice in succession and the loosening flag is OFF (the towel Q4 is not squeezed) (step S407: YES), the user uses the towel Q4 in the towel application 1005. It is judged that the operation of squeezing is performed. In this case, the towel application 1005 outputs the effect information (image, sound effect, etc.) in which the towel Q4 is squeezed to the projection device 11 and turns on the loosening flag (step S408). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. The effect information is, for example, an image and a sound effect in which water droplets fall into the bucket displayed on the projected image P. As a result, the user can be made aware that the operation of squeezing the towel Q4 has been properly executed.

If the pressure is less than the threshold value or the loosening flag is ON (step S407: NO), the process proceeds to step S411. Details of step S411 will be described later.

Next, the towel application 1005 determines whether or not the user has squeezed the towel Q4 a predetermined number of times (step S409).

When the user performs the operation of squeezing the towel Q4 a predetermined number of times (for example, three times) (step S409: YES), the towel application 1005 determines that the towel squeezing operation is completed, and the towel squeezing operation training is successful. The effect information (video, sound effect, etc.) indicating the above is output to the projection device 11 (step S410). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. Further, the towel application 1005 may cause the monitor 12 and the robot 14 to output effect information (text, image, sound, etc.) indicating the success of the towel squeezing motion training.

On the other hand, when the number of times it is determined that the user has performed the operation of squeezing the towel Q4 is less than a predetermined number of times (for example, 3 times) (step S409: NO), the process proceeds to step S411.

When the pressure is below the threshold value or the loosening flag is ON (step S407: NO),
Alternatively, if the number of times it is determined that the user has performed the operation of squeezing the towel Q4 is less than a predetermined number of times, the towel application 1005 determines whether or not the user has loosened the towel Q4 (step S411).
For example, when the pressure detected by the pressure sensor 15 is lower than the pressure detected last time by a predetermined value or more, the towel application 1005 determines that the user has performed the operation of loosening the towel Q4 (step S411: YES). In this case, the towel application 1005 turns off the loosening flag (step S412) and returns to step S403.
On the other hand, when the difference between the pressure detected by the pressure sensor 15 and the pressure detected last time is less than a predetermined value, the towel application 1005 determines that the user has not performed the operation of loosening the towel Q4. In this case, the towel application 1005 returns to step S403 without changing the loosening flag.

Next, the towel application 1005 determines whether or not the training for the number of sets (number of times) specified in the training content has been completed (step S413). If the entire set has not been completed (step S413: NO), the towel application 1005 returns to step S402 and causes the user to perform the towel squeezing operation training again. When all the sets are completed (step S413: YES), the towel application 1005 ends the process.

(Processing flow of character input application)
FIG. 10 is a flowchart showing an example of processing of a character input application according to an embodiment of the present invention.
FIG. 11 is a diagram for explaining the function of the character input application according to the embodiment of the present invention.
When the training content specifies that the next training is "character input (type) training", the main application 1001 causes the character input application 1006 to execute a series of processes related to the training in step S14 of FIG. .. Then, when the series of processes of the character input application 1006 is completed, the main application 1001 proceeds to step S15 of FIG. Here, a series of processing flows of the character input application 1002 in step S14 (FIG. 3) will be described in detail with reference to FIGS. 10 to 11.

As shown in FIG. 10, the main application 1001 activates the character input application 1006 (step S500).

Next, the character input application 1006 causes the monitor 12 and the robot 14 to output instruction information instructing the operation, the time limit, and the like performed by the user in the “character input (type) training” (step S501).
Then, the monitor 12 instructs the user to perform a type operation of inputting the specified character string via the keyboard displayed on the projected image P within a predetermined time limit (image, character, etc.). ) Is displayed. Further, the robot 14 outputs the same instruction information by voice.

When the output of the instruction information is completed, the character input application 1006 starts training (step S502).

Next, the character input application 1006 displays the projected image P including the keyboard M3 (FIG. 11) on the projection device 11, and displays the question content (character string to be input) on the monitor 12 (step S503).
The keyboard M3 can be changed according to the usage scene of the user. For example, the keyboard M3 is a numerical keyboard used for inputting a personal identification number, amount of money, etc. at an ATM or the like at level 1. Further, the keyboard M3 is a 50-tone keyboard for inputting Japanese at level 2, and a keyboard for inputting characters, numbers, function keys, etc. used in a personal computer or the like at level 3.
Further, the character input application 1006 starts counting the remaining time of the time limit when the question content is displayed on the monitor 12. The remaining time may be displayed on the monitor 12.

Next, the character input application 1006 detects whether an OnTouch event exists in the projected image P (step S504). The OnTouch event is a signal indicating that the position on the projected image P where the user's hand (finger) is touched is detected. The detection sensor 11b of the projection device 11 transmits a detection signal indicating the presence or absence of an OnTouch event to the control device 10 (character input application 1006). The character input application 1006 detects the existence of the OnTouch event based on the detection signal received from the projection device 11.

Next, the character input application 1006 determines whether or not there is remaining time (step S505).

When the remaining time becomes "0" (step S505: NO), the character input application 1006 causes the projection device 11, the monitor 12, and the robot 14 to output the effect information indicating that the time has expired (step S505: NO). Step S506). Then, the projection device 11 and the monitor 12 perform an effect expression (display of an image, characters, etc.) indicating that the type is not completed within the time limit. Further, the robot 14 outputs the same information by voice. After this, the character input application 1006 proceeds to step S512.

On the other hand, when there is remaining time (step S505: YES), the character input application 1006 determines whether or not the determination key R3 (FIG. 11) is touched (the OnTouch event exists) (step S507).
The determination key R3 is a key touched by the user when the input of all the character strings is completed, and is, for example, a "confirmation" key as shown in FIG.

When it is not detected that the determination key R3 is touched (step S507: NO), the character input application 1006 updates the input result display of the monitor 12 according to the touched key (the OnTouch event exists). (Step S508).
For example, the character input application 1006 displays "1" in the input result display field of the monitor 12 when the "1" key of the keyboard M3 is touched. At this time, when the previously input character is displayed in the input result display field, the monitor 12 adds a new character "1" after this character and displays it. Further, the character input application 1006 causes the monitor 12 to delete (hide) the previously input character when the "correction" key of the keyboard M3 is touched. In this way, the character input application 1006 changes the character string displayed in the input result display field of the monitor 12 according to the touched key.
When the display on the monitor 12 is updated, the character input application 1006 returns to step S504 and waits for the next OnTouch event.

Further, when the character input application 1006 detects that the determination key R3 is touched (step S507: YES), the character string input so far (the character string displayed in the input result display field of the monitor 12). ) Matches the character string specified in the question content (step S509).

If the character strings input so far do not match the character strings specified in the question content (step S509: NO), the character input application 1006 monitors the effect information indicating that the input character strings are incorrect. 12 and the robot 14 are made to output (step S510), and the process returns to step S504.

On the other hand, when the character strings input so far match the character strings specified in the question content (step S509: YES), the character input application 1006 determines that the user has correctly input characters, and performs type training. The effect information (video, sound effect, etc.) indicating the success of the above is output to the projection device 11 (step S511). Then, this effect information is projected onto the work surface T as a projected image P by the projection unit 11a of the projection device 11. Further, the character input application 1006 may cause the monitor 12 and the robot 14 to output effect information (text, image, voice, etc.) indicating the success of the type training.

Next, the character input application 1006 determines whether or not the training for the number of sets (number of times) specified in the training content has been completed (step S512). If the entire set has not been completed (step S512: NO), the character input application 1006 returns to step S502 and causes the user to perform the type training again. When the entire set is completed (step S512: YES), the character input application 1006 ends the process.

In the present embodiment, in the type training, a mode in which the user inputs characters by hand (finger) has been described as an example, but the present invention is not limited to this. For example, in another embodiment, the user may input characters using a simulated product Q such as a touch pen.

(Action effect)
As described above, the rehabilitation support system 1 according to the present embodiment includes a simulated product Q held by the target person (user) and an instruction means (instructing the target person to perform a predetermined operation using the simulated product Q). The projection device 11, the monitor 12, the robot 14) and the operation performed by the target person using the simulated product Q satisfy the judgment criteria set for the simulated product Q or the target person's hand holding the simulated product Q. Judgment means (control device 10) for determining whether or not, and notification means (projection device 11, monitor 12, 12) for notifying whether or not the target person is correctly performing a predetermined operation based on the determination result by the determination means. The robot 14) is provided.
With the above configuration, the rehabilitation support system 1 allows the target person (user) to actually grasp the simulated equipment Q (cup Q1, key Q2, table wipe Q3, towel Q4) and use it. The success or failure of the hand movement that occurs at that time is the judgment target.
As a result, the rehabilitation support system 1 can appropriately perform rehabilitation training for movements using tools required in daily life.

Further, the notification means (projection device 11) visually or audibly produces a phenomenon according to the operation performed by using the simulated product Q based on the determination result by the determination means.
By doing so, the rehabilitation support system 1 can give realistic feedback to the movement performed by the target person, and can easily recognize whether or not the target person is performing an appropriate movement. Can be done.

Further, the simulated product Q is a towel Q4 and includes a pressure sensor 15 inside. The instruction means instructs the subject to squeeze the towel Q4. The determination means (towel application 1005) uses whether or not the pressure detection result of the pressure sensor 15 is equal to or greater than a predetermined threshold value as a determination criterion.
By doing so, the rehabilitation support system 1 can appropriately train the operation of squeezing a towel or the like.

The simulated product Q is the key Q2. The instruction means aligns the key Q2 with a predetermined position in a predetermined insertion direction, and instructs the key Q2 to turn. The determination means (key turning application 1003) uses whether or not the hand holding the key Q2 has rotated by a predetermined amount or more in the insertion direction as a determination criterion.
By doing so, the rehabilitation support system 1 can appropriately train the operation of turning the key (unlocking operation).

The simulated product Q is a cup Q1. The instructing means instructs the cup Q1 to be placed in a predetermined position. The determination means (cup application 1004) uses the detection of the presence of an object at a predetermined position for a predetermined time as a determination criterion.
By doing so, the rehabilitation support system 1 can appropriately train the operation of arranging an object such as a cup at a predetermined position.

The simulated product Q is a table wipe Q3. The instruction means displays an image of the wipe operation range M1 that causes the wipe operation. The determination means (desk wiping application 1002) uses that the presence of the target person's hand is detected at a plurality of locations in the wipe operation range M1 as a determination criterion.
By doing so, the rehabilitation support system 1 can appropriately train the wipe operation such as wiping the desk.

In each of the above-described embodiments, the various processing processes of the CPU 100 described above are stored in a computer-readable recording medium in the form of a program, and the various processing is performed by the computer reading and executing this program. .. The computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, this computer program may be distributed to a computer via a communication line, and the computer receiving the distribution may execute the program.

The above program may be for realizing a part of the above-mentioned functions. Further, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

As described above, some embodiments according to the present invention have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope of the invention described in the claims and the equivalent scope thereof, as are included in the scope and gist of the invention.

1 Rehabilitation support system 10 Control device 11 Projection device 11a Projection unit 11b Detection sensor 12 Monitor 13 Motion reader 14 Robot 15 Pressure sensor 16 Inter-device communication device 100 CPU
1001 Main application 1002 Character input application 1002 Desk wiping application 1003 Key turning application 1004 Cup application 1005 Towel application 1006 Character input application 101 Connection interface 102 Memory 103 Recording medium P Projected image Q Simulation equipment Q1 Cup Q2 Key Q3 Table wipe Q4 Towel

Claims (6)

Simulated supplies held by the subject and
An instruction means for instructing the target person to perform a predetermined operation using the simulated product, and
A determination means for determining whether or not the operation performed by the target person using the simulated item satisfies the determination criteria set for the simulated item or the hand of the subject holding the simulated item.
A notification means for notifying whether or not the target person is correctly performing the predetermined operation based on the determination result by the determination means.
Rehabilitation support system equipped with. The rehabilitation support system according to claim 1, wherein the notification means visually or audibly produces a phenomenon according to an operation performed by using the simulated device based on a determination result by the determination means. The simulated product is a towel, which has a pressure sensor inside.
The instruction means instructs the subject to squeeze the towel.
The rehabilitation support system according to claim 1 or 2, wherein the determination means is based on whether or not the pressure detection result by the pressure sensor is equal to or higher than a predetermined threshold value. The simulated product is a key
The instruction means aligns the key with a predetermined position in a predetermined insertion direction, and instructs the key to be turned.
The rehabilitation support system according to any one of claims 1 to 3, wherein the determination means is based on whether or not the hand holding the key has rotated by a predetermined amount or more in the insertion direction. The simulated product is a cup,
The instructing means instructed the cup to be placed in place and
The rehabilitation support system according to any one of claims 1 to 4, wherein the determination means is based on detecting the presence of an object at a predetermined position for a predetermined time as a determination criterion. The simulated product is a table wipe,
The instruction means displays an image of the wipe operation range in which the wipe operation is performed.
The rehabilitation support system according to any one of claims 1 to 5, wherein the determination means is based on the fact that the presence of the target person's hand is detected at a plurality of locations in the wipe operation range.

Images