JP6932946B2 - Detection system, detection method, and detection program - Google Patents

Description

The present invention relates to a detection system, a detection method, a detection program, a processing device, and an exercise mat.

A technique for imaging a person in motion in an appropriate state has been proposed (see, for example, Patent Document 1 below).

Japanese Unexamined Patent Publication No. 2007-288682

According to the aspect of the present invention, the analysis unit includes a detection unit that optically detects a moving human body and an analysis unit that analyzes the pronation of the leg of the human body using the detection result of the detection unit. The position of the human body is detected from the detection result of the detection unit, at least a part of the leg is specified from the detected position of the human body, the position of at least a part of the specified leg is compared with the reference, and the pronation A detection system is provided that determines the condition.
According to the aspect of the present invention, a detection unit that optically detects a moving human body, a direction detection unit that detects the moving direction of the human body based on the detection result of the detection unit, and a human body using the detection result of the detection unit. The analysis unit includes an analysis unit that analyzes the pronation of the leg of the human body, and the analysis unit detects the position of at least a part of the leg of the human body with respect to the reference in the crossing direction with respect to the movement direction of the human body from the detection result. A detection system is provided that compares at least some positions with a reference to determine the state of pronation.
According to the aspect of the present invention, there is provided a detection system including a detection unit that optically detects a moving human body and an analysis unit that analyzes the pronation of the leg of the human body using the detection result of the detection unit. NS.
According to the first aspect of the present invention, a detection unit that optically detects a moving human body, a displacement calculation unit that calculates a time change of the detection result of the detection unit, and a human body based on the calculation result of the displacement calculation unit. A detection system including a landing determination unit for determining the landing of the vehicle is provided.

According to the first aspect of the present invention, a receiving unit that receives a landing timing that determines the landing of the human body based on a time change of a detection result that optically detects a moving human body, and a landing timing that the receiving unit receives. Provided is a processing device including an analysis unit that analyzes the pronation of the leg of the human body based on the above.

According to the aspect of the present invention, the detection of the pronation includes optically detecting the moving human body and detecting the pronation of the leg of the human body based on the detection result of detecting the human body. Then, the position of the human body is detected from the detection result of detecting the human body, at least a part of the leg is specified from the detected position of the human body, and the position of at least a part of the specified leg is compared with the reference. A detection method for determining the pronation status is provided.
According to an aspect of the present invention, there is provided a detection method including optically detecting a moving human body and detecting pronation of a leg of the human body based on the detection result of detecting the human body. NS.
According to the second aspect of the present invention, the moving human body is optically detected, the time change of the detection result of detecting the human body is calculated, and the landing of the human body is determined based on the time change. And, a detection method including.

According to the aspect of the present invention, the computer is made to detect the pronation of the leg of the human body based on the detection result of optically detecting the moving human body, and in the detection of the pronation, the human body is detected. The position of the human body is detected from the detected detection result, at least a part of the leg is specified from the detected position of the human body, the position of at least a part of the specified leg is compared with the reference, and the state of the pronation. A detection program is provided to determine.
According to an aspect of the present invention, there is provided a detection program that causes a computer to detect pronation of a leg of a human body based on a detection result of optically detecting a moving human body.
According to the third aspect of the present invention, the computer performs the calculation of the time change of the detection result of optically detecting the moving human body and the determination of the landing of the human body based on the time change. A detection program is provided.

According to the aspect of the present invention, the motion is arranged in the motion range of the moving human body, and the absorption rate of infrared light is higher than that of the human body in the wavelength band of infrared light in which the detection unit that optically detects the human body has sensitivity. Mats are provided.
According to the fourth aspect of the present invention, in the wavelength band of infrared light provided on the moving path of the moving human body and having a sensitivity of the detection unit for optically detecting the human body, the infrared light of the infrared light is higher than that of the human body. An exercise mat with a high absorption rate is provided.

It is a figure which shows the detection system which concerns on 1st Embodiment. It is a figure which shows the process of the detection part and the process of a processing part which concerns on 1st Embodiment. It is a figure which shows the process of the displacement calculation part and the process of a landing determination part which concerns on 1st Embodiment. It is a flowchart which shows the detection method which concerns on 1st Embodiment. It is a figure which shows the detection system which concerns on 2nd Embodiment. It is a figure which shows the processing by the processing part which concerns on 2nd Embodiment. It is a figure which shows the processing of the processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the detection method which concerns on 2nd Embodiment. It is a figure which shows the processing of the motion analysis part which concerns on the modification. It is a figure which shows the detection system which concerns on 3rd Embodiment. It is a figure which shows the arrangement of the detection part which concerns on embodiment. It is a figure which shows the detection system which concerns on 4th Embodiment. It is a figure which shows the exercise mat which concerns on embodiment.

[First Embodiment]
The first embodiment will be described. FIG. 1 is a diagram showing a detection system according to an embodiment. The detection system 1 according to the embodiment is used, for example, for footwear selection, person authentication security, fashion shows and sports (eg, baseball, soccer, golf, yoga) and the like. The detection system 1 according to the embodiment is, for example, a motion detection system, an exercise support system, or the like.

The detection system 1 detects the human body HM (body) of a target (eg, user) moving in a predetermined direction. The movement of the human body HM is, for example, an exercise involving the movement of the legs or arms of the human body HM (eg, movements such as walking, running, and posture change). The detection system 1 detects, for example, a human body HM moving (eg, moving) on a support surface G below the human body HM (eg, in the direction of gravity). The support surface G includes, for example, at least one of a surface that supports the human body HM, a surface on which the human body HM moves (moving surface), and a surface on which the human body HM lands (landing). The detection system 1 (or detection device 2) includes detecting an object, a living thing, or a robot (eg, a humanoid robot) moving in a predetermined direction.

The support surface G includes, for example, a predetermined surface such as the ground or floor surface, or an upper surface such as a sheet or mat laid on the predetermined surface. When the detection system 1 detects the human body HM, the human body HM may move barefoot or move with an attachment such as shoes (for example, an object moving with the human body HM). May be good. Further, the movement of the human body HM includes the movement of an object attached to the human body HM. For example, the detection of the human body HM by the detection system 1 may include the detection of an attachment moving with the human body HM.

The detection system 1 includes a detection device 2, a processing device 3, and a display device 4. The detection device 2 optically detects a moving human body HM (eg, a part such as a leg or an arm). The processing device 3 determines the landing of the human body HM (eg, landing state, operating posture during movement) based on the detection result of the detection device 2 (eg, pixel value of an image, received signal). For example, the processing device 3 determines whether or not the barefoot human body HM has landed on the support surface G. The processing device 3 may determine whether or not the attachment attached to the human body HM has landed on the support surface G. Hereinafter, each part of the detection system 1 will be described.

The detection device 2 includes, for example, a detection unit 5 and a storage unit 6. The detection unit 5 executes the detection process at a predetermined frequency (eg, sampling rate, frame rate), for example. The detection unit 5 includes, for example, a light receiving element such as an image sensor. The image sensor is, for example, a CCD image sensor or a CMOS image sensor. The detection unit 5 optically detects the human body HM by, for example, an imaging process. As the detection result, the detection unit 5 outputs, for example, data (eg, RGB data, pixel values such as gray scale) of an captured image (hereinafter, referred to as an captured image).

The storage unit 6 includes, for example, a non-volatile memory. The storage unit 6 stores the detection result of the detection unit 5. The storage unit 6 stores, for example, the detection result of the detection unit 5 in relation to the information of the timing when the detection unit 5 executes the detection. The above timing information may be, for example, a number (eg, frame number) assigned in the order in which detection is executed. Further, the above timing information may be, for example, a time based on the timing at which the detection for the human body HM is executed (hereinafter, referred to as a detection time). The detection time may be, for example, a time measured by a time measuring device (eg, an internal clock) built in the detection device 2, or a time acquired by information received from the outside (eg, a standard radio wave indicating standard time). ..

For example, the detection device 2 images the human body HM by the detection unit 5, and stores the captured image data (eg, RGB data, pixel values such as gray scale) in the storage unit 6. The detection device 2 can output the detection result of the detection unit 5 to the outside. For example, the detection unit 5 reads out the data of the captured image stored in the storage unit 6 and outputs the data to the outside. The detection device 2 may output the detection result to the outside without going through the storage unit 6 (for example, in real time). In this case, the detection device 2 does not have to include the storage unit 6.

The processing device 3 is, for example, an information processing device that processes information (eg, detection result) output by the detection device 2. The processing device 3 is communicably connected to the detection device 2, for example, by wire or wirelessly. The processing device 3 acquires (eg, receives) the data of the captured image from the detection device 2 as the detection result of the detection device 2, for example. The processing device 3 may acquire the detection result or the processing result of the detection device 2 without communication with the detection device 2. For example, the processing device 3 may acquire the detection result from the detection device 2 via a storage medium such as a non-volatile memory. For example, the storage unit 6 of the detection device 2 may be a storage medium such as a memory card that can be removed from the detection device 2. The processing device 3 may acquire the detection result of the detection device 2 by being connected to the storage unit 6 removed from the processing device 3, for example.

The processing device 3 includes a processing unit 7, a displacement calculation unit 8, a landing determination unit 9, and a storage unit 10. The processing unit 7 processes the information (eg, detection result, processing result) output from the detection device 2. The processing unit 7 is, for example, an image processing unit that processes the data of the captured image output from the detection device 2. The processing unit 7 detects the position information of the human body HM based on the detection result of the detection device 2, for example.

The position information (information about the position) according to the embodiment is the coordinates in the predetermined coordinate system (eg, the coordinates of the legs) and the amount corresponding to the time change of the coordinates (the time change amount of the position) (eg, the time change amount of the position). It includes at least one of (the speed of the legs) and an amount corresponding to this amount of time variation (eg, acceleration of the legs). The above coordinate system is, for example, a coordinate system set by the detection unit 5 or the processing unit 7. For example, the above coordinate system is a coordinate system in a captured image. For example, the coordinates of the human body HM are represented by the positions of the pixels in the horizontal scanning direction in the captured image and the positions of the pixels in the vertical scanning direction of the captured image. Further, the position information of the human body HM includes, for example, at least one position information of a part of the human body HM (eg, head, torso, legs, arms).

FIG. 2 is a diagram showing processing of the detection unit and processing of the processing unit according to the embodiment. In FIG. 2, reference numeral Im1 is an image captured by the detection unit 5. For example, the processing unit 7 performs edge detection processing on the captured image Im1 to specify the human body HM (eg, shape, contour) in the captured image Im1. The processing unit 7 identifies a part of the human body HM in the captured image Im1 by, for example, pattern recognition.

The processing unit 7 detects the position of the human body HM in the captured image Im1. For example, the processing unit 7 detects the position information of a portion of the human body HM supported by the support surface G (eg, a portion in contact with the support surface G). The processing unit 7 detects, for example, the position information of the leg portion Q1 of the human body HM detected by the detection device 2 in the captured image Im1. The leg Q1 is, for example, a portion from the waist Q2 to the toes Q3.

The position information of the leg portion Q1 may be the position of the left foot or the position of the right foot, or may be information in which the position of the left foot and the position of the right foot are combined. Further, the position information of the leg Q1 may be a position representing the leg Q1 (eg, the center of the contour of the leg Q1), or a part included in the leg Q1 (eg, toe Q3, heel Q4, knee Q5). ) May be at least one position. The leg Q1 includes a heel, an instep, and the like, and includes, for example, a region (eg, an instep, etc.) that moves integrally with the heel Q4. Further, the position information of the human body HM may be information including only the position of one part of the human body HM in the detection result (eg, captured image Im1), or may be information including a plurality of parts of the human body HM (eg, knee Q5, heel Q4). , Toe Q3) may be a set of information.

A mark (eg, feature portion, code) that can be distinguished from the surroundings is attached to the human body HM as position information in advance, and the processing unit 7 may specify this mark. The above mark may be a member having different optical characteristics (eg, reflectance, absorptivity) from the surroundings (eg, support surface G). Further, the above mark may include a figure having a shape that can be distinguished from the surroundings. The position information of the human body HM may include, for example, information on the position of a member (eg, attachment, shoes, wear) attached to the human body HM.

Returning to the description of FIG. 1, the processing unit 7 detects the position of the human body HM each time the detection unit 5 detects the human body HM (eg, imaging process). For example, the processing unit 7 detects the absolute or relative position of the human body HM in the captured image for each captured image captured by the detection unit 5. The processing unit 7 may detect the position of the human body HM each time the detection unit 5 detects the human body HM a plurality of times. For example, the processing unit 7 may detect the position of the human body HM in one captured image selected from the plurality of captured images captured by the detection unit 5. Further, the processing unit 7 may detect the position of the human body HM for each of the two or more captured images, perform arithmetic processing such as interpolation or averaging on the detected position, and calculate the position information of the human body HM. good. Then, the processing unit 7 outputs, for example, the detected position information of the human body HM to the outside. The processing unit 7 stores, for example, the detected position information of the human body HM in the storage unit 10.

The displacement calculation unit 8 calculates the time change (change amount with respect to time) of the detection result of the detection unit 5. For example, the displacement calculation unit 8 acquires the position information of the human body HM output by the processing unit 7 and calculates the time change of the position of the human body HM included in the position information. The displacement calculation unit 8 reads, for example, the position information stored in the storage unit 10 and calculates the time change of the position. The displacement calculation unit 8 determines, for example, a position detected by the processing unit 7 from the first captured image captured by the detection unit 5 and a position detected by the processing unit 7 from the second captured image captured by the detection unit 5. By comparison, the time change of the position is calculated. The first captured image and the second captured image are captured images in which the timing of imaging by the detection unit 5 is different from each other.

The displacement calculation unit 8 outputs displacement information indicating a time change of the detection result of the detection unit 5. For example, the displacement calculation unit 8 uses the time change (change amount) of the position information (eg, the spatial position of the part of the human body HM) detected by the processing unit 7 as the displacement information, or the displacement vector (displacement direction,) based on the time change. (Displacement distance, etc.) is output. The displacement calculation unit 8 stores, for example, the displacement information in the storage unit 10. The landing determination unit 9 estimates and determines the landing of the human body based on the calculation result of the displacement calculation unit 8.

FIG. 3 is a diagram showing processing of the displacement calculation unit and processing of the landing determination unit according to the embodiment. In FIG. 3A, the horizontal axis is time (elapsed time). The symbol TR (solid line) is the position of the heel of the right foot of the human body HM. The symbol TL is the position of the heel of the left foot of the human body HM. The data of FIG. 3A can be obtained, for example, by connecting the data points obtained by plotting the position of the heel Q4 detected by the processing unit 7 with respect to the time with a smooth line.

At time t1 in FIG. 3A, the left foot (see position TL) is in a landed state. In this state, the position TL of the left foot is almost constant. At time t1, the right foot (see position TR) is in a state of starting to move forward. For example, the right foot moves forward with the left foot as a support, and its position TR changes with time. When the right foot lands, the time change of the position TR becomes smaller than that when the right foot moves. When the right foot lands, the left foot moves forward with the right foot as a support, and the time change of the position TL increases as compared with the time when the left foot lands.

The displacement calculation unit 8 (see FIG. 1) calculates, for example, the time change of the position of the leg portion Q1 (eg, heel Q4) of the human body HM. For example, the displacement calculation unit 8 sets the difference between the position of the heel Q4 in the previous captured image captured by the detection unit 5 and the position of the heel Q4 in the current captured image captured by the detection unit 5 at the position of the heel Q4. Calculated as a time change. The time change of the position of the heel Q4 corresponds to, for example, the velocity of the heel Q4.

FIG. 3B is a diagram showing the time change (velocity) of the position TR of the heel Q4 of the right foot. The heel Q4 of the right foot separates from the support surface G (see FIG. 1) at time t1, and the speed increases with the passage of time. Further, the heel Q4 of the right foot lands at time t2 after the speed reaches the maximum. FIG. 3C is a diagram showing the time change (velocity) of the position of the heel Q4 of the left foot. The heel Q4 of the left foot is in a state of landing from time t1 to time t3. The heel Q4 of the left foot separates from the support surface G (see FIG. 1) at time t3, and the speed increases with the passage of time. Further, the heel Q4 of the left foot lands at time t4 after the speed reaches the maximum.

The landing determination unit 9 determines the landing of the human body based on the landing state of the leg Q1 with respect to the support surface G (landing) estimated from the time change. For example, when the time change of the position of the leg Q1 is equal to or less than the threshold value, it is presumed that the leg Q1 is in the landing state in contact with the support surface G. The landing state of the leg Q1 may include the landing posture of at least a part of the leg Q1 (eg, toe Q3, heel Q4).

The landing determination unit 9 (see FIG. 1) compares, for example, the time change of the position of the leg (eg, heel) calculated by the displacement calculation unit 8 with the threshold value, and the comparison result (eg, the result below the threshold value). , Results above the threshold) to determine landing. The landing determination unit 9 estimates that the heel of the right foot has landed, for example, when the time change (velocity in FIG. 3B) of the heel position of the right foot calculated by the displacement calculation unit 8 is equal to or less than the threshold value TH. To judge. The landing determination unit 9 estimates that the heel of the left foot has landed, for example, when the time change (velocity in FIG. 3C) of the heel position of the left foot calculated by the displacement calculation unit 8 is equal to or less than the threshold value TH. To judge. The landing determination unit 9 may estimate the landing state by interpolation or the like based on, for example, the time history of position information (eg, position (coordinates), velocity, acceleration).

Returning to the description of FIG. 1, the landing determination unit 9 stores, for example, the determination result (eg, presence / absence of landing) in the storage unit 10. The display device 4 is, for example, a liquid crystal display or the like. The display device 4 displays the processing result of the processing device 3. The processing device 3 supplies, for example, image data showing the determination result of the landing determination unit 9 to the display device 4. The processing device 3 causes the display device 4 to display, for example, an image showing the determination result of the landing determination unit 9.

Next, the detection method according to the embodiment will be described based on the operation of the detection system 1 described above. FIG. 4 is a flowchart showing a detection method according to the embodiment. For each part of the detection system 1, refer to FIG. 1 as appropriate. In step S1, the detection device 2 optically detects the moving human body HM. For example, the detection unit 5 images a moving human body HM. For example, the detection unit 5 repeatedly detects (eg, images) the human body HM in a preset period (hereinafter referred to as a detection set period), and then ends the detection.

The processing unit 7 detects the position of at least a part of the human body HM based on the detection result of the detection unit 5 (executes the position detection process). The processing unit 7 may execute the position detection process in parallel with at least a part of the detection process (eg, the imaging process) by the detection unit 5. Further, the processing unit 7 may execute the position detection process after the detection by the detection unit 5 is completed (for example, after the detection setting period is completed).

In step S2, the displacement calculation unit 8 calculates the time change of the detection result of the detection unit 5. The displacement calculation unit 8 calculates, for example, the time change of the position of the human body HM detected by the processing unit 7 (eg, the position of the heel Q4). For example, the displacement calculation unit 8 acquires all or a part of the processing result of the position detection processing after the position detection processing by the processing unit 7 is completed for the detection result of the detection unit 5 in the above detection setting period, and the human body HM. Calculate the time change of the position of.

In step S3 and step S4, the landing determination unit 9 determines the landing of the human body HM. For example, in step S3, the landing determination unit 9 determines whether or not the time change of the position of the human body HM detected by the processing unit 7 is equal to or less than the threshold value TH. When the landing determination unit 9 determines that the time change is equal to or less than the threshold value (step S3; Yes), it determines that the human body HM has landed (step S4). Further, when the landing determination unit 9 determines that the time change exceeds the threshold value (step S3; No), it determines that the human body HM has not landed (step S5).

In the above embodiments, at least a portion of the detection system 1 (eg, processing device 3) may include a computer system. The processing device 3 may read the detection program stored in the storage unit 10 and execute various processes according to the detection program. This detection program causes a computer to execute, for example, calculating the time change of the detection result of detecting the moving human body HM, and determining the landing of the human body HM based on the time change. The detection program may be recorded and provided on a computer-readable storage medium.

In the embodiment, the detection system 1 determines the landing of the human body HM based on the time change of the detection result of optically detecting the human body HM moving in a predetermined moving direction (eg, traveling direction, depth direction). In this case, the detection system 1 can determine, for example, the landing of the human body HM with high accuracy. The displacement calculation unit 8 calculates, for example, the time change of the position of a part (part) of the human body HM detected by the detection unit 5. In this case, the detection system 1 can calculate, for example, the time change of the position of the part of the human body HM with high accuracy. The displacement calculation unit 8 may calculate the time change of the position of a plurality of parts of the human body HM detected by the detection unit 5. The landing determination unit 9 may determine the landing based on the time change of the positions of the plurality of parts of the human body HM.

Further, the displacement calculation unit 8 calculates, for example, the time change of the position of the leg portion detected by the detection unit 5. For example, when the leg lands, the time change of the position of the leg becomes almost constant (eg, 0), and the detection system 1 can determine the landing with high accuracy. The displacement calculation unit 8 may calculate the time change of the position of the portion of the human body HM detected by the detection unit 5 other than the leg portion Q1. For example, the displacement calculation unit 8 may calculate the time change of the position of the portion of the human body HM detected by the detection unit 5 that moves in conjunction with the leg portion Q1 (eg, the arm portion Q6 in FIG. 1). The arm Q6 includes, for example, at least a part of the shoulder Q7 to the hand Q8 (eg, fingertip). The swing of the right arm Q6 is linked with the movement of the left leg Q1 during walking or running, and is stationary at least partly while the leg Q1 is landing. The landing determination unit 9 may determine that the left foot has landed, for example, when the time change of the position of the right arm Q6 becomes equal to or less than the threshold value.

Further, the landing determination unit 9 makes a landing of the human body HM based on the landing state of the arm Q6 with respect to the support surface G (landing) estimated from the time change of the position at, for example, the crawling start of the athletics or the start of the swimming competition. May be determined.

Further, the landing determination unit 9 determines the landing based on, for example, the time change of the position of the heel Q4 calculated by the displacement calculation unit 8. For example, in walking or the like, the heel Q4 of the leg Q1 lands before other parts (eg, toe Q3), and the detection system 1 can determine the landing with high accuracy. The landing determination unit 9 may determine the landing based on, for example, the time change of the position of the portion other than the heel Q4 calculated by the displacement calculation unit 8. The landing determination unit 9 may determine only the landing of one of the left foot and the right foot.

Further, in addition to the above-mentioned landing determination processing, the landing determination unit 9 detects the human body HM of a user (eg, himself / herself, a person who becomes a sample (teacher)) who has detected (measured) in the past such as the previous time (eg, past). Compared with the detection results currently detected using the captured images (eg, the current captured images and their landing determination results), the similarity between the captured images in the target portion (eg, legs and arms) The landing of the human body HM may be estimated from and determined. In this case, the storage unit 10 is configured to store the detection result (eg, past captured image) of the user's human body HM detected (measured) in the past for each human body HM at the detection timing or the like. ..

At least a part of the processing device 3 (eg, processing unit 7, displacement calculation unit 8, landing determination unit 9) may execute the processing after the detection of the human body HM by the detection device 2 is completed. Further, at least a part of the processing device 3 may execute the process in parallel with at least a part of the detection process of the human body HM by the detection device 2. For example, the detection unit 5 outputs the data of the captured image each time the human body HM is imaged, and the processing unit 7 outputs the data of the captured image each time the detection device 2 outputs the data of the captured image, in parallel with the next imaging process. The position of the human body HM may be detected.

[Second Embodiment]
The second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 5 is a diagram showing a detection device according to an embodiment. In the present embodiment, the detection device 2 optically detects the distance from each part of the human body HM. The detection device 2 may be, for example, a motion capture device or may detect the shape of the human body HM. The detection device 2 includes, for example, a detection unit 5, a processing unit 7, and a storage unit 6.

The detection unit 5 is, for example, a range finder such as a depth sensor. The detection unit 5 detects, for example, the distance from a predetermined viewpoint (eg, the position of the detection unit 5, the viewpoint) to each point on the surface of the human body HM. The detection unit 5 detects the distance by, for example, the TOF (time of flight) method. The detection unit 5 outputs, for example, position information indicating the distribution of distances as the detection result (eg, position information). The above position information includes, for example, depth information (eg, depth image data, depth information). The detection unit 5 outputs, for example, three-dimensional position information of a plurality of parts of the human body HM.

The detection unit 5 may be a device that detects the distance by a method other than the TOF method. The detection unit 5 may include, for example, a laser scanner (eg, a laser rangefinder) and may be a device that detects a distance by laser scanning. The detection unit 5 may be, for example, a device that projects a predetermined pattern on a region through which the human body HM passes and measures the distance based on the detection result of this pattern. The detection unit 5 may be, for example, a device including a phase difference sensor and detecting a distance by a phase difference method. The detection unit 5 may be, for example, a device that detects a distance by a DFD (depth from defocus) method. Further, the detection unit 5 may be a device that detects the distance by triangulation using captured images obtained by capturing the human body HM from a plurality of viewpoints by, for example, a stereo camera. The detection unit 5 may be a device that detects the human body HM by combining two or more of the plurality of detection methods described above. The detection unit 5 may include a sensor that detects a distance and an image sensor described in the first embodiment.

FIG. 6 is a diagram showing processing of the processing unit according to the embodiment. The detection unit 5 (see FIG. 5) outputs, for example, the data of the depth image Im2. In the depth image Im2, the pixel value of each pixel represents, for example, the distance between the detection unit 5 and the point in the real space corresponding to the position of the pixel on the depth image Im2. For example, the depth image Im2 of FIG. 6 is represented in grayscale, and the dark portion is farther from the detection unit 5 than the bright portion.

The processing unit 7 (see FIG. 5) detects (calculates) the position of the human body HM based on the detection result of the detection unit 5. The processing unit 7 generates (calculates) shape information indicating the shape of the human body HM by, for example, processing the data of the depth image Im2 (eg, perspective conversion processing, projection conversion processing). The shape information includes, for example, point cloud data. The point cloud data includes, for example, a plurality of point data on the surface of the human body HM. For example, each of the plurality of point data is represented by three-dimensional coordinates.

The processing unit 7 may calculate surface information as shape information. The surface information is, for example, polygon data, vector data, draw data, or the like. The surface information includes, for example, the coordinates of a plurality of points and the connection information between the plurality of points. The connection information includes, for example, information that associates points at both ends of a line corresponding to a ridgeline (eg, an edge) of an object (eg, human body HM) with each other. The connection information includes, for example, information that associates a plurality of lines corresponding to the contour of a surface on an object with each other.

For example, the processing unit 7 estimates a surface between a point selected from a plurality of points included in the point cloud data and a point in the vicinity thereof, and converts the point cloud data into polygon data having plane information between the points. You may. The processing unit 7 converts the point cloud data into polygon data by, for example, an algorithm using the least squares method. This algorithm may be, for example, an application of an algorithm published in a point cloud processing library.

The processing unit 7 generates position information PD of a feature portion (eg, feature point) of the human body HM based on the detection result of the detection unit 5. The characteristic part of the human body HM is, for example, a part that can be distinguished from other parts of the human body HM. Characteristic sites of the human body HM include, for example, at least one of the ends, joints, or between the ends and the joints or between the two joints of the human body HM. The terminal portion of the human body HM is, for example, toe Q3, head Q9, hand Q10, and the like. The joints of the human body HM are, for example, waist Q2, knee Q5, shoulder Q7, elbow Q11, wrist Q12, and the like. The intermediate portion of the human body HM is, for example, the neck Q13, the shoulder center Q14, the spine Q15, the waist center Q16, and the like.

The processing unit 7 generates the position information PD of the above-mentioned feature portion by, for example, recognition processing (eg, pattern recognition, shape recognition, skeleton recognition) using point cloud data. The position information of the feature portion includes, for example, the coordinates of a point representing the feature portion (eg, three-dimensional coordinates). For example, the position information of the feature portion generated by the processing unit 7 includes at least a part of the coordinates of the leg portion Q1. The leg Q1 is, for example, a portion from the waist Q2 to the toes Q3. The processing unit 7 generates position information PD including at least one of the coordinates of the waist Q2, the coordinates of the knee Q5, the coordinates of the heel Q4, and the coordinates of the toe Q3, for example. The processing unit 7 calculates the coordinates of the points representing the feature portions by, for example, the above recognition process.

For example, the processing unit 7 generates data in which the name of the feature portion, the position of the feature portion, and the detection timing of the detection unit 5 are associated with each of the feature portions. The processing unit 7 stores, for example, the coordinates of the feature portion in a variable associated with the name of the feature portion and the detection timing of the detection unit 5. For example, the variable name of the left heel Q4 is represented by ALn. For example, AL represents the left heel, and n is a number assigned based on the detection timing (eg, the order of detection) of the detection unit 5 which is the source of the position information (eg, 1, 2, 3 ...). The processing unit 7 stores, for example, the coordinates of the left heel Q4 obtained from the nth detection result in ALn. The processing unit 7 generates, for example, a position information PD (eg, table data) in which the position information of a preset feature portion is set. The position information PD includes, for example, the position information of a plurality of feature portions obtained from the detection results of the same detection timing.

Returning to the description of FIG. 5, the detection device 2 outputs the position information generated by the processing unit 7 to the outside. For example, the processing unit 7 stores the generated position information in the storage unit 6. The detection device 2 outputs, for example, the position information stored in the storage unit 6 to the processing device 3. The processing device 3 stores the position information output from the detection device 2 in the storage unit 10.

The displacement calculation unit 8 calculates the time change (time change amount) of the detection result of the detection unit 5. For example, the displacement calculation unit 8 calculates the time change of the detection result of the detection unit 5 by using the position information of the feature portion output from the detection device 2. The displacement calculation unit 8 reads, for example, the position information stored in the storage unit 10 and calculates the time change of the detection result of the detection unit 5. For example, the displacement calculation unit 8 has a left heel position (ALn-1) corresponding to the (n-1) th detection result and a left heel position (ALn) corresponding to the nth detection result. The difference is calculated as the amount of time change in the position of the left heel. The displacement calculation unit 8 stores, for example, the time change of the calculated position of the feature portion in the storage unit 10.

The landing determination unit 9 determines the landing of the human body HM based on the calculation result of the displacement calculation unit 8. The landing determination unit 9 reads, for example, the time change of the feature portion stored in the storage unit 10 and determines the landing of the human body HM. In the following description, the timing recognized by the landing determination unit 9 as the timing when the human body HM has landed is referred to as the landing timing. The landing timing may be represented by, for example, the number (eg, frame number) of the detection result of the detection unit 5 which is the source thereof, or may be represented by the time.

The landing determination unit 9 outputs the determination result. The determination result of the landing determination unit 9 includes, for example, information on whether or not the landing state is in the landing state at an arbitrary timing, and one or both of the above landing timings. The landing determination unit 9 outputs the determination result to the motion analysis unit 11, for example. The landing determination unit 9 may store the determination result in the storage unit 10. The determination result may include header information, an identification number, and the like. The landing determination unit 9 may store the determination result in the storage unit 10 in association with the header information, the identification number, and the like.

The landing determination unit 9 may detect a state of being out of landing (hereinafter referred to as takeoff) based on the calculation result of the displacement calculation unit 8. The takeoff (non-landing) is, for example, a state of floating away from the support surface G (eg, ground, floor) (non-landing state). The landing determination unit 9 may determine the takeoff (the takeoff determination process may be executed). In the following description, the timing recognized by the landing determination unit 9 as the timing when the human body HM takes off (the timing when the human body HM departs from the landing) is referred to as the takeoff timing. The takeoff timing may be represented by, for example, the number (eg, frame number) of the detection result of the detection unit 5 which is the source thereof, or may be represented by the time. The determination result of the landing determination unit 9 may include the takeoff timing.

In the present embodiment, the processing device 3 includes a motion analysis unit 11. The motion analysis unit 11 analyzes the motion of the human body HM using the detection result of the detection unit 5 by using the detection signal indicating that the landing determination unit 9 has detected the landing of the human body HM as a trigger. For example, the motion analysis unit 11 analyzes the motion of the human body HM using the detection result of the detection unit 5 according to the timing when the human body HM lands, based on the detection result of the landing determination unit 9. The motion analysis unit 11 analyzes the motion of the human body HM based on the detection result of the detection unit 5 selected according to the landing timing, for example. For example, the landing determination unit 9 determines that the human body HM has landed by using the detection result at the first detection timing of the detection unit 5. In this case, the motion analysis unit 11 analyzes the motion of the human body HM using the detection result of the detection unit 5 selected based on the above-mentioned first detection timing. The motion analysis unit 11 does not analyze the motion of the human body HM in the absence of the detection signal (non-landing state without landing).

The motion analysis unit 11 analyzes (eg, detects) the motion associated with the landing of the human body HM on a predetermined surface. For example, the motion analysis unit 11 analyzes the motion of the leg Q1 when the leg Q1 lands. The motion analysis unit 11 analyzes the motion of the human body HM by calculation using, for example, the position of a portion of the human body HM used by the landing determination unit 9 for the landing determination process and the position of another portion. For example, the landing determination unit 9 executes a landing determination process using the position of the first portion (eg, heel), and the motion analysis unit performs a second portion (eg, knee) different from the first portion. The motion is analyzed using the position of. The motion analysis unit may analyze the motion by using the position of the first portion and the position of the second portion (eg, knee).

The motion analysis unit 11 analyzes (eg, detects) the pronation (inward movement, supination movement) of the leg Q1, for example. Pronation is a series of movements (movements) in which the legs receive the impact of landing when the legs land, for example, during walking or running. Pronation is, for example, a movement in which the leg Q1 (eg, the part from the waist to the ankle) tilts to the same side (inside) or opposite side (outside) to the center of the human body HM at the time of landing.

Pronations are classified into three states, for example, overpronation, neutral, and underpronation, depending on the level at which the leg Q1 is tilted at the time of landing. Overpronation is a state in which the legs are tilted inward beyond a predetermined range so that the arch structure of the arch is crushed. Underpronation is a state in which the legs are tilted outward beyond a predetermined range. Neutral is a state in which the inclination of the leg is within a predetermined range inside or outside. In the following description, the level at which the leg Q1 tilts at the time of landing is appropriately referred to as the pronation level.

Depending on the level of pronation, the leg Q1 may be burdened. For example, overpronation may cause sprains and stress fractures of the leg Q1. To correct the level of pronation, for example, shoes that adjust the inclination of the legs during landing are used. For example, overpronation is neutralized by shoes with insoles that support the arch. For example, in a field such as running, an expert (eg, a trainer) visually observes the uncorrected pronation to determine the level of pronation. For example, the pronation is corrected by the selection of shoes based on the level of pronation judged by the expert.

The motion analysis unit 11 uses, for example, the position of the knee of the leg of the human body HM obtained from the detection result of the detection unit 5 based on the timing when the human body HM lands, and pronations the leg Q1 as the motion of the human body HM. Is detected. The motion analysis unit 11 compares, for example, the position of the knee Q5 obtained from the detection result of the detection unit 5 with the reference, and calculates an index value indicating the level of pronation.

FIG. 7 is a diagram showing processing of the processing apparatus according to the embodiment. In FIG. 7, the horizontal axis is time. The detection unit 5 starts detection according to a user command, for example, and repeats detection at predetermined time intervals. In FIG. 7, the symbol AL is the position of the heel Q4 (see FIG. 6) of the left foot obtained from the detection result of the detection unit 5. The position AL of the heel Q4 of the left foot is obtained by, for example, processing the data of the depth image Im2 (see FIG. 6) output by the detection unit 5 by the processing unit 7.

The displacement calculation unit 8 calculates the time change of the position AL of the heel Q4. The landing determination unit 9 monitors, for example, the time change of the position AL of the heel Q4 calculated by the displacement calculation unit 8. As described with reference to FIG. 3, the landing determination unit 9 determines the landing based on the time change of the position AL of the heel Q4 calculated by the displacement calculation unit 8. In FIG. 7, reference numeral t11 is a time (landing timing) recognized by the landing determination unit 9 as the time when the human body HM has landed. For example, the landing determination unit 9 detects the position AL of the heel Q4 obtained from the detection of the detection unit 5 at time t11 (eg, the current detection) and the detection unit 5 before the time t11 (eg, the previous detection). ) Is equal to or less than the threshold value TH (see FIG. 3), it is determined that the landing is made at time t11.

Further, the reference numeral t12 is a time (takeoff timing) recognized by the landing determination unit 9 as the time when the human body HM has taken off. For example, the landing determination unit 9 detects the position AL of the heel Q4 obtained from the detection of the detection unit 5 at time t12 (eg, the current detection) and the detection unit 5 before the time t12 (eg, the previous detection). ) Exceeds the threshold TH (see FIG. 3) from the position AL of Q4, and it is determined that the landing has occurred at time t12. Further, the reference numeral T1 is a period recognized by the landing determination unit 9 as a period during which the human body HM is landing (hereinafter, referred to as a landing period). The landing period T1 is a period from the landing timing (eg, time t11) to the takeoff timing (eg, time t12).

Further, in FIG. 7, the symbol ΔKL is the amount of change in the horizontal position of the left foot knee Q5 (see FIG. 6) (eg, parameters such as the difference value of the position of the knee Q5 and the time differential value). Here, the amount of change ΔKL when the knee Q5 moves inward with respect to the center of the human body HM is positive (+), and the amount of change ΔKL when the knee Q5 moves outward with respect to the center of the human body HM is negative (-). ). For example, the amount of change ΔKL indicates that it is an inward movement (eg, overpronation) when it is positive, and indicates that it is a supination movement (eg, underpronation) when it is negative.

The amount of change ΔKL is obtained, for example, by the displacement calculation unit 8 processing the position of the knee Q5 obtained by processing the data of the depth image Im2 (see FIG. 6) output by the detection unit 5. The motion analysis unit 11 detects, for example, motion (eg, pronation) during the period associated with the landing period. For example, the motion analysis unit 11 detects pronation in at least a part of the landing period T1. For example, the motion analysis unit 11 detects pronation in a predetermined period T2 for one or both of the landing timing (eg, time t11) and the takeoff timing (eg, time t12). The predetermined period T2 is, for example, a period having a predetermined length starting from the landing timing (eg, time t11).

The motion analysis unit 11 acquires (eg, extracts, monitors) the position information of the knee Q5 obtained from the detection result of the detection unit 5 during a predetermined period T2, for example. The detection result of the detection unit 5 in the predetermined period T2 includes the detection result in which the detection unit 5 detects the human body HM after the timing when the human body HM lands (eg, time t11). The position information of the knee Q5 is, for example, the position (eg, coordinates) of the knee Q5 obtained by processing the detection result of the detection unit 5 by the processing unit 7, and the displacement calculation unit 8 processing the position information of the knee Q5. It includes at least one of a time change of the position of the obtained knee Q5 (eg, change amount ΔKL, velocity of the knee Q5) and a time change of the change amount ΔKL (eg, acceleration).

The motion analysis unit 11 monitors, for example, the amount of change ΔKL of the position of the knee Q5 calculated by the displacement calculation unit 8 as the position information of the knee Q5. The knee position information (eg, change amount ΔKL) during the predetermined period T2 can be used, for example, as an index value indicating the level of pronation. For example, the state in which the amount of change ΔKL is positive corresponds to the state in which the leg portion Q1 is tilted so that the knee faces the inside of Q5. For example, a state in which the amount of change ΔKL is larger than a predetermined value corresponds to overpronation. For example, a state in which the amount of change ΔKL is negative corresponds to a state in which the leg portion Q1 is tilted so that the knee Q5 faces outward. For example, a state in which the amount of change ΔKL is smaller than a predetermined value corresponds to underpronation.

For example, the motion analysis unit 11 calculates a statistic (eg, maximum value, integrated value, average value) of the amount of change ΔKL in a predetermined period T2 as the above index value. The motion analysis unit 11 may calculate an index value for one landing, or may calculate an index value for each of a plurality of landings. Further, the motion analysis unit 11 calculates an individual index value for each of the plurality of landings, and further uses the calculated statistic of the individual index value (eg, maximum value, average value, integrated value) as the index value. It may be calculated. The motion analysis unit 11 may calculate the above index value as the position information of the knee Q5, for example.

The motion analysis unit 11 compares, for example, the position information of the knee Q5 (eg, the amount of change ΔKL) in the predetermined period T2 with the first threshold value TH1. The first threshold value TH is set to a positive value, where the value when the knee Q5 moves inward with respect to the center of the human body HM is positive (+). For example, as shown in FIG. 7A, the motion analysis unit 11 determines that the pronation corresponds to the overpronation when the amount of change ΔKL in the predetermined period T2 exceeds the first threshold value TH1.

Further, the motion analysis unit 11 compares, for example, the position information of the knee Q5 (eg, the amount of change ΔKL) in the predetermined period T2 with the second threshold value TH2. The second threshold value TH2 is set to a negative value, where the value when the knee Q5 moves outward with respect to the center of the human body HM is negative (−). For example, as shown in FIG. 7B, the motion analysis unit 11 determines that the pronation corresponds to the underpronation when the amount of change ΔKL in the predetermined period T2 is smaller than the second threshold value TH2. Further, as shown in FIG. 7C, for example, the motion analysis unit 11 pronation is neutral when the amount of change ΔKL in the predetermined period T2 is within the range of the second threshold value TH2 or more and the first threshold value or less. Judged as (normal).

The motion analysis unit 11 may determine the pronation by using a determination criterion different from the magnitude relationship between the change amount ΔKL and the threshold value. For example, the motion analysis unit 11 may determine the pronation based on the frequency with which the amount of change ΔKL exceeds the threshold value. For example, the motion analysis unit 11 counts the frequency (eg, number of times) when the amount of change ΔKL exceeds the first threshold value TH1, and determines that the pronation corresponds to the overpronation when the frequency exceeds a predetermined value. May be good. The motion analysis unit 11 may determine the pronation by using both the magnitude relationship between the amount of change ΔKL and the threshold value and the above frequency. Further, the motion analysis unit 11 may detect the pronation using a sample (eg, image, coordinates) of the position of the knee Q5 as the above reference.

The above threshold values (first threshold value TH1, second threshold value TH2) may be fixed values or variable values. For example, the above threshold value may be a variable value specified by user input or the like. The above threshold value may be set based on, for example, the dimensions (eg, height, foot length) of the human body HM to be detected. For example, the detection system 1 may accept input of information (eg, height) of the human body HM to be detected and set the above threshold value based on the input information. For example, the storage unit 10 may store reference information (eg, table data) in which the height of the human body is associated with the above threshold value. The motion analysis unit 11 may set the above threshold value by comparing the input information of the human body HM to be detected with the above reference information. One or both of the above threshold value and the above reference information may be supplied from an external device (eg, an external database) of the processing device 3.

The motion analysis unit 11 may detect the pronation by using a parameter different from the time change of the knee position (eg, the amount of change ΔKL). FIG. 8 is a diagram showing the processing of the motion analysis unit according to the modified example. In the XYZ Cartesian coordinate system of FIG. 8A, the Z direction is the moving direction of the human body HM. For the moving direction of the human body HM, for example, an angle with the detection direction of the detection unit 5 is preset. The detection direction of the detection unit 5 (later shown in FIG. 12) is a direction connecting the center of the detection range of the detection unit 5 (eg, the center of the field of view, the center of the angle of view) and the detection unit 5. The X direction is a direction that intersects (eg, is orthogonal) with the moving direction of the human body HM. The Y direction is, for example, the height direction of the human body HM (eg, the vertical direction).

In FIG. 8, the origin O is, for example, the position of the detection unit 5. Further, HLn, KLn, and ALn are position vectors indicating the positions of the characteristic parts of the human body HM in the XYZ Cartesian coordinate system, respectively. These position vectors can be obtained, for example, by appropriately converting the position information of the feature portion calculated by the processing unit 7 into coordinates. HLn is a position vector of the left waist Q2 (see FIG. 6) obtained from the nth detection result of the detection unit 5. KLn is a position vector of the left knee Q5 (see FIG. 6) obtained from the nth detection result of the detection unit 5. ALn is a position vector of the heel Q4 (see FIG. 6) of the left foot corresponding to the nth detection result of the detection unit 5.

The motion analysis unit 11 detects the pronation by comparing the position of the knee Q5 obtained from the detection result of the detection unit 5 with the reference BL. The reference BL is, for example, a line connecting the upper end (eg, waist Q2) and the heel Q4 of the leg Q1 of the human body HM detected by the detection unit 5. The motion analysis unit 11 detects, for example, the position of the knee Q5 with respect to the reference BL to detect the pronation. For example, the motion analysis unit 11 detects whether the knee Q5 is arranged with respect to the reference BL to determine whether the pronation corresponds to overpronation, neutral, or underpronation.

Further, the motion analysis unit 11 compares the position of the knee Q5 obtained from the detection result of the detection unit 5 with the reference BL, and detects an index value indicating the level of pronation. In FIG. 8A, the vector V1n is a vector (reference) connecting the heel Q4 and the waist Q2. Further, the vector V2n is a vector connecting the heel Q4 and the knee Q5. As shown in FIG. 8B, the vector V1n is the difference between the position vector HLn of the waist Q2 and the position vector ALn of the heel Q4. Further, the vector V2n is the difference between the position vector KLn of the knee Q5 and the position vector ALn of the heel Q4.

The motion analysis unit 11 calculates, for example, the value of a function whose variable is the angle θln between the vector V1n and the vector V2n or the angle θln as the index value of the pronation. For example, the motion analysis unit 11 calculates cosθln as an index value of pronation. As shown in FIG. 8C, cosθln is calculated by dividing the inner product of the vector V1n and the vector V2n by the magnitude of the vector V1n and the magnitude of the vector V2n. When the index value (cosθln) is close to 0, it corresponds to the absolute value of θln being close to 0, and for example, it means that the pronation is neutral. Further, when the index value (cosθln) is large, it corresponds to the fact that the absolute value of θln is large, and for example, it means that the pronation is an overpronation.

The motion analysis unit 11 may use the amount of time change of cosθln as the value of the function with the angle θln as a variable. Further, the motion analysis unit 11 may use an angle other than the angle θln as an index value. For example, the motion analysis unit 11 may use the angle α between the line connecting the knee Q5 and the heel Q4 (eg, vector) and the line connecting the knee Q5 and the waist Q2 (eg, vector) as an index value. .. Further, the motion analysis unit 11 may use the angle β between the line connecting the waist Q2 and the knee Q5 (eg, vector) and the line connecting the waist Q2 and the heel Q4 (eg, vector) as an index value. good. Further, the motion analysis unit 11 may use two or more of the angle θln, the angle α, and the angle β as index values.

The motion analysis unit 11 may evaluate at least one pronation state (overpronation, neutral, or underpronation) in two or more stages. For example, the motion analysis unit 11 may express the degree of overpronation at a plurality of levels (eg, 1, 2, 3, ...).

The motion analysis unit 11 compares the detection result in which the detection unit 5 detects the human body at the same time as the landing timing or before the landing timing with the detection result in which the detection unit 5 detects the human body after the landing timing. Then, the movement of the human body HM may be detected. For example, when the right foot is off the ground, the right foot is not directly restrained by the support surface G. The relative position between the reference and the knee when the right foot is off can be used, for example, as the relative position between the reference and the knee when there is no pronation due to landing. For example, the motion analysis unit 11 may detect the pronation by comparing the relative positions of the reference and the knee between the state of taking off and the state of landing. The motion analysis unit 11 may detect pronations for each of the right foot and the left foot, or may detect pronations for only one foot.

Returning to the description of FIG. 5, the motion analysis unit 11 stores, for example, the analysis result in the storage unit 10. The motion analysis unit 11 stores, for example, an analysis result indicating whether the pronation corresponds to overpronation, neutral, or underpronation in the storage unit 10. Further, the motion analysis unit 11 stores, for example, the index value of the pronation in the storage unit 10.

The display device 4 displays the processing result of the processing device 3 (eg, the analysis result of the motion analysis unit 11). For example, the processing device 3 supplies image data indicating pronation information to the display device 4. For example, the processing device 3 supplies the display device 4 with image data indicating whether the pronation corresponds to overpronation, neutral, or underpronation. Further, the processing device 3 supplies the display device 4 with image data indicating the index value of the pronation. The display device 4 displays an image based on the image data supplied from the processing device 3. The processing device 3 may supply pronation information to a device other than the display device 4. For example, the processing device 3 may transmit pronation information via a communication line (eg, an internet line).

Next, the detection method according to the embodiment will be described based on the operation of the detection system 1 described above. FIG. 9 is a flowchart showing a detection method according to the embodiment. The processing of steps S1 to S5 is the same as in FIG. After the landing determination unit 9 determines that the landing has occurred in step S4, the motion analysis unit 11 analyzes the motion of the human body HM in step S11. The motion analysis unit 11 detects the pronation for at least a part of the landing period T1, for example. The motion analysis unit 11 determines, for example, whether the pronation corresponds to overpronation, neutral, or underpronation. The motion analysis unit 11 calculates, for example, an index value indicating the level of pronation. In step S12, the display device 4 displays the processing result of the processing device 3. For example, the processing device 3 supplies image data indicating pronation information to the display device 4. The display device 4 displays this image based on the image data supplied from the processing device 3.

In the above embodiments, at least a portion of the detection system 1 (eg, processing device 3) may include a computer system. The processing device 3 may read the detection program stored in the storage unit 6 and execute various processes according to the detection program. This detection program calculates, for example, the time change of the detection result of detecting the moving human body HM on a computer, determines the landing of the human body HM based on the time change, and determines the landing of the human body HM. Based on, the movement of the human body HM is analyzed and executed. The detection program may be recorded and provided on a computer-readable storage medium.

The detection system 1 may execute detection for items other than pronation. For example, the detection system 1 may detect the moving speed of the human body HM. For example, the detection system 1 (eg, displacement calculation unit 8) may calculate the time change of the position of the characteristic portion of the human body HM (eg, waist center Q16, head Q9 in FIG. 6) as the moving speed.

Further, the detection system 1 may detect a stride in a predetermined direction. For example, the detection system 1 (eg, displacement calculation unit 8) moves from the position of one foot at an arbitrary landing timing of one foot (eg, right foot) in the moving direction of the human body HM to the other foot (eg, left foot). The distance to the position of the other foot at the next landing timing of) may be calculated as the stride length. The detection system 1 may calculate the overlapping stride (overlapping step length). For example, the detection system 1 (eg, displacement calculation unit 8) can change the position of one foot from the position of one foot at an arbitrary landing timing of one foot (eg, right foot) to the position of one foot at the next landing timing of one foot. The distance to may be calculated as an overlapping stride.

Further, the detection system 1 may detect the step distance in a predetermined direction. For example, for example, the detection system 1 (eg, displacement calculation unit 8) can be used from the position of one foot at an arbitrary landing timing of one foot (eg, right foot) in a direction orthogonal to the moving direction of the human body HM. The distance to the position of the other foot at the next landing timing of one foot (eg, left foot) may be calculated as the step distance.

Further, the detection system 1 may detect the shake of the upper body. For example, the detection system 1 (eg, processing unit 7) may calculate the relative position of the head Q9 with respect to the waist center Q16 shown in FIG. Further, the detection system 1 (eg, displacement calculation unit 8) may calculate the time change of the relative position of the head Q9 with respect to the waist center Q16 shown in FIG.

Further, the detection system 1 may detect the posture of the human body HM, for example, in a stationary state of the human body HM. For example, as described in FIG. 8, the detection system 1 detects the relative position between the reference BL and the knee Q5, and determines whether the left and right legs tend to be X-legs, O-legs, or parallel legs. You may judge. Further, the detection system 1 may detect the orientation of the toe with respect to the heel (inside: Toe-in, outside: Toe-out, parallel: neutral).

The detection system 1 may perform detection on one or more of the above items. Further, the detection system 1 does not have to execute the detection for the above-mentioned items. The processing device 3 may be provided separately from at least a part of the detection system 1 (eg, the detection device 2). Further, the processing device 3 may include at least a part of the detection device 2 (eg, the processing unit 7).

[Third Embodiment]
A third embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 10 is a diagram showing a detection system according to an embodiment. The detection system 1 includes a direction detection unit 12.

The direction detection unit 12 detects the moving direction D1 of the human body HM based on the detection result of the detection unit 5. The direction detection unit 12 detects, for example, the movement direction D1 (traveling direction, motion direction) of the human body HM based on the determination result of the landing determination unit 9. For example, the direction detection unit 12 acquires the position of the left foot (eg, heel Q4) at the first landing timing. Further, the direction detection unit 12 acquires the position of the left foot (eg, heel Q4) at the second landing timing after the first landing timing. The direction detection unit 12 calculates a change (eg, vector) in the position of the left foot between the first landing timing and the second landing timing. This vector corresponds to the moving direction of the left foot.

The direction detection unit 12 calculates, for example, the moving direction of the right foot in the same manner as the left foot. The direction detection unit 12 calculates the moving direction D1 of the human body HM by, for example, calculating the average of the moving direction of the left foot and the moving direction of the right foot. The direction detection unit 12 may calculate the moving direction of only one of the left foot and the right foot as the moving direction D1 of the human body HM. The direction detection unit 12 stores, for example, the detected moving direction D1 in the storage unit 10. The direction detection unit 12 outputs, for example, the detected movement direction to the motion analysis unit 11.

In the present embodiment, the motion analysis unit 11 analyzes the motion of the human body HM using the detection result of the direction detection unit 12. The moving direction of the human body HM may not be predetermined, for example. For example, in the process described with reference to FIG. 8, the motion analysis unit 11 converts the coordinates of the characteristic portion of the human body HM with the movement direction D1 detected by the direction detection unit 12 as the Z direction. The motion analysis unit 11 analyzes the motion using, for example, the coordinates of the converted feature portion.

The direction detection unit 12 may detect the moving direction D1 by using the position of a portion other than the leg portion Q1 detected by the detection unit 5. For example, the direction detection unit 12 may detect the moving direction D1 by using the time change of the position of the head Q9 of the human body HM. Further, the direction detection unit 12 may detect the moving direction at each landing timing. For example, the direction detection unit 12 may detect the moving direction between the two landing timings by comparing the position of the heel Q4 at an arbitrary landing timing with the position of the heel Q4 at the next landing timing. Further, the motion analysis unit 11 may analyze the motion by using the movement direction detected by the direction detection unit 12 at each landing timing. Further, the direction detection unit 12 may detect the moving direction D1 without using the landing timing.

Next, the detection direction of the detection unit 5 according to the embodiment will be described. FIG. 11 is a diagram showing the arrangement of the detection unit according to the embodiment. The detection direction D2 of the detection unit 5 is, for example, the direction of the center line of the detection range of the detection unit 5. The detection unit 5 is arranged so that the characteristic portion of the human body HM used for the landing determination passes through the detection range FV. When the motion analysis unit 11 detects the pronation, the detection unit 5 is arranged so that the displacement of the knee in the direction intersecting the moving direction D1 (eg, the X direction) can be detected. For example, when the human body HM is detected by imaging, the detection unit 5 is arranged so that the angle φ between the moving direction D1 and the detecting direction D2 is larger than 0 ° and smaller than 90 °.

The angle φ between the moving direction D1 and the detection direction D2 may be set to an arbitrary angle of 0 ° or more and 90 ° or less when the detection unit 5 detects the three-dimensional position of the human body HM. Further, the moving direction D1 may not be set in advance, and the detection system 1 may detect the moving direction D1.

Further, as shown in FIG. 11B, the detection unit 5 may include, for example, a plurality of detection units (eg, first detection unit 5a, second detection unit 5b) that detect the human body HM by imaging. .. The first detection unit 5a may be arranged, for example, so that the movement direction D1 and the detection direction D2a are parallel to each other. The second detection unit 5b may be arranged so that, for example, the moving direction D1 and the detecting direction D2b are orthogonal to each other. The landing determination unit 9 may perform the landing determination process based on the detection result of the second detection unit 5b, for example. Further, the motion analysis unit 11 may detect the pronation based on the detection result of the first detection unit 5a, for example.

[Fourth Embodiment]
A fourth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 12 is a diagram showing a detection system according to the embodiment. In the present embodiment, the processing device 3 includes a communication unit 21 (transmission unit). The communication unit 21 transmits at least a part of the processing result of the landing determination unit 9 to the outside. The landing determination unit 9 detects (eg, estimates) the landing timing of the human body HM based on the time change of the detection result of optically detecting the moving human body HM. The communication unit 21 transmits the landing timing of the human body HM as the processing result of the landing determination unit 9.

Further, the detection system 1 includes a processing device 25. The processing device 25 includes a communication unit 26 (reception unit and transmission unit) and an analysis unit 27. The communication unit 26 receives the landing timing at which the landing of the human body HM is determined based on the time change of the detection result of optically detecting the moving human body HM. The communication unit 26 is communicably connected to the communication unit 21 of the processing device 3 by wire or wirelessly. The communication unit 26 receives the landing timing of the human body HM detected by the landing determination unit 9 from the communication unit 21 of the processing device 3.

The analysis unit 27 analyzes the pronation of the leg Q1 of the human body HM based on the landing timing received by the communication unit 26. The analysis unit 27 uses the landing timing as the start signal of the analysis, and analyzes the pronation by the same processing as the motion analysis unit 11 described in FIG. 5, for example. For example, the communication unit 21 of the processing device 3 transmits the displacement information calculated by the displacement calculation unit 8 to the communication unit 26 of the processing device 25. The analysis unit 27 detects the pronation of the leg portion Q1 based on the landing timing and displacement information received by the communication unit 26. The communication unit 26 may receive at least a part of the detection result (detection result of the detection unit 5 and processing result of the processing unit 7) from the detection device 2. Then, the analysis unit 27 may detect the pronation based on the landing timing received by the communication unit 26 and the detection result of the detection device 2.

The processing device 25 stores, for example, the analysis result of the analysis unit 27 in the storage unit. The processing device 25 may output the analysis result of the analysis unit 27 to an external device (eg, a display device). The processing device 25 may be provided separately from at least a part of the detection system 1. For example, the processing device 25 may be provided separately from the processing device 3. Further, the processing device 25 may be provided separately from the detection device 2. Further, the processing device 25 may include at least a part of the detection device 2 (eg, the detection unit 5 and the processing unit 7).

Further, in addition to the above analysis processing, the motion analysis unit 11 analyzes the pronation of the leg of the human body HM of the user (eg, himself / herself, the person who becomes the sample (teacher)) detected (analyzed) in the past such as the previous time. Compared with the analysis results currently analyzed using (eg, past analysis results) (eg, current analysis results), the pronation of the legs of the human body HM is determined from the similarity of the legs between these analysis results. It may be detected. In this case, the storage unit 10 is configured to store the analysis result of the human body HM detected (analyzed) in the past for each leg of the human body HM at the analysis timing or the like.

Next, the exercise mat according to the embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 13 is a diagram showing an exercise mat according to the embodiment. The exercise mat 30 is arranged in the movement range (eg, movement range, support surface G) of the moving human body HM. The exercise mat 30 is, for example, a sheet-shaped or mat-shaped member. The exercise mat 30 is, for example, an exercise mat, a yoga mat, a training mat, a fitness mat, a swing mat for golf, baseball, or the like). The exercise mat 30 may be provided on a surface that supports the human body HM in exercise equipment such as a treadmill. For example, the human body HM exercises or moves on an exercise mat 30 laid on a support surface G (eg, the support surface G includes a surface of a competition track, a competition floor, and the like). The exercise mat 30 may be set on the entire surface or a part of the support surface G, or may be completely or partially embedded in the support surface G.

In the exercise mat 30, in order to reduce the noise of the reflected light from the mat itself, the wavelength band of light having the sensitivity of the detection unit 5 that optically detects the human body HM (eg, the wavelength band of infrared light, visible). In the wavelength band of light and the wavelength band of X-rays), the absorption rate of light (eg, infrared light) is higher than that of the human body HM. Further, the exercise mat 30 is configured to have a lower reflectance of light (eg, infrared light) than the human body HM. For example, the detection unit 5 irradiates the human body HM with light in a predetermined wavelength band, and detects the light emitted from the human body HM by the irradiation of the light. The above-mentioned predetermined wavelength band includes, for example, a wavelength band of infrared light (eg, near infrared light). The light emitted from the human body HM includes, for example, infrared light (eg, near-infrared light). For example, the exercise mat 30 has a higher absorption rate for the light emitted from the detection unit 5 than the human body HM. Further, the exercise mat 30 may be configured to have optical characteristics such that the difference in contrast with the human body HM is conspicuous in the wavelength band of light having the sensitivity of the detection unit 5.

FIG. 13B is a cross-sectional view showing an exercise mat according to the embodiment. The exercise mat 30 includes a base material 31, an absorption layer 32, and a base layer 33. The base material 31 is made of, for example, a resin material. The above resin material is, for example, at least one of nitrile rubber (NBR), natural rubber, thermoplastic elastomer (TPE), polyvinyl chloride (PVC), polymer environmental resin (PER), and ethylene vinyl acetate copolymer (EVA). including.

The absorption layer 32 contains a dye (material) that is uniformly arranged in the layer and has a higher absorption rate than the human body HM in a wavelength band in which the detection unit 5 has sensitivity (eg, a wavelength band of infrared light). The absorption layer 32 has, for example, an absorption maximum in a wavelength band of 800 nm or more and 1200 nm or less or a wavelength band of 750 nm or more and 3000 nm or less. The above dyes include, for example, phthalocyanine pigments, metal complex pigments, nickel dithiolene complex pigments, cyanine pigments, squarylium pigments, polymethine pigments, azomethine pigments, azo pigments, polyazo pigments, and diimonium pigments. , Aminium-based pigments, and anthraquinone-based pigments, including one or more. The absorption layer 32 may be formed of, for example, a resin material in which graphite is dispersed.

The base layer 33 is formed of, for example, a material having a coefficient of friction with respect to the support surface G larger than that of the base material 31. The base layer 33 may have a plurality of minute irregularities on its surface so as to suppress slippage with the support surface G.

The exercise mat 30 has a level of flexibility that allows it to be rolled around, for example. For example, the materials of the base material 31, the absorption layer 32, and the base layer 33 are selected so as to have the above-mentioned flexibility. The configuration of the exercise mat 30 shown in FIG. 13B is an example and can be changed as appropriate. For example, the exercise mat 30 has a protective layer (eg, antibacterial or antifouling coating) that protects the absorption layer 32 on the side opposite to the base material 31 (eg, the surface side of the exercise mat 30) with respect to the absorption layer 32. Layer) may be included. Further, the exercise mat 30 may include an antireflection layer (eg, AR coating) on the side opposite to the base material 31 (eg, the surface side of the exercise mat 30) with respect to the absorption layer 32.

The intensity of the light L1 incident on the detection unit 5 from the exercise mat 30 is weaker than, for example, the intensity of the light L2 incident on the detection unit 5 from the human body HM. The detection unit 5 can detect, for example, the human body HM moving on the exercise mat 30 with high accuracy. For example, the detection system including the detection unit 5 can discriminate between the legs of the human body HM moving on the exercise mat 30 and the exercise mat 30 with high accuracy. By using this (eg, the result of discriminating the legs with high accuracy), the detection system 1 can perform the above-mentioned landing determination of the human body HM (eg, landing determination of the legs) and motion analysis of the human body HM (eg, legs). It is possible to process the pronation analysis of the part with higher accuracy. This detection system may be the detection system 1 shown in FIG. 1 or the like, or may not be the detection system 1. Further, the detection system 1 shown in FIG. 1 or the like may include an exercise mat 30.

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, to the extent permitted by law, the disclosure of all documents cited in the above-mentioned embodiments and the like shall be incorporated as part of the description in the main text.

1 ... Detection system, 2 ... Detection device, 3 ... Processing device, 4 ... Display device 5 ... Detection unit, 8 ... Displacement calculation unit, 9 ... Landing determination unit, 11 ... Motion analysis unit, 12 ... Direction detection unit, 21 ... Optical member, BL ... Reference, D1 ... Movement direction, HM ... Human body, Q1 ... Legs, Q4 ... heel, Q5 ... knee

Claims (11)

A detector that optically detects the moving human body,
An analysis unit that analyzes the pronation of the legs of the human body using the detection results of the detection unit,
Equipped with a,
The analysis unit detects the position of the human body from the detection result of the detection unit, identifies at least a part of the leg from the detected position of the human body, and sets the position of at least a part of the identified leg. A detection system that determines the state of the pronation by comparing it with a reference. The detection system according to claim 1, wherein the analysis unit detects the pronation by using the position of the knee of the leg portion obtained from the detection result of the detection unit. Wherein the analysis unit compares the said reference position of said knee, calculates an index value that indicates the level of the pronation The detection system of claim 2. The analysis unit detects the position of the knee with respect to the reference in the crossing direction with respect to the moving direction of the human body, and determines whether the pronation corresponds to overpronation, neutral, or underpronation. Item 3. The detection system according to item 3. The detection system according to any one of claims 1 to 4, wherein the reference includes a line connecting the upper end and the heel of the leg of the human body detected by the detection unit. A receiving unit for receiving the landing timing for determining the landing of the human body based on the time change of the detection result of optically detecting the moving human body is provided.
The detection system according to any one of claims 1 to 5, wherein the analysis unit analyzes the pronation of the leg portion of the human body based on the landing timing received by the receiving unit. The detection system according to any one of claims 1 to 6, further comprising a display device for displaying the analysis result of the analysis unit. Claims 1 to 7 include an exercise mat that is arranged in the motion range of the human body and whose detection unit has a higher absorption rate of infrared light than the human body in the wavelength band of infrared light having sensitivity. The detection system according to any one item. Optically detecting the moving human body and
To detect the pronation of the leg of the human body based on the detection result of detecting the human body,
Only including,
In the detection of the pronation, the position of the human body is detected from the detection result of detecting the human body, at least a part of the leg is specified from the detected position of the human body, and at least a part of the specified leg is detected. A detection method for determining the state of the pronation by comparing the position of the head and the reference. On the computer
To detect the pronation of the leg of the human body based on the detection result of optically detecting the moving human body.
To run ,
In the detection of the pronation, the position of the human body is detected from the detection result of detecting the human body, at least a part of the leg is specified from the detected position of the human body, and at least a part of the specified leg is detected. A detection program that determines the state of the pronation by comparing the position of the pronation with the reference. A detector that optically detects the moving human body,
A direction detection unit that detects the moving direction of the human body based on the detection result of the detection unit, and
An analysis unit that analyzes the pronation of the legs of the human body using the detection results of the detection unit,
With
From the detection result, the analysis unit detects at least a part of the position of the leg of the human body with respect to the reference in the crossing direction with respect to the moving direction of the human body, and compares the position of at least a part of the leg with the reference. A detection system that determines the state of the pronation.

Images