KR20200025290A - System and method for analyzing exercise posture - Google Patents

Abstract

The present invention relates to a system and method for analyzing an exercise posture, comprising: a camera for capturing an image of an athlete to generate image information, a pose analyzer for recognizing a joint from the image information, and analyzing the angle of the joint to generate exercise pose information; And a control unit for generating correction information corresponding to the exercise posture information, storing the correction information for learning, performing a selected exercise program, and an output unit for outputting the correction information. According to the present invention, it is possible to improve the exercise posture correction efficiency by continuing to provide a real-time correction service by analyzing the exercise posture during exercise by the exerciser, and to correct the next exercise posture based on the learning information learned from the database-corrected information. By using this, not only the continuity of the exercise posture correction can be maintained but also the exercise efficiency can be maximized.

Description

System and method for analyzing exercise posture}

The present invention relates to an exercise posture analysis technique, and more particularly, to an exercise posture analysis system and method for providing a real-time correction service by analyzing an exercise posture, and to use for the next real-time correction through learning about an analysis result.

Humans can lead healthy lives simply by living in the right position. It takes a lot of effort and time to return your body, which has been hardened by wrong habits, to its correct position.

Similarly, exercise can maximize the exercise effect even with the correct exercise posture.

In general, for the correction of the exercise posture is helped by a trainer in a sports center.

However, the reality is that the trainer cannot teach the posture correction from the beginning to the end for one athlete. In other words, if the trainer demonstrates the correct exercise posture, there is only a brief posture correction instruction when the exercise posture is followed, and the exercise should be performed alone while actually remembering the exercise posture. At this time, with the help of a mirror, the exercise posture may be corrected by yourself.

In order to alleviate this inconvenience, a technique for servicing a workout posture has been disclosed.

Here, the conventional exercise posture correction technique will be described.

Republic of Korea Patent Publication No. 10-1784410 discloses a "movement posture recognition method and a system for applying the same". This technology, including a gesture measuring unit for measuring the motion of the exercise assistant subject, and a bone extracting unit for extracting the bone of the exercise assistant subject, the motion recognition unit for generating and outputting the body information coordinate value of the exercise assistant subject And receiving the body information coordinate values of the exercise assistant subject from the motion recognition unit to analyze the body information coordinate values of the exercise assistant subject to determine a change in the motion of the exercise assistant subject and determine whether the set target is reached. And a processing unit including an information determining unit and a data setting unit for initially setting the set target or resetting the set target according to a determination result of the performance information determining unit. According to this technique, it is possible to effectively assist the exercise of the exercise assistant subject by determining the exercise result of the exercise assistant subject, and changing the exercise condition from the exercise result determination.

However, this technique recognizes an exercise posture, but does not perform correction for the correct posture, and has a limit of counting only the number of exercises. That is, there is no function of recognizing posture when the user exercises and providing feedback accordingly.

Therefore, there will be a need for a way to receive immediate feedback on the exercise posture.

Republic of Korea Patent Publication No. 10-1784410 (Notice date 2017.11.06.), "A motion posture recognition method and system to apply it"

Accordingly, the present invention has been made to solve the above-mentioned problems of the prior art, an object of the present invention is to provide a real-time correction service by analyzing the exercise posture from the image taken through the camera, and the correction information that the real-time correction service was made The present invention provides a system and method for analyzing a posture of an exercise, which is used to correct a next exercise posture based on the learning information learned by making a database.

Exercise posture analysis system of the present invention for achieving the above object, the camera for generating the image information by taking an image of the exerciser; A posture analyzer for recognizing a joint from the image information and analyzing the angle of the joint to generate exercise posture information; A controller configured to generate calibration information corresponding to the exercise posture information, to store the calibration information to perform learning, and to execute a selected exercise program; And an output unit for outputting the calibration information.

The output unit preferably includes an image output unit and an audio output unit. In this case, the image output unit preferably outputs by overlaying the image information and the calibration image information taken by the camera.

The control unit includes a calibration information generation unit for generating calibration information from the exercise posture information; An exercise program setting unit supporting user selection for an exercise type, an exercise posture, and an exercise direction; A storage unit which stores an exercise program and stores calibration information; And a learning unit learning the calibration information based on the deep learning neural network to generate learning information.

On the other hand, exercise posture analysis method of the present invention, receiving the image information generated by photographing the exerciser, and analyzing the image information to generate the exercise posture information; Generating correction information from the exercise posture information and informing an exerciser; Database the calibration information and performing deep learning artificial neural network based learning on the calibration information to generate learning information; And using the learning information as the calibration information during the next exercise.

The exercise tendency of the exerciser may be determined by learning the calibration information about the same exercise type, and learning information corresponding to the exercise tendency may be generated.

Through user selection or recommendation, exercise program settings for exercise type, exercise posture, and exercise direction may be made.

The movement direction is a camera photographing direction, and the deep learning neural network-based learning may generate learning information by combining and analyzing the same type of motion photographed from various directions.

As described above, according to the exercise posture analysis system and method according to the present invention, it is possible to improve the exercise posture correction efficiency by continuously providing the real-time correction service by analyzing the exercise posture while the exerciser is exercising.

In addition, according to the present invention, by using the database-corrected correction information based on the learned learning information for the next exercise posture correction, not only can maintain the continuity of the exercise posture correction but also maximize the exercise efficiency.

1 is a block diagram of an exercise posture analysis system according to an exemplary embodiment of the present invention.
2 is a block diagram of a control unit according to an embodiment of the present invention.
3 is a flowchart illustrating a method of analyzing a workout posture according to an embodiment of the present invention.
4 is a class diagram according to an embodiment of the present invention.
5 is a flow chart according to an optimal embodiment of the present invention.
6 is an embodiment of the present invention, which is the initial screen of the exercise posture analysis program.
7 is a flowchart illustrating a routine execution process according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, the present invention will be described in detail with reference to preferred embodiments of the present invention and the accompanying drawings, in which the same reference numerals in the drawings will be described on the assumption that they refer to the same components.

When any component in the description or claims of the invention "includes" another component, unless otherwise stated, it is not limited to consisting only of the component, and other components are not interpreted. It should be understood that it may include more.

The present invention relates to a technology for checking, analyzing, and correcting a workout posture photographed by a camera when exercising, and to correct a workout posture through a device application in real time. As such, the present invention provides a technique for identifying a posture recognition program through a camera on one's device in order to enhance the exercise effect.

The present invention informs the athlete to check whether the posture that is exercising is correct, and provides a technique for correcting the exercise posture when the angle of the posture is out of the set value. Recognize the joint of the exerciser from the image taken using the camera to check the movement posture through the angle between the joints. That is, it focuses on improving the user's movement more efficiently by extracting bones from the user's appearance and analyzing the correct exercise posture through the angle calculation between joints. The feedback on the exercise according to the joint of the user to obtain the correct exercise posture can improve the exercise efficiency. At this time, through the camera to inform the user whether the exercise in the correct posture not only visually but also acoustically to increase the exercise effect, so that the user can easily provide services to help exercise at home.

1 is a block diagram of an exercise posture analysis system according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a camera 1 for capturing an image of an athlete to generate image information, a posture analyzer 2 for recognizing a joint from the image information, and analyzing the angle of the joint to generate exercise posture information; A control unit 3 for generating correction information corresponding to the exercise posture information, storing the correction information for learning, executing the exercise program including the selected exercise type, exercise posture and direction of movement, and outputting the correction information; Part 4 is included.

Here, the output unit 4 includes an image output unit and an audio output unit. The image output unit may output the overlay of the image information and the calibration image information photographed by the camera (1).

On the other hand, it is preferable that the posture analyzer 2, the control unit 3 and the output unit 4 is made of one module to simplify the system.

The exercise posture analysis system of the present invention configured as described above generates the exercise posture information by analyzing the exercise image of the athlete generated through the camera 1 in the posture analyzer 2. Accordingly, the controller 3 generates correction information corresponding to the exercise posture information and outputs the correction information through the output unit 4. The output unit 4 informs the exerciser of the calibration information through video and audio. The controller 3 stores the calibration information to perform learning. When the same learning is performed later, the learned learning information is used to correct the exercise posture using the learning information. On the other hand, the exercise program provides a screen on which the exercise type, exercise posture, exercise direction (shooting direction), and the like can be selected and can be selected and set on the screen. In response, the control unit 3 generates calibration information corresponding to the set state.

2 is a block diagram of a control unit according to an embodiment of the present invention.

Referring to FIG. 2, the control unit 3 of the present invention includes a calibration information generation unit 31 that generates calibration information from exercise posture information, and an exercise program that supports user selection for exercise type, exercise posture, and exercise direction. A setting unit 32, a storage unit 33 for storing an exercise program, storing the calibration information, and a learning unit 34 for learning based on the deep learning artificial neural network based on the calibration information to generate learning information. do.

The control unit 3 configured as described above checks the exercise posture information transmitted from the posture analyzer 2 in the calibration information generation unit 31 to generate calibration information, and stores the calibration information in the storage unit 33. Next, the learning unit 34 performs learning based on the deep learning neural network with respect to the calibration information. The learning includes grasping the exercise inclination of the exerciser by learning the calibration information for the same exercise type, or generating correction information corresponding to the exercise inclination. In addition, the learning may include a combination analysis of the same type of motion photographed in various directions. The calibration information generated through this learning can be used for the next exercise. On the other hand, the exercise program setting unit 32 supports the user selection for the exercise type, exercise posture, exercise direction. That is, the type of exercise, the exercise posture, the exercise direction, etc. are output as icons on the screen, and the exercise program is set according to the user's selection. Here, the movement direction means that the camera 1 is photographed in a direction in which the camera 1 photographs, that is, the front, side, rear, and bottom of the exerciser. As described above, respective calibration information may be generated through images captured from the front, side, rear, and bottom surfaces, and the learned calibration information may be generated by collecting and learning these calibration information.

Then, the exercise posture analysis method of the present invention using the device configured as described above will be described.

3 is a flowchart illustrating a method of analyzing a workout posture according to an embodiment of the present invention.

Referring to FIG. 3, the system is driven to set an exercise program by selecting an exercise type, an exercise posture, an exercise direction (shooting direction), and the like (S10). That is, the type of exercise, the exercise posture, the exercise direction, etc. are output as icons on the screen, and the exercise program is set according to the user's selection.

When the exercise is started in front of the camera 1, the camera 1 photographs the athlete to generate image information (S11).

The image information is transmitted to the posture analyzer 2, and the posture analyzer 2 analyzes the exercise posture to generate the exercise posture information (S12).

The exercise posture information is transmitted to the control unit 3, whereby the control unit 3 generates correction information corresponding to the exercise posture information (S13).

The calibration information is transmitted to the output unit 4, so that the output unit 4 notifies the exerciser of the calibration information through the image and the voice (S14).

On the other hand, the calibration information is a database, the deep learning artificial neural network based learning is performed on the calibration information (S15). That is, the exercise propensity of the athlete is determined by learning the correction information for the same exercise type, or the learning information corresponding to the exercise propensity is generated (S16). Alternatively, learning information is generated by combining and analyzing the same type of motion photographed from various directions.

Thus, by using the next exercise posture correction based on the learning information (S17), the continuity of the exercise posture correction can be maintained.

Example

In the present embodiment, a case where the exercise posture analysis service is provided through the server will be described.

Meanwhile, in the exercise pose analysis system and method of the present invention, when exercising in the Tizen-based device Z3 through the camera 1, the routine may be directly set or recommended by the routine.

Users who are familiar with the exercise and can do the exercise they want to do can make their own choices and set up routines. In this part, the numerical values of the exercise for the recognition and the information of the various movements are stored in advance in the DB of the web server and printed out.

On the other hand, a user who wants to start a workout without knowing the recommended routine can be recommended, and provides a routine to conveniently exercise without having to select the exercise. In this part, various routines are stored in the routine DB for recommendation through exercise information stored in the DB of the web server in advance. Thus, a member can have several routines.

4 is a class diagram according to an embodiment of the present invention.

In the exercise start and posture recognition exercise feedback part shown in FIG. 4 below, the user's exercise position is checked using a webcam (HD proWebcam C920r). Recognize people through Openpos, and use the Openpos skeleton to pick and use a human joint as a point, regardless of the human's movement in the image. The difference with OpenCV is that you can continue to hold human joints with Openpos alone without having to use the openCV_video library to capture the movement.

On the other hand, when the exercise is finished, a DB for storing the exercise amount is required. So you can save the momentum DB and the member can have several momentum DB.

Tizen devices should be 2.4 or higher and webcams should be HD webcams. It is recommended to use window10 as the operating environment, and you can use the application through Tizen Studio.

The DB server includes a member DB, an exercise DB, a routine DB, and an exercise DB. Generally, data software manufactured by Oracle Corporation is used to create and manage the DBs.

The member DB stores basic information such as name, address, mobile phone number, e-mail address, ID and password of the student and the lecturer.

The exercise DB stores the name of the exercise, the description of the exercise, the location and the angle of the exercise.

The routine DB stores the routine name, exercise name, and date.

Stores information about the user's exercise in the exercise DB. The exercise amount DB stores the exercise name, exercise amount, and date.

The web server is configured to check whether the user correctly performs the exercise when exercising properly, and is in charge of communication between the webcam and the device.

5 is a flow chart according to an optimal embodiment of the present invention.

System users access the application installed on the Tizen device and transmit their basic information through the information communication network at the initial screen of the application, and the web server receives the information and stores it in the member DB of the DB server. Perform the member registration procedure (S20).

In the present embodiment, the basic information of the system user stored in the member DB by the joining step may include, but is not limited to, the name, address, mobile phone number, e-mail address, ID and password of the system user.

On the other hand, the initial screen after login of the application shown in Figure 6 consists of a text box in which the member's name is written in the upper part, four menu buttons in the center of the screen, and the member's information menu button in the upper right corner. The text box displays the name of the application user. The four menu buttons in the center include an 'exercise search' menu, a 'routine setting' menu, a 'routine start' menu, and a 'exercise statistics' menu.

The "exercise search" menu is a menu for providing an application user with information about a workout according to the present invention. Information on exercise is provided by muscle group and difficulty. In addition, the 'routine setting' menu is a menu that allows the system user to set a desired routine by binding the exercise. In addition, the 'routine start' menu is a menu for starting a routine set by the system user who has completed the routine setup.

In addition, the 'exercise statistics' menu is a menu that allows the system user who has completed the start of the routine to check the exercise amount statistical information until the menu click time, weekly or monthly.

The system user who has registered as a member executes an application installed on the Tizen device, selects a workout that he / she wants to send, and transmits a workout list to the web server. The transmitted information of the exercise is stored in the routine list of the DB server by binding it as a routine in the server.

The system user selects the 'routine start' menu to retrieve a list of stored routines from the routine list table of the DB server. After selecting the routine to be started from the list of routines, the exercise recognition program is executed on the user's computer linked with the webcam and the routine start step is performed.

At the start of the routine, the system user recognizes his / her appearance on the shock cap and executes the routine according to the routine control sequence from the Tizen device. It recognizes the system user's motion on the computer linked with the webcam and increases the count only when exercising in the correct posture, and updates the data in the exercise result table by transmitting information on the progress of the exercise to the DB server in real time.

In the routine execution embodiment, the exercise recognition program of the computer linked with the webcam recognizes the user's movement and transmits one success or failure of one exercise in real time through the web server, and then stores it in the DB server. When the exercise success or failure is updated in the DB server, the exercise status is output to the Tizen device of the system user in real time so that the system user can check his / her exercise status during the start of the routine.

The system user can check his exercise statistics information by selecting the 'exercise statistics' menu. Exercise statistics provide daily calorie consumption by daily, weekly and monthly total exercise volume and frequency. In addition, by providing information on the amount of exercise for each muscle group, it is possible to identify the muscle group that has exercised less.

On the other hand, the routine may be set directly by itself or may be set by recommending the routine (S21 to S22).

Subsequently, when the exercise is started (S23), in response to posture recognition (S24), a notification sound of success or failure is made to help real-time feedback (S25).

7 is a flowchart illustrating a routine execution process according to an embodiment of the present invention.

Referring to FIG. 7, the motion recognition of the system user required for the routine execution step is performed by the motion recognition program of the computer linked with the webcam. Exercise recognition takes place in several stages. First, the user requests the server to start the routine by clicking the start routine button (S30).

After receiving the signal, the web server checks the user's routine list stored in the DB (S31) and transmits the list information of the routine set by the user to the user's Tizen device through the web server. To the screen (S32). After that, the user selects one routine and sends a signal to the server to exercise control request (S33).

The server receiving the signal makes an exercise start request to the user by showing an exercise start request to the Tizen device (S34). After receiving the exercise start request (S35), the user executes a motion recognition program on his computer to which the webcam is linked, and then stands in front of the webcam to recognize the system user.

The next step is to extract the user's bones using a motion recognition program to create a skeleton model. After that, the angle is measured based on the joint of the site used in the exercise using the skeleton model to determine and analyze whether it is the correct posture (S36). At this time, when it is determined that the exercise posture is correct (S37), the data of the server is updated (S38 to S39).

After all routines are finished, the server informs the user that the workout is over. The Tizen device outputs the exercise result stored in the DB server (S40 ~ S41).

The web server, on the other hand, transmits information about the user's ongoing exercise to the webcam. Information on the exercise in progress includes the name of the exercise, the effect of the exercise, exercise point and the like. When the user selects a routine set by the user using his device, the web server prepares to receive an image.

The web server checks whether the movement is properly performed through the angle of the posture with respect to the incoming image. The web server checks the information stored in the motion control DB of the database.

As a result of determining whether the posture of the web server is correct, if it is recognized that the exercise posture is correct and enters the server, the exercise amount DB stores the exercise information and the basic information of the user in the exercise amount DB.

The web server then transmits the information to the device based on the Tizen-based device Z3 that is determined to be correct by the user.

The device notifies the user visually and audibly of the result of receiving the information.

The device outputs the word success through the notification and audibly sounds the notification so that the user can check his posture in real time when exercising.

If it is determined that the received result is not correct posture, it will help to provide real-time feedback by sounding a failure notification and success and other notification sounds.

After the completion of the routine DB on the web server and the web server, when the completion of the exercise notification and notification sound output to the device to inform the user that the exercise is finished by ending the exercise.

According to the present invention as described above, not only to increase the exercise effect by providing an audio-visual exercise posture service using the camera (1) for the item that the user exercise, but the user wants to exercise without time and regional restrictions Exercise services can be provided for sports.

In one or more example implementations, the functions presented herein may be implemented through hardware, software, firmware, or a combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media and communication media including any medium for facilitating the transfer of a computer program from one place to another. A storage medium may be any available medium that can be accessed by a general purpose computer or a special purpose computer. For example, such computer readable media may be any required program code means in the form of RAM, ROM, EEPROM, CD-ROM or other optical disk storage media, magnetic disk storage media or other magnetic storage devices, or instructions or data structures. Include, but are not limited to, a general purpose computer, special purpose computer, general purpose processor, or any other medium that can be accessed by a particular processor.

In addition, any connecting means may be considered a computer readable medium. For example, if the software is transmitted from a website, server, or other remote source via wireless technologies such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or infrared radio, and microwave, such coaxial Wireless technologies such as cable, fiber optic cable, twisted pair, DSL, or infrared radio, and microwave may be included within the definition of such media. Disks and discs used herein include compact discs (CDs), laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs, where the disks magnetically reproduce data, but the discs are optically lasered. To play the data. Combinations of the above should also be included within the scope of computer-readable media.

Those skilled in the art will appreciate that the various exemplary elements, components, logic blocks, modules, and algorithm steps described above may be implemented as electronic hardware, computer software, or a combination thereof. To clarify the interoperability of hardware and software, various illustrative elements, blocks, modules, and steps have been described in terms of their functionality. Whether such functionality is implemented in hardware or software depends on the design constraints added for the particular application and the overall system. Skilled artisans may implement these functions in varying ways for each particular application, but such implementation decisions do not depart from the scope of the present invention.

Various example logic blocks and modules described in connection with the present disclosure may include general purpose processors, digital signal processors (DSPs), application specific semiconductors (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates. Or may be implemented or performed through transistor logic, discrete hardware components, or any combination designed to implement the functions described herein. A general purpose processor may be a microprocessor; In alternative embodiments, such a processor may be an existing processor, controller, microcontroller, or state machine. A processor may be implemented as a combination of computing devices, such as, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or a combination of these configurations.

With respect to a hardware implementation, various example logics, logic blocks, and modules of the processing units described in connection with aspects disclosed herein may include one or more application specific semiconductors (ASICs), digital signal processors (DSPs), Digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), discrete gate or transistor logic, discrete hardware components, general purpose processors, controllers, micro- It may be implemented in controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. A general-purpose processor may be a microprocessor, but in the alternative, may be any existing processor, controller, microcontroller, or state machine. A processor may also be implemented in a combination of computing devices (eg, a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors associated with a DSP core, or any other suitable configuration). Additionally, at least one processor may include one or more modules that may implement one or more of the steps and / or operations described herein.

In addition, various aspects or features described herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. In addition, the steps and / or operations of a method or algorithm described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination thereof. Additionally, in some aspects, steps or operations of a method or algorithm may exist as at least one or any combination of a set of codes or instructions on a machine-readable medium, or a computer-readable medium, which is Can be integrated into computer program objects. The term article of manufacture, as used herein, is intended to include a computer program accessible from any suitable computer-readable device or medium.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the invention. Thus, the present invention should not be limited to the embodiments set forth herein but should be construed in the broadest scope consistent with the principles and novel features set forth herein.

1: camera
2: posture analyzer
3: control unit
4: output unit

Claims (8)

A camera for photographing an exerciser to generate image information;
A posture analyzer for recognizing a joint from the image information and analyzing the angle of the joint to generate exercise posture information;
A controller configured to generate calibration information corresponding to the exercise posture information, to store the calibration information to perform learning, and to execute a selected exercise program; And
And an output unit for outputting the correction information. The method of claim 1,
The output unit exercise image analysis system including an image output unit and an audio output unit. The method of claim 2,
And the image output unit overlays and outputs the image information and the corrected image information captured by the camera. The method of claim 1,
The control unit,
A calibration information generator for generating calibration information from the exercise posture information;
An exercise program setting unit supporting user selection for an exercise type, an exercise posture, and an exercise direction;
A storage unit which stores an exercise program and stores calibration information; And
Exercise learning system including; learning unit for learning based on the deep learning artificial neural network correction information to generate learning information. Receiving image information generated by photographing an athlete, and analyzing the image information to generate exercise posture information;
Generating correction information from the exercise posture information and informing an exerciser;
Database the calibration information and performing deep learning artificial neural network based learning on the calibration information to generate learning information; And
Using the learning information as the correction information during the next exercise; exercise posture analysis method comprising a. The method of claim 5,
An exercise posture analysis method of determining an exercise tendency of an exerciser by learning correction information about the same exercise type and generating learning information corresponding to the exercise tendency. The method of claim 5,
An exercise posture analysis method in which an exercise program setting for exercise type, exercise posture, and exercise direction is made through user selection or recommendation. The method of claim 7, wherein
The movement direction is a camera shooting direction,
The deep learning neural network-based learning exercise pose analysis method for generating learning information by combining the analysis of the same type of motion picture taken from various directions.

Images