KR102534463B1 - Golf lesson system using a joint of human body and a golf club based on AI - Google Patents

Abstract

The system according to an embodiment of the present invention includes a first camera and a second camera capable of 2D shooting and installed at different positions or at different heights to photograph a subject; a 3D joint generation unit for generating joint positions of a human body, which is a subject, in 3D using captured images taken from each of the first and second cameras; and a posture guidance unit that analyzes a golf swing of a subject using the 3D joint positions generated by the 3D joint generation unit to provide a posture lesson.
According to the present invention, it is possible to accurately analyze the golf swing posture based on AI, and in particular, there is an advantage in that lessons for the current swing posture can be efficiently performed using position and rotation information of human joints.

Description

Golf lesson system using a joint of human body and a golf club based on AI}

The present invention relates to a golf lesson system using AI-based extraction of human joints and golf clubs. In particular, a plurality of 2D cameras are used to photograph human joints and golf clubs, and the photographed results are 3D based on AI. It is about a system that can provide an effective learning method through accurate swing analysis.

BACKGROUND OF THE INVENTION [0002] Golf is a sport played on a field where grass is formed, and proceeds by putting a small and hard golf ball into a hole cup while hitting it with a golf club. Golf has already become popular in Korea.

On the other hand, since it is very important in golf to keep the balance and posture of the body stable during the swing motion and to efficiently operate the body's flexibility and its dynamics, golf trainees use professional professional golfers or semi-professional golfers to learn the basics. receive diagnosis and correction of swing posture from However, in order to receive systematic guidance from experts, considerable time and economic investment are required. Due to this realistic reason, there are many difficulties in systematically correcting wrong swing posture.

Thus, a number of devices for analyzing golf swings have been released on the market. These devices, for example, take video of a user's swing to analyze the swing. In addition, by examining the change in the angle of the user's specific joint part in the photographed image, data for determining whether the posture of the swing motion is correct was provided.

However, among conventional swing analysis devices, analysis devices using optical and sensor-type motion capture can trace and analyze the angle of the user's joints only when the user attaches an optical marker or sensor to the body and takes pictures. An analysis device attached with an additional device has a problem in practical use because it is difficult for a user to wear.

As another device, the depth camera-type analysis device does not have to attach a marker to the user's body, but it recognizes and analyzes the 3D depth camera for the clothes worn by the user, so the accuracy of analysis depends on the clothes worn by the user. There is a problem with falling.

However, the conventional swing analysis device tracks and analyzes the angle change of the user's elbow and wrist directly from the captured image, but this has a problem that the angle analysis is inaccurate when the image is taken with backlight depending on the location or the shooting angle.

In addition, in winter, when a golfer wears a thick top or a long top with a wide arm, it is difficult to accurately measure the angle of a joint such as an elbow.

In golf lessons, it is the swing posture that needs correction the most, and in the present invention, in order to accurately analyze such a swing posture, a terminal capable of photographing a camera is attached to each joint or a user wears a wearable device such as a geomagnetic sensor. The purpose of this study is to propose a system capable of extracting and analyzing an accurate swing posture without providing a proper function.

The system according to the proposed embodiment of the present invention includes a first camera and a second camera capable of 2D shooting and installed at different positions or at different heights to photograph a subject; a 3D joint generation unit for generating joint positions of a human body, which is a subject, in 3D using captured images taken from each of the first and second cameras; and a posture guidance unit that analyzes a golf swing of a subject using the 3D joint positions generated by the 3D joint generation unit to provide a posture lesson.

Further, a 2D joint detection unit for detecting a 2D joint position based on AI technology from the captured images taken by the first and second cameras, wherein the 3D joint generation unit captures images taken by the first and second cameras. It is characterized in that the joint position for the unphotographed area is further added using an interpolation method considering golf movement characteristics for the unphotographed area.

And, a 3D joint rotation unit for providing rotation information including a rotation angle or rotation direction of the joint for each position detected using the 3D joint position generated by the 3D joint generator, and the 3D joint position and rotation information It is characterized in that it further comprises a 3D avatar creation unit for generating a 3D avatar using.

In addition, the 2D joint extractor further detects the position of the golf clip included in the captured image from the shape of the golf club based on AI technology, and the 3D joint generator uses the location of the golf club head and grip in the captured image. The 3D position of the golf club shaft is generated, and the 3D joint rotation unit calculates a golf club rotation value representing a rotation direction or rotation angle of the golf club using previously generated rotation information of human joints.

The proposed system of the present invention can accurately analyze the golf swing posture based on AI, and in particular, has the advantage of efficiently conducting lessons on the current swing posture using the position and rotation information of the human body joints. .

1 is a block diagram showing the configuration of a golf lesson system using AI-based human body joint and golf club extraction according to an embodiment of the present invention.
2 is an exemplary view showing 2D joint extraction and 3D joint generation through a camera according to an embodiment of the present invention.
3 is a diagram showing 10 tag positions that are considered important during golf exercise.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings. However, the scope of the inventive idea of this embodiment can be determined from the matters disclosed in this embodiment, and the inventive idea of this embodiment is the implementation of addition, deletion, change, etc. of components with respect to the proposed embodiment. will include transformation.

1 is a block diagram showing the configuration of a golf lesson system using AI-based human body joint and golf club extraction according to an embodiment of the present invention, and FIG. 2 is a 2D joint extraction and 3D joint extraction through a camera according to an embodiment of the present invention. It is an example diagram showing joint creation.

Referring to FIGS. 1 and 2 , in this embodiment, a first camera 10 and a second camera 20 are configured to capture a subject (person or golf club) at different positions or angles while enabling 2D shooting. , 2D joint detectors 11 and 21 for detecting human body joints for images or moving pictures (hereinafter, referred to as "photographed images") captured by the first camera and the second camera, respectively.

In addition, the 2D joint detection units 11 and 21 may detect joint positions of each part of the human body from pre-stored sample images based on AI technology, and create joint connection lines corresponding to the body parts for each detected joint position. there is. For example, as shown in FIG. 2 , each 2D joint detection unit designates a position of each joint of the human body included in a captured image using information on a reference image or a plurality of pre-stored AI-based images, and each designated joint Draw a connection line connecting each joint according to the body part.

Since the 2D captured images extracted from the 2D joint extractors are captured at different positions or at different angles, the 3D joint generator 30 extracts 3D joint positions from the captured images including joint position information. extract

The 3D joint generation unit 30 performs interpolation in consideration of the characteristics of the golf movement for a part/area not captured by the first camera 10 and the second camera 20 (ie, unphotographed area). Add/add via In order to create a 3D joint position, a position that is equally measured by at least two cameras is required, but due to the nature of the golf swing, which is a rotational motion, a part that is not visible from one camera occurs. For example, the side camera is filming to the right of the swinging golfer, and in some positions where the golfer has their back to the camera, the golfer's face or hands are not visible. Fortunately, since the golf swing shows motion trajectory characteristics that can be interpreted as a three-dimensional equation, it is possible to predict the unmeasured 2D position by substituting the measured position and the unmeasured position based on the time axis.

The joint generating unit 30 generates a 3D pose as shown in FIG. 2 by calculating a 3D position using the same position detected by two cameras or generated by interpolation.

In addition, the system of the present embodiment includes a 3D joint rotation unit 40, and the 3D joint rotation unit 40 obtains rotation information of each joint based on AI technology for each joint detected at a specific position, and Creates rotation information (rotation direction, rotation angle, etc.) for each joint at each location.

It can be expressed as the position of the joint of the human body and the rotation of the joint. Each of the 3D joints detected and generated by the 3D joint generator 30 has only the position value of the measured joint, and there is no data on how it rotates. Based on AI technology, rotation information for each joint at each position is generated.

In addition, the system of this embodiment includes a 3D avatar generator 50 that generates a 3D avatar using the position and rotation information of each joint generated by the 3D joint generator 30 and the 3D joint rotation unit 40. do. That is, the 3D avatar creation unit 50 serves to create a swing animation of the 3D avatar by applying the 3D joint position and rotation information to the 3D avatar as an input.

In addition, the system of the embodiment includes a posture guidance unit 60 for providing hints or lessons on the user's swing posture by referring to pre-stored reference motions/postures with reference to joints and rotational motions of the joints for each position, and a 2D or It includes a display unit 70 that displays 3D captured images, 3D avatars, or posture map information.

Meanwhile, it is also possible to extract the location of the golf club held by the user using the cameras 10 and 20 of this embodiment. In this case, the 2D joint detectors 11 and 21 detect the golf club based on AI technology. The position of the golf club is detected from the shape of . Common joint extraction AI-based technologies such as HRNet and PoseNet consist of a deep learning network that extracts only the joints of the human body, and relearning is performed by adding the location of the golf club head and golf club grip to this deep learning network. If this is done, it can be expanded and used as a deep learning network that detects the location of the joints of the human body and the golf club, which are the basic extraction results.

In addition, the 3D joint generating unit 30 generates the 3D position of the golf club shaft using the positions of the head and grip of the golf club, and the 3D joint rotation unit 40 generates rotation information of the already generated human joints ( Rotation information (rotation direction, rotation angle) of the golf club is calculated using the rotation direction and rotation angle.

That is, after extracting 2D golf club points from images taken by stereo cameras using AI technology, 3D golf club positions are extracted with 2D points of each camera, and 3D rotational motions of human joints are reflected on the golf club to create golf club points. generate a 3D rotational motion of

Through such extraction of golf clubs and generation of 3D golf clubs, golf clubs can also be created as 3D avatars, and it is also possible to construct a lesson system with 3D joint movement and club positions.

3 is a diagram showing 10 tag positions that are considered important during golf exercise.

An intuitive lesson system can be configured by comparing the ideal joint and club position in each position shown, or the professional golfer's joint and club position with the measured position of the golfer, and using a user's photograph taken directly is also possible. It is possible, and it is also possible to conduct a lesson using the generated 3D avatar.

Claims (5)

A first camera and a second camera capable of 2D shooting and installed at different locations or at different heights to photograph a subject;
a 3D joint generation unit for generating joint positions of a human body, which is a subject, in 3D using captured images taken from each of the first and second cameras;
a posture guidance unit that analyzes a golf swing of a subject using the 3D joint positions generated by the 3D joint generation unit to provide a posture lesson;
a 2D joint detection unit for detecting 2D joint positions based on AI technology from the captured images taken by the first and second cameras;
a 3D joint rotation unit for providing rotation information including a rotation angle or a rotation direction for each joint at each location detected using the 3D joint positions generated by the 3D joint generation unit; and
A 3D avatar creation unit generating a 3D avatar using the 3D joint position and rotation information;
The 3D joint generating unit further adds joint positions for the uncaptured areas not captured by the first and second cameras by using an interpolation method considering golf movement characteristics,
The 2D joint extraction unit further detects the position of the golf clip included in the captured image from the shape of the golf club based on AI technology,
The 3D joint generation unit creates a 3D position of the golf club shaft using the golf club head and grip positions in the captured image, and the 3D joint rotation unit rotates the golf club using already generated rotation information of the human body joints. A golf lesson system using an AI-based human joint and golf club extraction, characterized in that it calculates a golf club rotation value representing a direction or rotation angle. delete delete delete According to claim 1,
2D images generated by the first and second cameras, 3D body joint position images generated by the 3D joint generator, 3D avatars generated by the 3D avatar generator, and the posture lesson information A golf lesson system using an AI-based human joint and golf club extraction, further comprising a display unit displayed.

Images