KR20220100765A - Method of recognizing motion of golf ball and club in fast camera image and apparatus of analyzing golf motion using the same - Google Patents

Abstract

According to the present invention, a motion analysis device for golf is disclosed. The motion analysis device for golf comprises: a sensor unit (100) including a first camera (101), a second camera (102), and a strobe lighting device which are disposed at the top end of a user to obtain a motion of a golf ball; a motion unit (200) including a third camera (201) and a fourth camera (202) which are disposed on the front and sides of the user to obtain motion images of the user's whole body; a processing unit (300) that stores and processes images acquired by the sensor unit (100) and the motion unit (200); a display unit (400) that displays a result of the processing unit (300) on a screen and processes an input/output signal of the user; and a database (500) physically integrated with the processing unit (300) to read and store control settings and calculation results of the device. In addition, according to the present invention, disclosed is a method for recognizing motions of a golf ball and a golf club in a high-speed camera image by using the motion analysis device for golf. The method comprises: a setting step (S1000); a preparation step (S2000); a trigger step (S3000); a buffering step (S4000); a golf ball detection step (S5000); a golf club detection step (S6000); and a golf ball rotation amount calculation step (S7000). An image frame at an exact point in time when a golf club hits a golf ball is efficiently retrieved.

Description

A method for recognizing the movement of a golf ball and a golf club in a high-speed camera image and a motion analysis device for golf using the same

The present invention relates to a method for recognizing the movement of a golf ball and a golf club in a high-speed camera image and a motion analysis apparatus for golf using the same, and more particularly, to an accurate time point at which a golf club hits a golf ball only with an image of a high-speed camera. The image frame is efficiently searched, and at the same time, the movement speed of the golf ball and golf club and the rotation speed of the golf ball are calculated and provided to the user in real time at the moment of actual hitting. It relates to a motion analysis device for golf that can simultaneously analyze the cause and result of a swing motion by a user by providing an image together.

With the recent increase in fine dust and the worsening of outdoor sports participation environments such as abnormal climates, the demand for indoor sports training facilities is rapidly increasing. Indoor (in-door) virtual golf practice facilities are becoming popular. In particular, in the case of golf, it can be said that it is the most suitable sport for continuous practice and skill improvement using these indoor virtual sports facilities.

In this regard, according to Republic of Korea Patent No. 10-1855978 (registration on May 2, 2018, title of invention: golf shot analysis method and apparatus using a thermal sensor {DEVICE AND METHOD FOR ANALYSING GOLF SHOT USING HEAT SENSOR}), " The present invention relates to a golf shot analysis method and apparatus using a thermal sensor. According to a first aspect of the present invention, a golf ball recognition unit for recognizing a shot point, which is an initial position of a golf ball, and a club hitting the golf ball are It may include a divot recognition unit for recognizing a divot, which is an area passing through the ground on which the golf ball is placed, and a shot analysis unit for analyzing the type of the golf shot based on at least one of the shot point and the divot. have.

However, in the above patent document, the image frame at the exact point in time when the golf club hits the golf ball is efficiently retrieved only with the image of the high-speed camera, and at the same time, the movement speed of the golf ball and the golf club and the rotation speed of the golf ball at the actual hitting moment It is calculated and provided to the user in real time, and also provides a swing posture image synchronized with the hitting time of the golf ball and golf club, so that the user can analyze the cause and result of the swing motion by himself/herself. , a technique such as the present invention has not been disclosed at all.

In addition, Republic of Korea Patent Publication No. 10-2013-0047081 (published on May 8, 2013, title of invention: a golf simulation system using a high-speed machine vision camera and a golf simulation method using the system {GOLF SIMULATION SYSTEM USING HIGH SPEED MACHINE VISION CAMERAS AND GOLF SIMULATION METHOD USING THE SAME}) discloses "a system and method for calculating the physical properties of a golf ball using an image acquired through a high-speed machine vision camera".

However, the camera position of the system is limited in that it is impossible to measure the golf ball and the user's swing image because the golf ball is covered by the body in one camera depending on the left or right-handed user. it wasn't done In addition, an event for detecting the movement of the golf ball should be searched for in the corresponding method, but it is very inefficient to search for it in the images of all frames input from the high-speed camera, but a specific method for overcoming this is not presented. In addition, since the speed of the golf ball of the approach in this method is very slow compared to that of hitting the driver, it is very inefficient to search for it in the images of all frames input from the high-speed camera, but a specific method for overcoming this is not presented. . In addition, in the putting method, the movement of the golf ball is not recorded by one camera due to the occlusion of the body, so a specific method for overcoming this is not presented.

Next, according to the Republic of Korea Patent No. 10-1461144 (Registered on November 6, 2014, the title of the invention: a sensing device and a sensing method for a moving ball {DEVICE FOR SENSING MOVING BALL AND METHOD FOR THE SAME}), " A method of calculating the rotational movement of a moving ball by extracting features such as trademarks or logos displayed on the ball itself from successive images of the ball by the image acquired by the camera device.

However, the technology of the patent document specifies the coordinates of features such as trademarks or logos, but there may be disadvantages in that features cannot be clearly extracted due to changes in external lighting or contamination of the ball surface in the actual golfer's user environment. have. In addition, in order to extract the precise movement of the coordinates of the feature at 20,000 RPM per second for the hit rotating golf ball, an expensive, high-speed camera with a speed of 2,000 frames per second is absolutely necessary, so it is difficult to implement the actual technology.

Next, Republic of Korea Patent No. 10-1019902 (Registered on February 25, 2011, title of invention: Sensing processing device for a moving ball, sensing processing method, and virtual golf simulation device using the same {SENSING PROCESSING DEVICE AND METHOD FOR MOVING According to OBJECT, AND VIRTUAL GOLF SIMULATION DEVICE USING THE SAME}), "The present invention is a motion that enables high sensing processing power and sensing accuracy to be realized with a low-resolution, low-speed camera device and a strobe device for an image of a moving ball. Disclosed are a sensing processing apparatus for a ball, a sensing processing method, and a virtual golf simulation apparatus using the same.

However, since the technology of the corresponding patent document relies on the multiple exposure shooting method, there is a technical limitation in extracting the golf ball object because the golf club and a part of the user's body other than the golf ball can be photographed overlapping the golf ball. do.

Next, in the Republic of Korea Patent No. 10-1078975 (Registered on October 26, 2011, the title of the invention: a sensing device and a sensing method used in a virtual golf simulation device {SENSING DEVICE AND METHOD USED TO APPARATUS FOR VIRTUAL GOLF SIMULATION}) According to the present invention, "a method for tracing the shaft of a golf club of a straight component as a method for extracting a golf club trajectory" is disclosed.

However, in the patent document, since the shaft of the golf club is made of various materials and colors such as carbon fiber or plastic in addition to steel, there is a limitation that it is not accurately captured in the image of the camera, and a clear implementation method for this is not described. There are difficulties in implementing the technology.

Next, according to Republic of Korea Patent No. 10-1133150 (registered on March 28, 2012, title of invention: golf ball sensing device), golf capable of sensing the movement direction and height of a golf ball Although a swing analyzer is disclosed, there is an inconvenience of installing a multi-row sensor plate for detecting the movement of a golf ball on the floor.

Next, according to Republic of Korea Patent No. 10-1195497 (Registered on October 23, 2012, the title of the invention: the golf swing posture analysis system {THE GOLF SWING MOTION ANALYSIS SYSTEM}), a golf swing posture analysis system is provided and However, it is inconvenient to attach a wearable sensor for recognizing movement to a part of the user's body.

Republic of Korea Patent Registration No. 10-1855978 (Registered on May 2, 2018, title of invention: Golf shot analysis method and apparatus using a thermal sensor {DEVICE AND METHOD FOR ANALYSING GOLF SHOT USING HEAT SENSOR}) Republic of Korea Patent Publication No. 10-2013-0047081 (published on May 8, 2013, title of invention: Golf simulation system using high-speed machine vision camera and golf simulation method using the system {GOLF SIMULATION SYSTEM USING HIGH SPEED MACHINE VISION CAMERAS AND GOLF SIMULATION METHOD USING THE SAME}) Republic of Korea Patent No. 10-1461144 (Registered on November 6, 2014, title of invention: sensing device and sensing method for a moving ball {DEVICE FOR SENSING MOVING BALL AND METHOD FOR THE SAME}) Republic of Korea Patent No. 10-1019902 (Registered on February 25, 2011, title of invention: Sensing processing device for a moving ball, sensing processing method, and virtual golf simulation device using the same {SENSING PROCESSING DEVICE AND METHOD FOR MOVING OBJECT, AND VIRTUAL GOLF SIMULATION DEVICE USING THE SAME}) Republic of Korea Patent No. 10-1078975 (Registered on October 26, 2011, title of invention: Sensing device and sensing method used in a virtual golf simulation device {SENSING DEVICE AND METHOD USED TO APPARATUS FOR VIRTUAL GOLF SIMULATION}) Republic of Korea Patent No. 10-1133150 (Registered on March 28, 2012, title of invention: Golf ball sensing device) Republic of Korea Patent No. 10-1195497 (Registered on October 23, 2012, title of invention: golf swing posture analysis system {THE GOLF SWING MOTION ANALYSIS SYSTEM})

The present invention was created to solve the above problems, and an object of the present invention is to efficiently search for an image frame at the exact point in time when a golf club hits a golf ball only with an image of a high-speed camera, and at the same time, it is It is to provide a method of recognizing the movement of a golf ball and a golf club from a high-speed camera image, which calculates the movement speed of the golf club and the rotation speed of the golf ball and provides it to the user in real time.

In addition, it is another object of the present invention to provide a motion analysis device for golf in which a user can simultaneously analyze the cause and result of a swing motion by providing a swing posture image synchronized with the hitting time of the golf ball and the golf club. do.

A golf motion analysis apparatus according to the present invention for achieving the above object, a first camera 101, a second camera 102, and a strobe lighting device disposed on the upper end of the user to obtain the movement of the golf ball A sensor unit 100 including; a motion unit 200 including a third camera 201 and a fourth camera 202 disposed on the front and side surfaces of the user to acquire a full-body motion image of the user; a processing unit 300 for storing and processing images acquired by the sensor unit 100 and the motion unit 200; a display unit 400 for displaying the result of the processing unit 300 on a screen and processing the input/output signal of the user; and a database 500 that is physically integrated with the processing unit 300 to read and store control settings and calculation results of the device.

In addition, the first camera and the second camera of the sensor unit 100 transmit a photographing shutter signal to the strobe lighting device located outside, and the strobe lighting device receives the first camera and the second camera by the photographing shutter signal. The operation is controlled in synchronization with the shooting time of the camera, the processing unit 300 designates a shooting speed of the first camera of the sensor unit 100 , and the sensor unit 100 is the first camera 101 . transmits a synchronization signal of the second camera 102 and the third camera 201 and the fourth camera 202 of the motion unit 200, the second camera, the third camera, and the fourth The camera performs synchronized photography at the same time and at the same speed as the first camera by the synchronization signal of the first camera.

Meanwhile, the method for recognizing the motion of a golf ball and a golf club in a high-speed camera image according to another embodiment of the present invention uses the above-described motion analysis apparatus for golf to recognize the movement of a golf ball and a golf club in a high-speed camera image. As a method, set the golf club characteristics (S1100), set the camera calibration (S1200), set the measurement area (S1300), set the initial image frame (II) (S1400), set the image processing, and (S1500), setting the camera control (S1600), setting step (S1000); A preparation step (S2000) of confirming whether or not to hit the golf ball object placed in the hitting area (TZ); A trigger step (S3000) of searching for a trigger frame index (TIDX) at the point in time at which the golf ball is hit in the entire image frame; Buffering step ( S4000); The two-dimensional image coordinates of the golf ball object BO are detected from the trigger image frame TI of the first camera and the second camera of the sensor unit 100, and the golf ball object BO from the detected two-dimensional image coordinates. ) calculating the three-dimensional space coordinates of the three-dimensional space coordinates and the three-dimensional space velocity of the golf ball object (BO), golf ball detection step (S5000); Detect the two-dimensional image coordinates of the golf club object CO from the trigger image frame TI of the first camera and the second camera of the sensor unit 100, and the golf club object CO from the detected two-dimensional image coordinates ) of calculating the three-dimensional spatial coordinates and the three-dimensional space velocity of the golf club object (CO), a golf club detection step (S6000); and a golf ball rotation amount calculation step (S7000) of calculating the rotation information of the golf ball object with the three-dimensional space vector (V) of the calculated golf ball and the end point of the golf club before hitting the golf ball.

In addition, in the setting step (S1000), a trigger mode that is a mode capable of analyzing the movement of a golf ball and a golf club from the images of the first camera and the second camera, and the first, second, third, and fourth cameras and a preview mode, which is a mode that operates so that the user can visually check the installation and operation state of (i) when operating in the trigger mode, in order to analyze the movement of the golf ball and the golf club, the first , setting the second camera to a golf club characteristic setting value or a new value stored in the database 5000 (S1100); setting the first and second cameras to a calibration setting value or a new value stored in the database (5000) (S1200); setting (S1300) the first and second cameras to a measurement area setting value or a new value stored in the database (5000); setting the first, second, third, and fourth cameras as an initial image frame (II) or a new initial image frame (II) stored in the database (5000) (S1400); setting (S1500) the first and second cameras to image processing set values or new values stored in the database (5000); thereafter, setting the first, second, third, and fourth cameras to the camera control set value or new value stored in the database 5000 (S1600); and (ii) operating in the preview mode. In this case, the first, second, third, and fourth cameras are set to a camera control set value loaded from the database 5000 or a new value in order to obtain an image that can confirm the installation and operation status of the camera. (S1600) to; includes.

In addition, in the camera calibration setting step (S1200), the focal length (f _x , f _y ) and the origin coordinates (C _x , C _y ), which are internal parameters (M _cam ) of the first camera and the second camera, and Rotation information (R _cam ) and position information (t _cam ), which are external parameters, are stored in the database 500 .

In addition, in the measuring area setting step (S1300), the hitting area (TZ) and the human body area (BZ) of the golf ball are stored in the database 500, and in the initial image frame setting step (S1400), the first An initial image frame obtained by photographing only the background excluding the moving golf ball and the user's body from the image of the fourth camera is stored in the database 500, and in the image processing setting step (S1500), the preparation step (S2000) , store the image processing setting value applied in the image processing step of the trigger step (S3000), the golf ball detection step (S5000) and the golf club detection step (S6000) in the database 500, and set the camera control In step S1600, the exposure time and camera for controlling the first and second cameras 101 and 102 of the sensor unit 100 and the third and fourth cameras 201 and 202 of the motion unit 200 A brightness gain value indicating the sensitivity of the image sensor may be stored in the database 500 .

In addition, in the preparation step (S2000), when it is confirmed that the golf ball is ready to hit, the center point coordinate (R _p ) and the radius (R _r ), the pixel value ( R _v ) is stored in the database 500, and the first and second cameras of the sensor unit 100, and the third and fourth cameras of the motion unit prepare the last frame of the golf ball image acquired by the camera. The image frame RI may be stored in the database 500 .

In addition, in the trigger step (S3000), the center point coordinates (RO _p ) and the radius (RO _r ), the pixel value (RO _v ) of the golf ball ready object (RO) prepared to hit the hitting area (TZ) is loaded and the center point coordinates are loaded. Based on (RO _p ) and the radius (RO _r ), the binary threshold value (πR _r ² ) for determining that the golf ball-ready object (RO) has been hit is calculated and stored, and the processing unit 300 is the entire image Check and save the total number of frames (FQTY) of the video you want to search for the trigger frame index (TIDX) in the frame, and set the start index (SIDX) to 1, the end index (EIDX) to FQTY, and the middle index (MIDX) to (EIDX). -SIDX)/2, calculate and store the search index (KIDX) by adding the middle index (MIDX) to the start index (SIDX), and the hitting area (TZ) and the search index ( KIDX) by comparing the striking area (TZ) of the image frame to calculate a difference image frame with a pixel change, and the binarization threshold value of the trigger step (S3000) stored in the database 500 in the calculated difference image frame Calculate the pixel quantity (PQTY) with a pixel change of (πR _r ² ) or more, and if the calculated pixel quantity (PQTY) is greater than the binarization threshold (πR _r ² ), the ending index (EIDX) is set as the intermediate index (MIDX) ), and when the calculated pixel quantity (PQTY) is less than or equal to the binarization threshold (πR _r ² ), the starting index (SIDX) is changed to a value obtained by adding the middle index (MIDX) to the original starting index Then, change the intermediate index (MIDX) and the search index (KIDX) again, and repeat the process of calculating the pixel quantity (PQTY) from the difference image of the hitting area (TZ) of the image frame corresponding to the search index (KIDX) , if the value of the intermediate index (MIDX) is less than or equal to 1, the search can be terminated, the search index (KIDX) is stored as the trigger frame index (TIDX), and the search can be ended.

In addition, in the golf ball detection step (S5000), the two-dimensional image coordinates (p _l (x _l , y _l )) of the golf ball object (BO _l ) of the first camera and the golf ball object of the second camera ( A pair of two-dimensional image coordinates (p _l , p _r ) consisting of two-dimensional image coordinates p _r (x _r , y _r ) of BO _r ) as three-dimensional spatial coordinates (P(x) of the golf ball object BO , y, z)) when calculating the following Equation 4

When the golf ball object BO is not detected in the trigger image frame TI of one of the first camera or the second camera due to the occlusion of the user's body or the club during the golf swing, the first camera Only one of the two-dimensional image coordinates (p r) of the two-dimensional image coordinates (p _l ) of the golf ball object (BO _l ) of the second camera or the two-dimensional image coordinates (p _r ) of the golf ball object (BO _r ) of the golf ball object When calculating the three-dimensional spatial coordinate (P) of (BO), the following Equation 5

를 따를 수 있다.

can follow

In addition, in the golf club detection step (S6000), a foreground object list L is created for each trigger image frame TI, and the processing unit 300, when the user's golf club number is an iron and a putter club, first In the second trigger image frame TI, the golf ball and the golf club interfere with each other, so the list L of the foreground object is not created, the trigger image frame TI is converted into a binary image frame, and then the golf ball object BO ) and the golf ball shadow object (BS) to detect only the club golf club object (CO), and to calculate the two-dimensional image coordinates for removing the golf ball shadow object (BS) is the following Equation 6,

can follow

To detect the two-dimensional image coordinates of the detected golf club object (CO) and calculate the three-dimensional space coordinates of the golf club object (CO) and the three-dimensional space velocity of the golf club object (CO) from the detected two-dimensional image coordinates. can

The detected two-dimensional image coordinates of the golf ball object BO or the golf club object CO are detected, and the three-dimensional spatial coordinates of the golf ball object BO or the golf club object CO are detected from the detected two-dimensional image coordinates. In calculating the three-dimensional space velocity of the golf ball object (BO) or the golf club object (CO), the three-dimensional space coordinates of the golf ball object (BO) or the golf club object (CO) and the golf ball-ready object (RO) When calculating the angle between

Equation 7,

can follow

If the angle between the calculated by Equation 7 exceeds 10°, it is determined that the calculated three-dimensional spatial coordinate is invalid. In this case, the two-dimensionality of the detected golf ball object BO or golf club object CO In the method of calculating the three-dimensional movement speed of a golf ball or golf club even if the image coordinates are not detected for two or more consecutive frames,

Equation 8,

and FIG. 19 .

In addition, in the golf ball rotation amount calculation step (S7000), the processing unit 300, when calculating the moving speed (㎧) of the golf club based on the three-dimensional space vector (V), the following Equation 9-1

Equation 11 is a method of calculating the moving speed (ν _ci ) of the golf club object CO before the golf club hits the golf ball.

follow,

The golf ball rotation amount calculation step (S7000), Equation 12

can be calculated according to

In addition, in the buffering step (S4000), the processing unit 300, the first camera and the second camera of the sensor unit 100 in the image acquired by the golf ball object (BO) hit is detected trigger Indexes are sequentially assigned to 100 or more buffer image frames (BQTY) from the image frame of the frame index (TIDX), and they are collected and stored, and the processing unit 300 is the third camera of the motion unit 200 . And from the image frame of the trigger frame index (TIDX) in which the blow of the golf ball object (BO) is detected in the image acquired from the fourth camera, indexes are sequentially assigned to 100 or more buffer image frames (BQTY) and collected to sequentially assign indexes to 50 or more buffer image frames (BQTY) by skipping one frame at a time to save storage memory space, and collect and store them, the processing unit 300, the buffer image A frame time offset (Δto) that specifies the start time of a trigger image frame (TI) to detect a golf ball object (BO) based on the number of the user's golf club in the frame (BQTY), and three trigger image frames (TI) ), the frame time interval (Δti) specifying the time interval of

[Table 2]

can be specified according to

Here, W1 and W5 represent Wood 1 and Wood 5, I3, I8, and I9 represent iron 3, iron 8, and iron 9, PW represents a pitching wedge, and S2 is used for a distance of 50m to 25m. A sand wedge is indicated, A1 is the approach used for a distance of 25m to 10m, and PT is a putter. For more specific details on individual numbers of golf clubs, refer to Table 1 to be described later.

According to a method for recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention and a motion analysis apparatus for golf using the same,

First, by providing a motion analysis device for golf characterized by a structure that clearly separates and operates into a sensor unit consisting of two cameras for photographing the movement of a golf ball and a golf club, and a motion unit consisting of two cameras for photographing a user's swing, , it can be used without restrictions according to left or right-handed users.

Second, a method to quickly and accurately search the image frame at the trigger point of hitting the golf ball with the user's golf club from a large number of image frames input from the image of the high-speed camera, and a trigger that can efficiently recognize the movement of the golf ball It provides a method to designate and save an image frame.

Third, it efficiently retrieves the image frame at the exact point in time when the golf club hits the golf ball only with the video from the high-speed camera, and at the same time calculates the movement speed of the golf ball and the golf club and the rotation speed of the golf ball at the actual hitting moment. It provides a swing posture video synchronized with the golf club hitting time.

Fourth, as a method for tracking the precise movement of a golf ball, a method of calculating the three-dimensional spatial position of a golf ball object of a camera only with two-dimensional image coordinates detected from a photographed image of one camera, not two cameras. to provide.

Fifth, as a method for tracking the precise movement of a golf club, the characteristics and shooting area of the golf club are set in advance, and based on this, a method of removing a golf ball object unrelated to a golf club and a shadow object of a golf ball from a photographed image is provided. do.

Sixth, three-dimensional movement of a golf ball or a golf club is a method for tracking the precise movement of a golf ball or a golf club, even if the two-dimensional image coordinates of the golf ball and the golf club are not detected for two consecutive frames or more in the high-speed camera's captured image Provides a way to calculate speed.

1 is a system diagram illustrating the structure of a motion analysis apparatus for golf according to the present invention.
FIG. 2 is a diagram illustrating operating speeds of the first and second cameras of the sensor unit 100 and the third and fourth cameras of the motion unit 200 according to the present invention.
3 is a layout view illustrating the installation of a motion analysis device for golf according to the present invention.
4 is a flowchart illustrating a method of recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention.
5 is a flowchart illustrating a setting step (S1000) in a method for recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention.
6 is a diagram illustrating an internal parameter (M _cam ) that is a calibration setting, rotation information (R _cam ) and position information (t _cam ) that are external parameters.
7 is a view illustrating in detail rotation information (R _cam ) and position information (t _cam ) that are external parameters in the calibration setting of the camera.
FIG. 8 is a diagram illustrating a striking area TZ and a human body area BZ of a golf ball in the image area of the first and second cameras.
9 is a diagram illustrating an initial image frame II of the first, second, third, and fourth cameras.
10 is a flowchart illustrating a method of confirming whether to hit the preparation step (S2000).
11 is a diagram illustrating the preparation image frames RI of the first, second, third, and fourth cameras.
12 is a flowchart illustrating a method for retrieving an accurate trigger frame index (TIDX) in which a blow is detected in detail.
13 is an exemplary diagram for selecting and designating only three trigger image frames TI for detecting a golf ball object BO from 100 buffer image frames BQTY.
14 is a flowchart illustrating a method of creating a foreground object list (L) for every three trigger image frames (TI).
15 is a flowchart illustrating a method of filtering only the golf ball object (BO) in the foreground object list (L).
16 is a diagram illustrating a result of converting three trigger image frames TI into binary image frames and then detecting a golf ball object BO.
17 illustrates a method of calculating a pair of two-dimensional image coordinates p _l (x _l , y _l ) and p _r (x _r , y _r ) as three-dimensional spatial coordinates P(x, y, z) It is a drawing.
18 is a flowchart illustrating a method of selecting two three-dimensional spatial coordinates (P).
19 is a three-dimensional space of two three-dimensional space coordinates (x ₁ , y _{1 ,} z ₁ ), (x ₂ , y _{2 ,} z ₂ ) and a golf ball ready object (RO) ready to hit stored in the database 500 It is a flowchart illustrating a method of calculating the starting point P ₁ , the ending point P ₂ , and the movement time (Δt) of the three-dimensional space vector (V) with coordinates (x _{r ,} y _{r ,} z _r ).
20 is a flowchart illustrating a method of creating a foreground object list (L) for every three trigger image frames (TI).
21 is a flowchart illustrating a method of searching for a golf club shape object.
22 is a view illustrating the result of detecting only the club golf club object (CO) from which the three trigger image frames (TI) are converted into binary image frames and then the golf ball object (BO) and the golf ball shadow object (BS) are removed. to be.
23 shows the distance from P[0 to n], which is a set of coordinates of the outline of the object detected in the images of the first and second cameras of the sensor unit, and the coordinates P2, which is the maximum distance from the coordinates of the center of gravity (C) to the outline of the object. Dmax, which is the distance to the coordinate P1, which is the minimum distance, is Dmin.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, and the inventor should properly understand the concept of the term in order to best describe his invention. Based on the principle that it can be defined, it should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Therefore, the configuration shown in the embodiments and drawings described in the present specification is merely the most preferred embodiment of the present invention and does not represent all of the technical spirit of the present invention, so at the time of the present application, various It should be understood that there may be equivalents and variations.

(motion analysis device for golf)

1 is a system diagram illustrating the structure of a motion analysis apparatus for golf according to the present invention.

As shown in Figure 1, the golf motion analysis device according to the present invention, a first camera 101, a second camera 102 and a strobe lighting device disposed on the upper end of the user to obtain the movement of the golf ball a sensor unit 100 including; a motion unit 200 including a third camera 201 and a fourth camera 202 disposed on the front and side surfaces of the user to obtain an image of the user's whole body motion; a processing unit 300 for storing and processing images acquired by the sensor unit 100 and the motion unit 200; and a display unit 400 that displays the results of the processing unit 300 on the screen and processes the user's input/output signals; and a database 500 that is physically integrated with the processing unit 300 to read and store control settings and calculation results of the device ); includes.

Here, the first camera and the second camera of the sensor unit 100 transmit a photographing shutter signal to a strobe lighting device located outside, and the strobe lighting device determines the photographing timing and timing of the first and second cameras by the photographing shutter signal. Synchronize to control motion.

The processing unit 300 designates a shooting speed of the first camera of the sensor unit 100 . The sensor unit 100 transmits the synchronization signal of the first camera 101 to the remaining second camera 102 and the third and fourth cameras 201 and 202 of the motion unit 200, and the second and third , the fourth camera takes synchronized pictures at the same time and at the same speed as the first camera by the synchronization signal.

In addition, the processing unit 300 may set the shooting speed of the first camera of the sensor unit 100 as a trigger mode to operate to analyze the movement of the golf ball and golf club in the images of the first and second cameras. At this time, the shooting speed of the first camera of the sensor unit 100 operates at 300 frames per second, and the processing unit 300 detects the golf ball object and the golf club object from the image frames of the first and second cameras and analyzes the movement. and the display unit 400 may display the analysis result on the screen in real time.

Also, the processing unit 300 may save the memory used by skipping and storing the images acquired by the third and fourth cameras of the motion unit 200 one frame at a time. For reference, FIG. 2 is a diagram illustrating operating speeds of the first and second cameras of the sensor unit 100 and the third and fourth cameras of the motion unit 200 according to the present invention.

The processing unit 300 sets the shooting speed of the first camera of the sensor unit 100 to the preview mode so that the user can visually check the installation and operation status of the first, second, third, and fourth cameras. can operate to do so. At this time, the shooting speed of the first camera of the sensor unit 100 operates at 60 frames per second, and the display unit 400 displays images of the first, second, third, and fourth cameras on the screen in real time. can

The photographing speed of the camera has been described as an embodiment, and the present technology does not limit the photographing speed of the first camera of the sensor unit 100 to 60 frames per second, or 300 frames per second. For reference, Figure 3 is a layout view illustrating the installation of the motion analysis apparatus for golf according to the present invention.

(How to recognize the movement of golf balls and golf clubs in high-speed camera images)

4 is a flowchart illustrating a method of recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention.

As shown in FIG. 4 , the method for recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention includes a setting step (S1000), a preparation step (S2000), a trigger step (S3000), and a buffering step (S4000). ), a golf ball detection step (S5000), a golf club detection step (S6000), and a golf ball rotation amount calculation step (S7000). Here, the golf ball detection step (S5000) can be performed separately from the golf ball object detection (S51000), the golf ball coordinate calculation (S5200), and the golf ball speed calculation (S5300) in the order, and also the golf club detection step ( In detail, S6000) may be performed in the order of golf club object detection (S61000), golf club coordinate calculation (S6200), and golf club speed calculation (S6300).

(setting step)

5 is a flowchart illustrating a setting step (S1000) in a method for recognizing the movement of a golf ball and a golf club in a high-speed camera image according to the present invention.

As shown in FIG. 5 , in the setting step ( S1000 ) of the method for recognizing the movement of the golf ball and the golf club, the operation mode of the motion analysis apparatus for golf may be divided into “trigger mode” or “preview mode” and set. Here, the “trigger mode” is a mode in which the movement of the golf ball and golf club can be analyzed from the images of the first and second cameras, and the “preview mode” is the installation and operation of the first, second, third, and fourth cameras. It means a mode that operates so that the user can visually check the state.

In detail, the processing unit 300 uses the first and second cameras stored in the database 5000 to analyze the movement of the golf ball and the golf club when operating in the “trigger mode” in the setting step (S1000). Alternatively, a new value may be set (S1100), and the first and second cameras may be set to a calibration set value or a new value stored in the database 5000 (S1200), and the first and second cameras may be set to the database 5000 ) can be set (S1300) to the measurement area setting value or new value stored in the initial image frame (II) or the new initial image frame ( II) can be set (S1400), and the first and second cameras can be set to image processing set values or new values stored in the database 5000 (S1500), and then the first, second, and third , the fourth camera may be set to a camera control setting value stored in the database 5000 or a new value (S1600).

In addition, the processing unit 300, when operating in the “preview mode” in the setting step (S1000), the first, second, third, and fourth cameras in order to acquire an image that can check the installation and operation state of the camera A camera control setting value loaded from the database 5000 or a new value may be set (S1600).

In addition, if there is a change in the setting value loaded from the database 5000 after completing the camera control setting ( S1600 ), the processing unit 300 may store it again in the database 5000 . The processing unit 300 may start the operation of the camera after completing the camera settings ( S1100 to S1600 ).

Next, golf club characteristic setting will be described.

The processing unit 300 classifies the user's golf clubs by individual numbers in the setting step (S1000), "loft (˚)", which means the angle the golf club face makes with the ground, and the average weight of the golf club head “Head weight (kg)”, meaning “expected velocity (㎧)” meaning the average velocity of a golf ball when hitting a golf ball with the golf club, and “speed” meaning the expected movement speed of the golf club head A characteristic setting value of “coefficient (㎧)” may be stored in the database 500 . Table 1 below is a table illustrating characteristic setting values corresponding to individual numbers of golf clubs.

[Table 1]

Table 1 described above is for describing the characteristic setting value of a golf club as an embodiment according to the present invention, and is not necessarily limited or limited thereto.

In addition, the processing unit 300 classifies the user's golf clubs by group numbers in the setting step (S1000) and shoots the golf ball "expected movement speed (㎧)" of the golf ball expected when hitting the golf ball with the corresponding golf club. “Frame time interval-Δti(ms)” means the time interval of the trigger video frame in the high-speed camera video, and “frame” means the start time of the first frame of the trigger video frame in the high-speed camera video that shoots a golf ball When the golf ball moves at the “expected movement speed (㎧)” during the “time offset-Δto(ms)” and “frame time interval-Δti(ms)”, the expected “frame movement distance (mm)” of the golf ball is stored in the database ( 500) can be stored. Table 2 below is a table exemplifying characteristic setting values corresponding to group numbers of golf clubs.

[Table 2]

In Table 2 above, when the first and second cameras of the sensor unit 100 are acquired at a rate of 300 frames per second, the frame time interval Δti and the frame time offset Δto are described. It goes without saying that the frame time interval Δti and the frame time offset Δto in Table 2 may be changed according to the operating speed of the camera, since it is not necessarily limited or limited to the shooting speed of .

Next, the camera calibration settings will be described.

The processing unit 300 may store, in the setting step S1000 , internal parameters M _cam and external parameters of the first and second cameras of the sensor unit 100 in the database 500 . 6 is a diagram illustrating an internal parameter (M _cam ) that is a calibration setting, rotation information (R _cam ) and position information (t _cam ) that are external parameters. In FIG. 6, the camera coordinate system indicates a coordinate system in which the camera's focus is the origin, the front optical axis direction of the camera is the Z axis, the camera downward direction is the Y axis, and the right direction of the camera is the X axis. This refers to a coordinate system with the user's tee position as the origin, the left and right directions as the X-axis, the front-back direction as the Y-axis, and the up-down direction as the Z-axis.

Specifically, the processing unit 300, the internal parameters (M _cam ) of the first camera and the second camera of the sensor unit 100 , the focal length (f _x , f _y ) and the origin coordinates (C _x , C _y ) It is stored in the database 500 . Equation 1 below is a detailed example of the focal length (f _x , f _y ) and origin coordinates (C _x , C _y ) that are internal parameters (M _cam ) in the calibration setting of the camera.

[Equation 1]

In addition, the processing unit 300 stores the rotation information (R _cam ) and the position information (t _cam ), which are external parameters of the first camera and the second camera, in the database 500 . 7 is a view illustrating in detail rotation information (R _cam ) and position information (t _cam ) that are external parameters in the calibration setting of the camera.

Equation 2 below is a detailed example of rotation information (R _cam ) and position information (t _cam ) that are external parameters in the calibration setting of the camera.

[Equation 2]

Here, the world coordinates (tx, ty, tz) means that the position of the camera in the world coordinate system described above with respect to the X-axis (tx), Y-axis (ty), and Z-axis (tz) is expressed in mm units. indicates.

Next, measurement area setting will be described.

FIG. 8 is a diagram illustrating a striking area TZ and a human body area BZ of a golf ball in the image area of the first and second cameras. The processing unit 300 may store the hitting area TZ and the human body area BZ of the golf ball in the database 500 in the setting step S1000 . More specifically, as shown in Figure 8, the processing unit 300, the user's left or right-handed information from the golf ball image acquired by the first camera and the second camera of the sensor unit 100 and the user to use when hitting After grouping golf club numbers into wood and iron or putter, the tee position to hit may be stored in the database 500 as the hitting area TZ of the golf ball in rectangular coordinates. In addition, the processing unit 300 may store the body region BZ in the database 500 as rectangular coordinates of the movement range region of the human body during swing according to the user's left-handed or right-handed information.

Next, an initial image frame setting will be described.

The processing unit 300, in the setting step (S1000) of the method for recognizing the movement of the golf ball and the golf club, an initial image of only the background excluding the golf ball and the user's body having a movement in the images of the first camera to the fourth camera The frame II may be stored in the database 500 . 9 is a diagram illustrating an initial image frame II of the first, second, third, and fourth cameras.

Next, image processing settings will be described.

The processing unit 300 is a preparation step (S2000), a trigger step (S3000), a golf ball detection step (S5000), a golf club detection step (S6000) in the setting step (S1000) of the method for recognizing the movement of the golf ball and the golf club. An image processing setting value applied in the image processing step may be stored in the database 500 .

In more detail, the processing unit 300 may store a binarization threshold value for removing noise from the difference image frame in the preparation step S2000 in the database 500 . Also, the processing unit 300 may store a binarization threshold value for removing noise from the calculated difference image frame in the triggering step S3000 in the database 500 .

Also, the processing unit 300 may store a binarization threshold value for removing noise from the difference image frame of the golf ball detection step S5000 in the database 500 . Also, the processing unit 300 may store a binarization threshold value for removing noise from the difference image frame of the golf club detection step S6000 in the database 500 .

Also, the processing unit 300 may store, in the database 500 , the minimum expected size and the maximum expected size to be detected as the golf ball object BO in the images of the first and second cameras of the sensor unit. The processing unit 300 may store the minimum expected size and the maximum expected size for detecting the golf club object CO in the database 500 in the golf club detection step (S6000).

Also, the processing unit 300 may store, in the database 500 , the minimum circle fitting value to be detected as the golf ball object BO in the images of the first and second cameras of the sensor unit.

23 shows the distance from P[0 to n], which is a set of coordinates of the outline of the object detected in the images of the first and second cameras of the sensor unit, and the coordinates P2, which is the maximum distance from the coordinates of the center of gravity (C) to the outline of the object. Dmax, which is the distance to the coordinate P1, which is the minimum distance, is Dmin.

Referring to FIG. 23 , the circle fitting value is the center point of the detected object when the set of coordinates of the outline of the detected object is P[0 to n] and the coordinate of the center of gravity of the detected object is C. The difference between Dmax, the distance from the coordinates (C) to the coordinates (P2), which is the maximum distance, from the coordinates (C), and Dmin, which is the distance from the coordinates (P1), which is the minimum distance, from the coordinates of the center of gravity (C) to the set of outline coordinates (P[0) ~n]) is the ratio value divided by the average distance Davg. (Circle fitting value = {Dmax - Dmin} / Davg)

Next, camera control settings will be described.

The processing unit 300 may store the control setting values of the first, second, third, and fourth cameras in the database 500 in the setting step S1000 of the method for recognizing the movement of the golf ball and the golf club.

In more detail, the processing unit 300, the first and second cameras 101 and 102 of the sensor unit 100 and the third and fourth cameras 201 and 202 of the motion unit 200 in the setting step (S1000) ) and a brightness gain value indicating the sensitivity of the camera image sensor and the exposure time for controlling the data may be stored in the database 500 . The processing unit 300 may store the operating speeds of the “trigger mode” and “preview mode” of the first camera 101 of the sensor unit 100 in the database 500 in the setting step S1000 .

(preparation stage)

10 is a flowchart illustrating a method of confirming whether to hit the preparation step (S2000). 11 is a diagram illustrating the preparation image frames RI of the first, second, third, and fourth cameras.

In the preparation step (S2000) of the method for recognizing the movement of a golf ball and a golf club according to the present invention, a golf ball object placed in the hitting area (TZ) that detects a golf ball ready to hit only with the sensor unit camera of the golf motion analysis device ( You can check whether you are ready to strike against BO).

Specifically, the processing unit 300 may acquire images of the first and second cameras 101 and 102 of the sensor unit by initializing the previous coordinates BO _p in the preparation step S2000 . And the processing unit 300, based on the user's left or right-handed information, one of the first and second cameras 101 and 102 of the sensor unit 100 is ready to hit the sensor unit 100 It can be pre-selected with a trigger camera (TC) that detects a golf ball. And the processing unit 300, the user's golf club number among the wood hitting area (TZ WOOD), iron hitting area (TZ IRON), putter hitting area (TZ PUTT) of the trigger camera (TC) stored in the database 500 It is possible to preselect the area corresponding to the hitting area (TZ).

The processing unit 300 divides the golf ball image acquired by the trigger camera TC of the sensor unit 100 in the preparation step S2000 into image frame units, and pixels in the divided image frame and the initial image frame II. It is possible to calculate the difference image frame with change. The processing unit 300 may calculate a binary image frame equal to or greater than the binarization threshold value of the preparation step ( S2000 ) stored in the database 500 in order to remove noise from the calculated difference image frame.

The processing unit 300 detects the golf ball object (BO) within the hitting area (TZ) for detecting the golf ball prepared to hit in the calculated binary image frame, and the center point coordinates (BO _p ), the radius (BO _r ), the pixel value (BO _v ) and the current time (BO _t ) can be calculated. And the processing unit 300, if the comparison coordinates (BO _p' ) exist, the coordinates of the center point (BO _p ) of the current object as the comparison coordinates (BO _p' ), and also the current time (BO _t ) to the comparison time (BO) _t' ) can be updated.

The processing unit 300 may calculate a difference image frame having a pixel change in the next consecutive image frame and the initial image frame II acquired by the trigger camera TC of the sensor unit 100 in the preparation step S2000. . The processing unit 300 may calculate a binary image frame that is equal to or greater than the binarization threshold value of the preparation step ( S2000 ) stored in the database 500 in order to remove noise from the calculated difference image frame.

The processing unit 300, the center point coordinate (BO _p ) of the golf ball object (BO) within the hitting area (TZ) for detecting the golf ball prepared to hit in the calculated binary image frame is the distance between the comparison coordinates (BO _p' ) If it is within 2 pixels and the time difference between the current time (BO _t ) and the comparison time (O _t' ) is 1 second or more, it can be confirmed that the golf ball is ready to hit.

When it is confirmed that the golf ball is ready to hit in the preparation step (S2000), the processing unit 300 converts the golf ball object (BO) ready to hit in the preparation step (S2000) to the ready object (RO) as the center point coordinates (R) _p ), a radius (R _r ), and a pixel value (R _v ) are stored in the database 500 . And the processing unit 300, when it is confirmed that the golf ball is ready to hit in the preparation step (S2000) as described above, the first and second cameras of the sensor unit 100 and the third and fourth cameras of the motion unit acquire golf The last frame of the blank image is stored in the database 500 as a ready image frame (RI).

And the processing unit 300, when it is confirmed that the golf ball is ready to hit as described above in the preparation step (S2000), the center point coordinates (R _p ) of the golf ball-ready object (RO) prepared to hit the two-dimensional image coordinates in three-dimensional space The coordinates are calculated and stored in the database 500 . Equation 3 below is an equation illustrating a method of calculating two-dimensional image coordinates as three-dimensional coordinates.

[Equation 3]

Here, M is a 3X3 matrix storing the focal length of the camera and the image sensor origin information, and is the same as Mcam (Intrinsic Matrix) in Equation 1 above. Here, R is a 3X3 matrix storing rotation information about the X-axis, Y-axis, and Z-axis of the camera, and is the same as Rcam (Rotation Matrix) in Equation 2 above. Here, t is a 3X1 matrix storing position information on the X-axis, Y-axis, and Z-axis of the camera, and is the same as tcam (Translation Matrix) in Equation 2 above. Here, the meaning of R and t is illustrated in detail in FIG. 7 .

In addition, since Zconst is a height value in three-dimensional space, it is a value obtained by adding 21mm, which is the radius of a golf ball, to the height of the tee (mm).

(trigger phase)

12 is a flowchart illustrating a method for retrieving an accurate trigger frame index (TIDX) in which a blow is detected in detail.

As shown in Fig. 12, in the trigger step (S3000) of the method for recognizing the movement of a golf ball and a golf club according to the present invention, the golf ball is The trigger frame index (TIDX) at the time of hitting can be searched.

In more detail, the processing unit 300, the center point coordinates (RO _p ) and the radius (RO _r ) of the golf ball ready object (RO) prepared to hit the hitting area (TZ) of the trigger camera (TC) in the trigger step (S3000) , the pixel value (RO _v ) is loaded, and the threshold value (πR _r ² ) for determining that the golf ball-ready object (RO) has been hit is calculated and stored based on the center point coordinates (RO _p ) and the radius (RO _r ).

In addition, the processing unit 300 checks and stores the total number of frames (FQTY) of the image for which the trigger frame index TIDX is to be retrieved from all image frames acquired by the trigger camera TC. And the processing unit 300 stores the start index (SIDX) as 1, the end index (EIDX) as FQTY, the middle index (MIDX) as (EIDX-SIDX)/2, and the middle index (MIDX) in the start index (SIDX) Calculate and store the added search index (KIDX).

The processing unit 300 may calculate the difference image frame with a pixel change by comparing the hitting area TZ of the preparation image frame RI and the hitting area TZ of the image frame corresponding to the search index KIDX. The processing unit 300 may calculate a pixel quantity (PQTY) having a pixel change equal to or greater than the binarization threshold of the trigger step ( S3000 ) stored in the database 500 in the calculated difference image frame.

The processing unit 300, when the calculated pixel quantity (PQTY) is greater than the threshold value (πR _r ² ), changes the end index (EIDX) to the limiting value of the intermediate index (MIDX), and the calculated pixel quantity ( PQTY) is less than or equal to the threshold value πR _r ² , the start index SIDX is changed to a value obtained by adding the intermediate index MIDX.

The processing unit 300 changes the intermediate index (MIDX) and the search index (KIDX) again and calculates the pixel quantity (PQTY) in the difference image of the hitting area (TZ) of the image frame corresponding to the search index (KIDX) The process is repeated as shown in FIG. 12 below. If the value of the intermediate index MIDX is less than or equal to 1, the processing unit 300 terminates the search, stores the search index KIDX as the trigger frame index TIDX, and ends the search.

(buffering step)

13 is an exemplary diagram for selecting and designating only three trigger image frames TI for detecting a golf ball object BO from 100 buffer image frames BQTY.

As shown in Fig. 13, in the buffering step (S4000) of the method for recognizing the movement of the golf ball and golf club according to the present invention, the hit of the golf ball object (BO) is detected in the image of the camera of the golf motion analysis device. Only the trigger image frame TI in which the golf ball object BO is to be detected from the image frame of the trigger frame index TIDX may be selected and buffered.

Specifically, the processing unit 300, from the image frame of the trigger frame index (TIDX) in which the hit of the golf ball object (BO) is detected in the images acquired by the first and second cameras of the sensor unit 100, about 100 sheets Indexes are sequentially assigned to buffer image frames (BQTY), and they are collected and stored. In addition, the processing unit 300, from the image frame of the trigger frame index (TIDX) in which the hit of the golf ball object (BO) is detected in the images acquired from the third and fourth cameras of the motion unit 200, about 100 buffers You can assign indexes to image frames (BQTY) sequentially and collect them and store them, or skip one frame at a time to save storage memory space and assign indexes sequentially to about 50 buffered image frames (BQTY) and collect and store them. have.

The processing unit 300, the frame time offset (Δto) for designating the start time of the trigger image frame (TI) to detect the golf ball object (BO) based on the number of the user golf club in the buffer image frame (BQTY) and A frame time interval Δti designating a time interval of three trigger image frames TI may be specified with reference to Table 2 above.

Herein, as an embodiment of the present technology, the size of the buffer image frame (BQTY) is 100 sheets and the size of the trigger image frame (TI) is described as three sheets, but it is not necessarily limited or limited only to these numbers.

(Golf ball detection step)

In the golf ball detection step (S5000) of the method for recognizing the movement of a golf ball and a golf club according to the present invention, the golf ball object ( The two-dimensional image coordinates of BO are detected, and the three-dimensional space coordinates of the golf ball object BO and the three-dimensional space velocity of the golf ball object BO can be calculated from the two-dimensional image coordinates. Here, the three-dimensional space velocity of the golf ball object BO can be expressed concretely by a moving speed (m/s), a vertical elevation angle (˚), and a horizontal movement angle (˚).

First, golf ball object detection will be described.

14 is a flowchart illustrating a method of creating a foreground object list (L) for every three trigger image frames (TI).

As shown in FIG. 14 , the processing unit 300 changes pixels in the first image frame and the preparation image frame RI of the three trigger image frames TI of the first and second cameras of the sensor unit 100 . Calculate the difference image frame with . In addition, the processing unit 300, the second and third image frames of the three trigger image frames TI of the first and second cameras of the sensor unit 100, and a difference image frame having a pixel change in the first image frame to calculate

The processing unit 300 may calculate a binary image frame equal to or greater than the binarization threshold value of the golf ball detection step S5000 stored in the database 500 in order to remove noise from the calculated difference image frame. The processing unit 300 may create a foreground object list L by grouping pixels having the same value in the calculated binary image frame. However, if the user's golf club number is an iron and a putter club, the golf ball and the golf club interfere with each other in the first trigger image frame TI, so the list L of the foreground object is not created.

The processing unit 300 may filter only one golf ball object BO from the foreground object list L of the created trigger image frame TI. The detailed method of filtering only the golf ball object (BO) in the foreground object list (L) is as follows.

The processing unit 300,

1. In the list of foreground objects (L), objects whose coordinates are within a radius from the coordinates of the golf ball-ready object (RO) ready to hit are removed. (If the detected coordinates of the foreground object are within the area of the ready-to-attack object (RO))

2. In the list of foreground objects (L), objects whose radius of the foreground object is smaller than 1/2 of the radius of the golf ball ready object (RO) ready to hit are removed. (When the radius of the detected foreground object is less than 1/2 of the radius of the ready object (RO) stored in the database 500)

3. In the list of foreground objects (L), an object whose radius of the foreground object is larger than the maximum expected size for detecting the golf ball object stored in the database 500 or smaller than the minimum expected size is removed. (When the radius of the detected object is larger than the maximum expected radius to detect the golf ball object stored in the database 500 or smaller than the minimum expected radius)

4. In the list (L) of the foreground object, the object whose coordinates of the foreground object is located in the opposite direction of the expected progress of the ready object (RO) ready to hit is removed (the coordinates of the detected object are the center point of the ready object (RO) ready to hit) less than the x-coordinate)

5. An object having a circle fitting value of the foreground object greater than a minimum circle fitting value for detecting a golf ball object stored in the database 500 from the list L of foreground objects is removed. In this case, the circle fitting value is a ratio value obtained by dividing the difference between the maximum distance and the minimum distance from the coordinates of the center point of the detected object to the outline by the average distance of the foreground object.

6. In the foreground object list (L), the foreground object with the smallest circle fitting value is recognized as a golf ball object (BO) and stored together with the index number (TIIDX) of the trigger image frame (TI).

For reference, FIG. 15 is a flowchart illustrating a method of filtering only the golf ball object (BO) in the foreground object list (L).

16 is a diagram illustrating a result of converting three trigger image frames TI into binary image frames and then detecting a golf ball object BO. Here, the golf ball object BO detected from the three trigger image frames TI of the first camera is O _l 1, O _l 2, and O _l 3, and is detected from the three trigger image frames TI of the second camera. One golf ball object BO is _Or 1 , O _r 2 , _Or 3 .

Next, the spatial coordinate calculation of the golf ball object will be described.

17 illustrates a method of calculating a pair of two-dimensional image coordinates p _l (x _l , y _l ) and p _r (x _r , y _r ) as three-dimensional spatial coordinates P(x, y, z) It is a drawing.

As shown in FIG. 17 , the processing unit 300 is a golf ball object in a pair of two-dimensional image coordinates p of the golf ball object BO detected by the first camera and the second camera of the sensor unit 100 . The three-dimensional spatial coordinate (P) of (BO) can be calculated and stored together with the index number (TIDX) and restoration error (E) of the corresponding trigger image frame (TI). Here, a method of calculating the restoration error E will be described later.

In detail, the processing unit 300, the frame index (TIDX) of the golf ball object (BO _l ) detected in the trigger image frame (TI) of the first camera of the sensor unit 100, and the trigger image frame of the second camera ( When the frame index (TIDX) of the golf ball object (BO _r ) detected in TI) is the same, that is, when it is photographed at the same time, the two-dimensional image coordinate (p _l ) of the golf ball object (BO _l ) of the first camera ) and a pair of two-dimensional image coordinates (p _l , p _r ) consisting of two-dimensional image coordinates (p _r ) of the golf ball object (BO _r ) of the second camera, as a three-dimensional spatial coordinate of the golf ball object (BO) (P) can be calculated.

Equation 4 below is a method of calculating a pair of two-dimensional image coordinates p _l (x _l , y _l ) and p _r (x _r , y _r ) as three-dimensional spatial coordinates P(x, y, z). This is an example formula.

[Equation 4]

During a golf swing, the golf ball object BO may not be detected in the trigger image frame TI of one of the first camera or the second camera due to the user's body or the club being covered. In this case, the processing unit 300 is not photographed at the same time, that is, the two-dimensional image coordinates p _l of the golf ball object BO _l of the first camera or the golf ball object BO _r of the second camera. Abbreviated three-dimensional spatial coordinates (P) of the golf ball object (BO) with only one two-dimensional image coordinate among the two-dimensional image coordinates (p _r ) of the trigger image frame (TI) index number (TIDX), It can be saved together with the restoration error (E).

In detail, since the processing unit 300 rolls and moves on the floor when the golf club is a putter in the height value of the three-dimensional spatial coordinate (P) of the golf ball, it can be limited to the actual radius of the golf ball, 21 mm. And, when the golf club is not a putter, the pixel size of the golf ball object BO may be restricted to a ratio of 42 mm, which is the actual size. Equation 5 below is an equation illustrating a method of calculating three-dimensional spatial coordinates (X, Y, Z) with only one two-dimensional image coordinate (u, v).

[Equation 5]

The processing unit 300 may calculate a reconstruction error of the three-dimensional spatial coordinate P(x, y, z). In detail, the processing unit 300 converts the three-dimensional spatial coordinates P(x, y, z) of the golf ball object BO calculated as described above to the two-dimensional image coordinates p _l of the golf ball object BO of the first camera. '(u _l , v _l ) or the two-dimensional image coordinate p _r '(u _r , v _r ) of the golf ball object BO of the second camera can be reconstructed and calculated.

The processing unit 300 calculates the distance between the original two-dimensional image coordinates p _l (x _l y _l ) of the first camera and the restored two-dimensional image coordinates p _l '(u _l , v _l ) of the first camera with the restoration error ( |p _l -p _l '|). The processing unit 300 calculates the distance between the original two-dimensional image coordinates p _r (x _r y _r ) of the second camera and the restored two-dimensional image coordinates p _r '(u _r , v _r ) of the second camera as the restoration error of the second camera. It can be calculated as (|p _r -p _r '|).

The processing unit 300 sets a larger value among the restoration error of the first camera (|p _l -p _l '|) and the restoration error of the second camera (|p _r -p _r '|) as the three-dimensional spatial coordinate (P). It can be calculated as the restoration error (E) of Equation 6 below is an equation illustrating a method of calculating three-dimensional spatial coordinates (X _w , Y _w , Z _w ) by restoring them to two-dimensional image coordinates (u, v).

[Equation 6]

As described above, the present technology exemplifies a method of calculating 3D spatial coordinates with only one 2D image coordinate, but the method of calculating 3D spatial coordinates with the above-described pair of 2D image coordinates is a more accurate method and preferential It is self-evident that it should be applied as

Next, the calculation of the velocity of the golf ball object will be described.

The processing unit 300 selects two three-dimensional space coordinates P having a small value of the restoration error E from the stored list L of three-dimensional space coordinates, and calculates a three-dimensional space vector V based thereon. can In addition, the processing unit 300 may express the three-dimensional space vector V by specifying the moving speed (m/s), the vertical elevation angle (˚), and the horizontal movement angle (˚). For reference, FIG. 18 is a flowchart illustrating a method of selecting two three-dimensional spatial coordinates (P).

The processing unit 300, using the three-dimensional space coordinates (x _{r ,} y _{r ,} z _r ) of the golf ball-ready object RO stored in the database 500 as a viewpoint, two three-dimensional spaces selected as described above Two vectors (a, b) with coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) as endpoints can be calculated. The processing unit 300 may determine that the two vectors (a, b) are on a straight line when the angle between them is less than 10 degrees. Here, if the angle between the two vectors (a, b) is not less than 10 degrees, but less than 7 degrees or less than 5 degrees, it can be determined that they are on a straight line, but empirically setting it to 10 degrees desirable.

The processing unit 300, if it is determined that the two vectors (a, b) are on a straight line, two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) It may be determined that the golf ball object BO is correctly selected. Equation 7 below is a formula for calculating the angle between two vectors as a dot product.

[Equation 7]

When it is determined that the processing unit 300 has correctly selected two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) as the golf ball object BO, the selected first 3 Let the dimensional spatial coordinates (x ₁ , y _{1 ,} z ₁ ) be the starting point P ₁ , and the second three-dimensional spatial coordinates (x ₂ , y _{2 ,} z ₂ ) the endpoint P ₂ , and move between P ₂ and P ₁ . A three-dimensional space vector (V) whose magnitude is time (Δt) is calculated. Here, the movement time (Δt) is the time obtained by multiplying the index (TIDX) of the trigger image frame (TI) of P ₁ from the index (TIDX) of the trigger image frame (TI) of P ₂ multiplied by the frame time interval (Δti) (㎳).

When it is determined that the processing unit 300 has not correctly selected two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) as the golf ball object BO, the golf ball ready to hit Let the three-dimensional spatial coordinates (x _r y _r z _r ) of the empty ready object (RO) be the starting point P ₁ , and two spatial coordinates (x ₁ , y ₁ , z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) A three-dimensional space vector (V) with a small value of the center restoration error (E) as the end point P ₂ is calculated, and a movement time (Δt) between P ₂ and P ₁ as the size. Here, the movement time Δt is the time (ms) obtained by multiplying the index number TIDX of the trigger image frame TI of P ₂ by the frame time interval Δti and adding the frame time offset Δto.

In addition, the processing unit 300, by multiplying the calculated three-dimensional space vector (V) by the frame time interval (Δti) to a value obtained by dividing the calculated three-dimensional space vector (V) by the golf ball expected speed (m/s) according to the club number of Table 1 stored in the database 500 Calculate the correction time (δt).

The processing unit 300 adds the correction time δt to the movement time Δt when the calculated three-dimensional space vector V is greater than the expected golf ball velocity (m/s) according to the club number of Table 1 above, and the calculation is performed. When the calculated three-dimensional space vector (V) is smaller than the expected speed (m/s) of the golf ball according to the club number in Table 1, the correction time (δt) is subtracted from the movement time (Δt).

19 is a three-dimensional space of two three-dimensional space coordinates (x ₁ , y _{1 ,} z ₁ ), (x ₂ , y _{2 ,} z ₂ ) and a golf ball ready object (RO) ready to hit stored in the database 500 It is a flowchart illustrating a method of calculating the starting point P ₁ , the ending point P ₂ , and the movement time (Δt) of the three-dimensional space vector (V) with coordinates (x _{r ,} y _{r ,} z _r ).

Equation 8 below is two three-dimensional space coordinates (x ₁ , y _{1 ,} z ₁ ), (x ₂ , y _{2 ,} z ₂ ) and 3 of a golf ball ready object (RO) ready to hit stored in the database 500 This is an equation illustrating a method of calculating the start point P ₁ and the end point P ₂ , and the movement time (Δt) of the three-dimensional space vector (V) with the dimensional space coordinates (x _r y _r z _r ).

[Equation 8]

The processing unit 300 may calculate the moving speed (m/s), the vertical elevation angle (˚), and the horizontal movement angle (˚) of the golf ball based on the three-dimensional space vector (V). Equation 9 below is an equation illustrating a method of calculating the moving speed (㎧), the vertical elevation angle (˚), and the horizontal movement angle (˚) of the golf ball as a three-dimensional space vector (V) of the golf ball object.

[Equation 9]

(Golf club detection stage)

In the golf club detection step (S6000) of the method for recognizing the movement of a golf ball and a golf club according to the present invention, the golf club object ( CO) detects the two-dimensional image coordinates, calculates the three-dimensional space coordinates from the detected two-dimensional image coordinates, and based on this, the three-dimensional space velocity of the golf club object CO can be calculated. Here, the three-dimensional space velocity of the golf club object CO can be expressed in concrete terms as a movement speed (m/s), a vertical elevation angle (˚), and a horizontal movement angle (˚).

First, golf club object detection will be described.

20 is a flowchart illustrating a method of creating a foreground object list (L) for every three trigger image frames (TI).

As shown in FIG. 20 , the processing unit 300 includes the first, second, and third of the three trigger image frames TI of the trigger camera TC among the first and second cameras of the sensor unit 100 . A difference image frame having a pixel change in the second image frame and the preparation image frame RI may be calculated. In addition, the processing unit 300, the first image frame and the initial image frame (II) in the three trigger image frames (TI), not the trigger camera (TC) among the first and second cameras of the sensor unit 100 and in the second and third trigger image frames and in the first trigger image frame, a difference image frame with a pixel change can be calculated.

The processing unit 300 may calculate a binary image frame equal to or greater than the binarization threshold value of the golf club detection step ( S6000 ) stored in the database 500 in order to remove noise from the calculated difference image frame. The processing unit 300 may create a foreground object list L by grouping pixels having the same value in the calculated binary image frame. However, if the user's golf club number is an iron and a putter club, the processing unit 300 does not create a list L of foreground objects because the golf ball and the golf club interfere with each other in the first trigger image frame TI.

21 is a flowchart illustrating a method of searching for a golf club shape object.

As shown in FIG. 21 , the processing unit 300 may filter to only one golf club object CO from the foreground object list L of the created trigger image frame TI. The detailed method of filtering only the golf club object (CO) in the foreground object list (L) is as follows.

The processing unit 300,

1. In the foreground object area of the foreground object list (L), objects within the striking area (TZ) are removed.

2. In the foreground object area of the foreground object list (L), objects within the human body area (BZ) are removed.

3. In the foreground object area of the list of foreground objects (L), objects whose distance is within a radius from the coordinates of the golf ball-ready object (RO) ready to hit are removed.

4. In the foreground object area of the list L of foreground objects, an object whose radius is larger than the maximum expected size for detecting the golf club object stored in the database 500 or smaller than the minimum expected size is removed.

5. In the list of foreground objects (L), a region within a radius of the golf ball object (BO) of the trigger image frame (TI) having the same foreground object region is removed.

6. In the list of foreground objects (L), the area within the radius of the golf ball shadow object (BS) of the trigger image frame (TI) having the same foreground object area is removed.

At this time, the method of obtaining the position of the shadow object (BS) of the golf ball is,

(a) Calculate the two-dimensional image coordinates (u, v) of the golf ball object (BO) of the trigger image frame (TI) as the three-dimensional spatial position (X _w , Y _w , Z _w ) of the shadow object (BS) . At this time, since the height of the shadow object is the same as the floor surface, Z _w = 0, and the method of calculating the three-dimensional space coordinates using one two-dimensional image coordinate is the same as in Equation 5 above.

(b) The three-dimensional spatial position (X _w , Y _w , 0) of the shadow object BS is restored and calculated as the two-dimensional image coordinates of the camera to calculate the two-dimensional image coordinates of the shadow object BS. In this case, the method of restoring the 3D spatial position of the shadow object BS to 2D image coordinates is the same as in Equation 6 described above.

5. From the outline coordinates of the object in the list of foreground objects (L), the pixel distance is calculated from the center coordinates of the human body area (BZ) during the user's swing, and the farthest coordinate is recognized as the golf club object (CO), and the corresponding trigger image frame ( TI) with the index number (TIDX).

22 is a view illustrating the result of detecting only the club golf club object (CO) from which the three trigger image frames (TI) are converted into binary image frames and then the golf ball object (BO) and the golf ball shadow object (BS) are removed. to be. Here, the golf club object (CO) detected from the three trigger image frames (TI) of the first camera is CO _l 1, CO _l 2, and CO _l 3, and it is detected from the three trigger image frames (TI) of the second camera. One golf club object (CO) is CO _r 1 , CO _r 2 , CO _r 3 .

The processing unit 300 is a three-dimensional space of the golf club object CO in a pair of two-dimensional image coordinates p of the golf club object CO detected by the first camera and the second camera of the sensor unit 100 . The coordinates (P) can be calculated and stored together with the index number (TIDX) and the restoration error (E) of the corresponding trigger image frame (TI).

Next, the spatial coordinate calculation of the golf club object will be described.

In detail, the processing unit 300, the frame index ( _TIDX ) of the golf club object (Cl ) detected in the trigger image frame (TI) of the first camera of the sensor unit 100, and the trigger image frame of the second camera ( When the frame index (TIDX) of the golf club object (C _r ) detected in TI) is the same, that is, when it is photographed at the same time, the two-dimensional image coordinate (p _l ) of the golf club object (CO _l ) of the first camera ) and a pair of two-dimensional image coordinates (p _l , p _r ) consisting of two-dimensional image coordinates (p _r ) of the golf club object (CO _r ) of the second camera as a three-dimensional spatial coordinate of the golf club object (CO) (P) can be calculated. Here, the method of calculating a pair of two-dimensional image coordinates p _l (x _l , y _l ) and p _r (x _r , y _r ) with three-dimensional spatial coordinates P(x, y, z) is golf ball detection. The method of FIG. 17 and Equation 4 described above in step S5000 is the same.

The processing unit 300 may calculate a reconstruction error of the three-dimensional spatial coordinate P(x, y, z).

In detail, the processing unit 300 converts the three-dimensional spatial coordinates P(x, y, z) of the golf club object CO calculated as described above to the two-dimensional image coordinates p _l of the golf club object CO of the first camera. '(u _l , v _l ) or the two-dimensional image coordinate p _r '(u _r , v _r ) of the golf club object CO of the second camera can be restored and calculated. The processing unit 300 calculates the distance between the original two-dimensional image coordinates p _l (x _l y _l ) of the first camera and the restored two-dimensional image coordinates p _l '(u _l , v _l ) of the first camera with the restoration error ( |p _l -p _l '|). And the processing unit 300, the original two-dimensional image coordinates p _r (x _r , y _r ) of the second camera and the restored two-dimensional image coordinates p _r '(u _r , v _r ) The distance between the restoration of the second camera It can be calculated as the error (|p _r -p _r '|).

The processing unit 300 sets the three-dimensional spatial coordinate (P) as a larger value among the restoration error (|p _l -p _l '|) of the first camera and the restoration error (|p _r -p _r '|) of the second camera. ) can be calculated as the restoration error (E). Here, the method of calculating the three-dimensional spatial coordinates (X _w , Y _w , Z _w ) by restoring the two-dimensional image coordinates (u, v) is the same as the method of Equation 6 described above in the golf ball detection step (S5000). do.

Next, the calculation of the speed of the golf club object will be described.

The processing unit 300 selects two three-dimensional space coordinates P having a small value of the restoration error E from the stored list L of three-dimensional space coordinates, and calculates a three-dimensional space vector V based thereon. can In addition, the processing unit 300 may express the three-dimensional space vector V by specifying the moving speed (m/s), the vertical elevation angle (˚), and the horizontal movement angle (˚). 18 is a flowchart illustrating a method of selecting two three-dimensional spatial coordinates (P).

In detail, the processing unit 300 selects two three-dimensional space coordinates P having a small value of the restoration error E from the stored list L of three-dimensional space coordinates, and based on this, the three-dimensional space vector V ) can be calculated. Here, a method of selecting two three-dimensional spatial coordinates P is the same as that of FIG. 18 described above in the golf ball detection step (S5000).

The processing unit 300, taking the three-dimensional spatial coordinates (x _{r ,} y _{r ,} z _r ) of the golf ball-ready object RO stored in the database 500 as a starting point, the two three-dimensional space coordinates selected above Two vectors (a, b) can be calculated with endpoints (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ). The processing unit 300 may determine that the two vectors a and b are on a straight line when the angle between them is less than 10 degrees. The processing unit 300, if it is determined that the two vectors (a, b) are on a straight line, golf the two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) It can be determined that the club object (CO) is selected correctly. Here, the method of calculating the angle between the two vectors as the dot product is the same as Equation 7 described above in the golf ball detection step (S5000).

When it is determined that the processing unit 300 has correctly selected two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) as the golf club object CO, the selected first 3 Let the dimensional spatial coordinates (x ₁ , y _{1 ,} z ₁ ) be the starting point P ₁ , and the second three-dimensional spatial coordinates (x ₂ , y _{2 ,} z ₂ ) the endpoint P ₂ , and move between P ₂ and P ₁ . A three-dimensional space vector (V) whose magnitude is time (Δt) is calculated. Here, the movement time (Δt) is the time obtained by multiplying the index (TIDX) of the trigger image frame (TI) of P ₁ from the index (TIDX) of the trigger image frame (TI) of P ₂ multiplied by the frame time interval (Δti) (㎳).

When it is determined that the processing unit 300 has not correctly selected two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y _{2 ,} z ₂ ) as the golf club object CO, the golf ball prepared to hit Let the three-dimensional spatial coordinates (x _{r ,} y _{r ,} z _r ) of the ready object (RO) be the starting point P ₁ , and the two spatial coordinates (x ₁ , y _{1 ,} z ₁ ) and (x ₂ , y ₂ ) _, z ₂ ), the three-dimensional space vector (V) with the small value of the restoration error (E) among the three-dimensional space coordinates as the end point P ₂ , and the movement time (Δt) between P ₂ and P ₁ as the magnitude, is calculated. do. Here, the method of calculating the three-dimensional space vector (V) of the golf club is the same as in FIG. 19 and Equation 8 described above in the golf ball detection step (S5000).

The processing unit 300, based on the three-dimensional space vector (V), can calculate the movement speed (㎧), the vertical elevation angle (˚), and the horizontal movement angle (˚) of the golf club object (CO) after hitting the golf ball have. The method of calculating the movement speed (㎧), the vertical elevation angle (˚), and the horizontal movement angle (˚) of the golf ball as a three-dimensional space vector (V) of the golf club object (CO) after hitting the golf ball is a golf ball detection step. It is the same as Equation 9 described above in (S5000).

The processing unit 300 adds and subtracts the momentum weight to the movement angle (∠cf) of the golf club object (CO) after hitting the golf ball to correct the movement angle (∠ci) of the golf club object (CO) before hitting the golf ball. can Equation 10 below is a calculation formula illustrating a method of calculating the movement angle (∠ci) of the golf club object CO before hitting the golf ball.

[Equation 10]

The processing unit 300 adds and subtracts the momentum weight to the calculated movement speed (ν _cf ) of the golf club object (CO) after hitting the golf ball to the movement speed (ν _ci ) of the golf club object (CO) before hitting the golf ball can be calculated Equation 11 below is a calculation formula illustrating a method of calculating the moving speed (ν _ci ) of the golf club object CO before hitting the golf ball.

[Equation 11]

(Calculating the amount of golf ball rotation)

The processing unit 300 may calculate rotation information of the golf ball object using the calculated three-dimensional space vector V of the golf ball and the golf club end point before the golf ball is struck.

When the processing unit 300 calculates the moving speed (m/s) of the golf club based on the three-dimensional space vector (V), the following Equation 9-1 is

[Equation 9-1]

follow

That is, the method of calculating the movement speed (㎧), the vertical elevation angle (˚), and the horizontal movement angle (˚) of the golf club object CO after the golf ball is struck based on the three-dimensional space vector (V) is a golf ball It is the same as Equation 9 described above in the detection step (S5000).

In addition, the method of calculating the moving speed (ν _ci ) of the golf club object CO before the golf club hits the golf ball is Equation 11

follow

As a result, as one method of calculating the amount of golf ball rotation, Equation 12 below

[Equation 12]

can be calculated according to

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited thereto, and the technical spirit of the present invention and the following by those of ordinary skill in the art to which the present invention pertains. It goes without saying that various modifications and variations are possible within the equivalent scope of the claims to be described.

100...sensor part
101...first camera
102...second camera
200...Motion
201...3rd camera
202...4th camera
300...processor
400...display
500...database

Claims (9)

A sensor unit 100 including a first camera 101, a second camera 102, and a strobe lighting device disposed on the user's upper end to obtain the movement of the golf ball;
a motion unit 200 including a third camera 201 and a fourth camera 202 disposed on the front and side surfaces of the user to acquire a full-body motion image of the user;
a processing unit 300 for storing and processing images acquired by the sensor unit 100 and the motion unit 200;
a display unit 400 for displaying the result of the processing unit 300 on a screen and processing the input/output signal of the user; and
A database 500 that is physically integrated with the processing unit 300 to read and store control settings and calculation results of the device;
A motion analysis device for golf.
A method of recognizing the movement of a golf ball and a golf club in a high-speed camera image using the motion analysis device for golf according to claim 1,
Set the golf club characteristics (S1100), set the camera calibration (S1200), set the measurement area (S1300), set the initial image frame (S1400), set the image processing setting (S1500), and control the camera Setting (S1600), setting step (S1000);
A preparation step (S2000) of confirming whether or not to hit the golf ball object placed in the hitting area (TZ);
A trigger step (S3000) of searching for a trigger frame index (TIDX) at the point in time at which the golf ball is hit in the entire image frame;
Buffering step ( S4000);
The two-dimensional image coordinates of the golf ball object BO are detected from the trigger image frame TI of the first camera and the second camera of the sensor unit 100, and the golf ball object BO from the detected two-dimensional image coordinates. ) calculating the three-dimensional space coordinates of the three-dimensional space coordinates and the three-dimensional space velocity of the golf ball object (BO), golf ball detection step (S5000);
Detect the two-dimensional image coordinates of the golf club object CO from the trigger image frame TI of the first camera and the second camera of the sensor unit 100, and the golf club object CO from the detected two-dimensional image coordinates ) of calculating the three-dimensional spatial coordinates and the three-dimensional space velocity of the golf club object (CO), a golf club detection step (S6000); and
A golf ball rotation amount calculation step (S7000) of calculating the rotation information of the golf ball object with the three-dimensional space vector (V) of the calculated golf ball and the end point of the golf club before hitting the golf ball, including;
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the setting step (S1000), a trigger mode, which is a mode capable of analyzing the movement of a golf ball and a golf club in the images of the first camera and the second camera, and the installation of the first, second, third, and fourth cameras and a preview mode, which is a mode that operates so that the user can visually check the operating state,
When operating in the trigger mode,
setting the first and second cameras to a golf club characteristic setting value or a new value stored in a database 5000 in order to analyze the movement of the golf ball and the golf club (S1100); setting the first and second cameras to a calibration setting value or a new value stored in the database (5000) (S1200); setting (S1300) the first and second cameras to a measurement area setting value or a new value stored in the database (5000); setting the first, second, third, and fourth cameras as an initial image frame (II) or a new initial image frame (II) stored in the database (5000) (S1400); setting (S1500) the first and second cameras to image processing set values or new values stored in the database (5000); and then setting the first, second, third, and fourth cameras to the camera control set value or new value stored in the database 5000 (S1600);
When operating in the preview mode,
In order to acquire an image that can confirm the installation and operation status of the camera, the first, second, third, and fourth cameras are set to the camera control set value loaded from the database 5000 or a new value (S1600) including;
In the camera calibration setting step (S1200), the focal length (f _x , f _y ) and the origin coordinates (C _x , C _y ), which are internal parameters (M _cam ) of the first camera and the second camera, and external parameters In rotation information (R _cam ) and location information (t _cam ) is stored in the database 500,
In the measurement area setting step (S1300), the hitting area (TZ) and the human body area (BZ) of the golf ball are stored in the database 500,
In the initial image frame setting step (S1400), an initial image frame obtained by photographing only the background excluding the moving golf ball and the user's body from the images of the first to fourth cameras is stored in the database 500,
In the image processing setting step (S1500), the image processing applied in the image processing steps of the preparation step (S2000), the trigger step (S3000), the golf ball detection step (S5000), and the golf club detection step (S6000) Store the set value in the database 500,
In the camera control setting step (S1600), the first and second cameras 101 and 102 of the sensor unit 100 and the third and fourth cameras 201 and 202 of the motion unit 200 are controlled. characterized in that the brightness gain value, which means the exposure time and the sensitivity of the camera image sensor, is stored in the database (500),
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the preparation step (S2000), when it is confirmed that the golf ball is ready to hit, the center point coordinates (R _p ), the radius (R _r ), and the pixel value (R _v ) are the ready object (RO) ready to hit the golf ball object (BO). ) is stored in the database 500, the first camera and the second camera of the sensor unit 100, and the third camera and the fourth camera of the motion unit prepare the last frame of the golf ball image acquired by the image Stored in the database 500 as a frame (RI),
In the trigger step (S3000),
Load the center point coordinate (RO _p ), radius (RO _r ), and pixel value (RO _v ) of the golf ball ready object (RO) ready to hit in the hitting area (TZ), and load the center point coordinate (RO _p ) and radius (RO _r ) On the basis of , calculate and store the binarization threshold value (πR _r ² ) to determine that the golf ball-ready object (RO) has been hit,
The processing unit 300 checks and stores the total number of frames (FQTY) of the image for which the trigger frame index (TIDX) is to be retrieved from the entire image frame, the start index (SIDX) is 1, and the end index (EIDX) is FQTY. , save the intermediate index (MIDX) as (EIDX-SIDX)/2, calculate and store the search index (KIDX) by adding the middle index (MIDX) to the starting index (SIDX),
Comparing the striking area (TZ) of the preparation image frame (RI) and the striking area (TZ) of the image frame corresponding to the search index (KIDX) calculates the difference image frame with a pixel change, the calculated difference image frame Calculate the pixel quantity (PQTY) with a pixel change greater than or equal to the binarization threshold (πR _r ² ) of the trigger step (S3000) stored in the database 500 to remove noise from
When the calculated pixel quantity (PQTY) is greater than the binarization threshold (πR _r ² ), the ending index (EIDX) is changed to an intermediate index (MIDX), and the calculated pixel quantity (PQTY) is the binarization threshold ( If less than or equal to πR _r ² ), change the starting index (SIDX) to the original starting index plus the middle index (MIDX),
Change the intermediate index (MIDX) and the search index (KIDX) again and repeat the process of calculating the pixel quantity (PQTY) in the difference image of the hitting area (TZ) of the image frame corresponding to the search index (KIDX), When the value of the index (MIDX) is less than or equal to 1, the search is terminated, the search index (KIDX) is stored as the trigger frame index (TIDX), and the search is terminated,
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the golf ball detection step (S5000),
The two-dimensional image coordinates (p _l (x _l , y _l )) of the golf ball object (BO _l ) of the first camera and the two-dimensional image coordinates p _r (x) of the golf ball object (BO _r ) of the second camera When calculating the three-dimensional spatial coordinates (P(x, y, z)) of the golf ball object BO with a pair of two-dimensional image coordinates (p _l , p _r ) consisting of _r , y _r ), the following equation 4 people

follow,
When the user's golf swing, the golf ball object (BO) is not detected in the trigger image frame (TI) of one of the first camera or the second camera due to the occlusion of the user's body or the golf club, One of the two-dimensional image coordinates (p _l ) of the golf ball object (BO _l ) of the first camera or the two-dimensional image coordinates (p _r ) of the golf ball object (BO _r ) of the second camera When calculating the three-dimensional space coordinate (P) of the golf ball object (BO) with only the following Equation 5

characterized by following
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the golf club detection step (S6000),
A foreground object list L is created for each trigger image frame TI, and the processing unit 300, when the user's golf club number is an iron and a putter club, in the first trigger image frame TI, a golf ball and a golf club Since they interfere with each other, the list (L) of the foreground object is not created,
After converting the trigger image frame (TI) into a binary image frame, only the club golf club object (CO) from which the golf ball object (BO) and the golf ball shadow object (BS) are removed,
In calculating the two-dimensional image coordinates for removing the golf ball shadow object (BS), the following Equation 6,

characterized by following
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
7. The method according to claim 5 or 6,
The detected two-dimensional image coordinates of the golf ball object BO or the golf club object CO are detected, and the three-dimensional space of the golf ball object BO or the golf club object CO is detected from the detected two-dimensional image coordinates. In calculating the coordinates and the three-dimensional space velocity of the golf ball object (BO) or the golf club object (CO), the three-dimensional space of the golf ball object (BO) or the golf club object (CO) and the golf ball-ready object (RO) When calculating the angle between coordinates,
Equation 7 below

follow,
When the angle between the calculated by Equation 7 exceeds 10°, it is determined that the calculated three-dimensional spatial coordinates are not valid, and the two-dimensionality of the detected golf ball object (BO) or golf club object (CO) is In the method of calculating the three-dimensional movement speed of a golf ball or golf club even if the image coordinates are not detected for two or more consecutive frames,
Equation 8

characterized by following
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the golf ball rotation amount calculation step (S7000),
When the processing unit 300 calculates the moving speed (m/s) of the golf club based on the three-dimensional space vector (V), the following Equation 9-1 is

follow,
Equation 11 is a method of calculating the moving speed (ν _ci ) of the golf club object (CO) before the golf club hits the golf ball

follow,
The golf ball rotation amount calculation step (S7000), Equation 12

characterized in that it is calculated according to
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.
3. The method of claim 2,
In the buffering step (S4000),
The processing unit 300, from the image frame of the trigger frame index (TIDX) in which the hit of the golf ball object (BO) is detected in the images acquired by the first camera and the second camera of the sensor unit 100, 100 sheets or more buffer image frames (BQTY) are sequentially assigned indexes, collected and stored,
The processing unit 300, from the image frame of the trigger frame index (TIDX) in which the hit of the golf ball object (BO) is detected in the images acquired from the third and fourth cameras of the motion unit 200, 100 sheets Or more buffer image frames (BQTY) are sequentially indexed and aggregated and stored, or 50 or more buffer image frames (BQTY) are sequentially indexed by skipping one frame at a time to save storage memory space. and collect them and store them,
The processing unit 300, the frame time offset (Δto) for designating the start time of the trigger image frame (TI) to detect the golf ball object (BO) based on the number of the user golf club in the buffer image frame (BQTY) ) and the frame time interval (Δti) specifying the time interval of the three trigger image frames (TI) in Table 2 below.

specified according to
The W1 and W5 represent Wood 1 and Wood 5,
I3, I8, and I9 represent iron 3, iron 8, and iron 9,
PW stands for pitching wedge,
S2 indicates the sand wedge used for a distance of 50m to 25m,
A1 represents the approach used for a distance of 25m to 10m,
PT is characterized in that it represents a putter,
A method for recognizing the movement of golf balls and golf clubs in high-speed camera images.

Images