CN116152708B - Method and device for extracting effective actions of golf item, storage medium and equipment - Google Patents
Method and device for extracting effective actions of golf item, storage medium and equipment Download PDFInfo
- Publication number
- CN116152708B CN116152708B CN202310018934.7A CN202310018934A CN116152708B CN 116152708 B CN116152708 B CN 116152708B CN 202310018934 A CN202310018934 A CN 202310018934A CN 116152708 B CN116152708 B CN 116152708B
- Authority
- CN
- China
- Prior art keywords
- frame
- golf
- index list
- frame index
- list
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for extracting effective actions of golf items, a storage medium and equipment, wherein the method comprises the following steps: identifying human body key points of a golf item human body in each frame of video image of a video frame list by using a preset human body posture estimation model; traversing the result list, and generating a preparation frame index list meeting the preset golf preparation frame condition and an ending frame index list meeting the preset golf ending frame condition; selecting a target preparation frame index list and a target ending frame index list corresponding to the golf item effective action from the preparation frame index list and the ending frame index list based on prior constraint conditions; and generating a plurality of sets of golf item video sequence frame index combinations according to the corresponding combination of elements in the target preparation frame index list and the target ending frame index list, and obtaining the golf item effective action video. The invention can automatically extract the effective action sub-video sequence, and saves the complicated steps of manual extraction.
Description
Technical Field
The invention relates to the technical field of image vision, in particular to a method and device for extracting effective actions of golf projects, a storage medium and equipment.
Background
In recent years, with the development of image video technology and deep learning, applications in the field of image vision have been put together. The extraction of video sequences with repetitive motion from long videos is also a research hotspot in recent years, and in the prior art, the video sequences to be acquired are manually clipped by using a video clipping tool, so that the manual extraction step is complicated.
Disclosure of Invention
The present invention has been made in view of the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus, a storage medium, and a device for efficient action extraction of golf items, which overcome or at least partially solve the above-mentioned problems.
In one aspect of the present invention, there is provided a golf item effective action extraction method, the method comprising:
acquiring a video frame list;
traversing the video frame list, and identifying human body key points of a golf item human body in each frame of video image of the video frame list by using a preset human body posture estimation model to obtain a result list;
traversing the position distribution relation among the key points of the human body corresponding to each frame of video image in the result list, and generating a preparation frame index list meeting the preset golf preparation frame condition and an end frame index list meeting the preset golf end frame condition;
selecting a target preparation frame index list and a target ending frame index list corresponding to the golf item effective action from the preparation frame index list and the ending frame index list based on prior constraint conditions;
generating multiple sets of golf item video sequence frame index combinations according to the corresponding combinations of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring corresponding golf item effective action videos according to the sets of golf item video sequence frame index combinations.
Further, the acquiring the video frame list includes: and acquiring a video stream to be analyzed, and performing frame cutting processing on the video stream in a frame skipping mode to obtain a video frame list.
Further, the identifying the human body key points of the golf action human body in each frame of video image of the video frame list by using a preset human body posture estimation model to obtain a result list includes:
identifying point location information of a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left crotch, a right crotch, a left knee, a right knee, a left ankle and a right ankle of a golf action human body in each frame of video image of the video frame list by using a preset human body posture estimation model, and obtaining an identification result;
and packaging the bit data in the middle point of the identification result through an array to generate a result list.
Further, the human body posture estimation model is a human body key point position prediction model based on a single person.
Further, the preset golf preparation frame conditions include: the included angle between the straight line and the horizontal line, which are determined by the left shoulder coordinate point and the right shoulder coordinate point, is within a preset angle range; the included angle between the straight line and the horizontal line, which are determined by the left crotch coordinate point and the right crotch coordinate point, is within a preset angle range; the positions of the left foot and the right foot in the image coordinate system are the same as the positions of the left shoulder and the right shoulder in the image coordinate system in width; the X value of the left wrist coordinate is smaller than the X value of the left crotch coordinate and the X value of the right wrist coordinate is larger than the X value of the right crotch coordinate; the Y value of the left wrist coordinate is larger than the Y value of the double-span center point coordinate, and the Y value of the right wrist coordinate is larger than the Y value of the double-span center point coordinate;
the preset golf end frame condition includes: the X value of the left shoulder coordinate is smaller than that of the right shoulder coordinate; the wrist of both hands is above both shoulders, and the Y value of left wrist coordinate is less than the Y value of both shoulders central point coordinate and the Y value of right wrist coordinate is less than the Y value of both shoulders central point coordinate.
Further, the prior constraints include a first prior constraint and/or a second prior constraint;
the first a priori constraint is: the first time difference value is smaller than the second time difference value, wherein the first time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the upswing action frame, and the second time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the next preparation frame;
the second prior constraint is: the execution time of a complete set of golf item actions is within a preset time range.
Further, the selecting, based on the prior constraint condition, a target preparation frame index list and a target ending frame index list corresponding to the golf item valid action from the preparation frame index list and the ending frame index list includes:
subtracting two adjacent elements in the prepared frame index list based on a priori constraint condition, wherein when the subtraction result is larger than or equal to the interval frame number, the latter element is the prepared frame of the current golf action, a target prepared frame index list corresponding to the effective action of the golf item is generated according to the prepared frame, and when the subtraction result is smaller than the interval frame number, the last frame in the prepared frame index list is the prepared frame of the golf action;
and subtracting the current element in the ending frame index list from the element in the target preparation frame index list corresponding to the golf item effective action, and generating a target ending frame index list corresponding to the golf item effective action according to the ending frame when the subtraction result is minimum and the addition result of the element in the target preparation frame index list corresponding to the golf item effective action and the interval frame number is smaller than or equal to the current element in the ending frame index list.
In a second aspect of the present invention, there is provided a golf item effective action extracting apparatus, the apparatus comprising:
the acquisition module is used for acquiring a video frame list;
the identification module is used for traversing the video frame list, and utilizing a preset human body posture estimation model to identify human body key points of a golf item human body in each frame of video image of the video frame list so as to obtain a result list;
the traversing module is used for traversing the position distribution relation between the key points of the human body corresponding to each frame of video image in the result list and generating a preparation frame index list meeting the preset golf preparation frame condition and an end frame index list meeting the preset golf end frame condition;
the selecting module is used for selecting a target preparation frame index list and a target ending frame index list corresponding to the golf item effective action from the preparation frame index list and the ending frame index list based on prior constraint conditions;
the generating module is used for generating a plurality of sets of golf item video sequence frame index combinations according to the corresponding combination of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring corresponding golf item effective action videos according to each set of golf item video sequence frame index combinations.
In another aspect of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the golf item effective action extraction method as above.
In yet another aspect of the present invention, there is also provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the golf item effective action extraction method as above when the computer program is executed.
According to the method, the device, the storage medium and the equipment for extracting the effective actions of the golf item, provided by the embodiment of the invention, the action gesture analysis of the golf item is performed through the human body key point detection model, the actions of the preparation frame and the ending frame are extracted by utilizing the prior constraint condition, the video frame index is obtained, a plurality of complete golf action video sequences are automatically extracted from the long video, other invalid video sequences are filtered, the image video processing technology is fully utilized, the effective action sub-video sequences are automatically extracted, and the complicated steps of manual extraction are saved. The invention has the advantages of high efficiency, automation and high accuracy.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a method for extracting effective actions of a golf item according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a golf preparation frame according to an embodiment of the present invention;
FIG. 3 is a schematic view of a golf end frame according to an embodiment of the present invention;
FIG. 4 is a flowchart of another method for extracting golf effective actions according to an embodiment of the present invention;
fig. 5 is a block diagram of an effective action extracting device for golf according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Fig. 1 schematically shows a flow chart of a golf item valid action extraction method according to an embodiment of the present invention. Referring to fig. 1, the method for extracting golf effective actions according to the embodiment of the invention specifically includes the following steps:
s11, acquiring a video frame list;
further, the acquiring the video frame list includes: and acquiring a video stream to be analyzed, and performing frame cutting processing on the video stream in a frame skipping mode to obtain a video frame list.
In this embodiment, in order to improve the prediction efficiency, the video stream is subjected to frame slicing in a frame skipping manner, s=fps/30 (fps > =30), S is a frame interval, and fps is a frame rate of the current video.
S12, traversing the video frame list, and identifying human body key points of a golf item human body in each frame of video image of the video frame list by using a preset human body posture estimation model to obtain a result list;
further, the identifying the human body key points of the golf action human body in each frame of video image of the video frame list by using a preset human body posture estimation model to obtain a result list includes:
identifying point location information of a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left crotch, a right crotch, a left knee, a right knee, a left ankle and a right ankle of a golf action human body in each frame of video image of the video frame list by using a preset human body posture estimation model, and obtaining an identification result;
and packaging the bit data in the middle point of the identification result through an array to generate a result list. In this embodiment, array encapsulation is used for the point location data identified by each frame of video image, and a result list is generated so as to write into the memory for storage.
Further, the human body posture estimation model is a human body key point position prediction model based on a single person.
In this embodiment, the human body posture estimation model is a human body key point prediction model based on a single person, for example mediapipe, movenet or other single person posture estimation models.
S13, traversing the position distribution relation among the key points of the human body corresponding to each frame of video image in the result list, and generating a preparation frame index list meeting the preset golf preparation frame conditions and an end frame index list meeting the preset golf end frame conditions;
further, the preset golf preparation frame conditions include: the included angle between the straight line and the horizontal line, which are determined by the left shoulder coordinate point and the right shoulder coordinate point, is within a preset angle range; the included angle between the straight line and the horizontal line, which are determined by the left crotch coordinate point and the right crotch coordinate point, is within a preset angle range; the positions of the left foot and the right foot in the image coordinate system are the same as the positions of the left shoulder and the right shoulder in the image coordinate system in width; the X value of the left wrist coordinate is smaller than the X value of the left crotch coordinate and the X value of the right wrist coordinate is larger than the X value of the right crotch coordinate; the Y value of the left wrist coordinate is larger than the Y value of the double-span center point coordinate, and the Y value of the right wrist coordinate is larger than the Y value of the double-span center point coordinate;
the preset golf end frame condition includes: the X value of the left shoulder coordinate is smaller than that of the right shoulder coordinate; the wrist of both hands is above both shoulders, and the Y value of left wrist coordinate is less than the Y value of both shoulders central point coordinate and the Y value of right wrist coordinate is less than the Y value of both shoulders central point coordinate.
In the embodiment, the horizontal included angle between the left shoulder and the right shoulder is in the range of [0,5], and the horizontal included angle between the left span and the right span is in the range of [0,5 ]; the double wrist point is positioned in the middle and below the two spans;
preparing a frame index list of [ s_id1, s_id2, s_id3, s_idn … ], wherein idnum is a video frame index, and s represents start; the end frame index list is [ e_id1, e_id2, e_id3, e_idn … ], where idnum is the video frame index and e represents end.
S14, selecting a target preparation frame index list and a target ending frame index list corresponding to the golf item effective action from the preparation frame index list and the ending frame index list based on prior constraint conditions;
further, the prior constraints include a first prior constraint and/or a second prior constraint;
the first a priori constraint is: the first time difference value is smaller than the second time difference value, wherein the first time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the upswing action frame, and the second time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the next preparation frame;
the second prior constraint is: the execution time of a complete set of golf item actions is within a preset time range.
Further, the selecting, based on the prior constraint condition, a target preparation frame index list and a target ending frame index list corresponding to the golf item valid action from the preparation frame index list and the ending frame index list includes:
subtracting two adjacent elements in the prepared frame index list based on a priori constraint condition, wherein when the subtraction result is larger than or equal to the interval frame number, the latter element is the prepared frame of the current golf action, a target prepared frame index list corresponding to the effective action of the golf item is generated according to the prepared frame, and when the subtraction result is smaller than the interval frame number, the last frame in the prepared frame index list is the prepared frame of the golf action;
and subtracting the current element in the ending frame index list from the element in the target preparation frame index list corresponding to the golf item effective action, and generating a target ending frame index list corresponding to the golf item effective action according to the ending frame when the subtraction result is minimum and the addition result of the element in the target preparation frame index list corresponding to the golf item effective action and the interval frame number is smaller than or equal to the current element in the ending frame index list.
In this embodiment, in a golf course, there is a swing, and the striking operation is performed after the preparation-putting-preparation-putting cycle operation is performed a plurality of times, but the swing cannot be counted as the golf course operation, and there is a frame satisfying the preparation frame condition in the swing, and the preparation frame index in the swing must be removed.
A complete set of golf actions including a preparation-backswing-half swing-full swing-downswing-batting-backswing-take-up, a complete set of golf actions being performed for about 1-2s with an interval of frame number f=2 x fps;
the prepared frame index list s_id1, s_id2, s_id3, s_idn … is traversed, when s_idn+1-s_idn > =f, s_idn is the prepared frame of the current golf action, a target prepared frame index list s_id= [ s_id1, s_id2, s_id3, s_idn … ] satisfying the condition s_idn-1> =f is generated, and if the condition s_idn+1-s_idn > =f is not satisfied, the last frame in the s_id list is the prepared frame, for example s_id= [1,2,3,10,15,100,101,110,200], s_id= [15,200].
And traversing the ending frame index lists [ e_id1, e_id2, e_id3, e_idn … ], wherein the difference between the current element of the list and the S_id is the smallest and the S_id+2 x fps < = e_id is satisfied, the current e_id is the ending frame, and a target ending frame list E_id= [ E_id1, E_id2, E_id3, E_idn, … ] is generated, for example, e_id= [75,76,77,261,262,263], and E_id= [75,261 ].
S15, generating a plurality of sets of golf item video sequence frame index combinations according to the corresponding combination of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring corresponding golf item effective action videos according to each set of golf item video sequence frame index combinations.
In this embodiment, if the target preparation frame index list s_id and the target ending frame index list e_id are the complete multiple sets of golf item video sequence frames in the long video are [ s_id [0], e_id [0] ], [ s_id [1], e_id [1] ], [ s_id [2], e_id [2] ] …, e.g. s_id= [15,200], e_id= [75,261], then golf item 1 is [15,75], and golf item 2 is [200,261]. And generating a plurality of sets of golf item video sequence frame index combinations according to the corresponding combination of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring the corresponding golf item effective action video so as to remove other ineffective video sequences.
Fig. 2 is a schematic diagram of a golf preparation frame according to an embodiment of the present invention.
Fig. 3 is a schematic view of a golf end frame according to an embodiment of the present invention.
According to the method, the device, the storage medium and the equipment for extracting the effective actions of the golf item, provided by the embodiment of the invention, the action gesture analysis of the golf item is performed through the human body key point detection model, the actions of the preparation frame and the ending frame are extracted by utilizing the prior constraint condition, the video frame index is obtained, a plurality of complete golf action video sequences are automatically extracted from the long video, other invalid video sequences are filtered, the service application advantage is improved, the image video processing technology is fully utilized, the sub video sequences are automatically extracted, and the complicated steps of manual extraction are saved. The invention has the advantages of high efficiency, automation and high accuracy.
FIG. 4 is a flowchart of another method for extracting golf effective actions according to an embodiment of the present invention; as shown in fig. 4, multiple complete sets of golf action video sequences are automatically extracted from the long video, and other invalid video sequences are filtered out.
For the purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated by one of ordinary skill in the art that the methodologies are not limited by the order of acts, as some acts may, in accordance with the methodologies, take place in other order or concurrently. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Fig. 5 schematically shows a schematic structure of a golf item effective action drawing device according to an embodiment of the present invention. Referring to fig. 5, the golf item effective action extracting apparatus according to the embodiment of the present invention specifically includes an obtaining module 501, an identifying module 502, a traversing module 503, a selecting module 504, and a generating module 505, where:
an obtaining module 501, configured to obtain a video frame list;
the identifying module 502 is configured to traverse the video frame list, identify key points of a golf item in each frame of video image of the video frame list by using a preset human body posture estimation model, and obtain a result list;
a traversing module 503, configured to traverse the position distribution relationship between the key points of the human body corresponding to each frame of video image in the result list, and generate a preparation frame index list that meets a preset golf preparation frame condition and an end frame index list that meets a preset golf end frame condition;
a selection module 504, configured to select, based on a priori constraint conditions, a target preparation frame index list and a target ending frame index list corresponding to a golf item valid action from the preparation frame index list and the ending frame index list;
the generating module 505 is configured to generate multiple sets of golf item video sequence frame index combinations according to corresponding combinations of elements in the target preparation frame index list and elements in the target ending frame index list, and obtain corresponding golf item effective action videos according to each set of golf item video sequence frame index combinations.
Further, the obtaining module 501 is configured to obtain a video stream to be analyzed, and perform frame slicing processing on the video stream in a frame skipping manner to obtain a video frame list.
Further, the identifying module 502 is configured to identify, using a preset human body posture estimation model, point location information of a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left crotch, a right crotch, a left knee, a right knee, a left ankle, and a right ankle of a human body performing golf action in each frame of video image of the video frame list, so as to obtain an identification result;
and packaging the bit data in the middle point of the identification result through an array to generate a result list.
Further, the human body posture estimation model is a human body key point position prediction model based on a single person.
Further, the preset golf preparation frame conditions include: the included angle between the straight line and the horizontal line, which are determined by the left shoulder coordinate point and the right shoulder coordinate point, is within a preset angle range; the included angle between the straight line and the horizontal line, which are determined by the left crotch coordinate point and the right crotch coordinate point, is within a preset angle range; the positions of the left foot and the right foot in the image coordinate system are the same as the positions of the left shoulder and the right shoulder in the image coordinate system in width; the X value of the left wrist coordinate is smaller than the X value of the left crotch coordinate and the X value of the right wrist coordinate is larger than the X value of the right crotch coordinate; the Y value of the left wrist coordinate is larger than the Y value of the double-span center point coordinate, and the Y value of the right wrist coordinate is larger than the Y value of the double-span center point coordinate;
the preset golf end frame condition includes: the X value of the left shoulder coordinate is smaller than that of the right shoulder coordinate; the wrist of both hands is above both shoulders, and the Y value of left wrist coordinate is less than the Y value of both shoulders central point coordinate and the Y value of right wrist coordinate is less than the Y value of both shoulders central point coordinate.
Further, the prior constraints include a first prior constraint and/or a second prior constraint;
the first a priori constraint is: the first time difference value is smaller than the second time difference value, wherein the first time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the upswing action frame, and the second time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the next preparation frame;
the second prior constraint is: the execution time of a complete set of golf item actions is within a preset time range.
Further, the selecting module 504 is configured to subtract two adjacent elements in the prepared frame index list based on a priori constraint condition, when the subtraction result is greater than or equal to an interval frame number, the next element is a prepared frame of the current golf action, generate a target prepared frame index list corresponding to the valid golf item action according to the prepared frame, and when the subtraction result is less than the interval frame number, the last frame in the prepared frame index list is a prepared frame of the golf action; and subtracting the current element in the ending frame index list from the element in the target preparation frame index list corresponding to the golf item effective action, adding the element in the target preparation frame index list corresponding to the golf item effective action with the interval frame number when the subtraction result is minimum, and generating a target ending frame index list corresponding to the golf item effective action according to the ending frame when the addition result is smaller than or equal to the current element in the ending frame index list, wherein the current element is the ending frame.
Furthermore, embodiments of the present invention provide a computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements the steps of the method as described above.
In this embodiment, the modules/units integrated with the golf item valid action extraction device may be stored in a computer readable storage medium if implemented in the form of software functional units and sold or used as a stand alone product. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
In addition, the embodiment of the invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the method when executing the program. Such as steps S11-S15 shown in fig. 1. Alternatively, the processor may implement the functions of the modules/units in the embodiment of the golf item valid action extraction device described above when executing the computer program, such as the obtaining module 501, the identifying module 502, the traversing module 503, the selecting module 504, and the generating module 505 shown in fig. 5.
According to the method, the device, the storage medium and the equipment for extracting the effective actions of the golf item, provided by the embodiment of the invention, the action gesture analysis of the golf item is performed through the human body key point detection model, the actions of the preparation frame and the ending frame are extracted by utilizing the prior constraint condition, the video frame index is obtained, a plurality of complete golf action video sequences are automatically extracted from the long video, other invalid video sequences are filtered, the service application advantage is improved, the image video processing technology is fully utilized, the sub video sequences are automatically extracted, and the complicated steps of manual extraction are saved. The invention has the advantages of high efficiency, automation and high accuracy.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, any of the claimed embodiments can be used in any combination.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. A method of active action extraction for a golf item, the method comprising:
acquiring a video frame list;
traversing the video frame list, and identifying human body key points of a golf item human body in each frame of video image of the video frame list by using a preset human body posture estimation model to obtain a result list;
traversing the position distribution relation among the key points of the human body corresponding to each frame of video image in the result list, and generating a preparation frame index list meeting the preset golf preparation frame condition and an end frame index list meeting the preset golf end frame condition;
selecting a target preparation frame index list and a target ending frame index list corresponding to the golf item effective action from the preparation frame index list and the ending frame index list based on prior constraint conditions;
generating multiple sets of golf item video sequence frame index combinations according to the corresponding combinations of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring corresponding golf item effective action videos according to the sets of golf item video sequence frame index combinations;
the selecting, based on a priori constraint conditions, a target preparation frame index list and a target ending frame index list corresponding to the golf item valid action from the preparation frame index list and the ending frame index list, including:
removing frame indexes of video frames meeting preset golf preparation frame conditions in golf club shaking actions from the preparation frame index list based on prior constraint conditions;
subtracting two adjacent elements in the prepared frame index list, when the subtraction result is greater than or equal to the interval frame number, the latter element is the prepared frame of the current golf action, a target prepared frame index list corresponding to the effective action of the golf item is generated according to the prepared frame, and when the subtraction result is less than the interval frame number, the last frame in the prepared frame index list is the prepared frame of the golf action;
and subtracting the current element in the ending frame index list from the element in the target preparation frame index list corresponding to the golf item effective action, and generating a target ending frame index list corresponding to the golf item effective action according to the ending frame when the subtraction result is minimum and the addition result of the element in the target preparation frame index list corresponding to the golf item effective action and the interval frame number is smaller than or equal to the current element in the ending frame index list.
2. The method of claim 1, wherein the obtaining a list of video frames comprises:
and acquiring a video stream to be analyzed, and performing frame cutting processing on the video stream in a frame skipping mode to obtain a video frame list.
3. The method according to claim 1, wherein the identifying the human body key points of the human body performing the golf action in each frame of the video image of the video frame list by using the preset human body posture estimation model to obtain the result list includes:
identifying point location information of a left shoulder, a right shoulder, a left elbow, a right elbow, a left wrist, a right wrist, a left crotch, a right crotch, a left knee, a right knee, a left ankle and a right ankle of a golf action human body in each frame of video image of the video frame list by using a preset human body posture estimation model, and obtaining an identification result;
and packaging the bit data in the middle point of the identification result through an array to generate a result list.
4. The method of claim 1, wherein the human body posture estimation model is a single person-based human body keypoint prediction model.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the preset golf preparation frame conditions include: the included angle between the straight line and the horizontal line, which are determined by the left shoulder coordinate point and the right shoulder coordinate point, is within a preset angle range; the included angle between the straight line and the horizontal line, which are determined by the left crotch coordinate point and the right crotch coordinate point, is within a preset angle range; the positions of the left foot and the right foot in the image coordinate system are the same as the positions of the left shoulder and the right shoulder in the image coordinate system in width; the X value of the left wrist coordinate is smaller than the X value of the left crotch coordinate and the X value of the right wrist coordinate is larger than the X value of the right crotch coordinate; the Y value of the left wrist coordinate is larger than the Y value of the double-span center point coordinate, and the Y value of the right wrist coordinate is larger than the Y value of the double-span center point coordinate;
the preset golf end frame condition includes: the X value of the left shoulder coordinate is smaller than that of the right shoulder coordinate; the wrist of both hands is above both shoulders, and the Y value of left wrist coordinate is less than the Y value of both shoulders central point coordinate and the Y value of right wrist coordinate is less than the Y value of both shoulders central point coordinate.
6. The method according to claim 1, wherein the a priori constraints comprise a first a priori constraint and/or a second a priori constraint;
the first a priori constraint is: the first time difference value is smaller than the second time difference value, wherein the first time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the upswing action frame, and the second time difference value is a time difference value between the video time corresponding to the current preparation frame and the video time corresponding to the next preparation frame;
the second prior constraint is: the execution time of a complete set of golf item actions is within a preset time range.
7. An apparatus for efficient action extraction of golf items, the apparatus comprising:
the acquisition module is used for acquiring a video frame list;
the identification module is used for traversing the video frame list, and utilizing a preset human body posture estimation model to identify human body key points of a golf item human body in each frame of video image of the video frame list so as to obtain a result list;
the traversing module is used for traversing the position distribution relation between the key points of the human body corresponding to each frame of video image in the result list and generating a preparation frame index list meeting the preset golf preparation frame condition and an end frame index list meeting the preset golf end frame condition;
the selecting module is configured to select, based on a priori constraint condition, a target preparation frame index list and a target ending frame index list corresponding to a golf item valid action from the preparation frame index list and the ending frame index list, and specifically includes: removing frame indexes of video frames meeting preset golf preparation frame conditions in golf club shaking actions from the preparation frame index list based on prior constraint conditions; subtracting two adjacent elements in the prepared frame index list, when the subtraction result is greater than or equal to the interval frame number, the latter element is the prepared frame of the current golf action, a target prepared frame index list corresponding to the effective action of the golf item is generated according to the prepared frame, and when the subtraction result is less than the interval frame number, the last frame in the prepared frame index list is the prepared frame of the golf action; subtracting the current element in the ending frame index list from the element in the target preparation frame index list corresponding to the golf item effective action, and generating a target ending frame index list corresponding to the golf item effective action according to the ending frame when the subtraction result is minimum and the addition result of the element in the target preparation frame index list corresponding to the golf item effective action and the interval frame number is less than or equal to the current element in the ending frame index list;
the generating module is used for generating a plurality of sets of golf item video sequence frame index combinations according to the corresponding combination of the elements in the target preparation frame index list and the elements in the target ending frame index list, and acquiring corresponding golf item effective action videos according to each set of golf item video sequence frame index combinations.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-6.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1-6 when the computer program is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310018934.7A CN116152708B (en) | 2023-01-06 | 2023-01-06 | Method and device for extracting effective actions of golf item, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310018934.7A CN116152708B (en) | 2023-01-06 | 2023-01-06 | Method and device for extracting effective actions of golf item, storage medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116152708A CN116152708A (en) | 2023-05-23 |
| CN116152708B true CN116152708B (en) | 2023-07-07 |
Family
ID=86340179
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310018934.7A Active CN116152708B (en) | 2023-01-06 | 2023-01-06 | Method and device for extracting effective actions of golf item, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116152708B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473530A (en) * | 2013-08-30 | 2013-12-25 | 天津理工大学 | Adaptive action recognition method based on multi-view and multi-mode characteristics |
| CN103688275A (en) * | 2011-06-27 | 2014-03-26 | 陈信恒 | Method of analysing video of sports motion |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108564596B (en) * | 2018-03-01 | 2022-01-25 | 南京邮电大学 | Intelligent comparison analysis system and method for golf swing video |
| CN112422863B (en) * | 2019-08-22 | 2022-04-12 | 华为技术有限公司 | Video shooting method, electronic equipment and storage medium |
| CN114220119B (en) * | 2021-11-10 | 2022-08-12 | 深圳前海鹏影数字软件运营有限公司 | Human body posture detection method, terminal device and computer-readable storage medium |
| CN115170998A (en) * | 2022-05-12 | 2022-10-11 | 无锡范特智能科技有限公司 | Human behavior and action recognition method and device, computer equipment and storage medium |
-
2023
- 2023-01-06 CN CN202310018934.7A patent/CN116152708B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103688275A (en) * | 2011-06-27 | 2014-03-26 | 陈信恒 | Method of analysing video of sports motion |
| CN103473530A (en) * | 2013-08-30 | 2013-12-25 | 天津理工大学 | Adaptive action recognition method based on multi-view and multi-mode characteristics |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116152708A (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110751224B (en) | Training method of video classification model, video classification method, device and equipment | |
| CN108229282A (en) | Key point detection method and device, storage medium and electronic equipment | |
| CN111190939A (en) | User portrait construction method and device | |
| CN112866799B (en) | Video frame extraction processing method, device, equipment and medium | |
| CN112818955B (en) | Image segmentation method, device, computer equipment and storage medium | |
| CN111161314B (en) | Target object position area determination method and device, electronic equipment and storage medium | |
| CN112016682A (en) | Video representation learning and pre-training method and device, electronic equipment and storage medium | |
| CN112929695A (en) | Video duplicate removal method and device, electronic equipment and storage medium | |
| CN110991298A (en) | Image processing method and device, storage medium and electronic device | |
| CN107529071B (en) | A kind of video data handling procedure and device | |
| CN108125678B (en) | Electrocardiosignal direction detection method and device and electronic equipment | |
| CN116152708B (en) | Method and device for extracting effective actions of golf item, storage medium and equipment | |
| CN111488779B (en) | Video image super-resolution reconstruction method, device, server and storage medium | |
| CN113051430B (en) | Model training method, device, electronic equipment, medium and product | |
| CN115174823B (en) | Checkpoint special effect video generation method and device and storage medium | |
| CN108153863B (en) | Video information representation method and device | |
| CN116129319A (en) | Weakly-supervised time-series boundary location method, device, electronic equipment, and storage medium | |
| CN115424253A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN110781342B (en) | Recommendation system recall method and system based on user behavior sequence and data fusion | |
| CN110555182A (en) | User portrait determination method and device and computer readable storage medium | |
| CN114357248A (en) | Video retrieval method, computer storage medium, electronic device, and computer program product | |
| CN111126113B (en) | Face image processing method and device | |
| CN114372169A (en) | Method, device and storage medium for searching homologous videos | |
| CN113269658A (en) | Method, device and equipment for estimating production time of core data and storage medium | |
| CN111680499A (en) | Article application analysis method and system based on natural language processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |