[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN113792646A - Dance action auxiliary generation method and device and dance equipment - Google Patents

Dance action auxiliary generation method and device and dance equipment Download PDF

Info

Publication number
CN113792646A
CN113792646A CN202111065512.2A CN202111065512A CN113792646A CN 113792646 A CN113792646 A CN 113792646A CN 202111065512 A CN202111065512 A CN 202111065512A CN 113792646 A CN113792646 A CN 113792646A
Authority
CN
China
Prior art keywords
dance
user
action
virtual
dancing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111065512.2A
Other languages
Chinese (zh)
Other versions
CN113792646B (en
Inventor
谢华为
范广祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aimyunion Technology Ltd
Original Assignee
Aimyunion Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aimyunion Technology Ltd filed Critical Aimyunion Technology Ltd
Priority to CN202111065512.2A priority Critical patent/CN113792646B/en
Publication of CN113792646A publication Critical patent/CN113792646A/en
Application granted granted Critical
Publication of CN113792646B publication Critical patent/CN113792646B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0075Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/74Browsing; Visualisation therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/75Clustering; Classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The application relates to a dance action auxiliary generation method, a dance action auxiliary generation device, dance equipment, computer equipment and a computer storage medium; the method comprises the following steps: acquiring motion data generated by dancing of a user through motion capture equipment; shooting image data of the user for dancing through a camera; analyzing the image data to obtain a bone motion track of the user, and obtaining three-dimensional bone data of dance of the user according to the bone motion track and the motion data; acquiring a virtual character selected by a user, rendering the three-dimensional skeleton data into virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time; according to the technical scheme, the dancing difficulty and cost are reduced, the space-time isolation between a user and the dancer is ensured by utilizing the virtual dance animation display, the personal privacy of the dancer is practically protected, and the dance device is suitable for wide spreading and sharing.

Description

Dance action auxiliary generation method and device and dance equipment
Technical Field
The application relates to the technical field of dance choreography, in particular to a dance action auxiliary generation method and device, dance equipment, computer equipment and a computer storage medium.
Background
At present, people mainly have modes of dance music, video, traditional on-site in-person demonstration and the like for learning and choreography; the dance music and the video modes can be copied, edited and shared, and are the main application modes of the current dance.
In the dance scheduling, a dance music mode is adopted, and dance movements are described mainly in a plane symbol or image mode; dancing machine foot register for easy reference content all uses the editor to customize the dance foot register for easy reference through keyboard mouse input mode by professional according to categorised, it is higher to the degree of specialty of people to make the dance foot register for easy reference, it is correct when the user edits the foot register for easy reference, need the dancer to verify according with the dance register for oneself and whether accord with the expectation, to the dancer of dancing, the use degree of difficulty is higher, the learning cost is high, and the dance action in the reality is three-dimensional, the dance register for easy reference dance action, be unsuitable for generally using widely.
The dance scheduling adopts a video mode, mainly describes dance motions in a plane image and sound mode, and the dance motions are three-dimensional, so that the three-dimensional motions can be described by adopting 360-degree videos, but the 360-degree videos are high in shooting cost and large in storage data amount, so that the dance scheduling is difficult to popularize, and the motions of a practicer cannot be corrected in the learning process. This makes dance study, exchange receive very big restriction, and the video mode causes privacy such as individual appearance to leak easily, is unfavorable for spreading and sharing.
Therefore, the dance music or video mode has the defects of high dance cost and easiness in privacy disclosure.
Disclosure of Invention
In order to solve one of the technical defects, the application provides a dance action auxiliary generation method, a dance action auxiliary generation device, dance equipment, computer equipment and a computer storage medium, so that dance arrangement cost can be reduced, and user privacy can be protected.
A dance action auxiliary generation method comprises the following steps:
acquiring motion data generated by dancing of a user through motion capture equipment;
shooting image data of the user for dancing through a camera;
analyzing the image data to obtain a bone motion track of the user, and obtaining three-dimensional bone data of dance of the user according to the bone motion track and the motion data;
and acquiring a virtual character selected by a user, rendering the three-dimensional skeleton data into virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time.
In one embodiment, the dance action auxiliary generation method further includes:
obtaining dance actions selected by a user to be played back in the virtual dance animation;
determining a plurality of visual angles at which to playback the dance movement of the virtual character;
respectively acquiring dance action pictures of virtual characters at all visual angles;
simultaneously playing back the dance action pictures on a display device;
in one embodiment, the dance action auxiliary generation method further includes:
and obtaining the dance action selected by the user to be played back in the virtual dance animation, and carrying out local amplification display on the dance action on display equipment.
In one embodiment, the dance action auxiliary generation method further includes:
extracting a section of dance action selected by a user in the virtual dance animation;
calculating state parameters of each action part of the dance action; wherein the state parameters include amplitude, speed, and/or angle;
calculating the action part needing playback display and the visual angle thereof according to the state parameters;
performing playback display on the corresponding action part needing playback display according to the visual angle;
in one embodiment, the dance action auxiliary generation method further includes:
extracting action parts appointed by a user in the virtual dance animation;
calculating state parameters of the action part; wherein the state parameters include amplitude, speed, and/or angle;
calculating and displaying the visual angle of the action part according to the state parameter;
and carrying out playback display on the appointed action part according to the visual angle.
In one embodiment, the dance action auxiliary generation method further includes:
obtaining dance actions selected by a user in the virtual dance animation;
displaying corresponding three-dimensional skeleton data on the virtual character of the dance action;
and acquiring the editing operation executed by the user on the three-dimensional bone data, and updating and storing the three-dimensional bone data according to the editing operation.
In one embodiment, the dance action auxiliary generation method further includes:
uploading the three-dimensional skeleton data of the dance of the user to a database of a cloud server, and authorizing the operation permission of other users on the three-dimensional skeleton data;
the other users download the three-dimensional bone data through dance equipment, render the three-dimensional bone data on the dance equipment into virtual dance animation, and operate and store the three-dimensional bone data; the operations include reading, copying, and/or editing.
In one embodiment, the dance action auxiliary generation method further includes:
when a user dances, analyzing historical data of dancing actions of the current user dancing, and calculating dancing actions with higher frequency according to the historical data; and recommending the dance motion with higher frequency to the current user for the user to select.
In one embodiment, the dance action auxiliary generation method further includes:
when a user dances, searching the next dance action matched with the current dance action in the serial actions of the database of the current user according to the edited previous dance action of the current user;
recommending a selection list of a next dance action to the current user according to the searched dance action result according to the occurrence frequency and the difficulty level of the current user, and displaying the difficulty score of the dance action;
and receiving the dance action selected by the current user, and replacing the dance action designated by the current user with the dance action.
In one embodiment, the dance action auxiliary generation method further includes:
uploading the three-dimensional bone data of the dance of the user to a database of a cloud server;
the cloud server performs the following steps:
analyzing three-dimensional skeleton data of dance editing uploaded by each user to acquire a plurality of dance actions with higher frequency; wherein the dance action comprises a single action or a plurality of groups of serial actions;
classifying the dance movements;
calculating difficulty scores by utilizing the action parameters of the dance actions to obtain the difficulty scores of the dance actions; wherein the motion parameters include motion amplitude, frequency and/or speed;
and associating the difficulty scores with corresponding dance movements.
In one embodiment, the dance action auxiliary generation method further includes:
respectively acquiring dance encodings of a plurality of users, and combining the dance encodings to obtain a dance encodings;
in one embodiment, the dance action auxiliary generation method further includes:
acquiring dance of one user, and sharing three-dimensional skeleton data of the dance to the next user; and the next user inserts the three-dimensional bone data of another dance in the dance to obtain a combined dance.
In one embodiment, the dance action auxiliary generation method further includes:
respectively acquiring single dances of a plurality of users dancing respectively;
copying the dance action sets of the single dances to designated time points of multi-person dances, and putting the initial positions of the virtual characters of the dance action sets into designated coordinate positions of a multi-person dance scene;
and rendering and playing the panorama of the multi-person dance scene to form a multi-person dance.
In one embodiment, the dance action auxiliary generation method further includes:
acquiring multi-person dancing of a multi-person dancing scene;
appointing a scene position for adding real-time dance movement in the multi-person dance;
and playing multi-person dancing and recording dancing actions in real time, displaying the dancing pictures of the virtual figures of the multi-person dancing, and displaying the real-time dancing actions of the virtual figures of the user at the corresponding scene positions.
A dance action assistance generation apparatus comprising:
the acquisition unit is used for acquiring motion data generated by dancing of a user through motion capture equipment;
the shooting unit is used for shooting image data of dancing of the user through a camera;
the acquisition unit is used for analyzing the image data to obtain a bone motion track of the user and acquiring three-dimensional bone data of dance of the user according to the bone motion track and the motion data;
and the rendering unit is used for acquiring the virtual character selected by the user, rendering the three-dimensional skeleton data into virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time.
A dance device, comprising: the system comprises a main control board, motion capture equipment and a camera; the main control board is used for executing the steps of the dance action auxiliary generation method.
A computer apparatus, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the dance motion auxiliary generation method is executed.
A computer-readable storage medium storing at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded by a processor and executes the dance action assistance generation method described above.
According to the technical scheme, the dance three-dimensional skeleton data of the user is obtained by collecting the dance motion data and the image data of the user, analyzing the image data to obtain the skeleton motion track and combining the dance motion data; finally, rendering the virtual dance animation to be displayed on display equipment; this technical scheme has realized integrating dance editor and dancing equipment function on a dancing equipment, and the user can only need to dance to the camera on the action capture equipment at the dance in-process, can obtain the virtual dance animation of dance in real time, has reduced the dance degree of difficulty and cost, utilizes virtual dance animation to show simultaneously and has guaranteed user and dancer's space-time isolation, has protected dancer's personal privacy conscientiously, is adapted to extensively propagating and sharing.
In addition, by generating the three-dimensional skeleton data of the dancing of the user, when the dancing is shared and transmitted, only the three-dimensional skeleton data needs to be transmitted, a large amount of data transmission is avoided, bandwidth occupation is reduced, and the sharing transmission efficiency is improved.
Moreover, different users can copy, check and edit the three-dimensional skeleton data of dance editing, such as dance action modification, dance action deletion/dance action insertion and the like, so that different dancers can share and communicate the dance conveniently, the dance creation and sharing enthusiasm of the users is improved, and the user experience is improved.
Furthermore, the user can carry out multipicture, local amplification and multi-angle playback to arbitrary one section dance action of dance editing, can carry out 360 playback, has promoted dance editing's result of use, promotes user experience.
Furthermore, a plurality of users can collaboratively dance, dance encodings of the users are combined in sequence to obtain a dance encouraging, dance encouraging of the users can also be inserted into the same multi-person dance scene, and therefore the dance encouraging mode is expanded, and the dance encouraging effect is improved.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is an application environment diagram of a dance action auxiliary generation method;
FIG. 2 is a schematic diagram of a dancing device;
FIG. 3 is a flow diagram of a dance motion assistance generation method of an embodiment;
FIG. 4 is a schematic diagram of a dance-generating process;
FIG. 5 is a schematic diagram of a multi-angle multi-picture synchronous playback scheme;
FIG. 6 is a schematic diagram of a partially enlarged playback scheme;
FIG. 7 is a schematic diagram of an automatic multi-angle playback scheme;
FIG. 8 is a schematic illustration of sharing three-dimensional skeletal data to other users;
FIG. 9 is a schematic diagram of a recommended dance action;
FIG. 10 is a schematic diagram of adding a multi-person dance action set;
FIG. 11 is a schematic view of adding real-time dance movements;
FIG. 12 is a schematic structural diagram of a dance motion assistance generating apparatus;
FIG. 13 is a circuit configuration diagram of the dance device.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, or operations, but do not preclude the presence or addition of one or more other features, integers, steps, operations, or groups thereof.
Referring to fig. 1, fig. 1 is an application environment diagram of a dance movement auxiliary generation method, and it can be found through fig. 1 that the technical scheme can be applied to dance devices, and each user can perform entertainment sports such as dancing by using the dance devices.
For the dancing device, referring to fig. 2, fig. 2 is a schematic structural diagram of the dancing device, which mainly comprises a main control board (not shown in the figure), a camera, a motion capture device (a dancing blanket, a dancing pedal, a bracelet sensor, etc.) and a display device (preferably a touch screen), as shown in fig. 2, in a dancing mode, a user can dance on the motion capture device, the main control board of the dancing device can record a section of dance of the user, through image data and motion data of the camera and the motion capture device, a character displayed in the dance is a virtual character image selected by the user, and the dance is displayed in a virtual dance animation form, so that the privacy of the user can be ensured, and the virtual character image can be selected according to personal preferences; moreover, the dance-editing three-dimensional skeleton data is stored and transmitted, so that the dance-editing three-dimensional skeleton data can be rapidly shared by the user and other users, the shared three-dimensional skeleton data can be conveniently edited, and the defects that the conventional video dance data is large in volume and difficult to perform secondary editing processing are overcome.
In the following, an embodiment of a dance movement auxiliary generation method according to the present application is described with reference to an application to dance equipment as an example, and in this embodiment, a technical scheme of how dance movement auxiliary generation is provided.
Referring to fig. 3, fig. 3 is a flowchart of a dance action auxiliary generation method according to an embodiment, and mainly includes:
and S10, acquiring motion data generated by dancing of the user through the motion capture device.
In this step, the user can dance through dance equipment, and the user records a section of dance action to dance equipment, and dance equipment gathers the action data that the user danced through the action capture device of connection, like dance footboard, dance blanket etc. catches the step data of dancing, like the action data that can detect limbs such as response bracelet, response handle.
And S20, shooting image data of the dancing of the user through a camera.
In the step, the dancing equipment shoots real-time image data of dancing actions of the user through a camera, and the dancing actions comprise single actions or a plurality of groups of serial actions and the like; for the camera, can adopt degree of depth camera or wide angle camera etc. the user dances in camera shooting range and carries out dancing, dancing equipment.
And S30, analyzing the image data to obtain the bone motion trail of the user, and obtaining the three-dimensional bone data of the dance of the user according to the bone motion trail and the motion data.
Specifically, the image data are analyzed based on a 3D modeling technology, the skeleton motion trail of the user in the dancing process is obtained, three-dimensional skeleton data of dancing of the user are formed by combining the user motion data captured by the motion capture device, and the three-dimensional skeleton data describe the skeleton key points of each dancing motion.
And S40, acquiring a virtual character selected by a user, rendering the three-dimensional skeleton data into virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time.
In the step, various virtual scenes can be prestored, the dance device can download corresponding virtual scenes according to user selection, various virtual characters are provided for the user to select, then the three-dimensional bone data are rendered into virtual dance animation according to the virtual characters preferred by the user, dance dancing is displayed through the virtual characters, and the user can see action demonstration of the dance dancing in real time through the display device.
For example, in the above embodiments, referring to fig. 4, fig. 4 is a schematic diagram of a dance-editing generation process, in which functions of a dance editor and a dance device are integrated on a dance device, and a user switches the dance device to a dance editing mode; for the switching control mode, the switching control mode can comprise voice control switching, touch screen operation switching or handheld device control switching, and the handheld device comprises a watch, a bracelet, a remote controller and the like; recording a dance action by a user facing the dance equipment; the dance equipment analyzes user action image data collected by the camera and calculates three-dimensional skeleton data of the user by combining the action data; the dance equipment is rendered into a virtual character selected by a user according to the three-dimensional skeleton data, and the virtual character is output to display equipment in real time; the user may save the action three-dimensional skeletal data for repeated opening and editing.
As described above, in the dance process, a user can obtain dance virtual dance animation in real time only by dancing against a camera on the motion capture equipment, the dance difficulty and cost are reduced, and meanwhile, the space-time isolation between the user and a dancer is ensured by utilizing the virtual dance animation display, so that the personal privacy of the dancer is practically protected, and the dance device is suitable for wide spread and sharing; through the three-dimensional skeleton data of generating user's dance, when sharing the transmission with this dance, only need transmit its three-dimensional skeleton data, avoid a large amount of transmission data, reduced the bandwidth and taken up, promoted and shared transmission efficiency. Different users can copy, look over and edit the three-dimensional skeleton data of dance editing, if modify dance movements, delete/insert dance movements and the like, different dancers can share and exchange the dance conveniently, user creation and dance sharing enthusiasm is improved, and user experience is improved.
In order to make the technical solutions of the present application clearer, several other embodiments of the present application are further described below with reference to the accompanying drawings.
In an embodiment, in order to facilitate the user to review dance editing and modify dance movements, the dance movement auxiliary generation method of the application may further include the following playback scheme:
(1) the multi-angle multi-picture synchronous playback scheme specifically comprises the following steps:
and s11, acquiring the dance motion selected by the user to play back in the virtual dance animation.
s12, determining a plurality of visual angles at which to play back the dance movement of the virtual character.
And s13, respectively acquiring dance action pictures of the virtual characters at the visual angles.
s14, simultaneously playing back the dance motion pictures on the display device; preferably, the dance motion picture can be displayed in an enlarged manner.
Referring to fig. 5, fig. 5 is a schematic diagram of a multi-angle and multi-picture synchronous playback scheme, for example, a user may adjust a playback action angle of a virtual character in real time through voice, a touch screen, a handheld device, and the like, and may also play back a dance action shown by the virtual character at the same time in multiple angles and multiple pictures.
(2) The partial amplification display playback scheme specifically comprises the following steps:
and s21, acquiring the dance motion selected by the user to play back in the virtual dance animation.
And s22, displaying the dance action on the display device in a local magnification mode.
Referring to fig. 6, fig. 6 is a schematic view of a partially enlarged playback scheme, for example, a user can control a playback screen to be enlarged and display a specific position so as to know the action details of the specific position.
(3) The method for automatically selecting the multi-angle playback scheme specifically comprises the following steps:
s31, extracting a section of dance motion selected by the user in the virtual dance animation.
s32, calculating the state parameters of each action part of the dance action; wherein the state parameters include amplitude, speed, angle, etc.
And s33, calculating the action part and the visual angle thereof which need to be played back and displayed according to the state parameters.
And s34, performing playback display on the corresponding action part needing playback display according to the visual angle.
In the scheme, a user selects a section of action in dance, calculates the part needing to be amplified and displayed and a plurality of visual angles by analyzing parameters such as amplitude, speed and angle of each action part of the action, and then carries out automatic multi-angle playback.
And s41, extracting the action part appointed by the user in the virtual dance animation.
s42, calculating the state parameter of the action part; wherein the state parameters include amplitude, speed, angle, etc.
And s43, calculating and displaying the visual angle of the action part according to the state parameter.
s44, displaying the designated action part according to the visual angle.
In the scheme, the user specifies the part, calculates the visual angle needing to be amplified and displayed by analyzing the parameters such as the amplitude, the angle and the like of the specified part, and then carries out automatic multi-angle playback.
Referring to fig. 7, fig. 7 is a schematic diagram of an automatic multi-angle playback scheme, which automatically selects a multi-angle magnified picture for playback by analyzing parameters such as amplitude, speed, angle, etc. of an action part according to a user's request, so that the user can know the action details of a certain position.
As mentioned above, according to the technical scheme of the application, the user can carry out multi-picture, local amplification and multi-angle playback on any section of dance action of dance editing, and can carry out 360-degree playback, so that the use effect of dance editing is improved, and the user experience is improved.
In one embodiment, in order to be convenient for the dance for recording to modify, when the dance action needs to be edited to modify the dance action, the satisfactory dance action is obtained; accordingly, the dance action auxiliary generation method of the application may further include the steps of:
s51, obtaining the dance motion selected by the user in the virtual dance animation;
s52, displaying corresponding three-dimensional skeletal data on the virtual character of the dance movement;
and s53, acquiring the editing operation executed by the user on the three-dimensional bone data, and updating and storing the three-dimensional bone data according to the editing operation.
For example, for the editing operation, several ways can be included but not limited to:
the first method is as follows: the user may modify the three-dimensional skeletal data of an arbitrary dance, thereby modifying dance movements.
The second method comprises the following steps: the user may delete a section of dance action in the dance routine.
The third method comprises the following steps: the user can insert a dance action at a designated time point of dance in the following specific mode: inserting a section of dance action recorded in real time into the dance editing; and opening another dance, selecting dance motion segments and inserting the dance motion segments into the current dance.
According to the technical scheme of the embodiment, the user can modify the choreographed dance in real time, the three-dimensional skeleton data can be directly edited when the dance is modified, and the user can directly modify the virtual dance animation without learning a professional complex editing technology in a visual mode.
In one embodiment, in order to facilitate other users to use dances recorded by the users on other dancing devices, the users may upload three-dimensional skeleton data of the dances to a cloud server, and the other users may download the three-dimensional skeleton data on their dancing devices and render the three-dimensional skeleton data on the dancing devices, and may perform related operation applications, and accordingly, the dance action auxiliary generation method of the present application may further include:
and s61, uploading the three-dimensional bone data of the dance of the user to a database of a cloud server, and authorizing the operation authority of other users on the three-dimensional bone data.
s62, the three-dimensional skeletal data is downloaded by other users through the dance device, and rendered as a virtual dance animation on the dance device.
s63, performing operations on the three-dimensional bone data and saving; wherein the operation includes reading, copying, editing, and the like.
Illustratively, referring to fig. 8, fig. 8 is a schematic diagram of sharing three-dimensional skeletal data to other users; the user can upload the three-dimensional skeleton data to the cloud server and authorize other users to use, the other users can download the three-dimensional skeleton data through the dance equipment, the three-dimensional skeleton data are rendered into virtual characters on the dance equipment and displayed, and the other users can directly edit, copy, edit the dance and the like on the authorized three-dimensional skeleton data of the dance.
In the scheme of the embodiment, other users are authorized to copy, check, edit and modify the three-dimensional skeleton data of the dance of the user through the dance device, so that different dancers can share and communicate the dance conveniently, the enthusiasm of the user for creating and sharing the dance is improved, and the user experience is improved.
In one embodiment, in order to facilitate the analysis of dance motion data of a user, the application also provides a scheme for classifying dance motions; accordingly, the dance action auxiliary generation method of the present application may further include the steps of:
and s71, uploading the three-dimensional bone data of the dance of the user to a database of a cloud server.
s72, the cloud server analyzes the dance-editing three-dimensional skeleton data uploaded by each user to obtain a plurality of dance actions with high frequency; wherein the dance action comprises a single action or a plurality of groups of serial actions.
s73, classifying the dance motion.
s74, calculating difficulty scores by using the action parameters of the dance actions to obtain difficulty scores of the dance actions; wherein the motion parameters include motion amplitude, frequency, speed, and the like.
s75, associating the difficulty score with a corresponding dance action.
According to the scheme of the embodiment, the dance motions are classified and divided into the difficulty levels, so that the dance motions with different difficulty levels can be selected to replace the designated dance motions in the dance editing process.
Further, in order to facilitate assisting a user in creating or modifying a dance action, on the basis of classifying dance actions, the dance action auxiliary generation method of the application may further include the following steps:
s81, when the user dances, according to the previous dance motion edited by the current user, searching the series of motions in the database of the current user for the next dance motion matched with the previous dance motion.
And s82, recommending the searched dance action result to the current user according to the appearance frequency and the difficulty level of the current user, and showing the difficulty score of the dance action.
s83, receiving the dance motion selected by the current user, and replacing the dance motion with the dance motion specified by the current user.
Referring to FIG. 9, FIG. 9 is a schematic view of a recommended dance movement; according to different user difficulty levels, the dance device can recommend 180-degree turning dance actions with difficulty scores of 1.0, 2.0 and 8.0 to a user, and the user can select the dance actions to replace the dance actions edited by the user according to requirements.
According to the scheme of the embodiment, when the user edits the dance or the related application scene, the user can be assisted to finish creation or modification according to the action classification data, active pushing is carried out when the user edits the dance, the user can be assisted to find dance actions with personal styles, so that the user is assisted to create dance actions with personal styles, and the application experience of the user is improved.
In one embodiment, in order to assist a user in discovering a dance motion having a personal style, the dance motion auxiliary generation method of the application may further include the following steps:
when a user dances, analyzing historical data of dancing actions of the current user dancing, and calculating dancing actions with higher frequency according to the historical data; and recommending the dance motion with higher frequency to the current user for the user to select.
According to the scheme of the embodiment, when the user edits a dance or related application scene, the user selection is recommended, so that the user is assisted to create dance motions with personal styles.
As described in the above embodiment, the cloud server analyzes dance motion data of all users to obtain motions with high occurrence frequency or some groups of serial motions, and classifies the motions; then, the difficulty score is obtained by comprehensively calculating parameters such as the action amplitude, the frequency, the speed and the like; and analyzing the historical data of the dance movements recorded by the current user to obtain movements with higher occurrence frequency or some groups of serial movements, and recommending the user to select when the user edits a dance or related application scene.
As described in the above embodiment, in the dance action auxiliary generation method according to the present application, three-dimensional skeleton data of dance editing is stored, and thus, when editing a dance, a single dance can be edited by multiple persons in a cooperative manner, or multiple dances can be edited by multiple persons.
In one embodiment, when a plurality of persons collaboratively edit a single dance, dance dances of a plurality of users are respectively acquired, and the dance dances are combined to obtain a dance dancing.
Specifically, a plurality of users edit a dance respectively, and then insert a dance motion at a specified time point, for example, insert an instantly recorded dance motion into a dance routine, or open another dance routine, insert the dance motion routine into the dance routine, and combine the dances into a dance. In addition, after the dance editing is finished and stored by the first user, the three-dimensional bone data file of the dance editing is shared to the next editor for editing, and the dance editing is sequentially iterated to finally finish the dance editing.
According to the scheme of the embodiment, the multiple users can collaboratively dance, the dance encodings of the multiple users are combined in sequence to obtain a dance encodings, the dance encodings are expanded, and the dance encodings effect is improved.
In addition, when a plurality of persons edit multi-person dances, dance dancing of one user can be obtained, and three-dimensional skeleton data of the dance dancing is shared to the next user; the next user can insert the three-dimensional bone data of another dance in the dance to obtain a combined dance; it should be noted that, in the merging process, the next user may also perform operations such as replacement/deletion on several dance movements in the dance routine completed by the previous user.
In one embodiment, referring to FIG. 10, FIG. 10 is a schematic view of adding a multi-person dance action set; when a plurality of persons edit multi-person dances, respectively acquiring single dances of a plurality of users dancing respectively; copying the dance action sets of the single dances to designated time points of multi-person dances, and putting the initial positions of the virtual characters of the dance action sets into designated coordinate positions of a multi-person dance scene; and rendering and playing the panorama of the multi-person dance scene to form a multi-person dance.
Specifically, a plurality of single dances are combined into a multi-person dance by appointing different positions; opening the existing single dance, copying the single dance action set to a designated time point of multi-person dance, and putting the initial position of the virtual character of the action set into a designated coordinate position of a scene; the user can play the multi-person dance panorama and also can designate one or more virtual characters to play so as to check the adding effect; the user can modify the placing position of each virtual character in the scene and can also modify the dance action of any virtual character; the user can repeat the above operations, and a plurality of single dances are added to form a multi-person dance in a scene.
In addition, dance actions can be recorded in real time and added to the multi-person dance pointing position; referring to FIG. 11, FIG. 11 is a schematic view of adding real-time dance movements; firstly, opening the existing multi-person dance; a user firstly appoints to add real-time dance actions to a scene position of multi-person dance; the user plays multi-person dancing and records dancing actions in real time, at the moment, the display equipment can display the real-time dancing actions of the virtual character of the user at the corresponding scene position besides displaying the dancing picture of the existing virtual character for multi-person dancing, and the user can dance for multiple persons personally on the scene.
According to the scheme of the embodiment, in multi-person dance editing by multiple persons, dance editing of multiple users is inserted into the same multi-person dance scene, so that the dance editing mode is expanded, and the dance editing effect is improved.
Referring to fig. 12, fig. 12 is a schematic structural diagram of a dance motion assistance generating apparatus, including:
the device comprises an acquisition unit 10, a display unit and a control unit, wherein the acquisition unit is used for acquiring motion data generated by dancing of a user through motion capture equipment;
a shooting unit 20 for shooting image data of the user dancing through a camera;
the obtaining unit 30 is configured to analyze the image data to obtain a bone motion trajectory of the user, and obtain three-dimensional bone data of dance of the user according to the bone motion trajectory and the motion data;
and the rendering unit 40 is used for acquiring a virtual character selected by a user, rendering the three-dimensional skeleton data into a virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time.
The dance action auxiliary generation device of this embodiment may execute the dance action auxiliary generation method provided in this embodiment of this application, and the implementation principles thereof are similar, the actions executed by the modules in the dance action auxiliary generation device in this embodiment of this application correspond to the steps in the dance action auxiliary generation method in this embodiment of this application, and for the detailed functional description of the modules in the dance action auxiliary generation device, reference may be specifically made to the description in the corresponding dance action auxiliary generation method shown in the foregoing, and details are not repeated here.
An embodiment of the dance device of the present application is described below, and referring to fig. 13, fig. 13 is a circuit configuration diagram of the dance device, including: the system comprises a main control board, motion capture equipment and a camera; the main control board is used for executing the steps of the dance action auxiliary generation method of the embodiment; furthermore, the dance equipment can be connected with a display device and a sound equipment, so that the audio-visual playing function can be realized; for motion capture devices, including dance pedals, bracelets, remote controls, and the like.
An embodiment of a computer device of the present application is set forth below, the computer device comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the dance motion assistance generation method according to any of the above embodiments is performed.
Embodiments of a computer-readable storage medium of the present application are set forth below, the storage medium storing at least one instruction, at least one program, a set of codes, or a set of instructions that are loaded by the processor and that perform the dance action assistance generation method of any of the above embodiments.
According to the dance equipment, the computer equipment and the computer readable storage medium of the embodiment, a user can dance on the action capturing equipment in the dance process only by facing to the camera, so that dance virtual dance animation can be obtained in real time, dance difficulty and cost are reduced, space-time isolation between the user and a dancer is guaranteed by utilizing the virtual dance animation display, personal privacy of the dancer is practically protected, and the dance equipment, the computer equipment and the computer readable storage medium are suitable for wide spread and sharing.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.

Claims (10)

1. A dance action auxiliary generation method is characterized by comprising the following steps:
acquiring motion data generated by dancing of a user through motion capture equipment;
shooting image data of the user for dancing through a camera;
analyzing the image data to obtain a bone motion track of the user, and obtaining three-dimensional bone data of dance of the user according to the bone motion track and the motion data;
and acquiring a virtual character selected by a user, rendering the three-dimensional skeleton data into virtual dance animation according to the virtual character, and outputting the virtual dance animation to display equipment in real time.
2. A dance action auxiliary generation method according to claim 1, further comprising:
obtaining dance actions selected by a user to be played back in the virtual dance animation;
determining a plurality of visual angles at which to playback the dance movement of the virtual character;
respectively acquiring dance action pictures of virtual characters at all visual angles;
simultaneously playing back the dance action pictures on a display device;
or
And obtaining the dance action selected by the user to be played back in the virtual dance animation, and carrying out local amplification display on the dance action on display equipment.
3. A dance action auxiliary generation method according to claim 1, further comprising:
extracting a section of dance action selected by a user in the virtual dance animation;
calculating state parameters of each action part of the dance action; wherein the state parameters include amplitude, speed, and/or angle;
calculating the action part needing playback display and the visual angle thereof according to the state parameters;
performing playback display on the corresponding action part needing playback display according to the visual angle;
or
Extracting action parts appointed by a user in the virtual dance animation;
calculating state parameters of the action part; wherein the state parameters include amplitude, speed, and/or angle;
calculating and displaying the visual angle of the action part according to the state parameter;
and carrying out playback display on the appointed action part according to the visual angle.
4. A dance action auxiliary generation method according to claim 1, further comprising:
obtaining dance actions selected by a user in the virtual dance animation;
displaying corresponding three-dimensional skeleton data on the virtual character of the dance action;
and acquiring the editing operation executed by the user on the three-dimensional bone data, and updating and storing the three-dimensional bone data according to the editing operation.
5. A dance action auxiliary generation method according to claim 1, further comprising:
uploading the three-dimensional skeleton data of the dance of the user to a database of a cloud server, and authorizing the operation permission of other users on the three-dimensional skeleton data;
the other users download the three-dimensional bone data through dance equipment, render the three-dimensional bone data on the dance equipment into virtual dance animation, and operate and store the three-dimensional bone data; the operations include reading, copying, and/or editing.
6. A dance action auxiliary generation method according to claim 1, further comprising:
when a user dances, analyzing historical data of dancing actions of the current user dancing, and calculating dancing actions with higher frequency according to the historical data; recommending the dance action with higher frequency to the current user for the user to select;
or
When a user dances, searching the next dance action matched with the current dance action in the serial actions of the database of the current user according to the edited previous dance action of the current user;
recommending a selection list of a next dance action to the current user according to the searched dance action result according to the occurrence frequency and the difficulty level of the current user, and displaying the difficulty score of the dance action;
and receiving the dance action selected by the current user, and replacing the dance action designated by the current user with the dance action.
7. A dance action auxiliary generation method according to claim 6, further comprising:
uploading the three-dimensional bone data of the dance of the user to a database of a cloud server;
the cloud server performs the following steps:
analyzing three-dimensional skeleton data of dance editing uploaded by each user to acquire a plurality of dance actions with higher frequency; wherein the dance action comprises a single action or a plurality of groups of serial actions;
classifying the dance movements;
calculating difficulty scores by utilizing the action parameters of the dance actions to obtain the difficulty scores of the dance actions; wherein the motion parameters include motion amplitude, frequency and/or speed;
and associating the difficulty scores with corresponding dance movements.
8. A dance action auxiliary generation method according to claim 1, further comprising:
respectively acquiring dance encodings of a plurality of users, and combining the dance encodings to obtain a dance encodings;
or
Acquiring dance of one user, and sharing three-dimensional skeleton data of the dance to the next user;
and the next user inserts the three-dimensional bone data of another dance in the dance to obtain a combined dance.
9. A dance action auxiliary generation method according to claim 1, further comprising:
respectively acquiring single dances of a plurality of users dancing respectively;
copying the dance action sets of the single dances to designated time points of multi-person dances, and putting the initial positions of the virtual characters of the dance action sets into designated coordinate positions of a multi-person dance scene;
rendering and playing the panorama of the multi-person dance scene to form a multi-person dance;
or
Acquiring multi-person dancing of a multi-person dancing scene;
appointing a scene position for adding real-time dance movement in the multi-person dance;
and playing multi-person dancing and recording dancing actions in real time, displaying the dancing pictures of the virtual figures of the multi-person dancing, and displaying the real-time dancing actions of the virtual figures of the user at the corresponding scene positions.
10. A dance device, comprising: the system comprises a main control board, motion capture equipment and a camera; wherein the main control board is used for executing the steps of the dance action auxiliary generation method of any one of claims 1 to 9.
CN202111065512.2A 2021-09-10 2021-09-10 Dance motion auxiliary generation method and device and dance equipment Active CN113792646B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111065512.2A CN113792646B (en) 2021-09-10 2021-09-10 Dance motion auxiliary generation method and device and dance equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111065512.2A CN113792646B (en) 2021-09-10 2021-09-10 Dance motion auxiliary generation method and device and dance equipment

Publications (2)

Publication Number Publication Date
CN113792646A true CN113792646A (en) 2021-12-14
CN113792646B CN113792646B (en) 2024-07-02

Family

ID=79183262

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111065512.2A Active CN113792646B (en) 2021-09-10 2021-09-10 Dance motion auxiliary generation method and device and dance equipment

Country Status (1)

Country Link
CN (1) CN113792646B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114266265A (en) * 2021-12-30 2022-04-01 广东中科凯泽信息科技有限公司 Exercise system based on 3D cartoon figure interaction
CN114928767A (en) * 2022-05-19 2022-08-19 湖南快乐阳光互动娱乐传媒有限公司 Method for rendering virtual reality VR (virtual reality) glasses visualization and related equipment

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150008345A (en) * 2013-07-13 2015-01-22 (주)앤에스티 Online dance training system
WO2015030334A1 (en) * 2013-08-28 2015-03-05 Ha Dong Won System and method for automatic generation of dance notation
CN107341351A (en) * 2017-07-06 2017-11-10 京东方科技集团股份有限公司 Intelligent body-building method, apparatus and system
CN109447020A (en) * 2018-11-08 2019-03-08 郭娜 Exchange method and system based on panorama limb action
CN110109532A (en) * 2018-06-11 2019-08-09 成都思悟革科技有限公司 A kind of human action Compare System obtaining system based on human body attitude
WO2019177870A1 (en) * 2018-03-15 2019-09-19 Magic Leap, Inc. Animating virtual avatar facial movements
CN110414434A (en) * 2019-07-29 2019-11-05 努比亚技术有限公司 Dancing exercising method, mobile terminal and computer readable storage medium
CN110660017A (en) * 2019-09-02 2020-01-07 北京航空航天大学 Dance music recording and demonstrating method based on three-dimensional gesture recognition
CN111563487A (en) * 2020-07-14 2020-08-21 平安国际智慧城市科技股份有限公司 Dance scoring method based on gesture recognition model and related equipment
CN111798548A (en) * 2020-07-15 2020-10-20 广州微咔世纪信息科技有限公司 Control method and device of dance picture and computer storage medium
CN111798549A (en) * 2020-07-15 2020-10-20 广州微咔世纪信息科技有限公司 Dance editing method and device and computer storage medium
CN112882575A (en) * 2021-02-24 2021-06-01 宜春职业技术学院(宜春市技术工人学校) Panoramic dance action modeling method and dance teaching auxiliary system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150008345A (en) * 2013-07-13 2015-01-22 (주)앤에스티 Online dance training system
WO2015030334A1 (en) * 2013-08-28 2015-03-05 Ha Dong Won System and method for automatic generation of dance notation
CN107341351A (en) * 2017-07-06 2017-11-10 京东方科技集团股份有限公司 Intelligent body-building method, apparatus and system
WO2019177870A1 (en) * 2018-03-15 2019-09-19 Magic Leap, Inc. Animating virtual avatar facial movements
CN110109532A (en) * 2018-06-11 2019-08-09 成都思悟革科技有限公司 A kind of human action Compare System obtaining system based on human body attitude
CN110674794A (en) * 2018-11-08 2020-01-10 郭娜 Panoramic dance action modeling method and dance teaching auxiliary system
CN109447020A (en) * 2018-11-08 2019-03-08 郭娜 Exchange method and system based on panorama limb action
CN110414434A (en) * 2019-07-29 2019-11-05 努比亚技术有限公司 Dancing exercising method, mobile terminal and computer readable storage medium
CN110660017A (en) * 2019-09-02 2020-01-07 北京航空航天大学 Dance music recording and demonstrating method based on three-dimensional gesture recognition
CN111563487A (en) * 2020-07-14 2020-08-21 平安国际智慧城市科技股份有限公司 Dance scoring method based on gesture recognition model and related equipment
CN111798548A (en) * 2020-07-15 2020-10-20 广州微咔世纪信息科技有限公司 Control method and device of dance picture and computer storage medium
CN111798549A (en) * 2020-07-15 2020-10-20 广州微咔世纪信息科技有限公司 Dance editing method and device and computer storage medium
CN112882575A (en) * 2021-02-24 2021-06-01 宜春职业技术学院(宜春市技术工人学校) Panoramic dance action modeling method and dance teaching auxiliary system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SALVA KIRAKOSIAN等: "Immersive Simulation and Training of Person-to-3D Character Dance in Real-Time", 《INTERNATIONAL CONFERENCE ON VIRTUAL WORLDS AND GAMES FOR SERIOUS APPLICATIONS》, 6 September 2019 (2019-09-06), pages 1 - 4, XP033629507, DOI: 10.1109/VS-Games.2019.8864530 *
赵梦: "基于混合密度网络的音乐编舞算法研究", 《中国优秀硕士学位论文全文数据库哲学与人文科学辑》, no. 8, 15 August 2019 (2019-08-15), pages 086 - 328 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114266265A (en) * 2021-12-30 2022-04-01 广东中科凯泽信息科技有限公司 Exercise system based on 3D cartoon figure interaction
CN114928767A (en) * 2022-05-19 2022-08-19 湖南快乐阳光互动娱乐传媒有限公司 Method for rendering virtual reality VR (virtual reality) glasses visualization and related equipment

Also Published As

Publication number Publication date
CN113792646B (en) 2024-07-02

Similar Documents

Publication Publication Date Title
CN109462776B (en) Video special effect adding method and device, terminal equipment and storage medium
EP3631681B1 (en) Augmented reality smartglasses for use at cultural sites
US10902676B2 (en) System and method of controlling a virtual camera
KR101936692B1 (en) Dance training apparatus and method using automatic generation of dance key motion
CN113822970B (en) Live broadcast control method and device, storage medium and electronic equipment
CN103258338A (en) Method and system for driving simulated virtual environments with real data
JP2023501832A (en) Realization method, apparatus and related products for lens division
Essid et al. A multi-modal dance corpus for research into interaction between humans in virtual environments
Reimat et al. Cwipc-sxr: Point cloud dynamic human dataset for social xr
Ribeiro et al. 3D annotation in contemporary dance: Enhancing the creation-tool video annotator
CN113792646B (en) Dance motion auxiliary generation method and device and dance equipment
CN114363689B (en) Live broadcast control method and device, storage medium and electronic equipment
Rizzo et al. WhoLoDancE: whole-body interaction learning for dance education
Mase et al. Socially assisted multi-view video viewer
Wang et al. Hyper-lapse from multiple spatially-overlapping videos
CN114598819A (en) Video recording method and device and electronic equipment
JP2008010966A (en) Moving picture generation system and method
CN111652986B (en) Stage effect presentation method and device, electronic equipment and storage medium
Pongnumkul et al. Creating map-based storyboards for browsing tour videos
KR20160069663A (en) System And Method For Producing Education Cotent, And Service Server, Manager Apparatus And Client Apparatus using therefor
US20210287718A1 (en) Providing a user interface for video annotation tools
US20230326161A1 (en) Data processing method and apparatus, electronic device, computer-readable storage medium, and computer program product
KR102324579B1 (en) Method and system for matching a dance
CN116017082A (en) Information processing method and electronic equipment
Greuter et al. The immersive archive

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant