TWI402784B - Music detection system based on motion detection, its control method, computer program products and computer readable recording media - Google Patents

Description

Motion detection-based music creation play system, control method thereof, computer program product and computer readable recording medium

The music creation play system using motion detection of the present invention relates to a system for creating different musical expressions by detecting children's body movements, resulting in children's interest in music. It is a music creation and play system that changes the volume of the child by changing the body posture of the child, or adjusts the volume of the child in the direction of movement of the child, or the control method of the music creation playing system by the speed of the child's movement. Computer program products and computers can read recording media.

Music is born from the melody and rhythm of nature, and has a close relationship with human potential. Therefore, in education, music plays an indispensable role. Since the birth of the child, there has been a great interest in the surrounding voices. Relevant research has pointed out that the sooner children are exposed to music, the children's language, creativity, coordination and social skills are significantly helpful. Therefore, the sooner children's music cells are developed, the creative thinking of children will be extremely helpful.

In response to this inspiring need, especially for preschool children, a variety of musical toys have been developed on the market. Among them, there are play-type players who are free to be beaten by children, such as bells, sand bells, etc., as long as there is action to make a sound; there are also those who let children feel the actual operation of the instrument, such as a variety of reduced-size instruments , flute, horn, etc. There are also a variety of music bells, music boxes, and musical toys made with electronic music. This type of forest is always using its vocal music function, with a bright color, or a smile, in order to attract the attention of children. Taking Patent Document 1 as an example, the playback of music is controlled only by a switch, and a rotating charm is added to increase the interest. Thus, attempts have been made to give children a sense of inspiration in the process of playing these musical toys and to bring joy to children.

(Patent Document 1) Republic of China Patent Publication No. D107524

However, musical toys in the prior art have several major drawbacks. First, the operating conditions of the toy are too rigid. In addition to random tapping and tapping, most of them use keyboard or button-type operation. Therefore, before the music is felt, the connection between the specific action, the specific button and the music sound is established, and the child becomes a child. The prerequisite challenge that must be experienced. In contrast, the protagonist becomes an exercise of movement, and the inspiration between music and the brain becomes a secondary supporting role. Secondly, whenever the potential of the music is stimulated, most people associate it with classical music, and especially emphasize the rhythm of the music, the stimulating effect of the melody on the brain, and therefore most toys with beautiful music. With classical music as the background. However, such a simple switching operation to start and stop listening, lacking the feedback mechanism after the child learns, can not further stimulate the connection between music and the body. In addition, in terms of melodious music, no matter the rhythm or melody, there must be considerable effort to achieve something. Only as a means of play for children, the existing music toys are difficult to achieve a good level, over time. It will lose interest because of setbacks. Once you lose your patience, the meticulous design will not work.

The invention provides a music creation playing system using motion detection, which aims to create children's favorite music by detecting children's body movements, thereby developing children's creations and thereby developing children's creations. force. That is, the music creation system based on the action capture of the present invention not only allows children to create their own music through the swing of the body, but also can change their favorite music through the speed of the body movement. This intuitive way allows children to learn about music and increase their interest in learning music so that children can enjoy music. It is not necessary to require children to listen to certain music, practice specific instruments, or learn the music of music, and the same can achieve the inspiration of using music. That is to say, as long as the children perform their favorite actions, the music creation and play system using the motion detection of the present invention can help the child interact with the music.

According to a first aspect of the present invention, a music creation play system using motion detection is a music creation system based on detecting a child's body movement posture, so that a child can create his own music through the swing of the body. The above-mentioned music creation playing system using motion detection includes the following devices: an image input device for photographing an image of a user's motion; a basic gesture memory device for memorizing a plurality of basic gestures; and a gesture recognition device for inputting the image input device Comparing the captured image with the basic postures stored in the basic posture memory device, and identifying the basic posture to which the input image belongs; the music corresponding device, according to the type of the basic posture recognized by the gesture recognition device a duration to determine a corresponding note to be output and a length of time; and a music output device that outputs music composed of the notes and the length of time determined by the music corresponding device.

The music creation play system using motion detection according to the present invention is a limb virtual musical instrument system, and the children's music creation and play system can emit a note sound as long as the children perform their favorite actions. The music creation play system is configured by the user to set a plurality of favorite basic postures in the basic posture memory device, and each basic posture is assigned to a note, which represents a different scale and is stored in the music corresponding to the music. In the device. When the operation is in progress, it is only necessary to digitize the image of the user's motion through the image input device, and the posture recognition device recognizes the basic posture to which the input image belongs, and when the user stays at a certain basic The pose represents the length of the beat of each note. Then, the music output device causes the music creation play system to emit music corresponding to the musical instrument according to the basic posture made by the user with a preset virtual musical instrument.

A music creation playing system using motion detection according to a second aspect of the present invention, further comprising: a basic posture setting device that sets and memorizes a basic posture in the basic posture memory device; and a music setting device, in the music corresponding device, Set the correspondence between the basic posture and the notes. With the above-described basic posture setting device and music setting device, the user of the music creation playing system of the present invention can set or change the basic posture and the note corresponding to each basic posture according to his or her own preference.

A music creation playing system using motion detection according to a third aspect of the present invention further includes: a gesture corresponding device corresponding to the type and duration of the basic posture recognized by the gesture recognition device to determine a corresponding output to be outputted The posture and the length of time; and the motion output device output an action composed of the posture determined by the posture corresponding device and the length of time. That is, the music creation play system using motion detection of the present invention can also give the music creation play system the ability to react to actions. For example, when a humanoid robot appearance is employed, the humanoid robot can be used to imitate a basic posture made by a user or a specific corresponding action by using the posture corresponding device and the motion output device in the above-described music creation process. Through this interactive process, the fun and interaction of the whole process is increased.

According to a fourth aspect of the present invention, a music creation play system using motion detection is a music play system based on detecting a child's body movement speed, so that a child can interact with his or her familiar music through the swing of the body. The above-mentioned music creation playing system using motion detection includes the following devices: an image input device for photographing an image of a user's motion; a basic gesture memory device for memorizing a plurality of basic gestures; and a gesture recognition device for inputting the image input device Comparing the captured image with each of the basic gestures stored in the basic posture memory device, and recognizing the basic posture to which the input image belongs, and recognizing the left and right movement of the user; the music memory device, remembering to play a music adjustment device that changes a speed or a pitch of the music memorized by the music output device by using the music output device according to a change speed of the basic posture recognized by the gesture recognition device; and a volume adjustment device corresponding to the gesture recognition The user recognized by the device moves to change the output volume of the music output device.

According to the present invention, the music creation playing system using motion detection is a limb virtual musical instrument system, and the child can move movements and speeds through the body, and the music creation and playing system can change the playing volume, speed and tone of the favorite music. Among them, the adjustment of the playback speed and the pitch is adjusted by the music adjustment device according to the speed of the child's body movement. The child's body movement speed can still be obtained by using the above-described posture recognition device, and by inferring the speed of change between different basic postures, the movement speed of the child can be indirectly inferred. Here, since it is not necessary to accurately recognize the correspondence between the basic posture and the notes, the intention is the change of the basic posture, so that the number of basic postures can be increased, and the speed of change of the small movement can be detected.

As for the volume of playing music, the movement of the user's body position is the focus of detection, for example, one step is to increase the volume to the right, and one step to the left is to decrease the volume. Further, the volume adjusting device described above can also be applied to the music creation playing system using the motion detection according to the first aspect of the present invention, and is also provided with a function of volume adjustment.

According to a fifth aspect of the present invention, a music creation playing system using motion detection further includes: a music recording device that memorizes the music to be played in the music memory device. With the music recording device described above, the user of the music creation playing system of the present invention can select the music to be played according to his or her preference, and perform the above-mentioned music recording operation in advance before playing.

A music creation playing system using motion detection according to a sixth aspect of the present invention, further comprising: a plurality of position sensors mounted on the user for sensing a posture position of the user body; and posture measurement And receiving the body posture position information transmitted by the plurality of position sensors. That is, the gesture detecting means used in the music creation playing system using the motion detection of the present invention can not only use the above-mentioned manner of recognizing the user's posture by the image, but can be performed without any sensor. Gesture recognition; the position sensor can be used to detect the absolute position of the limbs of the sensor or the relative position of the motion change, as in the prior art.

A music creation playing system using motion detection according to a seventh aspect of the present invention, further comprising: a plurality of motion sensors mounted on the user for sensing a motion change of the user body; and motion measurement The device receives body motion speed information transmitted by the plurality of motion sensors. That is, the motion detecting means used in the music creation playing system using the motion detection of the present invention can not only use the above method for calculating the user's motion speed by using the image, without any sensor. The speed of the motion is obtained; as in the prior art, the action sensor, such as a gyroscope, can be used to directly sense the motion changes of the limbs where the motion sensor is located.

The music creation playing system using motion detection according to the eighth aspect of the present invention further includes: a voice input device for recording a voice input by the user with a voice; and a voice recognition device recording the voice input device. Voice recognition for voice; and voice corresponding device corresponding to the voice recognized by the voice recognition device, using the action output device to perform a specific action, or making the music creation play system Perform specific actions. According to the eighth aspect of the present invention, the music creation playing system using motion detection can instantly determine the content of the child's speech, extract the key vocabulary, and use the motion detection by presetting the response corresponding to the vocabulary. The music creation play system either performs a corresponding action or executes a command content corresponding to the vocabulary.

A method for controlling a music creation play system using motion detection according to a ninth aspect of the present invention, wherein the apparatus included in the system performs the following steps: photographing an image of a user motion by the image input device; The memory device memorizes a plurality of basic gestures; the gesture recognition device compares the image captured by the image input device with each of the basic gestures stored in the basic gesture memory device, and recognizes the basic posture to which the input image belongs; a music corresponding device, according to the type and duration of the basic posture recognized by the gesture recognition device, to determine a corresponding note and a length of time to be outputted; and a music output device that outputs a note determined by the music corresponding device The length of time and the composition of the music.

At the same time, the method further includes the following steps: sensing a posture position of the user body by a plurality of position sensors mounted on the user; and receiving, by the posture measuring device, a body posture position transmitted by the plurality of position sensors The basic posture setting device sets and memorizes the basic posture in the basic posture memory device; the music setting device sets a correspondence between the basic posture and the note in the music corresponding device; and the user recognizes the user by the posture recognition device Moving left and right; changing the output volume of the music output device by the volume adjustment device corresponding to the user movement recognized by the gesture recognition device; and the type of the basic posture recognized by the gesture corresponding device corresponding to the gesture recognition device a duration to determine a corresponding posture and a length of time to be outputted; an action output device outputs an action composed of the posture determined by the posture corresponding device and a length of time; and the voice input device inputs the sound to the user Voice recording; The device performs voice recognition on the voice recorded by the voice input device; and the voice response device corresponds to the voice recognized by the voice recognition device, and uses the action output device to perform a specific action, or It is to make the above music creation play system perform a specific operation.

A method for controlling a music creation play system using motion detection according to a tenth aspect of the present invention, wherein the apparatus included in the system performs the following steps: photographing an image of a user motion by the image input device; The memory device stores a plurality of basic postures; the gesture recognition device compares the images captured by the image input device with the basic postures stored in the basic posture memory device, and recognizes the basic posture to which the input image belongs, And recognizing the left and right movement of the user; remembering, by the music memory device, the music piece to be played; and changing, by the music adjusting device, the change speed of the basic posture recognized by the gesture recognition device, changing the memory played by the music output device by using the music output device The speed or pitch of the music; and the volume adjustment means, corresponding to the user movement recognized by the gesture recognition means, changing the output volume of the music output means.

At the same time, the method further includes the following steps: sensing a posture position of the user's body by a plurality of position sensors mounted on the user; and receiving, by the posture measuring device, the body posture transmitted by the plurality of position sensors Position information; sensing a change in motion of the user's body by a plurality of motion sensors mounted on the user; receiving body motion speed information transmitted by the plurality of motion sensors by the motion measuring device; The recording device memorizes the music to be played on the music memory device; the voice input device records the voice input by the user with the voice; and the voice recognition device performs voice recognition on the voice recorded by the voice input device.

The computer program product program according to the eleventh aspect of the present invention is the control method for the ninth aspect or the tenth aspect of the music creation playing system using motion detection.

A computer-readable recording medium according to the twelfth aspect of the present invention, which stores the program of the music creation playing system using the motion detection. The program can be recorded on a computer-readable information recording medium such as a compact disc, a floppy disk, a hard disk, an optical disk, a digital video disk, a magnetic tape, or a semiconductor memory.

According to the present invention, a music creation play system using motion detection can provide a musical expression that can be created by detecting a child's body movements, such as by detecting a child's body posture, Adjust the volume of the child's movement direction, or change the playing speed of the music by playing the speed of the child, or change the speed of playing the music, etc., which will cause children to be interested in music and have the effect of inspiring to create thinking.

Embodiments of the present invention will be described below. The following is a description of the implementation of the music creation and play system using the motion detection of the present invention for ease of understanding. However, the embodiments described below are for illustrative purposes only and are not intended to limit the scope of the invention. Therefore, those skilled in the art can adopt embodiments in which the various elements or all of the elements are replaced by equivalent embodiments, and such embodiments are also within the scope of the invention.

As an example, referring to the relationship between the music creation play system using motion detection and the child in the embodiment of FIG. 1, the music creation play system has a humanoid robot shape and is used in the following situations: (1) in music At the time of creation, the child makes a series of actions in front of the robot, and the music creation play system plays the music corresponding to the action, and causes the robot to perform corresponding actions, so that the child can create the belonging through the swing of the body. (2) When playing music, when the music creation and playing system starts playing the pre-recorded music, the child makes a series of actions in front of the robot to change the volume and speed of playing music so that the child can Interact with your familiar music through the swing of your body.

(Composition of music creation and play system using motion detection)

Fig. 2 is a schematic view showing a schematic configuration of a music creation play system using motion detection according to an embodiment of the present invention. Hereinafter, description will be made with reference to this figure.

The music creation play system 100 using motion detection includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a microphone 104, and a CCD camera (Charge-coupled Device Camera). Coupling device camera 105, joint drive device 106, sound output device 107, interface 108, and personal computer 109.

When the power of the music creation play system 100 using the motion detection is turned on, and the music creation play system program using the motion detection is executed, the music creation play system using the motion detection of the present embodiment can be realized.

The CPU 101 controls the overall operation of the music creation play system 100 using motion detection to be connected to each constituent element, and transmits control signals and data.

The IP address (Initial Program Loader) executed immediately after the power is turned on is recorded in the ROM 102, and when the IPL is executed, the music creation play system program recorded by the ROM 102 is read into the RAM 103, and Execution is started by the CPU 101. Further, in the ROM 102, a program and various materials of an operation system required for the overall motion control of the music creation play system 100 using motion detection are recorded.

The RAM 103 is used for temporarily storing data and programs, and stores programs and data read from the ROM 102, and other input/output devices (microphone 104, CCD camera 105, joint driving device 106, and sound output device 107). ) Information necessary for control or processing.

The microphone 104 is used to record a user's voice command. The analog sound data captured by the CPU 101 or the microphone 104 is converted into digital sound data, and processed, and then recorded into the RAM 103. .

The CCD camera 105 is used to photograph a user's gesture. The captured image data is processed by the image processor (not shown) provided in the CPU 101 or the CCD camera 105, and then recorded in the RAM 103.

The joint driving device 106 is a driving device that performs a corresponding action during use in accordance with the shape adopted by the music creation playing system using motion detection. For example, in the present embodiment, a humanoid robot is used, and the joint driving device 106 is used to drive and control a servo motor (not shown) attached to the humanoid robot joint. In order to enable the robot to imitate the movement of the child, a servo motor is mounted on each of the main joints of the robot to enable the buckling rotation. The main joints on the body include at least 17 necks (1), left and right hands (3 each), and left and right feet (5 each). And can be increased or decreased according to actual needs, such as increasing the waist rotation or bending joints, you can make a turn or bend. When the imitation operation is actually performed, the servo motor for controlling each joint is driven to be a specific bending angle in accordance with a command from the CPU 101, thereby achieving the purpose of posture control.

The sound output device 107 converts the processed digital sound data into an analog sound signal, and outputs it from a speaker (not shown) connected to the sound output device 107. Moreover, under the control of the CPU 101, the effect sound and the music data which should be generated during the operation of the music creation playing system 100 using motion detection can also be generated, and the corresponding sound is output from the speaker.

The interface 108 has the function of network connection; and the personal computer 109 connected via the interface 108 is used for necessary installation, maintenance and update of the music creation play system 100 using motion detection. For example, a music creation/playing system program using motion detection is downloaded to the ROM 102 described above, or information such as a basic gesture already described in the ROM 102 is updated. In the personal computer 109, the maintenance function of the position creation, fault detection and elimination of the music creation play system 100 using motion detection can also be performed through the connection of the interface 108. When the music creation play system 100 using motion detection using the motion detection system is normally used, it is not necessary to connect with the personal computer 109.

Further, in the music creation play system 100 using motion detection according to the present embodiment, in addition to the CPU 101, the ROM 102, and the RAM 103 described above, a microprocessor (Micro Proessor) architecture having a data processing and storage function can be used as a utilization action. The core control device of the detected music creation play system 100.

(Implementation of music creation and play system using motion detection)

FIG. 3 is a functional block diagram of a music creation play system using motion detection according to the embodiment. The function of the music creation play system 100 using motion detection includes the following devices: an image input device 201, a music recording device 202, a voice input device 203, a basic gesture memory device 211, a gesture recognition device 212, a music corresponding device 213, and a gesture corresponding device. 214. The basic posture setting device 215, the music setting device 216, the music storage device 221, the volume adjusting device 222, the music adjusting device 223, the voice recognition device 231, the voice corresponding device 232, the music output device 251, and the action output device 252.

First, the music creation play system using the motion detection of the present embodiment allows a child to create his own music through the swing of the body. When the child makes a variety of action postures by waving the hand and foot in front of the music creation play system using motion detection in the present embodiment of the present invention, the image input device 201 is used to photograph the user's image. Since the subsequent processing is performed by a digital logic algorithm, the image input device 201 must output the information to the next processing program in a predetermined digital format after recording the related information.

Furthermore, the CCD camera 105 can function as such a video input device 201.

In the image processing flow, the image input device 201 has acquired the digitized posture image. Then, the gesture recognition device 212 is used to acquire a feature of the body posture from the posture image, for example, by comparing the contours of the image, and identifying the basic posture closest to the posture. As shown in FIGS. 4(a)-(g), the "basic posture" refers to a plurality of basic posture setting devices 215. The postures are distinct from each other, set and memorized in the basic posture memory device 211. In the present embodiment, it is further contemplated that the plurality of base postures can be smoothly achieved by the robot so that the robot can imitate the user's action posture. Therefore, the gesture recognition device 212 compares the acquired gesture image with the base gesture recorded in the basic gesture memory device 211 by a similar gesture, thereby identifying which base gesture the gesture image belongs to. The basic posture described above is not limited to one person, and may be performed as a basic posture by a screen composed of two or more persons in a specific posture.

In addition, since the person to be recognized by the gesture recognition device 212 is a feature of the body posture, and is not the image itself after the photography, the feature of the user posture does not have to be acquired by the image input device 202, and multiple position sensors may be used. A combination with a posture measuring device. The plurality of position sensors are mounted at a specific position on the user, and the posture measuring device receives the body posture position information transmitted by the plurality of position sensors, thereby obtaining the same posture recognition. Body posture features that device 212 can handle.

Furthermore, the ROM 102 can function as such a basic gesture memory device 211. The RAM 103 and the CPU 101 can function as such a gesture recognition device 212. The ROM 102, the interface 108, and the personal computer 109 can function as such a basic gesture setting device 215.

The music corresponding device 213 determines the corresponding note and time length to be output according to the type and duration of the basic gesture recognized by the gesture recognition device 212. In more detail, the user can set the correspondence between the basic gesture and the note in the music corresponding device 213 through the music setting device 216 in advance, and the gesture recognition device 212 recognizes the basic posture made by the user. After that, the note that the user wants to emit can be known through the correspondence. Further, by using the type and duration of the basic posture described above, the corresponding note type and note length are obtained, so that specific music can be formed. Then, the music composed of the above-described music corresponding device 213 is output by the music output device 251.

Furthermore, the RAM 103 and the CPU 101 can function as such a music corresponding device 213. The ROM 102, the interface 108, and the personal computer 109 can function as such a music setting device 216. The above-described sound output device 107 can function as such a music output device 251.

In the embodiment of the music creation play system 100 using motion detection according to the present invention, the shape of the humanoid robot is employed in design, and the posture corresponding device 214 and the motion output device 252 are provided. The gesture corresponding device 214 corresponds to the type and duration of the basic gesture recognized by the gesture recognition device 212 to determine the corresponding posture and time length to be outputted, and the action output device 252 outputs the corresponding gesture. The action of the posture determined by the device and the length of time. The purpose is to make the children feel more interactive during the process of making a series of actions by the music creation play system 100 and letting the music creation play system 100 play the music corresponding to the action. Interesting. Since the present embodiment has the appearance of a humanoid robot, the humanoid robot can be directly imitated to the basic posture made by the user. However, it is not limited to this. Other animal shapes can be used, such as the shape of a dog, and the tail movement can be made at the right time, which can enhance the overall appeal to children.

Furthermore, the RAM 103 and the CPU 101 can function as the gesture corresponding device 214. The joint drive device 106 and the servo motor driven thereby can function as such an action output device 252.

The music creation play system using motion detection in this embodiment can also be used as a music playback device, allowing children to change their favorite music through the speed of the body movement. The child can use the music recording device 202 to pre-memorize the music to be played on the music memory device 221. Here, the music recording device 202 can directly receive the music recorded on the scene through the microphone, or download the music file (wav file, mp3 file, etc.). The music creation play system of the present invention is different from the music play device of the prior art, and the child can change the volume and speed or tone of the played music by a series of actions. That is, the volume of the music memorized by the music storage device 222 is changed by the music output device 251 by the volume adjustment device 222 corresponding to the left and right movement of the body position made by the child; and, by the music adjustment device 223 The speed or tone of the music memorized by the music storage device 222 is changed by the music output device 251 in response to the change speed of the gesture made by the child. Unlike the aforementioned gesture recognition for the basic posture, the "action" here emphasizes the temporal change, that is, the speed of the motion. The detection of such a speed change can also be combined with a plurality of motion sensors and motion measuring devices. The plurality of motion sensors are mounted at a specific position on the user, and the motion measuring device receives the speed information of the motion change transmitted by the plurality of motion sensors. However, based on the basic posture recognized by the gesture recognition device 221, the speed information of the motion change can be obtained by calculating the speed of change of the basic postures. However, in the present embodiment, the posture change speed characteristic of the child is further calculated by using the gesture recognition device 221, so that no additional motion sensor or motion measurement device is added. In addition, the music adjusting device 223 can also change different tones in the operation of creating the music according to the body position recognized by the gesture recognition device 221.

Furthermore, the microphone 104 or the ROM 102, the interface 108, and the personal computer 109 can function as such a music recording device 202. The RAM 103 described above can function as such a music storage device 221. The RAM 103 and the CPU 101 can function as such a volume adjusting device 222. The RAM 103 and the CPU 101 can function as such a music adjusting device 223.

In addition, the main functions of the embodiment of the music creation play system 100 using the motion detection of the present invention can be roughly classified into creating music and playing music. Therefore, for the operation of the music creation play system 100, at least three different commands of "creation start", "play start" and "end" are required. The present embodiment further includes: a voice input device 203 for recording voices input by the user with voice; a voice recognition device 231 for voice recognition of voices recorded by the voice input device 203; and a voice corresponding device 232 Corresponding to the voice recognized by the voice recognition device 231, the preset reaction is performed by the action output device 252, or the music creation play system 100 performs a specific operation. Therefore, it is possible to use the different expressions of children, replacing the "creation start" command with the "play start" command, the "play start" command replaced by "play music", and the "stop playing" command instead of the "end" command. At the same time, the robot can be made to perform a specific action, so that the child knows that the robot has received the corresponding command.

Furthermore, the microphone 104 can also function as such a voice input device 203. The RAM 103 and the CPU 101 can function as such a voice recognition device 231 and a voice corresponding device 232.

(The music creation and play system of this embodiment is applied to the creation of music)

The following is a diagram for explaining the flow of the music creation play system of the present embodiment applied to creating music.

In the music creation play system 100, as shown in FIG. 5(a), a child can design seven postures as a basic posture, and the plurality of basic postures are stored in the basic posture memory device by using the basic posture setting device 215. 211. Further, the music setting means 216 is used to associate the plurality of basic gestures with seven notes such as Do, Re, Mi, Fa, So, La, and Si.

The child can say to the music creation play system 100 that when the "beginning of performance" is performed, the voice recorded by the voice input device 203 will be recognized by the voice recognition device 231, and the voice creation device 232 can be used to make the music creation system. 100 Execute the command "Creation Start". As shown in FIG. 5(b), when the child starts dancing in front of the music creation play system 100, or performs a series of actions, the gesture recognition device 212 performs the identification of the type and duration of the basic posture, and simultaneously obtains the child. Mobile information about the body position. The type and duration of the basic gesture will be converted into corresponding notes and time lengths by the music corresponding device 213; and the movement information of the body position is changed by the volume adjusting device 222 to change the volume of the output. In the process of creating music, the body position can also represent different tones, such as C major or D major, and the music adjustment device is used to adjust the pitch. Thereby, the composed music can be output by the music output device 251. Finally, the child can use the "do not play!" command to execute the "end" command.

According to the music creation and play system of this embodiment, children do not need to learn difficult music fingerings and music theory, and can use the most familiar movements to create their own music according to the way they like, so that the creation of music can be more fun. Not only can you create music alone, but also many people can participate in the creation of music. Use competition and cooperation to create music together. See which team's music is the most beautiful, the music is the most beautiful, and the overall effect is the best. Music and the purpose of knowing music.

(The music creation play system of this embodiment is applied to play music)

The following is a diagram for explaining the flow of the music creation play system of the present embodiment applied to playing music.

In the music creation play system 100, the child can use the music recording device 202 to memorize the favorite music in the music storage device 221. Then, the "play start" command can be executed by "playing music", and the music created by the music output device 100 will be started by the music output device 251. At this time, as shown in FIG. 6, when the child makes a series of actions in front of the music creation play system 100, the posture recognition device 212 performs calculation to obtain the left and right movement of the body position and the posture change speed of the composition action. Further, the volume adjustment means 222 and the music adjustment means 223 are used to change the volume and speed or pitch of the played music.

The music creation play system of the present embodiment is applied to the function of playing music, and is particularly suitable for a music bell for a crib. Babies naturally like to make some sounds, and they are easy to be attracted by sounds. As the baby's movements can make various sounds through the system, the baby can find that if other movements occur, different sounds will appear, and the baby can be trained on music. Sensitive, and cultivate music preferences by exploring music. With the music creation and play system of this embodiment, it is possible to instantly detect the movement of the baby while playing music. When the baby moves faster, the speed of the music is accelerated; when the baby moves slower, the speed of the music is slowed down; and the position of the baby can also change the size of the music. In conjunction with the aforementioned creative music function, the baby posture can also be turned into a note, and the series of actions of the baby can be turned into a piece of music composed of a series of notes. When the baby makes any action and finds that the concert becomes different, the baby is attracted to attract the baby's attention through the change of music. In addition, through the image input device of the system, it is also possible to provide a method for parents to observe the baby from a distance.

(industrial availability)

The present invention provides a music creation play system using motion detection, which is a method for creating different musical expressions by detecting a child's body movements, that is, playing a note by detecting a body posture made by a child, Adjusting the volume of the child's movement direction, or changing the music creation and playing system with the speed of the child's movement, so that the system that causes children's interest in music will be of great help to children's creative thinking.

100. . . Music creation play system

101. . . CPU

102. . . ROM

103. . . RAM

104. . . Microphone

105. . . CCD camera

106. . . Joint drive

107. . . Sound output device

108. . . interface

109. . . personal computer

201. . . Image input device

202. . . Music recording device

203. . . Voice input device

211. . . Basic posture memory device

212. . . Posture recognition device

213. . . Music counterpart

214. . . Posture corresponding device

215. . . Basic posture setting device

216. . . Music setting device

221. . . Music memory device

222. . . Volume adjustment device

223. . . Music adjustment device

231‧‧‧Voice recognition device

232‧‧‧Voice counterpart

251‧‧‧Music output device

252‧‧‧Action output device

FIG. 1 is a diagram showing the relationship between a music creation play system using motion detection and a child according to an embodiment of the present invention.

Fig. 2 is a schematic view showing a schematic configuration of a music creation play system using motion detection according to an embodiment of the present invention.

FIG. 3 is a functional block diagram of a music creation play system using motion detection according to the embodiment.

4(a)-(g) are diagrams showing examples of basic postures.

Figures 5(a)-(b) are schematic diagrams of the music creation play system of the present embodiment applied to the creation of music.

Fig. 6 is a schematic view showing the application of the music creation play system of the present embodiment to playing music.

201. . . Image input device

202. . . Music recording device

203. . . Voice input device

211. . . Basic posture memory device

212. . . Posture recognition device

213. . . Music counterpart

214. . . Posture corresponding device

215. . . Basic posture setting device

216. . . Music setting device

221. . . Music memory device

222. . . Volume adjustment device

223. . . Music adjustment device

231. . . Speech recognition device

232. . . Voice corresponding device

251. . . Music output device

252. . . Motion output device

Claims (30)

A music creation playing system using motion detection, the system comprising: an image input device for photographing an image of a user's motion; a basic gesture memory device for memorizing a plurality of basic gestures; and a gesture recognition device for photographing the image input device The image is compared with each of the basic postures stored in the basic posture memory device, and the basic posture to which the input image belongs is recognized; the music corresponding device memorizes the correspondence between the basic posture and the note, and is identified according to the posture. a type of the basic posture recognized by the device and a duration before the basic posture change to determine a corresponding note and a length of time to be outputted; and a music output device that outputs a note and a length of time determined by the music corresponding device And the music that makes up. The music creation playing system using motion detection according to claim 1 of the patent scope further includes: a plurality of position sensors mounted on the user for sensing a posture position of the user body; and posture measurement And receiving the body posture position information transmitted by the plurality of position sensors. For example, the music creation playing system using motion detection in the first application of the patent scope includes: a basic posture setting device, which is set and memorized in the basic posture memory device. a basic posture; and a music setting device that sets a correspondence relationship between the basic posture and the note in the music corresponding device. The music creation playing system using motion detection according to the first aspect of the patent application, wherein the gesture recognition device further recognizes the movement of the user; and further includes a volume adjustment device corresponding to the gesture recognition device The user moves and changes the output volume of the music output device by the preset amplitude. For example, the music creation playing system using the motion detection in the first application of the patent scope further includes: a posture corresponding device corresponding to the type and duration of the basic posture recognized by the gesture recognition device to determine the corresponding output. The posture and the length of time; and the motion output device output an action composed of the posture determined by the posture corresponding device and the length of time. The music creation playing system using motion detection according to item 5 of the patent application scope further includes: a voice input device for recording a voice input by the user with a voice; and a voice recognition device recording the voice input device; Speech language And the voice corresponding device, corresponding to the voice recognized by the voice recognition device, using the action output device to perform a specific action, or causing the music creation play system to perform a specific operation . The music creation playing system using the motion detection according to the first aspect of the patent application, wherein the gesture recognition device further recognizes the movement of the user; and further includes: a plurality of position sensors mounted on the user, a posture position device for sensing a posture of the user body; a posture measuring device that receives body posture position information transmitted by the plurality of position sensors; and a basic posture setting device that sets and memorizes the base posture in the basic posture memory device; a setting device, in the music corresponding device, setting a correspondence between a basic posture and a note; the volume adjusting device corresponding to the user movement recognized by the gesture recognition device, and changing the use of the music output device according to a preset amplitude The output volume; the gesture corresponding device corresponds to the type and duration of the basic gesture recognized by the gesture recognition device to determine the corresponding posture and time length to be outputted; The action output device outputs an action composed of the posture determined by the posture corresponding device and the length of time; the voice input device records the voice input by the user with the voice; and the voice recognition device records the voice input device The voice is voice-recognized; and the voice-corresponding device corresponds to the voice recognized by the voice recognition device, and the preset response is performed by the action output device, or the music creation and play system is executed. Specific operations. A control method for a music creation play system using motion detection, the method comprising the steps of: photographing an image of a user motion by an image input device; memorizing a plurality of basic gestures by the basic gesture memory device; The image captured by the image input device is compared with each of the basic postures stored in the basic posture memory device, and the basic posture to which the input image belongs is recognized; and the music corresponding device is based on the recorded basic posture and the note. Corresponding relationship, the type of the basic posture recognized by the gesture recognition device and the duration before the basic posture change to determine the corresponding note and time length to be outputted; and the music output device outputting the music corresponding device The music composed of the determined notes and the length of time. The method for controlling a music creation play system using motion detection according to claim 8 of the patent application, the method further comprising the steps of: sensing a body of the user by a plurality of position sensors mounted on the user a posture position; and receiving, by the posture measuring device, body posture position information transmitted by the plurality of position sensors. The method for controlling a music creation play system using motion detection according to item 8 of the patent application, the method further comprising the steps of: setting and memorizing the basic posture by the basic posture setting device in the basic posture memory device; and the music setting device, In the music corresponding device described above, the correspondence between the basic posture and the note is set. The method for controlling a music creation play system using motion detection according to claim 8 of the patent application, the method further comprising the steps of: recognizing the movement of the user by the gesture recognition device; and corresponding to the gesture recognition by the volume adjustment device The user's movement recognized by the device changes the output volume of the music output device by the preset amplitude. The method for controlling a music creation play system using motion detection according to Item 8 of the patent application, the method further comprising the steps of: the type and duration of the basic posture recognized by the gesture corresponding device corresponding to the gesture recognition device, To determine the corresponding posture to be output And an operation of the time length; and the motion output device outputs the posture determined by the posture corresponding device and the length of time. For example, in the method for controlling the music creation play system using motion detection according to Item 12 of the patent application, the method further includes the following steps: recording, by the voice input device, the voice input by the user by using the voice; The voice recorded by the voice input device performs voice recognition; and the voice corresponding device corresponds to the voice recognized by the voice recognition device, and the preset reaction is used to perform a specific action by using the action output device, or The above music creation play system performs a specific operation. The method for controlling a music creation play system using motion detection according to claim 8 of the patent application, the method further comprising the steps of: sensing a posture of the user body by a plurality of position sensors mounted on the user body; Positioning; receiving, by the posture measuring device, body posture position information transmitted by the plurality of position sensors; setting, by the basic posture setting device, the basic posture memory device and memorizing the basic posture; and the music setting device, in the music corresponding device, Set the correspondence between the basic posture and the notes; Identifying, by the gesture recognition device, the movement of the user; the volume adjustment device corresponding to the movement of the user recognized by the gesture recognition device changes the output volume of the music output device according to a preset amplitude; corresponding to the gesture corresponding device The type and duration of the basic posture recognized by the gesture recognition device to determine a corresponding posture and time length to be outputted; and the action output device outputs a posture and a time length determined by the posture corresponding device. Acting; recording, by the voice input device, the voice input by the user by voice; performing voice recognition on the voice recorded by the voice input device by the voice recognition device; and identifying by the voice recognition device corresponding to the voice recognition device The voice, the preset response, using the action output device to perform a specific action, or causing the music creation play system to perform a specific operation. A computer program product for storing a music creation play system program for motion detection, which is loaded into the program and executed after the motion detection-based music creation play system is completed, as in the patent application range 8 to 14 The method of any of the items. A computer-readable recording medium for storing a program, which can be completed after loading and executing the program using the motion detection music creation and playing system The method of any one of claims 8 to 14. A music creation playing system using motion detection, the system comprising: an image input device for photographing an image of a user's motion; a basic gesture memory device for memorizing a plurality of basic gestures; and a gesture recognition device for photographing the image input device Comparing with the basic postures stored in the basic posture memory device, and identifying the basic posture to which the input image belongs, and identifying the change speed of the basic posture and the movement of the user; the music memory device And memorizing the music to be played; the music adjusting device changes the speed or pitch of the music memorized by the music memory device by using the music output device according to the preset amplitude according to the change speed of the basic posture recognized by the gesture recognition device; And a volume adjusting device that changes the output volume of the music output device according to the preset amplitude corresponding to the user movement recognized by the gesture recognition device. The music creation playing system using motion detection according to claim 17 of the patent application, further comprising: a plurality of position sensors mounted on the user for sensing a posture position of the user body; and posture measurement And receiving the body posture position information transmitted by the plurality of position sensors. Music creation using motion detection, as in claim 17 The play system further includes: a plurality of motion sensors mounted on the user for sensing a change in motion of the user's body; and an action measuring device receiving the body motion transmitted by the plurality of motion sensors Speed information. For example, the music creation playing system using motion detection according to the scope of claim 17 includes: a music recording device that memorizes the music to be played in the music memory device. For example, the music creation playing system using motion detection according to claim 17 of the patent scope further includes: a voice input device for recording voices input by the user with sound; and a voice recognition device for recording the voice input device The voice is speech recognized. The music creation playing system using motion detection according to claim 17 of the patent application, further comprising: a plurality of position sensors mounted on the user for sensing a posture position of the user body; the posture measuring device Receiving body posture position information transmitted by the plurality of position sensors; a plurality of motion sensors mounted on the user for sensing the The action of the user's body changes; the motion measuring device receives the body motion speed information transmitted by the plurality of motion sensors; the music recording device stores the music to be played in the music memory device; the voice input device, The user performs recording by using the voice input by the voice; and the voice recognition device performs voice recognition on the voice recorded by the voice input device. A control method for a music creation play system using motion detection, the method comprising the steps of: photographing an image of a user motion by an image input device; memorizing a plurality of basic gestures by the basic gesture memory device; And comparing the image captured by the image input device with each of the basic postures stored in the basic posture memory device, and identifying the basic posture to which the input image belongs, and identifying the change speed of the basic posture, and the user's Moving the music piece to be played by the music memory device; the music adjusting device plays the music memorized by the music memory device by using the music output device according to the preset amplitude according to the change speed of the basic posture recognized by the gesture recognition device Speed or pitch; and by the volume adjustment means corresponding to the gesture recognition device The user moves and changes the output volume of the music output device by the preset amplitude. The method for controlling a music creation play system using motion detection according to claim 23 of the patent application, the method further comprising the steps of: sensing a body of the user by a plurality of position sensors mounted on the user a posture position; and receiving, by the posture measuring device, body posture position information transmitted by the plurality of position sensors. The method for controlling a music creation play system using motion detection according to the scope of claim 23, the method further comprising the steps of: sensing a physical body of the user by a plurality of motion sensors mounted on the user The action changes; and the body motion speed information transmitted by the plurality of motion sensors is received by the motion measuring device. For example, the method for controlling a music creation play system using motion detection according to claim 23 of the patent scope further includes the following steps: the music to be played is memorized by the music recording device to the music memory device. For example, the method for controlling a music creation play system using motion detection according to claim 23 of the patent scope further includes the following steps: recording, by the voice input device, the voice input by the user by voice; The voice recorded by the voice input device is voice-recognized by the voice recognition device. The method for controlling a music creation play system using motion detection according to claim 23 of the patent application, the method further comprising the steps of: sensing a body of the user by a plurality of position sensors mounted on the user a posture position; receiving, by the posture measuring device, body posture position information transmitted by the plurality of position sensors; sensing a movement change of the user body by a plurality of motion sensors mounted on the user; The measuring device receives the body motion speed information transmitted by the plurality of motion sensors; the music recording device memorizes the music to be played in the music memory device; and the voice input device records the voice input by the user with the voice And performing speech recognition on the voice recorded by the voice input device by the voice recognition device. A computer program product for storing a music creation play system program using motion detection, when the motion detection-based music creation play system system is loaded into the program and executed, the patent application scope is as follows: The method of any of the 28 items. A computer-readable recording medium of a built-in program, which can be completed as described in any one of claims 23 to 28, after the program is loaded and executed by the motion-detecting music creation and play system.