JPH09190181A - Method for generating and playing music - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain unity between video and music by converting element figure data into musical note data on the pitch, length, and timbre of a sound and connecting the musical note data according to figure connection data. SOLUTION: A figure musical note conversion processing part 1 converts some or all of the shape, direction, position, size, color, and other information of an element figure into the pitch, length, and timbre of a sound and other musical note data. A musical score generating process part 2 connects a musical note sequence as the converted musical note data to generate a musical score. At the time of this musical score generation processing, the connection order and connecting method are determined according to the figure connection data. Namely, the connection order of the note sequence is based upon the connection order of corresponding element figures. When the element figures are connected in series, the musical note sequence is arrayed in series with time. When the element figures are connected in parallel, the musical note sequence is also arrayed in parallel so that the 1st musical note or rest is put in order. Generated musical score data are played by being inputted to an automatic musical performance processing part 3.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of creating and playing music, and more particularly, to television programs, video software,
The present invention relates to the making and method of music used in movies, multimedia software and other video works.

[0002]

2. Description of the Related Art Video, movies, multimedia software and other video works are not only video images, but also background music accompanying them is an important element of the video works. Conventionally, such background music is created separately from the video and edited according to the video to form a video work. That is, the background music suitable for the image of the image is newly created and played, or the selected background music is selectively collected from the existing music and the timing and music of the movement of the image or the scene are changed by using the editing device. A video work is constructed by editing the video and / or music so that the two match. A similar editing process is adopted also in the case of computer graphics in which an image is created by a computer based on graphic data and motion data.

[0003]

When the above-mentioned music making method is adopted, the following problems occur. There is a risk that the relationship between video and music will be weakened, the unity of the video work will be lost, and its value will be reduced. In particular, when the existing music is applied while reducing the cost for music, it is difficult to find music suitable for the image, and this fear becomes particularly serious.

Further, it takes a lot of time and labor to perform editing work on both video and music, particularly fine adjustment of timing between video and music. Also, when the user views the completed video work, if the playback speed of the video is changed, that is, if fast-forwarding, rewinding, or slow-motion playback is performed, the playback speed of the sound is similarly changed,
The music is played as high-pitched sounds such as fast piro-piro, or low-pitched sounds like slow wah-wah,
Even if it is not similar to the original music, it loses its effect as background music.

Further, when the display range of the image is moved, the partial area is enlarged and zoomed in, or the wide area is reduced and zoomed out, the impression given by the image is changed despite the change. It is difficult to change the music to suit. Particularly, in multimedia software, when the user interactively performs these operations, the music must be changed and played in real time, which is even more difficult. As a result, the unity between video and music is lost.

The present invention provides a method of making and playing music which solves the above problems.

[0007]

[MEANS FOR SOLVING THE PROBLEMS] The element graphic data consisting of the shape, size, direction, or color of each element graphic forming the graphic is converted into musical note data of pitch, length and timbre,
A method of creating and playing music is constructed, in which musical score data of music to be generated is created by connecting note data based on graphic connection data that defines how to connect element graphics.

The musical score data is generated based on the note data and the graphic connection data, the drawing timing data is generated, and the music is created and played sequentially based on the element graphic data and the drawing timing data. In addition, a method of creating and playing music is constructed in which the figure data is composed of element figure data composed of fractal figures which are self-similar figures and figure connection data connecting these element figures.

Further, before performing the graphic note conversion processing, a method of creating and playing music is constructed in which visual field conversion and clipping processing are performed on element graphic data based on visual field data indicating a range in which a graphic is to be displayed.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described with reference to embodiments. A first embodiment of the present invention will be described with reference to the drawings. First, referring to FIG. 1 and FIG. 2, FIG. 1 is a diagram showing a flow of processing of the first embodiment as a whole, and FIG. 2 is a diagram showing figures used in creating music.

The graphic shown in FIG. 2 is graphic data.
A straight line with an arrow is used as an element figure, and information about a change in the direction of the element figure is associated with the pitch of the sound, and information about the length of the element figure is associated with the length of the sound. The numbers attached to the elemental figures are elemental figure numbers indicating the generation order of sounds to be generated. As will be described with reference to FIG. 3 as well, (a) when the orientation of the elemental figure changes 60 ° counterclockwise with respect to the immediately preceding elemental figure, it increases by one note in the scale and changes 60 ° clockwise. One note is descended, and (b) the figure-note conversion processing for converting the element figure into a note is performed by applying two rules that the sound length is proportional to the length of the element figure. The result of graphic note conversion processing by applying these rules to the graphic data shown in FIG. 2 is as shown in FIG. Here, the tonality is assumed to be C major, and the element graphic 1 is assumed to be a quarter note of one point C sound. Since the element figure 2 has changed by 60 ° counterclockwise with respect to the immediately preceding element figure 1, it rises by one sound and is a one-point double sound, and since the length is twice that of the element figure 1, the sound length is S is a double note that is twice the quarter note of element graphic 1. Element graphic 3 is 6 clockwise with reference to the immediately preceding element graphic 1.
Since it changes by 0 °, it drops by one sound and is a low sound. Since the length is one time that of the element graphic 1, the sound length is also a quarter sound in the element graphic 1. Hereinafter, the same applies.

The graphic data is composed of the above-described element graphic data and graphic connection data described later. As shown in FIG. 2, the element graphic data is divided into element graphic units in advance and stored together with the graphic connection data, or the element graphic data and the graphic connection data are implemented based on the graphic data of another format. Generate in time. The element graphic forming the graphic data has a line segment including a curve, a polygon, or another surface as a basic unit, but a combination of a plurality of these can be a unit. And as the information of the element figure,
In addition to the above orientation and length, the shape, position, and
You can focus on size, color, and other information.

The graphic connection data will be described with reference to FIG. The graphic connection data holds, as data, the connection order of element graphics and whether the connection status of element graphics is serial connection or parallel connection. The graphic connection data corresponding to the graphic data shown in FIG. 2 is as shown in FIG. Here, the chain line --- indicates that the element figures are connected in series, and the parallel line ‖ indicates that the element figures are connected in parallel. The serial connection means that element graphics are sequentially connected in series so that the generated notes are sequentially arranged on the same staff. The parallel connection means that the generated notes are branched and connected so as to be arranged on a staff different from the staff.

The graphic note conversion processing unit 1 uses part or all of the shape, direction, position, size, color, and other information of the element graphic, such as pitch, length, strength, tone color, and other notes. Convert to data. Both the pitch and the length of the sound can be treated as either a continuous quantity or a discrete quantity. For example, the pitch or frequency of the tone can be continuously changed in proportion to the length of the element graphic, or can be changed stepwise according to a predesignated scale as shown in FIG. In this case, the major scale, the minor scale, the chromatic scale, the pentatonic scale, and other irregular scales can be arbitrarily determined.
Furthermore, one element graphic can be converted into a note string in addition to being limited to only one note. The note sequence includes a single note composed of only one note, a compound note or chord in which a plurality of notes are arranged in parallel, and a note string in which a plurality of single notes or compound notes are arranged in series. The note string may include rests. As an example of associating the elemental figure with the tone sequence, it is possible to assume an example in which the combination of the lengths of the individual notes is changed or the combination of the pitches of the notes is changed according to the shape.

The musical score generation processing unit 2 connects musical note strings, which are converted musical note data, to generate a musical score. In this musical score generation processing, the connection order and connection method are determined based on the above-mentioned graphic connection data. That is, the connection order of the musical note sequence follows the connection order of the corresponding element figures. When the element figures are connected in series, note strings are also arranged in time series. When the element figures are connected in parallel, the note sequence is also arranged in parallel with the first note or rest aligned. When the musical score data obtained by connecting the note strings of FIG. 3 based on the graphic connection data of FIG. 4 is expressed in a staff, it becomes as shown in FIG.

The created musical score data can be played by inputting it to the automatic performance processing section 3. This automatic performance processing can use a commercially available MIDI sound source and software and hardware for playing the same by creating score data in compliance with the MIDI standard. When it is desired to add a high degree of musicality, which is difficult to achieve by automatic performance, the musical score data can be printed and a human can play the music in accordance with the printed musical score data.

FIG. 6 is a diagram showing the overall processing flow of the second embodiment. In the second embodiment, when the score / drawing timing generation processing unit 4 generates the score data from the note data and the figure connection data, the drawing timing data indicating the timing of drawing the element graphic is simultaneously generated. It corresponds to the one added to the first embodiment. in this case,
The drawing timing data is created so that the drawing of the element graphic is started in synchronization with the start of the performance of the corresponding note sequence. Various concrete structures of the drawing timing data are conceivable. For example, calculate the time from the moment of starting playing and drawing to the start of playing the note sequence corresponding to each element figure, and use the number of element figure and time group sorted in chronological order as drawing timing data. You can FIG. 7 shows drawing timing data corresponding to the examples of FIGS. Here, the unit of time is 1 for the performance time of the quarter note in FIG.

The sequential drawing processing unit 5 draws the element graphic data at a designated timing by referring to the element graphic data and the drawing timing data. The automatic performance processing section 3 refers to the score data to perform the performance. In this way, the drawing and the music can be synchronized with each other, and the labor of manual editing can be omitted. FIG. 8 is a diagram showing a processing flow of the entire third embodiment.

The third embodiment differs from the first embodiment in that
This corresponds to the addition of the process of generating the element graphic data and the graphic connection data by the fractal graphic generation processing unit 6. A method of generating a fractal figure having self-similarity is described in "Fractal Geometry" (mandelpro, translated by Heisuke Hironaka, published by Nikkei Science, 1985). This will be briefly described with reference to FIG.
Various fractal figures can be obtained by recursively replacing the unit shape included in the “initiator” with the “generator”. FIG.
Shows the change in the shape of the initiator when the line segment which is the initiator, that is, the unit figure, is replaced by the generator consisting of two line segments. Here, the level 0 for the initiator and the generator for all the unit figures constituting the level 0 will be referred to as level 1, and the one for all the unit figures for level 1 will be referred to as level 2. The same applies hereinafter. In FIG. 9, the level 0 initiator is replaced by a generator without displacing the left and right ends of the line segment 0 as it is, thereby forming a level 1 graphic in which the line segment 11 and the line segment 12 are connected at a right angle. be able to. The level 1 graphic is generated by the generator without displacing the left and right ends of the downward-sloping line segment 11 of the level 1 graphic, and without displacing the left and right ends of the right-down graphic segment 12. By replacing it, line segment 21 or line segment 2
It is possible to form level 2 in which 4 are sequentially connected at right angles. The level 3 graphic can be similarly formed based on the level 2 graphic.

FIG. 10 is a figure formed by overlapping the figures from level 1 to level 5 in FIG.
The element graphic data is composed of a graphic formed by superposing several continuous graphic levels. The figure connection data for the element figure data in FIG. 10 is serially connected for the lowest level figure among the overlapped figures, and is additionally connected in parallel every time the generator is replaced. The lowest level graphic in this case is the level 1 graphic. A schematic diagram of the graphic connection data of the element graphic of FIG. 10 is as shown in FIG. To explain this concretely, in FIG. 10, the thick line segment 11 and the thick line segment 12 are line segments that compose the element graphic of level 1. However, these two line segments are connected in series in FIG. It is indicated by the black circle 11 and the black circle 12 which are related. Next, in FIG. 10, a thick line segment 21 to a thick line segment 24
Is a line segment forming the level 2 element graphic. Two of these line segments 21 and 22 are composed of a thick line segment 11. In FIG. 11, a black circle 11 corresponding thereto is branched and connected in parallel, while two lines are connected. The line segment 23 and the line segment 24 are composed of the thick line segment 12, and in FIG. 11, the black circle 12 corresponding thereto is branched and connected in parallel. Below, Level 2 and Level 3
The same applies to the connection relationship between the and, the connection relationship between the level 3 and the level 4, and the connection relationship between the level 4 and the level 5.

Here, in order to generate music having self-similarity, the graphic note conversion processing section 1 needs the following two constraint conditions. Generator 1 for a level n graphic
The playing time of the figure A corresponding to the number of pieces is T _A , the playing time corresponding to one of the unit figures B is T _B , and one generator corresponding to the unit figure B at the next level (n + 1) is set. Assuming that the playing time of the corresponding figure C is T _c , (constraint condition 1) T _B = T _c .

(Restriction condition 2) The ratio between the reference frequency when the figure A is played and the reference frequency when the figure C is played is (1 /
T _A ): (1 / T _c ). In this case, if the reproduction speed is multiplied by the original T _C / T _A times, the pitch of the note string corresponding to level n becomes equal to the pitch of the note string of level (n + 1) in the original music. In the case of the figure in FIG. 9, by replacing one line segment with a generator, the line segment becomes two equal-length line segments that are 1 / √2 times the original length. Here, each time the level advances by 1, the length of the sound corresponding to each line segment is 1 / √
Although it is twice, it is necessary to set 1/2 times correspondingly, and it is necessary to set the frequency to twice, which corresponds to a sound one octave higher.

Under the above-mentioned constraint conditions, if music is generated from a figure in which an infinite number of levels are superposed, no matter how the reproduction speed is changed, T _c / T _A times or T _A / T.
Exactly the same music appears with different pitches in octave units every _c times. However, in practice, the audible frequency is 20 to 2
If it overlaps only within the range of ten thousand hertz, the same music will sound almost without problems. The above conditions are limited to the length and pitch of the sound. In order to make the timbres the same by changing the reproduction speed, it is necessary to use a musical instrument sound in which the timbre of the low-pitched sound reproduced at high speed and the timbre of the high-pitched sound are close to each other.

An example of the musical score of the music generated from the element graphic of FIG. 10 in this manner is shown in FIG. In this example, the change in the clockwise or counterclockwise direction of the line segment corresponds to the upper and lower pitches of the scale. The scale is
It uses the five-tone scale of Mi, So, La, De, and Re. In addition,
In the third embodiment, when the display image is formed by using the fractal figure, music may be generated based on the fractal figure. By generating music based on a graphic formed using another suitable fractal graphic, it is possible to generate music that is not affected by the playback speed.

FIG. 13 is a diagram showing the overall processing flow of the fourth embodiment. The fourth embodiment corresponds to the one obtained by adding the configuration of performing the visual field conversion / clipping processing to the element graphic data and then performing the graphic note conversion processing in the first embodiment. When drawing a figure, the field-of-view conversion / clipping processing unit 7 performs field-of-view conversion and clipping processing by giving field-of-view data indicating a drawing range or an enlargement ratio to the original elemental figure data, and performs the specified size and designation. The drawing processing unit 8 draws only the specified range. When such visual field conversion and clipping processing are performed, only the apparent elemental figures visible on the screen are regarded as elemental figure data. Therefore, by performing graphic note conversion processing, musical score generation processing, and automatic performance processing on this element graphic data to play music, it is possible to always perform a performance that is unified with an apparent image. In this case, even when the user changes the visual field data in real time, the image and the music follow the change.

If the original figure is a three-dimensional figure,
When drawing, it is usual to set a viewpoint and perform perspective projection. This processing can be carried out by being included in the visual field conversion / clipping processing section 3, but can be carried out by the drawing processing section 8 if the change of the notes becomes too complicated and the musicality is impaired.

[0027]

As described above, according to the first embodiment of the present invention, by connecting the note data obtained by the graphic note conversion processing based on the figure connection data indicating the connection between the element figures. The score data of the music to be generated is generated. In this way, music is created and played based on the graphic data used for drawing, so that the unity between the image and the music is maintained.

In the second embodiment of the present invention, when the musical score data is generated, at the same time, the drawing timing data indicating the timing of drawing each element graphic is also generated, and this is used to draw the graphic. And music performance are synchronized. In this way, the timing between the drawing of the figure and the performance of the music can be synchronized based on the same figure data, so that the editing of the sound, especially the timing adjustment with the image is manually performed. Is unnecessary.

The third embodiment of the present invention uses a fractal graphic generator to generate element graphic data and graphic connection data having self-similarity. By doing so, self-similarity also occurs in the generated music, and the same music can be obtained although the pitch is different regardless of the reproduction speed. Furthermore, the fourth aspect of the present invention
In the embodiment, the visual field conversion / clipping process is applied to the element graphic data in advance to move, enlarge, or reduce the display range, and convert it to the range and size to be actually displayed. By doing so, the note string to be played can be changed and the music corresponding to the displayed image can be automatically played. Even when the user operates the visual field data in real time, by applying the visual field conversion / clipping process to the element graphic data each time, it is possible to deal with the change of the image and music in real time. When the view field data is changed and the figure is enlarged and displayed, the music is played slowly and at low pitches. However, since the figures have self-similarity, the same figure appears even when enlarged, and the same music can be heard at the same time. As a result, the self-similarity, which is a feature of fractals, can be viewed at the same time for both sight and hearing.

The present invention is particularly effective in the production of computer graphic video works, and it is a method of creating and playing music which maintains a close relationship between video and music and maintains the unity of the work. Can be provided. Then, the editing work of video and sound can be reduced. Further, it is possible to provide a music creating and playing method for playing music close to the original music regardless of the playing speed. Furthermore, the music changes in response to the movement of the display range of the image, the enlarged display of a part of the area, and the reduced display of a wide area, and the unity between the displayed image and the music can be maintained. It is possible to provide a method of creating and playing music that also follows real-time operation.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a first embodiment.

FIG. 2 is a diagram showing element graphics.

FIG. 3 is a diagram showing note data.

FIG. 4 is a diagram showing graphic connection data.

FIG. 5 is a diagram showing musical score data.

FIG. 6 is a diagram illustrating a second embodiment.

FIG. 7 is a diagram showing drawing timing data.

FIG. 8 is a diagram illustrating a third embodiment.

FIG. 9 is a diagram illustrating a fractal figure generation process.

FIG. 10 is a diagram showing a fractal figure.

FIG. 11 is a diagram showing graphic connection data of a fractal graphic.

12 is a diagram showing musical score data based on the fractal figure of FIG. 11. FIG.

FIG. 13 is a diagram for explaining the fourth embodiment.

[Explanation of symbols]

 1 Graphic note conversion processing unit 2 Music score generation processing unit 3 Automatic performance processing unit 4 Music score / drawing timing generation processing unit 5 Sequential drawing processing unit 6 Fractal figure generation processing unit 7 Visual field conversion / clipping processing unit 8 Drawing processing unit

Claims (4)

[Claims] 1. The shape of individual elemental figures that make up the figure,
Convert element figure data consisting of size, direction, or color into note data of pitch, length, and tone color, and connect note data based on figure connection data that defines how to connect between element figures A method of creating and playing music, characterized in that the score data of the music to be generated is generated. 2. The method for creating and playing music according to claim 1, wherein the musical score data is generated based on the note data and the graphic connection data, and the drawing timing data is generated to generate the element graphic data and the drawing timing data. A method of creating and playing music, which is characterized by sequentially drawing based on. 3. The music creating and playing method according to claim 1 or 2, wherein the graphic data is element graphic data composed of a fractal graphic which is a self-similar graphic, and between these graphic elements. A method of making and playing music, characterized in that it is composed of graphic connection data for connecting music. 4. The method for creating and playing music according to claim 1, wherein the graphic note conversion processing is performed based on visual field data indicating a range in which a graphic is to be displayed. A method of creating and playing music, characterized by subjecting element graphic data to visual field conversion and clipping processing.