JP6227456B2 - Music performance apparatus and program - Google Patents

Description

  The present invention relates to a music performance apparatus and program for performing performance output of a music based on music data selected from a large number of music data, and in particular, a video transmission process and performance during video shooting with a relatively small amount of data. The present invention relates to an improvement for realizing the reproduction.

  2. Description of the Related Art Music performance devices that perform music performance output based on music data selected from a large number of music data are known. For example, a karaoke device used in a karaoke box or the like. According to such a karaoke apparatus, while outputting a performance sound based on the performance information included in the music data selected from a large number of music data (karaoke performance music) stored in the storage device in advance, the lyrics information of the music Can be displayed on the screen in synchronism with the output of the performance sound, so that karaoke performance of the desired music can be enjoyed.

  By the way, the present applicant incorporated the karaoke device as a communication terminal device into a video information distribution system on the Internet, and recorded video information captured by an imaging device and audio input by a voice input device when a performance song was output by the karaoke device. A technology that accepts and allows browsing of information has been devised and put into practical use. As such, a technique for extracting a predetermined region in an image captured by an imaging device has been proposed. For example, this is the contour extraction method described in Patent Document 1. According to this technology, hierarchical edge data of a selected area is created, and using this, the tracking start point is identified and the contour tracking process is performed to extract the contour data, and then the correction process using the dynamic contour model Thus, by smoothly correcting the contour data, it is possible to extract a smooth contour line that matches the original image pattern even when there is noise or shading unevenness.

JP 2003-216959 A

  By the way, in the video (performance video) taken when the performance music is output by the karaoke device, the scene of the room where the performance is performed as well as the singer, or the performance is heard in the same room (or waiting for the performance). The image of other users is also reflected, but it may be desired to shoot the video of the singer exclusively as the video for posting to the video information distribution system. As a technique for cutting out a predetermined area, for example, a part corresponding to a video of a singer, from a moving image photographed by the imaging device, a technique of applying transmission information such as a luminance key image, a chroma key image, and a depth mat image is known. However, in addition to the moving image shot by the imaging device, uploading the moving image corresponding to the transmission information for performing the transmission processing on the moving image and the information related to the music played during the moving image shooting to the video information distribution system Considering this, there is a harmful effect that the network is loaded due to the large amount of data. In addition, although a technique for compressing a moving image is publicly known, depending on the format of the moving image, it is necessary to upload the moving image and the transparent information separately, which requires a database for associating the moving image with the transparent information, which is burdensome. The evil that occurs. Such a conventional technique cannot solve such a problem. Such a problem has been newly found in the course of continuous research by the present inventor with the intention of improving the performance of the music performance device.

  The present invention has been made against the background of the above circumstances, and the object of the present invention is to provide a music performance device and program that realizes a transparent processing of moving images and reproduction of performance during moving image shooting with a relatively small amount of data. Is to provide.

  In order to achieve such an object, the gist of the first aspect of the present invention is that a performance means for playing music data and the music data obtained by the performance means are acquired from the imaging device. An acquisition unit that acquires moving image information, a transmission information acquisition unit that acquires transmission information for performing transmission processing on the acquired moving image information, and a first moving image for each of a plurality of frames constituting one moving image file An area and a second moving image area are defined, moving image information captured by the imaging device is pasted on the first moving image area, and transmission information acquired by the transmission information acquiring unit is pasted on the second moving image area. Paste means, music data played by the performance means, and correspondence means for storing in association with the moving image file pasted by the paste means. Is a music playing device according to claim.

  According to the first aspect of the present invention, the performance means for playing music data, the acquisition means for acquiring moving image information acquired from the imaging device in response to the music data being played by the performance means, and the acquired Transparent information acquisition means for acquiring transparent information for performing transparent processing on the moving image information, a first moving image region and a second moving image region are defined for each of a plurality of frames constituting one moving image file, and the first The moving image information taken by the imaging device is pasted to the moving image area, and the second moving image area is played by the pasting means for pasting the transparent information acquired by the transparent information acquiring means and the performance means. Since it includes music data and association means for storing in association with the moving image file pasted by the pasting means, In addition, the moving image information for performing the transparency processing on the moving image is integrated into one moving image file, and further, the information on the music played at the time of moving image shooting is associated with the moving image transparency processing and moving image with the smallest possible data amount. The performance of the music that was output at the time of shooting can be realized. In other words, it is possible to provide a music performance device that realizes moving image transmission processing and reproduction of performance during moving image shooting with a relatively small amount of data.

  In the first aspect of the invention, preferably, from the moving image file, reading means for reading out the moving image pasted in the first moving image area and the transparent information pasted in the second moving image area, and reading by the reading means. Based on the issued moving image and the transmission information, the display control means for displaying the moving image that has been subjected to the transmission processing on the moving image on the moving image on the display device, in parallel with the display of the moving image by the display control means, Performance output control means for performing performance output of music based on music data associated with the moving image file is provided. In this way, for example, a moving image file uploaded to a predetermined server, such as being shot by another musical performance device, is downloaded by the predetermined musical performance device, and the transparent processing of the moving image and the moving image are performed in the musical performance device. Reproduction of performance at the time of shooting can be realized.

  Preferably, the music performance device is a karaoke device connectable to the imaging device. In this way, it is possible to provide a karaoke apparatus that realizes moving image transmission processing and reproduction of performance during moving image shooting with a relatively small amount of data.

  Preferably, the pasting unit changes the size of the moving image to be pasted to the first moving image area and the size of the transparent information to be pasted to the second moving image area, and They are pasted to the moving image area and the second moving image area, respectively. In this way, it is possible to further reduce the data amount of the moving image file that realizes the transparent processing.

  Preferably, recording means for recording a sound signal input from the input device, second correspondence means for storing the sound signal recorded by the recording means in association with the moving image file, and the display control And a sound signal output control means for outputting a sound signal associated with the moving image file in parallel with the display of the moving image by the means. In this way, in addition to integrating the moving image and the moving image information for performing the transparency process on the moving image into one moving image file, the singing voice of the singer at the time of moving image recording is recorded and associated, Therefore, it is possible to realize moving image transmission processing and reproduction of karaoke performance at the time of moving image shooting with a small amount of data.

  In order to achieve the above object, the gist of the second aspect of the present invention is to provide a control device for a music performance device that performs musical performance output based on music data selected from a large number of music data. A performance means for performing the performance, an acquisition means for acquiring the moving image information acquired from the imaging device in response to the music data being played by the performance means, and a transmission for performing a transmission process on the acquired moving image information Transparent information acquisition means for acquiring information, a first moving image area and a second moving image area are defined for each of a plurality of frames constituting one moving image file, and moving image information photographed by the imaging device in the first moving image area Pasting means for pasting transparent information acquired by the transparent information acquiring means to the second moving image area, and music data being played by the performance means, Is a program for causing to function as correlation means for storing in association with the attached by serial paste means the video file. In this way, a moving image and moving image information for performing transparency processing on the moving image are integrated into one moving image file, and information relating to the music played at the time of moving image shooting is associated with each other so that the amount of data is as small as possible. Thus, it is possible to realize the performance of the music that was output at the time of moving image transmission processing and moving image shooting. That is, it is possible to provide a program that realizes moving image transmission processing and reproduction of performance during moving image shooting with a relatively small amount of data.

It is a block diagram which illustrates the composition of the karaoke device which is a suitable example of the music performance device of the present invention. It is a figure which shows a mode that the karaoke apparatus of FIG. 1 was connected with the other apparatus via the communication line. It is a functional block diagram explaining the principal part of the control function with which the karaoke apparatus of FIG. 1 was equipped. It is a figure which illustrates the image image | photographed with the digital camera with which the karaoke apparatus of FIG. 1 was equipped. FIG. 5 is a diagram illustrating a transmission information image corresponding to transmission information for performing transmission processing on the image of FIG. 4. It is a figure which illustrates the partial image | video obtained by performing the permeation | transmission process with the permeation | transmission information of FIG. 5 with respect to the image of FIG. It is a figure which illustrates the image | video with which the partial image shown in FIG. 6 is synthesize | combined. It is a figure which illustrates the image | video which synthesize | combined the partial image | video shown in FIG. 6 with the front side of the image | video shown in FIG. It is a figure explaining the specific example of pasting of the entity picture and transparent information picture to a movie file by the karaoke device of FIG. It is a figure which shows the example which changes and pastes a size as a specific example of the attachment of the entity image and transparent information image to a moving image file by the karaoke apparatus of FIG. It is a figure which shows the example which changes and pastes a size as a specific example of the attachment of the entity image and transparent information image to a moving image file by the karaoke apparatus of FIG. It is a flowchart explaining the principal part of an example of the moving image file creation / transmission control of a present Example by CPU of the karaoke apparatus of FIG. It is a flowchart explaining the principal part of an example of the moving image file read-out / reproduction | regeneration control of a present Example by CPU of the karaoke apparatus of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating the configuration of a karaoke apparatus 10 which is a preferred embodiment of a music performance apparatus of the present invention. As shown in FIG. 1, a karaoke apparatus 10 according to this embodiment includes a CPU 12 that is a central processing unit, a ROM 14 that is a read-only memory, a RAM 16 that is a write / read memory as needed, and a hard disk 18 that is a storage device. And the video processing unit 20 such as a graphics chip (graphic board), the audio processing unit 22 such as a sound chip (sound board), the operation panel 24, the display control unit 26, and the input control unit 28. A touch panel display 30 connected to the CPU 12, a communication interface 36, a LAN interface 38, a wireless communication unit 40, and a display 44 which is a video display device connected to the video processing unit 20 via a video output terminal 42. Audio connected to the audio processing unit 22 via the audio input terminal 46 A microphone 48 that is a power device, an amplifier 52 that is a sound amplifying device connected to the sound processing unit 22 via an audio output terminal 50, and a speaker 54 that is a sound output device provided in the amplifier 52. A digital camera 70 that is an imaging device connected via a terminal 72 is provided.

  The CPU 12 is a so-called microcomputer that processes and controls electronic information based on a predetermined program stored in advance in the ROM 14 using the temporary storage function of the RAM 16, and executes various controls in the karaoke apparatus 10. . That is, when a predetermined music piece (karaoke performance music piece) is selected and input by the operation panel 24, the touch panel display 30, or the electronic quick sample device 68, the reserved music provided in the RAM 16 or the like is input to the selected music piece. Music selection reservation control to be registered in the table, music data read control for reading performance information and lyrics information (music data) of the karaoke performance music selected from the hard disk 18 in accordance with the performance order of the reserved music table, music performance reading progress In response to the performance output control for transmitting performance information from the RAM 16 to the sound processing unit 22, a lyric character image is generated based on the lyric information developed in the RAM 16 during the performance output control, and sent to the video processing unit 20. The lyric character video output control to be transmitted, the video processing in the performance output control 20 is a background video output control that reproduces a predetermined background video, and inter-song information that outputs inter-song information such as new music information, music selection rankings, store advertisements, etc. while karaoke performance is not being performed. In addition to basic control such as output control, various types of control such as moving image file creation control according to the present embodiment to be described later are executed. In this embodiment, the CPU 12 corresponds to the control device of the karaoke apparatus 10.

  The video processing unit 20 performs various controls related to drawing of a screen (video) displayed on the display 44. For example, based on the data supplied from the CPU 12, the data is written into the graphics memory, and the data is read out so that a predetermined screen is displayed on the display 44 through the video output terminal 42. Specifically, when the karaoke apparatus 10 outputs a musical performance (karaoke performance), a text image (telop) such as a lyric character image generated by the CPU 12 is output, or MPEG (stored in the hard disk 18) is stored. Based on background video information such as Moving Picture Experts Group) data, a predetermined background video is reproduced (decoded), the lyrics character video is synthesized on the front side of the background video and displayed on the display 44, Various display controls are performed such as displaying lyric text images in a color-changed manner as the music progresses.

  The sound processing unit 22 includes various sound sources such as an FM sound source and a PCM sound source, and performs various controls related to sound output by the karaoke apparatus 10. Preferably, a synthesizer that synthesizes sound by an electronic circuit and generates various timbres is provided. The synthesizer generates a music signal such as a musical instrument performance signal based on performance information of a karaoke performance song read from the hard disk 18 and sent. The synthesizer preferably includes a MIDI (Musical Instrument Digital Interface) terminal, and the performance information is, for example, data in the MIDI format. The music signal generated by the synthesizer based on the MIDI data is mixed with the singing voice of the user (performer) input from the microphone 48 via the audio input terminal 46, and via the audio output terminal 50. Is supplied to the amplifier 52, amplified by the amplifier 52, and output from the speaker 54.

  The operation panel 24 selects a song that the user of the karaoke apparatus 10 wants to sing, adjusts the pitch related to the performance output of the song, adjusts the volume balance between the performance and the song, The input device includes operation buttons (switches) or knobs for performing various adjustments such as volume and tone. The touch panel display 30 is a device that displays an image (video) and inputs an operation to the karaoke device 10 according to a user's operation. The touch panel display 30 displays a predetermined image (video). And a touch panel 34 that performs input by touching the touch panel display 30 with a user's finger, an unillustrated pen, or the like. The display control unit 26 is a video processing unit that controls drawing of a screen (video) displayed on the display device 32 based on information supplied from the CPU 12. The input control unit 28 is an input processing unit that supplies operation input information input from the touch panel 34 to the CPU 12 and the like. With the above configuration, the touch panel display 30 functions as a second video display device separately from the display 44, and the user of the karaoke device 10 selects a song that the user wants to sing, It functions as an input device for adjusting the pitch related to the performance output, adjusting the volume balance between the performance and the song, and performing various adjustments such as echo, volume and tone.

  The karaoke apparatus 10 is connected to a communication line 60 via the communication interface 36 and a network connection terminal 56. The communication line 60 is a wide area information communication network connected to the Internet such as a WWW (World Wide Web) constituted by, for example, a public telephone line, an ADSL line, or an optical fiber line. Similarly, the karaoke apparatus 10 can communicate information with the other devices connected to the communication line 60 via the communication line 60. FIG. 2 is a diagram illustrating a state in which the karaoke apparatus 10 is connected to another device via the communication line 60. As shown in FIG. 2, the karaoke apparatus 10 is preferably connected to a server (host apparatus) 80 via the communication line 60, and music data (karaoke data), background video is transmitted from the server 80. Information and distribution of digital contents such as information between songs are accepted. Further, as will be described later, the moving image file created in the karaoke apparatus 10 is transmitted (uploaded) to the server 80 via the communication line 60 and stored in the moving image database 78 provided in the server 80. It is like that. That is, the karaoke apparatus 10 is preferably a communication karaoke apparatus that is connected to a predetermined communication line and transmits / receives various information to / from the server, but is not connected to such a communication line. The present invention is also suitably applied to devices and the like.

  Various communication terminal devices are preferably connected to the communication line 60 so that information can be transmitted and received between the server 80 and the like. For example, as shown in FIG. 2, the communication line 60 includes a plurality of mobile phones 82 (one is illustrated in FIG. 1) and a plurality of tablets (one is illustrated in FIG. 1). A terminal 84 and a plurality of personal computers 86 (one is illustrated in FIG. 1) are connected. The mobile phone 82 is preferably a so-called smartphone having a touch panel display and a function of displaying various contents on the Internet on the touch panel display. The tablet terminal 84 preferably includes a touch panel display and a function for displaying various contents on the Internet on the touch panel display. The personal computer 86 is a known home personal computer that includes an input device and a display device, and has a function of displaying various contents on the Internet on the display device. The mobile phone 82 and the tablet terminal 84 are connected to the communication line 60 via a relay base station 88, and information can be communicated between the server 80 and the mobile phone 82 or the tablet terminal 84. It is said that. The personal computer 86 is connected to the communication line 60 so that information can be communicated with the server 80 via the communication line 60.

  The LAN interface 38 is a connector for connecting the karaoke device 10 to a local area network (LAN) 64 via a LAN connection terminal 62, and thus connected to the LAN 64, the karaoke device 10 is also capable of transmitting / receiving information to / from other devices such as the electronic sample device 68 connected to the LAN 64. Further, when a plurality of karaoke apparatuses are provided in a store or the like where the karaoke apparatus 10 is installed, information can be transmitted and received between karaoke apparatuses connected to the LAN 64 in the same manner. For example, the music selection input from the electronic sample device 68 received via the access point 66 connected to the LAN 64 is received and stored in the reserved music table provided in the RAM 16, or via the access point 66. Mutual information exchange is performed between the karaoke device 10 and the electronic quick sample device 68 via radio waves, such as transmitting predetermined information from the karaoke device 10 to the electronic quick sample device 68.

  The wireless communication unit 40 performs wireless communication between the karaoke device 10 and an input device such as the electronic quick sample device 68. For example, it functions as a remote control receiving unit that receives a remote control signal transmitted from an input device such as the electronic quick sample device 68. The association (sticking) process between the karaoke apparatus 10 and the electronic quick sample apparatus 68 is preferably performed via the wireless communication unit 40 by such a remote control signal (infrared signal). That is, the electronic sample device 68 has an individual serial number, and in the association process, for example, a signal including the serial number (for example, the last four digits) and a predetermined connection code is notified of the connection. The electronic quick sample device 68 is associated with the karaoke device 10 that is transmitted to the karaoke device 10 and receives the connection signal by the wireless communication unit 40. The electronic quick sample device 68 thus associated with the karaoke device 10 functions as an input device (remote control device) of the karaoke device 10, and a signal transmitted from the electronic quick sample device 68 is the CPU 12. , The user of the karaoke apparatus 10 selects a song that the user wants to sing, adjusts the pitch related to the performance output of the song, adjusts the volume balance between the performance and the song, and so on. Input for making various adjustments such as volume, tone, etc. is accepted. After the association process is performed, communication between the electronic sample device 68 and the karaoke apparatus 10 is performed by LAN communication via the LAN 64, the access point 66, and the like. Further, in the present embodiment, the following description will be given on the assumption that the input device such as the electronic quick sample device 68 subjected to the association process with the karaoke device 10 also constitutes a part of the karaoke device 10.

  The digital camera 70 includes, for example, an image sensor such as a CCD (charge coupled device) and a lens, detects an image incident from the lens by the image sensor, and acquires the image as electronic information (video data). A video camera that can shoot at least moving images (moving images that change over time), but is configured to shoot still images (still photos) as needed. Also good. Video information photographed by the digital camera 70 is supplied to the CPU 12 and the like via an interface such as the connection terminal 72 and the like. For example, AVI (Audio-Video Interleaved) format, MPEG (Moving Picture Experts Group) format, FLV (FLV) Flash video) is stored in the RAM 16 or the like as a video file.

  The digital camera 70 is preferably a known video camera with a depth sensor. That is, it is a video camera that can detect the three-dimensional distance of an object to be photographed, and can detect a partial video of a person in a predetermined distance range from the digital camera 70 in the photographed moving image, for example. More preferably, concurrently with the shooting of the moving image, transmission information such as a luminance key image or a chroma key image, which will be described later, is acquired as information for cutting out, for example, a human image from the moving image. The digital camera 70 does not necessarily have to be provided as a part of the karaoke apparatus 10, for example, a separate video camera or the like fixed at a predetermined position in a room where the karaoke apparatus 10 is installed. A mode in which an image captured by an imaging device or the like provided in the mobile phone 82 or the like is input to the karaoke apparatus 10 via a predetermined interface is also conceivable.

  The hard disk 18 is provided with various databases including a music database 76 (see FIG. 3). The music database 76 stores a large number (for example, tens of thousands of songs) of music data (karaoke data) for the performance output (karaoke performance) of the music by the karaoke apparatus 10. The music data stored in the music database 76 corresponding to each music output in the karaoke apparatus 10 includes, for example, metadata, MIDI data as performance information, and telop data as lyrics information. This metadata includes, for example, information such as song number (music selection number), song name, singer name, registration date and time as identification information of each song data. The telop data is information for performing lyric character video display control of each song via the video processing unit 20. For example, each of the lyrics of a song (performance song) corresponding to the song data is divided into a plurality of pieces. The block data corresponding to the section is included. The MIDI data is information for performing performance output control of each piece of music via the audio processing unit 22, and includes a plurality of tracks (channels) respectively corresponding to a plurality of types of musical instruments. The output timing of the performance sound is defined.

  FIG. 3 is a functional block diagram for explaining a main part of the control function provided in the karaoke apparatus 10. Each control means shown in FIG. 3 is preferably functionally provided in the CPU 12 of the karaoke apparatus 10. In other words, the CPU 12 is caused to function as each control unit shown in FIG. 3 by the program according to the embodiment of the present invention. A program according to an embodiment of the present invention is downloaded by the mobile phone 82, the tablet terminal 84, or the like through the communication line 60, for example, so that their CPUs function as the respective control means shown in FIG. It may be. The reading means 116, the display control means 118, the performance output control means 120, and the sound signal output control means 122 shown in FIG. 3 are not necessarily provided in the CPU 12 of the karaoke apparatus 10, and are not connected to the communication line 60. It may be provided in the control device of the connected communication terminal device. For example, it may be provided in a control device of a communication terminal device such as the mobile phone 82, the tablet terminal 84, and the personal computer 86. In the following embodiment, a mode in which the reading unit 116, the display control unit 118, the performance output control unit 120, and the sound signal output control unit 122 are provided in the CPU 12 of the karaoke apparatus 10 will be described.

  The karaoke apparatus 10 of the present embodiment includes the transparent information acquisition unit 104 and the moving image file creation unit 106 shown in FIG. 3, so that, for example, FIG. As will be described later, a process for obtaining a transmission image with a portion other than the image corresponding to the user (singer) as a transmission area is realized. As a conventional technique for obtaining such a transmission image, an alpha channel method is known. In this alpha channel method, image (moving image) data is transferred to a color channel of R (red) G (green) B (blue) or Y (luminance) U (color difference component) V (color difference component) A (transmission information). RGBA channel or YUVA channel with additional alpha channel added. That is, by having four channels including an alpha channel corresponding to transmission information, a moving image having transmission information can be realized. However, in this method, mounting of only RGB or YUV color channels is defined in transmission or recording standards, and a transparent image cannot be realized in a format in which an alpha channel cannot be mounted.

  Further, as another conventional technique for obtaining the transparent image, a method of separately transmitting and recording an actual image and a transparent information image is known. This method is a method of performing transmission processing using moving images of two RGB or YUV color channels for image transmission and recording methods of only RGB or YUV color channels that do not have transmission information. Two images, a target entity image to be processed and a transparent information image, are prepared separately. This transparent information image is, for example, a known Luminance Key image or Chroma key image. Based on the luminance key image or chroma key image obtained from the transmission information image, a luminance image or chroma key process is performed on the actual image to cut out the transmission range to obtain a transmission image. However, in this method, two images, that is, an entity image and a transparent information image must be prepared, and information that associates (links) the entity image and the transparent information image to each other is required.

  The karaoke apparatus 10 according to the present embodiment includes each control unit illustrated in FIG. 3, thereby realizing a screen division simultaneous transmission and recording method for an entity image and a transmission information image. That is, it realizes a system that can provide transmission information with only one image transmission and recording for an image transmission and recording format that does not have an RGB channel of only RGB or YUV color channels. Specifically, one image is divided into left and right or up and down to divide the image into a substantial image area and a transparent information image area, and a substantial image and a transparent information image (such as a luminance key image or a chroma key image) are divided into each area. Record. In the transparency processing, the entity image area and the transparency information image area are separately read out from the one image, and based on the luminance key image or chroma key image obtained from the transparency information image, Chroma key processing is performed to cut out the transmission range and obtain a transmission image. In the method of this embodiment, transmission information can be held in one image even in a format that cannot implement the alpha channel, and in the conventional method, two separate images are required in one image. A transmission image can be obtained. Hereinafter, in conjunction with the description of each control function shown in FIG. 3, the screen division simultaneous transmission and recording method of the real image and the transparent information image of this embodiment will be described in detail.

  The music performance control means 100 shown in FIG. 3 controls the performance (reproduction) of music by the karaoke apparatus 10. Specifically, in the karaoke performance based on the music data stored in the music database 76, the reserved music stored in the reserved music table in order from the upper reserved music in the reserved music table such as the RAM 16 (in order of input). The music data corresponding to the music selection number of the music is read from the music database 76 of the hard disk 18, and the output of the performance music is controlled based on the performance information included in the music data. For example, based on MIDI data as performance information, a synthesizer provided in the sound processing unit 22 outputs performance sounds (music information) of musical instruments corresponding to each track as score information in the MIDI data, and the amplifier The sound is output from the speaker 54 via 52 or the like. Moreover, the output of the lyric character video concerning the performance music is controlled based on the lyric information included in the music data. That is, a lyric character image is generated based on the lyric information, and the lyric character image is displayed on the display 44 via the image processing unit 20. Further, as the karaoke performance progresses, the lyric character image displayed on the display 44 is switched and displayed, and the lyric character image is sequentially color-changed and displayed based on the lyric color change information. In this embodiment, the music performance control means 100 corresponds to a performance means for playing music data.

  The recording means 102 records a sound signal (singing voice signal) input from the microphone 48 as an input device (speech input device) at the time of performance output based on the music data by the music performance control means 100. For example, the sound signal input from the microphone 48 is stored in the RAM 16 or the like, and stored as audio data in the MPEG (Moving Picture Experts Group) format or WAV format after the performance is completed. Preferably, the sound signal input from the microphone 48 is recorded at the time when the moving image to be pasted on the first moving image area by the pasting means 108 described later is photographed by the digital camera 70. That is, the sound signal input from the microphone 48 is recorded when a moving image corresponding to an entity image to be processed by the moving image file creation unit 106 described later is captured by the digital camera 70.

  The transmission information acquisition unit 104 acquires transmission information for performing transmission processing on a moving image shot by the digital camera 70 that is an imaging device. In other words, a moving image photographed by the digital camera 70 is used as an entity image to be subjected to transparency processing, and a transmission information image for clipping a predetermined partial image (for example, corresponding to a user image) from the entity image is acquired. To do. Specifically, when a moving image photographed by the digital camera 70 is an image (still image) corresponding to a plurality of consecutive wrinkles, a sequence for cutting out a predetermined partial video from each image as a substantial image A transmission information image corresponding to a plurality of eyelids is acquired. In other words, a plurality of continuous transmission information images for performing transmission processing on a moving image shot by the digital camera 70 that is an imaging device are acquired. Information captured by the digital camera 70 may be transmitted to a device such as the karaoke apparatus 10 and the transparent information may be generated on the device side.

  Preferably, when the digital camera 70 is a video camera with a depth sensor, the transmission information acquisition unit 104 detects the transmission information acquisition unit 104 together with the moving image (substance image) being captured by the digital camera 70. From the three-dimensional distance information, a transmission information image for cutting out a predetermined partial image in the moving image is created. More preferably, transmission information such as a luminance key image, chroma key image, or depth matte image acquired by the digital camera 70 is acquired in parallel with the shooting of the moving image by the digital camera 70. That is, when a moving image is shot by the digital camera 70, the moving image corresponding to the actual image and the transmission information (a plurality of continuous transmission information images) corresponding to the moving image are acquired from the digital camera 70.

  The transmission information acquiring unit 104 is preferably based on a difference between an image captured by the digital camera 70 that does not include a partial video to be clipped from a moving image and an image that includes a partial video to be clipped. The transmission information for cutting out the partial video in the moving image is created. For example, when an image of a user who is performing using the karaoke device 10 is cut out as the partial image, the digital camera 70 uses the image that does not include the user image (for example, the karaoke device 10 is installed). A room landscape video) and a video including a user image (for example, a video of a user performing in a room where the karaoke apparatus 10 is installed), and based on the difference between them, Transmission information (a plurality of continuous transmission information images) for cutting out the image of the user from the moving image taken by the digital camera 70 is created. In the case where the digital camera 70 does not include a depth sensor, such processing is suitably executed in order to acquire the transmission information.

  FIG. 4 is a diagram illustrating an actual image 130 corresponding to a moving image photographed by the digital camera 70. FIG. 5 is a diagram illustrating a transparent information image 132 corresponding to the transparent information for performing the transparent process on the entity image 130 of FIG. The transparent information image 132 shown in FIG. 5 is, for example, a known Luminance Key image or Chroma key image, and detects specific signals such as luminance, hue, color, and motion difference on the screen. This is an embodiment of a keyer that extracts it. Specifically, the transparent information image 132 is, for example, the same size as the entity image 130 to be cut out of a predetermined portion, and the portion to be cut out and the other portion have different brightness, hue, It is an image divided by color or the like. FIG. 5 illustrates a transparent information image 132 for cutting out the partial image 130a corresponding to the person image in the entity image 130 shown in FIG. By performing transparent processing (luminance key processing, chroma key processing, etc.) on the entity image 130 shown in FIG. 4 using the transmission information image 132 shown in FIG. 5, the partial image is displayed from the entity image 130 as shown in FIG. Only 130a can be cut out. In FIG. 6, a range other than the partial image 130 a is a transmissive region 134, and the transmissive region 134 is indicated by diagonal lines from the upper right to the lower left. Note that in FIGS. 4 to 6 (FIGS. 7 to 11 and the like to be described later), the transparency process is described using still images for convenience, but a plurality of processes corresponding to such still images are continuously performed. By being performed on the image, the transmission process is similarly realized for the moving image, and a predetermined partial image is cut out (extracted) from the moving image, and an image in which the other portion is a transmissive region can be created.

  The image as shown in FIG. 6 in which the partial image 130a is cut out from the entity image 130 as described above and the other portion is the transmissive region 134, is synthesized as if it is combined with the front side of the other image. An image in which the partial image 130a is included in the image can be created. For example, the partial image 130a corresponding to the image of the person shown in FIG. 6 is synthesized on the front side of the burning flame image 136 as shown in FIG. 7, so that the partial image 130a is combined with the partial image 130a as shown in FIG. An image 138 in which the image of the corresponding person carries a flame on the back can be realized.

  The moving image file creating means 106 shown in FIG. 3 corresponds to the moving image taken by the digital camera 70, and includes one moving image file including the moving image and transparency information for performing the transparency process on the moving image. Create In order to realize such processing, the moving image file creation unit 106 includes a pasting unit 108, a music association unit 110, and a recorded data association unit 112.

  The pasting means 108 determines a first moving image area and a second moving image area for each of a plurality of frames (齣) constituting the moving image file with respect to one moving image file. An entity image 130 corresponding to the captured moving image is pasted, and a transparent information image 132 corresponding to the transparent information acquired by the transparent information acquiring unit 104 is pasted to the second moving image area. FIG. 9 is a diagram illustrating a specific example of pasting of the entity image 130 and the transparent information image 132 to the moving image file by the pasting unit 108. In FIG. 9, a frame constituting a moving image file to which the entity image 130 and the transparent information image 132 are pasted is divided into left and right (side-by-side division), and a normal ratio image of the entity image 130 and the transparent information image 132 is recorded. The mode to do is illustrated. As shown in FIG. 9, the moving image file creating means 106 prepares an empty moving image file (moving image file that does not include substantial video information) and records, for example, each of a plurality of frames constituting the moving image file. A first moving image area 140 and a second moving image area 142 are defined by dividing the screen area equally on the left and right. Then, the pasting means 108 pastes (inserts) the actual image 130 corresponding to the moving image photographed by the digital camera 70 into the first moving image area 140, and pastes (inserts) the substantial image 130 into the first moving image area 140. A transparent information image 132 for cutting out a predetermined partial image 130 a in the entity image 130 is pasted (inserted) into the second moving image area 142. It is necessary to cut out a predetermined partial video 130a from the moving image taken by the digital camera 70 by pasting the entity image 130 and the transparent information image 132 to each of a plurality of frames constituting the moving image file. One moving image file including information can be created.

  The pasting means 108 preferably corresponds to the entity image 130 corresponding to the moving image to be pasted to the first moving image area 140 and the transparent information to be pasted to the second moving image area 142. The size of the transparent information image 132 to be changed is changed and pasted to the first moving image area 140 and the second moving image area 142, respectively. 10 and 11 are diagrams for explaining a specific example of processing for changing the sizes of the entity image 130 and the transparent information image 132 and pasting them into one moving image file. In FIG. 10, the frames constituting the moving image file to which the entity image 130 and the transparent information image 132 are to be pasted are divided into left and right (side-by-side division). The aspect which records the anamorphic ratio image made into 50% of right and left is illustrated. In FIG. 11, the frames constituting the moving image file to which the entity image 130 and the transparent information image 132 are to be pasted are vertically divided (top and bottom division), and the entity image 130 and the transparent information image 132 are respectively divided into the upper and lower 50 parts. The aspect which records the anamorphic ratio image made into 100% and% 100% is illustrated. With such a moving image file, as shown in FIG. 6, in addition to being able to perform a transparent process of cutting out a predetermined partial image from the moving image captured by the digital camera 70, data of the moving image file that realizes such a transparent process There is an advantage that the amount can be further reduced.

  The music association means 110 shown in FIG. 3 is played by the karaoke apparatus 10 when a moving image to be pasted on the first moving image area 140 by the pasting means 108 is shot by the digital camera 70. Information related to the music data corresponding to the current music is stored in association with the moving image file. Here, the information related to the music data to be associated with the moving image file is for specifying the music played by the karaoke device 10 at the time of being photographed by at least the digital camera 70. Information, for example, a music selection number, a song name, and a singer name included in the metadata of the song data. Alternatively, the music data that has been played by the karaoke apparatus 10 at the time of being photographed by the digital camera 70 may be stored in association with the moving image file.

  The recorded data association unit 112 stores the sound signal recorded by the recording unit 102 in association with the moving image file. That is, the sound that is input from the microphone 48 and recorded by the recording unit 102 when the moving image to be pasted on the first moving image area 140 by the pasting unit 108 is captured by the digital camera 70. Recording data corresponding to the signal is stored in association with the moving image file. Instead of storing the recording data itself in association with the moving image file, a user ID related to the recording data, a management number for managing recording information, and the like may be stored in association with the moving image file. The video file may be associated with both information relating to music data by the music association means 110 and recording data correspondence by the recording data correspondence means 112, or the music correspondence. Only the association by either one of the attaching unit 110 and the recording data association unit 112 may be performed. That is, in this embodiment, the recorded data association unit 112 corresponds to a second association unit that stores the sound signal recorded by the recording unit 102 in association with the moving image file.

  The transmitting unit 114 transmits (uploads) the moving image file created by the moving image file creating unit 106 to the server 80 via the communication line 60 and stores it in the moving image database 78. Preferably, the user ID as the identification information of the user who is the transmission subject of the moving image file (for example, the user who was performing with the karaoke device 10 when shooting with the digital camera 70), the shooting date and time, and the shooting location (for example, Information such as the identification information of the karaoke apparatus 10, the title of the moving image, and the description of the moving image (for example, description by short text) is stored in the moving image database 78 in association with the moving image file. As the format of the file recorded in the moving image database 78 (that is, the format of the file created by the moving image file creating means 106), the well-known H.264 standard is used. H.264 file, H.264 265 files, MPEG-2 files, etc. are conceivable.

  The moving image file stored (accumulated) in the moving image database 78 as described above is used by the reading means 116, the display control means 118, the performance output control means 120, and the sound signal output control means 122 shown in FIG. . As described above, these control means may be provided in a control device of a communication terminal device such as the mobile phone 82, the tablet terminal 84, and the personal computer 86. That is, the control described in detail below may be executed in a communication terminal device such as the mobile phone 82, the tablet terminal 84, and the personal computer 86.

  The reading unit 116 acquires (downloads) the moving image file stored in the moving image database 78 via the communication line 60, and from the moving image file, the moving image pasted in the first moving image area 140 and the second moving image file. The transmission information pasted on the moving image area 142 is read out. That is, the transparent image corresponding to the transmission information for cutting out the partial video in the entity image 130 while reading out the entity image 130 corresponding to the moving image to be clipped of the predetermined partial image from the first moving image area 140. The image 132 is read from the second moving image area 142. As a moving image information acquisition format (video transmission method) from the server 80, moving image streaming or moving image download by known IP communication is conceivable. A signal transmitted via a known HDMI (registered trademark) interface, an SDI signal, a composite signal, a broadcast radio wave, or the like may be used. In such an aspect, once the moving image information is downloaded to the karaoke apparatus 10 or the like, the mobile phone is used by using a signal, an SDI signal, a composite signal, wireless communication, or the like transmitted via an HDMI (registered trademark) interface. Distribution of moving image information to the telephone 82 and the like is performed. The acquisition of the moving image file from the moving image database 78 by the reading unit 116 is preferably performed in accordance with designation of a moving image title, a user ID of a contributor, a music selection number, and the like. That is, for example, when a predetermined movie title, user ID, song selection number, etc. are specified by an input operation via the electronic quick sample device 68 or the like, the movie database 78 stores the movie. Acquire (download) a video file corresponding to each information from the video file.

  Based on the moving image and the transmission information read by the reading unit 116, the display control unit 118 displays the moving image obtained by performing transmission processing on the moving image with the transmission information on the display 44 or the like as a display device. . For example, the transparent information image 132 as shown in FIG. 5 read by the reading means 116 is used, and the substantial image 130 shown in FIG. By performing key processing or chroma key processing), only the partial image 130a is cut out from the entity image 130 as shown in FIG. Here, if the size of the entity image 130 and the transparent information image 132 has been changed in the creation of the moving image file to be read out of the moving image and the transparent information, the display control unit 118 preferably The substantial image 130 and the transparent information image 132 are returned to their original sizes, and a transparent process is performed. After performing the transparency process, the entity image 130 and the transparency information image 132 may be returned to their original sizes. Regarding the restoration of the size, information (for example, change ratio) related to the size change performed in the creation of the moving image file is stored in association with the moving image file, and the size is restored based on the information. The file creation unit 106 and the display control unit 118 may share in advance information related to the size change in the movie file creation stage, and the size may be restored based on the information. You may do it.

  The display control means 118, for example, based on the moving image read by the reading means 116 and the transmission information, the moving image, that is, the partial image obtained by performing the transmission processing on the moving image with the transmission information, as described above with reference to FIG. As shown in FIG. 4, display control is performed such that the image is synthesized on the front side of another image or moving image and displayed on the display 44 or the like. As the images or moving images synthesized on the back side of the partial video subjected to the transparent processing, those previously determined and stored in the hard disk 18 or the like may be selectively used. Any one of the moving image data stored in the moving image database (for example, another moving image database different from the moving image database 78 shown in FIG. 3) is selected and used. For example, when a moving image in which another user performs a karaoke performance using the karaoke device 10 is stored in the other moving image database, the moving image is read and the partial image subjected to the transmission process is displayed on the front side. By synthesizing and displaying the images, it is possible to display the video as if those users (the user related to the partial video and the user related to the other moving image data) are performing the karaoke performance in the same room. Can be displayed.

  The performance output control means 120 performs the performance output of the music based on the music data associated with the video file in parallel with the display of the video by the display control means 118. Preferably, a music selection number associated with the moving image file is read out, and music data corresponding to the music selection number is read out from the music database 76. Then, in parallel with the reproduction (display control) of the moving image corresponding to the moving image file by the display control means 118, the performance output of the music corresponding to the read music data is performed. That is, in the present embodiment, the music performance control means 100 may function as the performance output control means 120.

  The sound signal output control unit 122 outputs a sound signal associated with the moving image file in parallel with the display of the moving image by the display control unit 118. Preferably, simultaneously with the start of the display control of the moving image by the display control means 118, the output of the sound signal associated with the moving image file is started, and the display of the moving image corresponding to the moving image file by the display control means 118 is started. In parallel with the reproduction (display control), the sound signal associated with the moving image file is output (recorded data is reproduced). For example, the recorded data obtained from a predetermined database in the server 80 may be reproduced together with the reproduction of the recorded data associated with the moving image file. For example, when a sound signal corresponding to a karaoke performance performed by another user using the karaoke apparatus 10 is stored in the database, the recorded data is read out together with the recorded data associated with the moving image file. By reproducing, it is possible to realize sound output as if those users (the user related to the partial video and the user related to the other recorded data) are performing the karaoke performance together. Furthermore, as described above, by outputting a video in which the partial video corresponding to the user is combined with the other moving image data, it is as if those users (the user related to the partial video and the other moving image data are included). Such a user) can realize a video as if performing a karaoke performance in the same room and an audio output corresponding to the video.

  FIG. 12 is a flowchart for explaining a main part of an example of the moving image file creation / transmission control of the present embodiment by the CPU 12 of the karaoke apparatus 10 and is repeatedly executed at a predetermined cycle.

  First, in step (hereinafter, step is omitted) SA1, whether or not the karaoke performance corresponding to the predetermined music data is started in the order of performance of the reserved music table stored in the RAM 16 or the like. To be judged. If the determination at SA1 is negative, the routine is terminated. If the determination at SA1 is affirmative, the music data relating to the start of performance is read from the music database 78 at SA2. Then, the music is played, and the digital camera 70 shoots along with the performance output. In parallel with the processing of SA2, the sound signal input from the microphone 48 is recorded in SA3. This SA3 process need not necessarily be executed. Next, in SA4, it is determined whether or not the performance corresponding to the music data is terminated, for example, by performing music data related to the performance to the end. When the determination at SA4 is negative, the process after SA2 is executed. When the determination at SA4 is affirmative, the process after SA5 is executed.

  In SA5, a transmission information image for performing transmission processing on the moving image taken by the digital camera 70 in SA2 is acquired. This SA5 process may be executed in parallel with the karaoke performance corresponding to the music data. Next, in SA6, for one moving image file, a first moving image area 140 and a second moving image area 142 are determined for each of a plurality of frames constituting the moving image file, and the moving image taken by the digital camera 70 in SA2 Is pasted on the first moving image area 140, and the transmission information image acquired in SA5 is pasted on the second moving image area 142. Next, in SA7, information such as the music selection number related to the music data corresponding to the music output in SA2 is stored in association with the moving image file created in SA6. Next, in SA8, the recording data recorded in SA3 is stored in association with the moving image file created in SA6. The SA8 process need not necessarily be executed. Next, in SA9, it is determined whether or not an operation for transmitting (uploading) the moving image file created in SA6 to the server 80 is performed by an input operation through the electronic quick sample device 68 or the like. Is done. If the determination at SA9 is negative, the routine is terminated. If the determination at SA9 is affirmative, the routine is created at SA6, and the information about the music data at SA7 and SA8. After the moving image file associated with the recording data or the like is uploaded to the server 80 via the communication line 60 and stored in the moving image database 78, this routine is terminated.

  FIG. 13 is a flowchart for explaining a main part of an example of the moving image file reading / reproducing control of the present embodiment by the CPU 12 of the karaoke apparatus 10, and is repeatedly executed at a predetermined cycle. The control shown in FIG. 13 is also preferably executed by a control device of a communication terminal device such as the mobile phone 82, tablet terminal 84, personal computer 86, and the like.

  First, in SB1, it is determined whether or not an operation for reading a predetermined moving image stored in the moving image database 78 of the server 80 has been performed by an input operation through the electronic quick sample device 68 or the like. If the determination at SB1 is negative, the routine is terminated. If the determination at SB1 is affirmative, the target video file is retrieved from the video database 78 of the server 80 at SB2. It is read out and acquired (downloaded) via the communication line 60. Next, in SB3, a moving image is read from the first moving image area 140 and a transmission information image is read from the second moving image area 142 in the moving image file read in SB2. Based on the information, a transmission process is performed on the moving image with the transmission information. Next, in SB4, video processing such as the video processed by SB3 being combined with, for example, the front side of another moving image is performed via the video processing unit 20, and the moving image related to the processing is performed. Is displayed on the display 44. In parallel with the process of SB4, in SB5, the performance output of the music related to the music data associated with the moving image file downloaded in SB2 is performed. In parallel with the processing of SB4 and SB5, in SB6, the recorded data associated with the moving image file downloaded in SB2 is reproduced. The process of SB6 is not necessarily executed. Next, in SB7, it is determined whether or not the reproduction of the moving image is terminated, for example, the moving image corresponding to the moving image file downloaded in SB2 is output to the end. When the determination at SB7 is negative, the processing after SB4 is executed. When the determination at SB7 is affirmative, the routine is terminated.

  In the above control, SA1, SA2 and SA4 are the operation of the music performance control means 100, SA3 is the operation of the recording means 102, SA5 is the operation of the transparent information acquisition means 104, and SA6 to SA8 are the moving images. In the operation of the file creation means 106, SA6 is the operation of the pasting means 108, SA7 is the operation of the music association means 110, SA8 is the operation of the recording data correspondence means 112, and SA10 is the operation of the transmission means 114. In operation, SB1 to SB3 are operations of the reading means 116, SB4 is operations of the display control means 118, SB5 is operations of the performance output control means 120, and SB6 is operations of the sound signal output control means 122. , Respectively.

  According to the present embodiment, the transmission information acquisition unit 104 (SA5) that acquires transmission information for performing transmission processing on a moving image captured by the digital camera 70 that is an imaging device, and one moving image file, A first moving image area 140 and a second moving image area 142 are defined for each of a plurality of frames constituting the moving image file, and a moving image shot by the digital camera 70 is pasted on the first moving image area 140, and the second moving image is pasted. In the area 142, the pasting means 108 (SA6) for pasting the transparent information acquired by the transparent information acquiring means 104, and the moving picture to be pasted on the first moving picture area 140 by the pasting means 108 is the digital image. Corresponds to the musical piece being played by the karaoke apparatus 10 at the time of being photographed by the camera 70 Since the apparatus has the music association means 110 (SA7) for storing information relating to the music data in association with the video file, the video and the video information for performing the transparent process on the video are stored in one piece. By integrating the information into the moving image file and further associating the information related to the music played at the time of moving image shooting, it is possible to realize the transmission processing of the moving image and the performance of the music output at the time of moving image shooting with the smallest possible data amount. That is, it is possible to provide the karaoke apparatus 10 that realizes moving image transmission processing and performance reproduction during moving image shooting with a relatively small amount of data.

  Reading means 116 (SB1 to SB3) for reading out the moving picture pasted in the first moving picture area 140 and the transparent information pasted in the second moving picture area 142 from the moving picture file, and reading by the reading means 116 Display control means 118 (SB4) for displaying on the display 44, which is a display device, a moving image obtained by performing transmission processing on the moving image with the transmission information based on the moving image and the transmission information, and the display control means 118 Since it is provided with performance output control means 120 (SB5) for performing the performance output of the music based on the music data associated with the video file in parallel with the display of the video, for example, other karaoke A moving image file uploaded to a predetermined server 80 by shooting or the like by the apparatus 10 is stored in the predetermined karaoke apparatus 10. Download Ri, it is possible to realize a transmission process and reproducibility of the performance of the moving image shooting of the moving image at the karaoke device 10.

  The pasting means 108 changes the size of the moving image to be pasted to the first moving image area 140 and the size of the transparent information to be pasted to the second moving image area 142, and the first moving image area 140. And the second moving image area 142, the data amount of the moving image file for realizing the transparency processing can be further reduced.

  Recording means 102 for recording a sound signal input from the microphone 48 as an input device when a moving image to be attached to the first moving image area 140 by the attaching means 108 is photographed by the digital camera 70. (SA3), recording data association means 112 (SA8) as second association means for storing the sound signal recorded by the recording means 102 in association with the moving image file, and the moving image by the display control means 118 Since the sound signal output control means 122 (SB6) for outputting the sound signal associated with the moving image file is provided in parallel with the display of the moving image file, the moving image and the moving image are subjected to transparency processing. In addition to integrating the video information for the video into one video file, record the singing voice of the singer at the time of video recording In Rukoto, it is possible to realize the transmission processing and reproduction of karaoke performance for movie recording videos in a relatively small amount of data.

  According to the present embodiment, the CPU 12 as a control device of the karaoke apparatus that includes the digital camera 70 as an imaging device that captures a moving image and outputs the performance of music based on music data selected from a large number of music data. The transmission information acquisition means 104 (SA5) for acquiring transmission information for performing transmission processing on a moving image photographed by the digital camera 70, with respect to one moving image file, each of a plurality of frames constituting the moving image file A first moving image area 140 and a second moving image area 142 are defined, a moving image taken by the digital camera 70 is pasted on the first moving image area 140, and the transmission information acquisition unit 104 is added to the second moving image area 142. Pasting means 108 (SA6) for pasting the transmission information acquired by the above and pasting means 108 Therefore, when the moving image to be pasted on the first moving image area 140 is shot by the digital camera 70, information related to the music data corresponding to the music played by the karaoke apparatus 10 is stored in the moving image file. Since it is a program characterized by functioning as the music association means 110 (SA7) to be stored in association with each other, the moving image and moving image information for performing transparency processing on the moving image are integrated into one moving image file. By associating the information related to the music played at the time of shooting, it is possible to realize the transmission processing of the moving image and the performance of the music output at the time of moving image shooting with as small a data amount as possible. That is, it is possible to provide a program that realizes moving image transmission processing and reproduction of performance during moving image shooting with a relatively small amount of data.

  The preferred embodiments of the present invention have been described in detail with reference to the drawings. However, the present invention is not limited to these embodiments, and may be implemented in other modes.

  For example, in the above-described embodiment, the karaoke apparatus 10 is provided as an example of a music performance apparatus that includes an imaging device that captures a moving image and outputs a musical performance based on music data selected from a large number of music data. Although the example to which the present invention is applied has been described, the present invention is not limited to this, and can be applied to another form of music performance device. For example, a mobile phone such as a so-called smartphone or a karaoke game in which a karaoke application that installs a digital camera that shoots a moving image and performs music performance output based on music data selected from a large number of music data is installed. The present invention is also suitably applied to an aspect in which a home game machine to be executed is used as a music performance device.

  In the above-described embodiment, the processing of the moving image file creation means 106, the display control means 118, and the like is executed by the CPU 12 provided in the karaoke apparatus 10. Substantial control of the present embodiment by 106 and the display control means 118 may be executed by a server control device connected to the music performance device via the communication line 60. For example, when a moving image corresponding to a moving image file stored in the moving image database 78 of the server 80 is played by a communication terminal device such as the karaoke device 10, the transparent processing is performed on the server 80 side. For example, a moving image in which the partial video that has been subjected to the transparency processing is synthesized on the front side of another video is played back and streamed to the communication terminal device such as the karaoke device 10 via the communication line 60. Embodiments are also conceivable.

  Although not particularly mentioned in the above-described embodiments, the moving image information created by the moving image file creating means 106 can be used in various ways. For example, a usage form such as combining a plurality of pieces of moving picture information in which transparent information is already incorporated, storing the plurality of pieces of moving picture information in association with each other, and distributing the information via the communication line 60 is conceivable.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

  10: Karaoke device (music performance device), 12: CPU (control device), 44: Display (display device), 48: Microphone (input device), 70: Digital camera (imaging device), 100: Music performance control means ( Performance means), 102: recording means, 104: transparent information acquisition means, 106: moving image file creation means, 108: pasting means, 110: music correspondence means, 112: recorded data correspondence means (second correspondence means), 116: reading means, 118: display control means, 120: performance output control means, 122: sound signal output control means, 140: first moving image area, 142: second moving image area

Claims (6)

A performance means for playing music data;
An acquisition means for acquiring moving image information acquired from the imaging device in response to music data being played by the performance means;
Transparency information acquisition means for acquiring transparency information for performing transparency processing on the acquired moving image information;
A first moving image area and a second moving image area are defined for each of a plurality of frames constituting one moving image file, and moving image information photographed by the imaging device is pasted on the first moving image area, and the second moving image area A pasting means for pasting the transparent information acquired by the transparent information acquiring means;
A music performance apparatus comprising: music data played by the performance means; and association means for storing the music data in association with the moving image file pasted by the pasting means. Reading means for reading out the moving image pasted in the first moving image area and the transparent information pasted in the second moving image area from the moving image file;
Display control means for displaying on the display device a moving image obtained by performing transmission processing on the moving image with the transmission information based on the moving image and the transmission information read by the reading unit;
The performance output control means which performs the performance output of a music based on the music data matched with the said moving image file in parallel with the display of the moving image by the said display control means. Music performance device.   The music performance device according to claim 1, wherein the music performance device is a karaoke device connectable to the imaging device. The pasting means changes the size of the moving image to be pasted to the first moving image area and the size of the transparent information to be pasted to the second moving image area, and the first moving image area and the second moving image area. The music performance device according to any one of claims 1 to 3, wherein the music performance device is affixed to each moving image area. A recording means for recording a sound signal input from the input device;
Second correspondence means for storing the sound signal recorded by the recording means in association with the moving image file;
The music performance device according to claim 2 , further comprising sound signal output control means for outputting a sound signal associated with the moving image file in parallel with display of the moving image by the display control means. . A control device for a music performance device that performs performance output of music based on music data selected from a large number of music data,
Performance means for playing music data;
An acquisition means for acquiring moving image information acquired from the imaging device in response to the music data being played by the performance means;
Transparency information acquisition means for acquiring transparency information for performing transparency processing on the acquired moving image information;
A first moving image area and a second moving image area are defined for each of a plurality of frames constituting one moving image file, and moving image information photographed by the imaging device is pasted on the first moving image area, and the second moving image area A pasting means for pasting the transparent information acquired by the transparent information acquiring means,
And a program for causing the music data played by the performance means to correspond to the moving image file pasted by the pasting means to be stored in association with each other.