JP5249546B2 - Karaoke system using the Internet - Google Patents

Description

  The present invention relates to a karaoke system using the Internet, and more particularly to an improvement for realizing synchronous distribution of karaoke information to a plurality of users.

  2. Description of the Related Art Music playback apparatuses that output a predetermined performance song selected from a large number of performance songs are known. For example, a karaoke device used in a karaoke box or the like. According to such a karaoke apparatus, music information of a predetermined karaoke performance song selected from a large number of karaoke performance songs stored in advance in a storage device is output, and an image including lyrics information of the karaoke performance song is output. Karaoke performance of a desired song can be enjoyed by displaying it on the screen in synchronization with.

  As an example of a karaoke system using the karaoke apparatus, a karaoke system using the Internet has been proposed. For example, this is the duet / choral karaoke control system in multiple locations described in Patent Document 1. According to this technology, by connecting a plurality of mobile / fixed music performance communication devices to the Internet and constructing a communication network environment in which video / audio signals can be transmitted and received between the music performance communication devices. It is said that by sharing the video and audio signals of the space at multiple locations, it is possible to realize karaoke duets and chorals between users in remote locations.

JP 2005-77485 A

  By the way, in recent years, online games have been widely used in which a virtual world is constructed using the Internet and role playing games and action games are enjoyed in the virtual world. Also, unlike such online games, the purpose is to live in a virtual world built on the Internet without any particular purpose, or to communicate with other residents who also live in the virtual world by chat etc. New systems such as Second Life (second registered trademark: Second Life (C) Linden Research Inc.) have been developed.

  The present inventor has conceived of building a karaoke system in the virtual world using the Internet, and has been proceeding with specific studies for its realization. That is, intensive research has been continued in order to realize a karaoke system in which a karaoke facility such as a karaoke box is provided in the virtual space, and karaoke performance can be performed among a plurality of users in the karaoke facility. In the process, the conventional technology as described above has raised a problem that it is difficult to synchronize information about karaoke. That is, when performing a karaoke performance in the virtual world, it is required that a plurality of users who are present at the same time view information such as a karaoke performance or video accompanying the performance at the same timing. Depending on the performance of the corresponding PC, communication environment, etc., such information distribution cannot be synchronized, and there is a problem that a time lag occurs between the users and it is impossible to enjoy realistic karaoke performance. For this reason, in the karaoke system using the Internet, development of the technique which implement | achieves synchronous delivery of the information regarding karaoke with respect to the some user was calculated | required.

  The present invention has been made against the background of the above circumstances, and the purpose of the present invention is to provide a karaoke system using the Internet that realizes the synchronous distribution of karaoke information to a plurality of users. It is to provide.

In order to achieve such an object, the gist of the present invention is to store information for performing karaoke performance of a large number of performance songs and to a karaoke server connected to the Internet, and also connected to the Internet. A plurality of terminal devices capable of transmitting / receiving information to / from the karaoke server, and a virtual world server connected to the Internet and capable of transmitting / receiving information to / from the karaoke server and the plurality of terminal devices; The karaoke system using the Internet, which performs karaoke performance of a predetermined performance song selected from among a large number of performance songs stored in the karaoke server, using the terminal device, Streams and distributes information for performing the karaoke performance to the terminal device that is the main body of the karaoke performance 1st streaming delivery means is provided, The said terminal device for performing the karaoke performance delivered by the said 1st streaming delivery means from the said karaoke server, when the terminal device is a main body of a karaoke performance Host streaming playback means for streaming and reproducing audio information and video information based on the information, and a musical composition in which audio information and video information reproduced by the host streaming playback means are combined with at least second audio information different from the audio information Second streaming distribution means for streaming information to other terminal devices, and performance music information distributed by the second streaming distribution means from the main terminal device when the terminal device is not the main body of karaoke performance Streaming audio and video information based on A guest streaming reproduction means for reproducing, which includes the second streaming means state, and are not provided to the terminal device as a subject of karaoke performance, the virtual world server, the terminal device Virtual world video generation means for generating a virtual world video in which a human-type video corresponding to each user moves relative to a predetermined structure video in response to an input from the video, and the virtual world video generation Virtual world video distribution means for distributing the virtual world video generated by the means to the plurality of terminal devices, and for performing operations related to the karaoke performance as part of the virtual world video generated by the virtual world video generation means Interface video generation means for generating the interface video, and the interface video generated by the interface video generation means. Distribution request means for requesting distribution of information for performing the karaoke performance to the plurality of terminal devices in response to an input operation by the terminal device in accordance with a video image. It is.

If it does in this way, the said karaoke server will be provided with the 1st streaming delivery means which carries out the streaming delivery of the information for performing the karaoke performance to the said terminal device used as the subject of karaoke performance, Host streaming playback means for streaming playback of audio information and video information based on information for performing the karaoke performance delivered from the karaoke server by the first streaming delivery means when the terminal device is the subject of karaoke performance; Second streaming delivery means for streaming and delivering to the other terminal device the musical composition information obtained by synthesizing at least second audio information different from the audio information to the audio information and video information reproduced by the host streaming reproduction means; The terminal device is not the subject of karaoke performance. In this case, the second streaming is provided with guest streaming reproduction means for streaming reproduction of audio information and video information based on performance music information distributed by the second streaming distribution means from the terminal device as the main body. delivering device state, and are not provided to the terminal device as a subject of karaoke performance, the virtual world server, in response to an input from the terminal device, each user for a predetermined structure image A virtual world image generating means for generating a virtual world image in which a humanoid image corresponding to a person moves relatively, and a virtual world for distributing the virtual world image generated by the virtual world image generating means to the plurality of terminal devices As a part of the virtual world video generated by the video distribution means and the virtual world video generation means, an operation related to the karaoke performance is performed. Interface video generating means for generating an interface video for performing the karaoke performance on the plurality of terminal devices in response to an input operation by the terminal device according to the interface video generated by the interface video generating means Since a distribution request means for requesting distribution of information is provided, in a virtual world using the Internet, a terminal device that is the main player of karaoke performance and a terminal device on the side of viewing the karaoke performance Synchronization can be achieved in a practical manner. That is, it is possible to provide a karaoke system using the Internet that realizes synchronous distribution of information about karaoke to a plurality of users.

Here , it is preferable that the interface video generation unit generates the interface video as a video in a predetermined room as a structure video included in the virtual world video generated by the virtual world video generation unit. The second streaming distribution means is configured such that when a humanoid image corresponding to a user of the terminal device that is the subject of the karaoke performance exists in the room, a user corresponding to the humanoid image existing in the room is also provided. The music piece information is streamed and distributed to the terminal device. In this way, it is possible to realize a karaoke system in which a store such as a virtual karaoke box is set up in a virtual world using the Internet, and a realistic karaoke performance can be performed in the room of the store.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a karaoke system 10 using the Internet 18 according to an embodiment of the present invention. As shown in FIG. 1, a karaoke system 10 according to the present embodiment includes a plurality of personal computers 12 (hereinafter, two are illustrated in FIG. 1) as terminal devices (hereinafter referred to as PCs 12) and a virtual world service. A virtual world server 14 operated by a provider company and a karaoke server 16 operated by a karaoke service provider company are provided. The plurality of PCs 12, the virtual world server 14, and the karaoke server 16 are all connected to the Internet 18, and information can be communicated with each other via the Internet 18. Here, the plurality of PCs 12 can receive a virtual world service, which will be described later, by transmitting and receiving information to and from the virtual world server 14. In other words, the karaoke system 10 of this embodiment is For example, it is a virtual world service system using the Internet 18. The Internet 18 is a network environment in which a large number of information networks are connected to each other, and a large number of PCs, servers, digital home appliances, and the like are connected to each other in a wired or wireless manner. It is configured to be able to communicate information.

  FIG. 2 is a diagram illustrating the configuration of the PC 12. The PC 12 is a so-called Neumann computer that performs signal processing according to a program stored in advance in a ROM 22 that is a read-only memory while using a temporary storage function of a RAM 24 that is a read / write memory by a CPU 20 that is a central processing unit. There are home personal computers generally called desktop personal computers or notebook personal computers. Further, as shown in FIG. 2, a video display device 26 such as a CRT (Cathode-ray Tube) or a TFT (Thin Film Transistor Liquid Crystal) functioning as a video display unit, and the video display by the video display device 26 are controlled. A video board 28 as a video display control device, an input device 30 such as a keyboard, an input interface 32 for processing input by the input device 30, and the CPU 20 and the like for connecting to the Internet 18 A modem 34, a hard disk 36 as a storage device, a microphone 38 as an audio input device, and audio information as an analog signal input from the microphone 38 is converted into a digital signal and supplied to the CPU 20 or the like. Supplied from the converter 40, the speaker 42 as an audio output device, the CPU 20 and the like Audio information as digital signals and a D / A converter 44 to be outputted from the speaker 42 into an analog signal, and is configured to include.

  FIG. 3 is a diagram illustrating the configuration of the virtual world server 14, and FIG. 4 is a diagram illustrating the configuration of the karaoke server 16. As shown in these drawings, in this embodiment, for convenience of explanation, a mode in which the virtual world server 14 and the karaoke server 16 are servers of the same type that make elements such as a central processing unit equal will be described. That is, the virtual world server 14 and the karaoke server 16 use the temporary storage function of the RAM 50 that is a writing / reading memory as needed by the CPU 46 that is a central processing unit according to a program stored in advance in a ROM 48 that is a read-only memory. It is a so-called Neumann computer that performs signal processing, and is a server that executes various controls such as file management and input / output control, management of access through the Internet 18, communication control through the Internet 18, etc. . As shown in FIGS. 3 and 4, a video display device 52 such as a CRT or TFT, a video board 54 which is a video display control device for controlling display of video by the video display device 52, a keyboard, etc. , An input interface 58 for processing input by the input device 56, and a modem 60 for connecting the CPU 46 and the like to the Internet 18.

  As shown in FIG. 3, the virtual world server 14 is provided with various databases such as a virtual world database 62 and a customer database 64. The virtual world database 62 stores various information for providing a virtual world service for the PC 12 or the like. That is, video information necessary for generating a virtual world image in which a humanoid image corresponding to each user moves relative to a predetermined structure image in response to an input from the PC 12 or the like, 3D model information (polygon model information) corresponding to a structure image or a humanoid image, information on the position of each structure image, control information for operating a humanoid image based on the 3D model information, audio Information and other various information are stored. Here, the above-mentioned structure image is an image corresponding to various structures other than the humanoid image constituting the virtual world, such as the ground, buildings, trees, furniture, and water and smoke in the virtual world. Preferably, it is a three-dimensional image, that is, an image defined by coordinates of three axes orthogonal to each other. The human-type image is controlled so that it can move relative to the structure image, that is, can move freely in the virtual world constituted by the structure image except for the area occupied by the structure image. 3D video. This humanoid image is basically defined in a one-to-one correspondence with the user of the virtual world service, and behaves as a substitute for the user in the virtual world, and is generally called an avatar or the like. Is done.

  The customer-specific database 64 relates to the virtual world service for the PC 12 or the like, and for each user, personal information such as the user's identification information (ID), login password, user's name, gender, age, etc. The IP address on the Internet 18 of the PC 12 corresponding to the user, information defining the above-mentioned human type image corresponding to the user, and the like are stored. The information defining the human-type video is information such as the name (nickname), gender, appearance, clothes, occupation, and money of the human-type video corresponding to each user. That is, the humanoid image corresponding to each user is defined based on the information stored in the customer database 64 corresponding to the user. In other words, the human-type video corresponding to each user is stored in the customer database 64 corresponding to each user from the information for generating the human-type video stored in the virtual world database 62. Generated based on information defining a human-type video.

  As shown in FIG. 4, the karaoke server 16 is provided with various databases such as a karaoke database 66 and a customer-specific database 68. The karaoke database 66 stores a large number (for example, tens of thousands of pieces) of karaoke information for performing a karaoke performance by the terminal device such as the PC 12. This karaoke information is composed of performance information for generating performance sounds corresponding to each performance song, lyrics information for generating lyrics character video (lyric telop) corresponding to the lyrics of the performance songs, and the like. The music ID is identified by a music selection number unique to each performance music piece that is a content ID.

  The customer-specific database 68 is related to the karaoke service for the PC 12 or the like. For each user, identification information (ID) of the user, a login password, personal information such as the user's name, gender, age, And the IP address on the Internet 18 of the PC 12 corresponding to the user. The identification information and login password of each user stored in the customer database 68 are the same as the identification information and login password of each user stored in the customer database 64 of the virtual world server 14. Also good. Moreover, in the aspect which performs login authentication of the karaoke service by the karaoke server 16 by the information stored in the customer database 64 of the virtual world server 14, the customer database 68 is not necessarily provided in the karaoke server 16. .

  FIG. 5 is a functional block diagram illustrating the main parts of the control functions provided in the CPU 20 of the PC 12, the CPU 46 of the virtual world server 14, and the CPU 46 of the karaoke server 16. The PC 12a shown in FIG. 5 is a terminal device (host PC) that is the main player of karaoke performance, and the PC 12b is a terminal device (guest PC) that is not the main body of karaoke performance, both of which have the same control function. Yes, the terminal device which is the main body of karaoke performance can be replaced at any time. Further, the virtual world video display control means 70, the information transmission control means 72, the streaming reproduction means 88, and the second streaming distribution means 90 shown in FIG. 5 are sent to the CPU 20 of the PC 12 by the virtual world service authentication means 74, the virtual world. The video generation means 76, the virtual world video distribution means 80, and the karaoke distribution request means 82 are sent to the CPU 46 of the virtual world server 14, and the karaoke service authentication means 84 and the first streaming delivery means 86 are sent to the CPU 46 of the karaoke server 16. It is functionally provided.

  The virtual world video display control means 70 displays the virtual world video 100 (see FIG. 8 etc.) corresponding to the virtual world video information distributed from the virtual world server 14 via the Internet 18 on the video display device 26. Let Specifically, a video corresponding to the virtual world video information distributed from the virtual world server 14 is generated by the video board 54, and the video generated by the video board 54 is displayed on the video display device 26. This display control is preferably executed repeatedly in a very short time period, and therefore the virtual world video displayed on the video display device 26 is a moving image. Alternatively, the virtual world video information distributed from the virtual world server 14 is one in which video corresponding to the virtual world video information is streamed (a method in which data is divided into a plurality of packets and distributed), The virtual world video display control means 70 performs streaming playback (a system in which playback is performed whenever a packet is received) of the virtual world video thus distributed.

  The information transmission control means 72 transmits various information related to the virtual world service to the virtual world server 14 via the Internet 18. Specifically, user identification information and a password are transmitted when logging in to the virtual world service. In addition, operation information for operating a humanoid image 96 (see FIG. 8 and the like) corresponding to the user in the virtual world image 100, which is input via the input device 30 or the like, is transmitted. Also, as the music selection input information, the music selection number of a predetermined performance music inputted through the input device 30 is transmitted in association with the identification information of the user. In addition, when text chat or voice chat or the like is performed with other users in the virtual world, text information corresponding to the text chat or voice information corresponding to the voice chat is transmitted as needed. Note that such character chat or voice chat may be controlled by a server device different from the virtual world server 14.

  The virtual world service authentication unit 74 performs login authentication to the virtual world service of the user corresponding to the identification information based on the identification information and password transmitted from the PC 12. Specifically, when identification information and a password for logging in to the virtual world service are transmitted from the PC 12 via the Internet 18, whether the identification information is stored with reference to the customer database 64. Further, it is determined whether or not the password stored in association with the identification information matches the transmitted one. When the authentication by the virtual world service authentication means 74 is formally performed, the user corresponding to the identification information is logged in to the virtual world service, and a humanoid image corresponding to the user is generated. For example, the virtual world video is started to be distributed from the customer database 64 to the PC 12 corresponding to the user by the virtual world video generation means 76 described later. On the other hand, when formal authentication is not performed by the virtual world service authentication means 74, an error is returned and the result is returned to the target PC 12.

  The virtual world image generation means 76 generates a virtual world image including at least the structure image and a humanoid image corresponding to each user based on the information stored in the virtual world database 62. Specifically, based on the video information stored in the virtual world database 62, 3D model information corresponding to a structure image or a humanoid image, information on the position of each structure image, and the 3D model information. Based on the control information for operating the human-type image and other various information, the image of the virtual world corresponding to each user, that is, the three-dimensional displayed on the PC 12 corresponding to the viewpoint of the user. Generate a video of the virtual world. 8 and 9 are diagrams illustrating the virtual world video 100 generated by the virtual world video generation means 76. FIG. As shown in FIGS. 8 and 9, the virtual world image generation means 76 includes a karaoke box image 92 corresponding to a building, a karaoke device image 94 corresponding to a karaoke device, etc. as a part of the virtual world image 100. And a humanoid image 96 corresponding to each user who can move relative to the component image. The generation of the virtual world video 100 is repeatedly executed in an extremely short time period, and thus the generated virtual world video (video displayed on the video display device 26 of the PC 12) is a moving image. The virtual world image generation means 76 is a person corresponding to each user for the structure images 92, 94, etc., according to the input from the PC 12, that is, the operation information transmitted via the Internet 18. The generation (update) control of the virtual world image 100 is performed so that the type image 96 moves relatively (three-dimensional movement).

  The virtual world video distribution means 80 distributes the virtual world video 100 generated by the virtual world video generation means 76 to the PC 12 via the Internet 18. Preferably, the virtual world video 100 generated by the virtual world video generation means 76 is distributed by streaming. The operation of the virtual world video distribution unit 80 is associated with the operation of the virtual world video display control unit 70 provided in the PC 12, and thus the virtual distribution streamed from the virtual world video distribution unit 80 is performed. The world video 100 is streamed and reproduced by the virtual world video display control means 70 in the PC 12, so that the structure video 92, 94, etc. is sent to the video display device 26 in the PC 12 according to the input from the input device 30. On the other hand, a virtual world image is displayed as a moving image in which the humanoid image 96 corresponding to each user moves relatively.

  Here, the virtual world video generation means 76 includes an interface video generation means 78 for generating an interface video for performing an operation related to karaoke performance by the PC 12 as a part of the virtual world video. The interface video generation means 78 preferably generates the interface video and generates a virtual world video corresponding to the user when a karaoke performance start request is transmitted from the PC 12 via the Internet 18. Synthesize. FIG. 9 is a diagram showing a virtual world video 100 including an example of the interface video 98 generated by the interface video generation means 78. As shown in FIG. 9, the interface video 98 generated by the interface video generation means 78 includes, for example, 10 numeric buttons from 0 to 9, a music selection button, a performance cancel button, a re-singing button, and a performance stop button. , An interrupt music selection button, key operation buttons, an input display unit, etc., and according to the operation of the input device 30 of the PC 12, the music selection input operation, the performance canceling operation, the re-singing operation, the performance stop operation, the interrupt Input operations such as a music selection operation and a key operation are made possible. That is, in a state where the interface video 98 is displayed on the video display device 26 of the PC 12, when a predetermined operation is performed by pressing each button by the operation of the input device 30 of the PC 12, the information Information corresponding to the operation is transmitted to the virtual world server 14 via the Internet 18 by the transmission control means 72. In the virtual world, the human-type image 96 may be realized by actually pressing a button on the interface image 98 with its finger or the like. The interface video 98 generated by the interface video generation means 78 is distributed to the PC 12 via the Internet 18 by the virtual world video distribution means 80 as a part of the virtual world video 100, as described above. The image is displayed on the video display device 26 in the PC 12. Further, the interface video generation means 78 is preferably a predetermined room in a structure video included in the virtual world video 100 generated by the virtual world video generation means 76 as shown in FIG. The interface image 98 is generated as an image in the frame 92.

  When the PC 12 sends (requests) information related to karaoke performance to the virtual world server 14 from the PC 12, the karaoke delivery request means 82 requests the karaoke server 16 for information corresponding to the information. . For example, when a karaoke performance start request is transmitted from the PC 12, the user identification information and password relating to the karaoke performance start request are transmitted to the karaoke server 16, and the karaoke server 16 is provided with the karaoke server 16. The service authentication means 84 executes login authentication to the karaoke service. When the music selection number of the performance music is transmitted from the PC 12 as the music information distribution request for the predetermined performance music, the music data of the performance music corresponding to the music selection number is the target user's PC 12. Request to be delivered. That is, in the karaoke service by the karaoke server 16, various requests from the PC 12 to the karaoke server 16 are preferably made through the karaoke distribution request means 82 provided in the virtual world server 14.

  The karaoke service authentication means 84 performs login authentication to the karaoke service of the user corresponding to the identification information based on the identification information and password transmitted from the virtual world server 14 (PC 12). Specifically, when identification information and a password for logging in to the karaoke service are transmitted from the virtual world server 14 via the Internet 18, the identification information is stored with reference to the customer database 68. And whether or not the password stored in association with the identification information matches the transmitted one. When the authentication by the karaoke service authentication unit 84 is formally performed, the user corresponding to the identification information is logged in to the karaoke service, and the virtual world video generation unit 76 (interface) of the virtual world server 14 is logged in. The video generation means 78) displays the interface video 98 as a part of the virtual world video 100 corresponding to the user, and the user's karaoke service is started. On the other hand, if formal authentication is not performed by the karaoke service authentication means 84, an error is generated and the result is returned to the virtual world server 14 and the target PC 12.

  The first streaming distribution means 86 performs streaming distribution of information for performing the karaoke performance to the PC 12a which is a terminal device that is the main body of the karaoke performance. Here, the PC 12a which is the main body of the karaoke performance is the PC 12 corresponding to the user who is a singer. In other words, the PC 12a is used via the microphone 38 in synchronization with the output of the performance song corresponding to the karaoke performance. PC 12 to which the person's voice is to be input. Specifically, when there is a request for distribution of music data of a predetermined performance song from the virtual world server 14, the first streaming distribution means 86 is the user's PC 12 corresponding to the request, that is, the karaoke performance subject. The music information (music data) including information for performing the karaoke performance, that is, the performance information and the lyric character video image information is streamed to the PC 12a via the Internet 18 to the PC 12a. Preferably, mpeg (Moving Picture Experts Group) data including performance information of the performance music and lyric character video image information generated based on the karaoke information corresponding to the performance music stored in the karaoke database 66, etc. Stream information.

  The streaming playback means 88 in the PC 12a performs streaming playback of audio information (performance information) and video information based on information for performing a karaoke performance streamed from the karaoke server 16. That is, it functions as host streaming playback means. Specifically, music information such as mpeg data streamed from the karaoke server 16 is decoded and a video corresponding to the information is displayed on the video display device 26, and performance information (performance data) corresponding to the information is displayed. Sound) is output from the speaker 42. Here, the streaming playback means 88 preferably displays a karaoke performance video 94a based on information streamed from the karaoke server 16 as a part of the virtual world video 100, for example, as shown in FIG. . In the example shown in FIG. 8, a karaoke performance video 94a that is streamed and played back by the streaming playback means 88 is displayed in a portion corresponding to the screen of the karaoke device video 94 in the virtual world video 100.

  The second streaming distribution means 90 combines the musical composition information obtained by synthesizing at least the second audio information different from the audio information and the audio information and video information to be streamed and reproduced by the streaming reproduction means 88 with another terminal device, that is, karaoke. Streaming is delivered to the PC 12b that is not the main subject of the performance. Specifically, the audio information of the user who is the subject of the karaoke performance input from the microphone 38 via the A / D converter 40 to the audio information and video information streamed and reproduced by the streaming reproduction means 88. The synthesized information is streamed and delivered to the PC 12b in the form of mpeg data or the like. That is, the performance music information (performance music data) distributed by the second streaming distribution means 90 is preferably changed from the microphone 38 to the A / D converter into information to be distributed by the first streaming distribution means 88. The voice information input via 40 is added (synthesized). The distribution by the second streaming distribution means 90 is preferably executed as so-called peer-to-peer communication between the PCs 12a and 12b via the Internet 18.

  The streaming playback means 88 in the PC 12b performs streaming playback of audio information and video information based on performance music information streamed from the PC 12a. That is, it functions as a guest streaming playback means. Specifically, performance music information such as mpeg data streamed from the PC 12a is decoded and video corresponding to the information is displayed on the video display device 26, and performance information (performance sound) corresponding to the information is displayed. Voice corresponding to the singing voice) is output from the speaker 42. Here, as in the PC 12a, the streaming playback means 88 preferably displays a karaoke performance video 94a based on information streamed from the PC 12a, for example, as shown in FIG. As part of In the example shown in FIG. 8, a karaoke performance video 94a that is streamed and played back by the streaming playback means 88 is displayed in a portion corresponding to the screen of the karaoke device video 94 in the virtual world video 100. In this way, the performance sound corresponding to the performance music to be processed and the karaoke performance video 94a as shown in FIG. 8 are output substantially synchronously by the PC 12a which is the main body of the karaoke performance and the PC 12b which is not the main body. . The performance sound, karaoke performance video 94a, and the like output from the PC 12b are distributed from the PC 12a, and in principle are slightly delayed from the output from the PC 12a, but are based on streaming distribution. The time difference is almost negligible. Further, there is no time difference between the PCs 12b that are not the main players of the karaoke performance, and the karaoke performances at the main PC 12a can be enjoyed synchronously.

  Here, the second streaming delivery means 90 is preferably the same when the humanoid image 96 corresponding to the user of the PC 12a who is the subject of the karaoke performance exists in a predetermined room, for example, the karaoke box image 92. The performance tune information is stream-distributed to the user's PC 12 corresponding to the humanoid image 96 existing in the karaoke box image 92. FIG. 7 shows that as an example of the relative positional relationship between the structure image and the humanoid image in the virtual world image 100, a humanoid image 96 corresponding to three users enters the karaoke box image 92. It is a figure which shows the state which exists. As shown in FIG. 7, the karaoke box image 92 corresponds to one room in a building image including the karaoke box, and is separated from the adjacent room by a wall image 92a. The humanoid image 96 cannot pass through the wall image 92a, and enters the karaoke box image 92, which is a space (closed room) that is virtually formed by being surrounded by the wall image 92a. It has become. In such a state, the users corresponding to the three humanoid images 96 in the karaoke box image 92 are performing karaoke performance, and in this case, the second streaming distribution means 90 performs the karaoke performance. Streaming distribution of the performance tune information is performed only for users existing in the box video 92, and is not performed for users not existing in the karaoke box video 92. For example, the streaming distribution limited to the target is realized by distributing the URL for streaming distribution only to the PC 12 corresponding to the user existing in the target room by the virtual world server 14. Is done.

  As described above, the PCs 12a and 12b basically have the same configuration. As shown in FIG. 5, the PC 12b that is not the main player of the karaoke performance is also provided with the second streaming distribution means 90. This guarantees the possibility that all PCs 12 will be the main players of the karaoke performance. When a karaoke performance is performed by the selected PC 12, the PC 12 becomes the PC 12a as the main subject of the karaoke performance. The other PC 12 becomes the PC 12b as a so-called viewer who is not the subject of the karaoke performance. In other words, the PC 12a that was the main player of the karaoke performance at the time of the predetermined karaoke performance becomes the PC 12b when the karaoke performance is ended and the karaoke performance mainly of the other PC 12 is started. PC12 which becomes becomes PC12a newly.

  FIG. 6 is a diagram for explaining transmission / reception of information via the Internet 18 among the PC 12, the virtual world server 14, and the karaoke server 16 in the karaoke performance by the karaoke system 10 of the present embodiment. As shown in FIG. 6, first, a predetermined music selection input operation is performed by the input device 30 in accordance with the display of the interface video 98 on the PC 12a, and as shown in (a), the music selection input information is stored in the virtual world. It is transmitted to the server 14. Next, as shown in (b), the music selection number corresponding to the music selection input information and the user identification information are transmitted from the virtual world server 14 to the karaoke server 16, and the music data for the target PC 12a is transmitted. Delivery is required. Next, as shown in (c), the music data of the performance music requested from the karaoke server 16 to the target PC 12a is streamed. Next, as shown in (d) and (e), the URL for streaming delivery by peer-to-peer from the virtual world server 14 to the PC 12 corresponding to the user existing in the room where the karaoke performance is performed Is distributed. Next, as shown in (f), the PC 12a that has received the streaming distribution of the music data from the karaoke server 16 performs streaming reproduction of the music data related to the streaming distribution, as shown in (g). The audio information input from the microphone 38 is combined with the music data and streamed to the PC 12b as performance music data. Then, the PC 12b that has received the streaming distribution of the performance music data performs streaming playback of the performance music data. It should be noted that the streaming distribution of the performance data from the PC 12a to the PC 12b may be started in response to a distribution request from the PC 12b to the PC 12a.

  FIG. 10 is a flowchart for explaining a main part of the virtual world service execution control by the CPU 20 of the PC 12, which is repeatedly executed at a predetermined cycle.

  First, in step (hereinafter, step is omitted) SA1, it is determined whether or not a login operation for logging in to the virtual world service is performed by the input device 30 or the like. If the determination at SA1 is negative, the routine is terminated accordingly. If the determination at SA1 is affirmative, the identification information (ID) of the user to be logged in and the input are input at SA2. The password thus transmitted is transmitted to the virtual world server 14 via the Internet 18. Next, in SA3, it is determined whether or not an authentication result indicating that the formal authentication has been performed is returned from the virtual world server 14. If the determination of SA3 is negative, the routine is terminated after the fact that an error has occurred is displayed on the video display device 26 or the like in SA4, but if the determination of SA3 is positive In SA5 corresponding to the operation of the virtual world video display control means 70, the virtual world video 100 distributed from the virtual world server 14 is displayed on the video display device 26. Further, the virtual world video 100 is updated as needed in accordance with the distribution from the virtual world server 14. Next, in SA6, it is determined whether or not an action operation, which is an operation for operating the humanoid video 96 of the user included in the virtual world video 100, has been performed by the input device 30 or the like. If the determination at SA6 is negative, the processing after SA8 is executed. If the determination at SA6 is affirmative, the input action information is associated with the identification information of the user at SA7. After being transmitted to the virtual world server 14, it is determined in SA8 whether or not a logout operation for logging out from the virtual world service has been performed by the input device 30 or the like. When the determination at SA8 is negative, the processing after SA5 is executed again. When the determination at SA8 is affirmative, the routine is terminated.

  FIG. 11 is a flowchart for explaining a main part of the virtual world service execution control by the CPU 48 of the virtual world server 14 and is repeatedly executed at a predetermined cycle.

  First, in SB1, it is determined whether or not identification information (ID) and a password for logging in to the virtual world service are received from the PC 12. If the determination at SB1 is negative, the processing from SB6 is executed. If the determination at SB1 is affirmative, the customer database 64 is queried at SB2, and formally at SB3. It is determined whether or not authentication has been performed, that is, whether or not the received identification information is stored, and whether or not the password stored corresponding to the identification information matches the received password. If the determination at SB3 is negative, the process after SB6 is executed after the error is returned to the target PC 12 at SB4, but the determination at SB3 is affirmed. In SB5, the target user is logged in to the virtual world service, and a message to that effect is returned to the target PC 12. In addition, generation and distribution of the virtual world video 100 targeted for the user (PC 12) is started. Next, in SB6, it is determined whether or not action information for operating the human-type image 96 is received from the PC 12. When the determination of SB6 is negative, the processing after SB8 is executed, but when the determination of SB6 is affirmative, the received action information is set as a target in the virtual world video 100 at SB7. The video display control to be reflected in the operation of the human-type video 96 is performed, and the virtual world video 100 updated (generated) in this way is distributed to the PC 12. Next, in SB8, it is determined whether information corresponding to the logout operation is received from the PC 12. If the determination at SB8 is negative, the processing below SB6 is executed. If the determination at SB8 is affirmative, after the target user is logged out from the virtual world service at SB9. This routine is terminated.

  FIG. 12 is a flowchart for explaining a main part of karaoke performance control by the CPU 20 of the PC 12, and is repeatedly executed at a predetermined cycle.

  First, in SC1, it is determined whether or not a karaoke performance start operation for starting a karaoke performance is performed in the virtual world service by the input device 30 or the like. If the determination at SC1 is negative, the routine is terminated. If the determination at SC1 is affirmative, the information corresponding to the karaoke performance start operation is identified at SC2 by the user. It is associated with information and transmitted to the virtual world server 14 via the Internet 18. Next, in SC3, it is determined whether or not the virtual world server 14 has returned that the formal authentication has been performed. If the determination in SC3 is negative, the routine is terminated after the fact that an error has occurred is displayed on the video display device 26 or the like in SC4. However, if the determination in SC3 is positive. In SC5, in accordance with the interface video 98 displayed on the video display device 26, it is determined whether or not a predetermined music selection input operation has been performed by the input device 30 or the like. If the determination at SC5 is negative, the processing from SC7 onward is executed. If the determination at SC5 is affirmative, at SC6, the music selection number of the music piece that has been input is selected by the user. The identification information is transmitted to the virtual world server 14 in association with the identification information. Next, at SC7, it is determined whether or not the performance of the player, that is, the karaoke performance mainly of the user's PC 12, is started. If the determination in SC7 is negative, the processing from SC11 is executed. If the determination in SC7 is positive, the music data streamed from the karaoke server 16 is streamed in SC8. Is done. Next, in SC9 corresponding to the operation of the second streaming distribution means 90, the user's voice information input from the microphone 38 via the A / D converter 40 is combined with the information streamed and reproduced in SC8. The synthesized information is streamed to the PC 12 corresponding to the user (humanoid image 96) existing in the room related to the karaoke performance. Next, in SC10, it is determined whether or not the karaoke performance has ended, that is, whether or not the streaming distribution of the music data from the karaoke server 16 has ended. When the determination of SC10 is negative, the processing after SC8 is executed again. However, when the determination of SC10 is positive, the performance of another person, that is, the karaoke performance that is not based on own PC 12 is performed at SC11. It is determined whether or not is started. If the determination at SC11 is negative, the processing from SC13 onward is executed. If the determination at SC11 is affirmative, at SC12, the other PC 12 that is the subject of the karaoke performance is connected to the Internet 18. The performance music data of the karaoke performance is received via the received data, and the received data is streamed and reproduced. Next, in SC13, it is determined whether or not a karaoke performance end operation for ending the karaoke performance in the virtual world service has been performed by the input device 30 or the like. If the determination of SC13 is negative, the processing from SC5 onward is executed again. If the determination of SC13 is positive, the virtual world server 14 responds to the karaoke performance end operation in SC14. After the behavior information is transmitted, this routine is terminated.

  FIG. 13 is a flowchart for explaining a main part of karaoke performance control by the CPU 46 of the virtual world server 14, and is repeatedly executed at a predetermined cycle.

  First, in SD1, it is determined whether or not information corresponding to the karaoke performance start operation is received from the PC 12. If the determination of SD1 is negative, the processing from SD6 is executed. If the determination of SD1 is positive, the identification information and password of the user related to the start of karaoke performance are stored in SD2. The data is transmitted to the karaoke server 16 and the karaoke server 16 performs login authentication to the karaoke service. Next, in SD3, it is determined whether or not a reply indicating that the karaoke server 16 has been formally authenticated has been returned. If the determination of SD3 is negative, the process after SD6 is executed after the error is returned to the target PC 12 in SD4, but when the determination of SD3 is positive In SD5 corresponding to the operation of the interface video generation means 78, the interface video 98 for performing various input operations relating to karaoke performance is generated as a part of the virtual world video 100 corresponding to the target PC 12, and the interface video 98 is generated. The virtual world image 100 including the interface image 98 is distributed to the target PC 12 via the Internet 18. Next, in SD6, it is determined whether or not, for example, the music selection number of the performance music related to the music selection input and the identification information of the user are received as information corresponding to the music selection input operation from the PC 12. If the determination of SD6 is negative, the processing from SD8 is executed, but if the determination of SD6 is positive, the music selection number and identification information related to the music selection input to the karaoke server 16 in SD7. Is transmitted, and a distribution request for the performance song is made. Next, in SD8, it is determined whether or not the karaoke performance is started. If the determination of SD8 is negative, the processing below SD10 is executed. If the determination of SD8 is positive, in SD9, the PC 12a that is the subject of the karaoke performance and the karaoke performance are performed. A URL for performing peer-to-peer streaming distribution is distributed to the PC 12b corresponding to the human-type image 96 existing in the room. Next, in SD10, it is determined whether or not information corresponding to the karaoke performance end operation is received from the PC 12. When the determination of SD10 is denied, the processing after SD6 is executed again. However, when the determination of SD10 is affirmed, in SD11, a virtual corresponding to the PC 12 of the user related to the end of the karaoke performance. After the interface video 98 is erased from the world video 100, this routine is terminated.

  FIG. 14 is a flowchart for explaining a main part of the karaoke song data distribution control by the CPU 46 of the karaoke server 16 and is repeatedly executed at a predetermined cycle.

  First, in SE1, it is determined whether or not there is an inquiry request for login authentication from the virtual world server 14 to the karaoke service of a predetermined user. If the determination of SE1 is negative, the processing from SE6 is executed. If the determination of SE1 is positive, the customer database 68 is queried in SE2, and formally in SE3. It is determined whether or not authentication has been performed, that is, whether or not the received identification information is stored, and whether or not the password stored corresponding to the identification information matches the received password. If the determination at SE3 is negative, the process from SE6 onward is executed after the fact that an error has been returned to the virtual world server 14 at SE4, but the determination at SE3 is positive. In SE5, the fact that the target user is formally authenticated is returned to the virtual world server 14. Next, in SE6, it is determined whether or not the virtual world server 14 has requested distribution of song data of a predetermined performance song. If the determination at SE6 is negative, the routine is terminated accordingly. If the determination at SE6 is affirmative, the routine is requested in SE7 corresponding to the operation of the first streaming distribution means 86. The music data is streamed via the Internet 18 to the PC 12a that is the main player of the karaoke performance. Next, in SE8, it is determined whether or not the music data related to streaming delivery has been completed (reproduction completed). When the determination at SE8 is negative, the processing after SE7 is executed again, but when the determination at SE8 is affirmative, the routine is terminated.

  In the above control, SA2, SA7, SC2, SC6, and SC14 are the operations of the information transmission control means 72, SB2 and SB3 are the operations of the virtual world service authentication means 74, and SB7 and SD5 are the virtual world video generation. In the operation of the means 76, SB7 and SD5 are the operation of the virtual world video distribution means 80, SD6 and SD7 are the operations of the karaoke distribution request means 82, SE2 and SE3 are the operations of the karaoke service authentication means 84, SE8 And SE12 correspond to the operation of the streaming playback means 88, respectively.

  Thus, according to the present embodiment, the karaoke server 16 performs the first streaming delivery means 86 (SE7) for streaming delivery of the information for performing the karaoke performance to the PC 12a as the terminal device that is the main body of the karaoke performance. And the PC 12 is audio information based on information for performing the karaoke performance delivered from the karaoke server 16 by the first streaming delivery means 86 when the PC 12 is the subject of the karaoke performance. And a host streaming playback means 88 (SC8) for streaming playback of video information, and a musical composition in which the audio information and video information played back by the host streaming playback means 88 are combined with at least second audio information different from the audio information. Stream information to other PCs 12b Second streaming delivery means 90 (SC9), and audio information and video information based on performance song information delivered by the second streaming delivery means 90 from the PC 12a which is the subject when the PC 12 is not the subject of the karaoke performance Is provided with the guest streaming playback means 88 (SC12) for streaming playback of the karaoke, so that it is practically used between the PC 12a as the main body of the karaoke performance and the PC 12b as the terminal device on the side of viewing the karaoke performance. Can be synchronized in various ways. In other words, it is possible to provide the karaoke system 10 using the Internet 18 that realizes synchronous distribution of karaoke information to a plurality of users.

  In addition, a virtual world server 14 connected to the Internet 18 and capable of transmitting / receiving information to / from the karaoke server 16 and the plurality of PCs 12 is provided, and the virtual world server 14 responds to an input from the PC 12. Virtual world image generation means 76 (SB7 and SD5) for generating a virtual world image 100 in which a humanoid image 96 corresponding to each user moves relative to predetermined structure images 92, 94, etc. , Virtual world video distribution means 80 (SB7 and SD5) for distributing the virtual world video 100 generated by the virtual world video generation means 76 to the plurality of PCs 12, and a virtual world generated by the virtual world video generation means 76 An interface that generates an interface video 98 for performing operations related to the karaoke performance as a part of the video 100 Distribution of information for performing the karaoke performance to the plurality of PCs 12 in response to the operation of the humanoid image 96 on the interface image 98 generated by the image generation unit 78 (SD5) and the interface image generation unit 78. Since the requested karaoke distribution request means 82 (SD6 and SD7) is provided, the karaoke system 10 in a practical form can be realized in the virtual world using the Internet 18.

  The interface video generation means 78 is the interface video 98 as a video in a karaoke box video 92 which is a predetermined room as a structure video included in the virtual world video 100 generated by the virtual world video generation means 76. The second streaming distribution means 90 also generates a karaoke box when a humanoid image 96 corresponding to the user of the PC 12a who is the subject of the karaoke performance exists in the karaoke box image 92. Since the performance music information is streamed to the user's PC 12 corresponding to the human-type video 96 existing in the video 92, a store such as a virtual karaoke box in the virtual world using the Internet 18 In the store's room It is possible to realize a karaoke system 10 to allow some karaoke performance.

  The preferred embodiments of the present invention have been described in detail with reference to the drawings. However, the present invention is not limited to these embodiments, and may be implemented in other modes.

  For example, in the above-described embodiment, the virtual world server 14 and the karaoke server 16 are provided, the virtual world server 14 controls the virtual world service, and the karaoke server 16 controls the karaoke service. The present invention is not limited to this. For example, the virtual world service and the karaoke service may be controlled by a single server device. That is, the virtual world server provided with the karaoke database 66 and the first streaming delivery means 86 may control the music data streaming transmission to the PC 12a. Further, the virtual world service may be controlled by two server devices, and various configurations are conceivable depending on the mode of the network.

  In the above-described embodiment, the single PC 12a, which is the main player of the karaoke performance, receives the streaming distribution (first streaming distribution) of the music data from the karaoke server 16, and the music data is streamed and input from the microphone 38. In the above-described embodiment, the synthesized voice information is streamed (second streaming delivery) to a plurality of PCs 12b that are not the subject of the karaoke performance. A case where there are two or more PCs 12a can be considered. In such a case, the first PC 12a, which is the main body of the karaoke performance, receives streaming distribution of the music data from the karaoke server 16 and streams the music data and synthesizes the audio information input from the microphone 38, Streaming is delivered to the second PC 12a, which is the main player of karaoke performance. The second PC 12a receives streaming distribution of the performance music data from the first PC 12a which is the subject of the karaoke performance, streams the performance music data and synthesizes the audio information input from the microphone 38, The synthesized musical composition data is streamed (second streaming delivery) to the first PC 12a, which is the subject of the karaoke performance, and to the plurality of PCs 12b, which are not the subject of the karaoke performance. In this way, karaoke performance by a plurality of singers can be realized in a practical manner.

  In the above-described embodiment, when the PC 12a is the main player of the karaoke performance, the host streaming playback means 88 for streaming the music data distributed from the karaoke server 16 and the PC 12 are not the main subject of the karaoke performance. The guest streaming playback means 88 for streaming playback of performance music data distributed from the PC 12a as the main body is provided in the CPU 20 of the PC 12 as the same control function. Each of the guest streaming playback means may be provided as an individual control function.

  In the above-described embodiment, the karaoke system 10 is realized as a part of the virtual world service controlled by the virtual world server 14, but the present invention is not necessarily performed in association with the virtual world service. It doesn't have to be a thing. For example, the present invention is preferably applied to a remote karaoke system using a videophone via the Internet 18.

  In the above-described embodiment, the example in which the present invention is applied to the karaoke system 10 using the PC 12 as a terminal device has been described. However, for example, the present invention is also applied to a karaoke system using a mobile phone or a dedicated karaoke device as a terminal device. Is preferably applied.

  In the above-described embodiment, streaming distribution / playback of music data between the karaoke server 16 and the PC 12a and streaming distribution of performance music data between the PC 12a which is the main body of the karaoke performance and the PC 12b which is not the main body. / Although the real-time encoding by mpeg data was illustrated as a reproduction | regeneration format, the aspect will not be ask | required if equivalent control can be performed. That is, of course, it is possible to perform streaming distribution / playback using data in other file formats.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

It is a figure explaining the structure of the karaoke system using the internet which is one Example of this invention. It is a figure which illustrates the structure of the personal computer with which the karaoke system of FIG. 1 was equipped. It is a figure which illustrates the structure of the virtual world server with which the karaoke system of FIG. 1 was equipped. It is a figure which illustrates the structure of the karaoke server with which the karaoke system of FIG. 1 was equipped. FIG. 5 is a functional block diagram illustrating a main part of a control function provided in each of the CPU of the personal computer in FIG. 2, the CPU of the virtual world server in FIG. 3, and the CPU of the karaoke server in FIG. 4. It is a figure explaining transmission / reception of the information via the internet between the personal computer in the karaoke performance by the karaoke system of FIG. 1, a virtual world server, and a karaoke server. As an example of the relative positional relationship between the structure image and the humanoid image in the virtual world image generated by the virtual world server of FIG. 3, the concept is a state in which three humanoid images enter the karaoke box image FIG. It is a figure which illustrates the virtual world image | video produced | generated by the virtual world server of FIG. It is a figure which illustrates the virtual world image | video containing the interface image | video for karaoke performance control produced | generated by the virtual world server of FIG. It is a flowchart explaining the principal part of the virtual world service execution control by CPU of the personal computer of FIG. It is a flowchart explaining the principal part of the virtual world service execution control by CPU of the virtual world server of FIG. It is a flowchart explaining the principal part of karaoke performance control by CPU of the personal computer of FIG. It is a flowchart explaining the principal part of karaoke performance control by CPU of the virtual world server of FIG. It is a flowchart explaining the principal part of karaoke music data delivery control by CPU of the karaoke server of FIG.

Explanation of symbols

10: Karaoke system 12: Personal computer (terminal device)
14: Virtual world server 16: Karaoke server 18: Internet 76: Virtual world video generation means 78: Interface video generation means 80: Virtual world video distribution means 82: Karaoke distribution request means 86: First streaming distribution means 88: Streaming reproduction means 90: Second streaming distribution means 92: Karaoke box image (structure image)
92a: Wall image (structure image)
94: Karaoke device image (structure image)
96: Humanoid video 98: Interface video 100: Virtual world video

Claims (2)

A plurality of terminal devices that store information for performing karaoke performance of a large number of performance songs and that are connected to the Internet, and that are also connected to the Internet and can transmit and receive information to and from the karaoke server. And a virtual world server that is also connected to the Internet and is capable of transmitting and receiving information between the karaoke server and a plurality of terminal devices, and from among a large number of performance songs stored in the karaoke server A karaoke system using the Internet that performs karaoke performance of a selected performance piece to be selected by the terminal device,
The karaoke server is provided with first streaming distribution means for streaming information for performing the karaoke performance to the terminal device that is the main body of the karaoke performance,
The terminal device performs streaming reproduction of audio information and video information based on information for performing the karaoke performance distributed from the karaoke server by the first streaming distribution means when the terminal device is a karaoke performance subject. Host streaming playback means;
Second streaming delivery means for streaming and delivering to the other terminal device performance music information obtained by synthesizing at least second audio information different from the audio information to the audio information and video information reproduced by the host streaming reproduction means;
Guest streaming playback means for streaming playback of audio information and video information based on the performance music information distributed by the second streaming distribution means from the terminal device as the main body when the terminal device is not the main body of karaoke performance; It is equipped with
The second streaming means state, and are not provided to the terminal device as a subject of karaoke performance,
The virtual world server generates a virtual world image in which a humanoid image corresponding to each user moves relative to a predetermined structure image in response to an input from the terminal device Generating means;
Virtual world video distribution means for distributing the virtual world video generated by the virtual world video generation means to the plurality of terminal devices;
Interface video generation means for generating an interface video for performing an operation related to the karaoke performance as a part of the virtual world video generated by the virtual world video generation means;
Distribution request means for requesting distribution of information for performing the karaoke performance to the plurality of terminal devices in response to an input operation by the terminal device in accordance with the interface video generated by the interface video generation means;
Karaoke system using the Internet, characterized in that the, those with. The interface video generation means generates the interface video as a video in a predetermined room as a structure video included in the virtual world video generated by the virtual world video generation means,
The second streaming distribution means includes a terminal of a user corresponding to a human-like image existing in the room when a human-type image corresponding to the user of the terminal device that is the subject of the karaoke performance exists in the room. The karaoke system using the Internet according to claim 1 , wherein the music piece information is streamed and distributed to a device.

Images