JP2022114309A - Online session server device - Google Patents

Abstract

To provide an online session server device which can perform a session performance online by using karaoke performance data used in a karaoke performance.SOLUTION: An online session server device comprises: a setting unit which sets parts of a performance; a generation unit which generates audio data by parts corresponding to each of the set parts; a transmission unit which transmits minus one audio data, to a terminal device in which one part is set; an acquisition unit which acquires live performance data from the terminal device in which the one part is set; and a storage processing unit which stores live performance data of a certain music piece acquired from each of the plurality of terminal devices.SELECTED DRAWING: Figure 2

Description

The present invention relates to an online session server device.

An online conference system using a cloud-type server device is in widespread use. Online events using such online conference systems are also being held.

Japanese Unexamined Patent Application Publication No. 2002-200001 discloses a video conference system that includes a plurality of terminals, a conference server, and a music server capable of distributing music to the terminals.

JP 2010-074199 A

As an online event using an online conference system, it is possible for a plurality of users to perform an ensemble (so-called session performance) for a certain piece of music.

Here, by applying the technique disclosed in Patent Document 1, the participants of the video conference can perform online karaoke singing along with the performance of the music distributed to the terminal device they use. However, the performance of a piece of music performed using the technology of Patent Document 1 includes performance sounds of all musical instrument parts (for example, a guitar part, a bass part, and a drum part). Therefore, for example, for a user who is in charge of a guitar part in a session performance, the performance sound of the guitar part included in the performance of the musical composition becomes an obstacle and makes it difficult for the user to perform the performance.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an online session server device that enables online session performances using karaoke performance data used in karaoke performances.

One invention for achieving the above object is an online session server device communicably connected to a terminal device used by each of a plurality of users, wherein each terminal device is assigned a performance part. a generating unit for generating part-specific audio data corresponding to each of the set parts by processing karaoke performance data of a piece of music used in karaoke performance; and one part is set. From a transmission unit that transmits minus one audio data, which is the part-specific audio data corresponding to another part different from the one part, to the terminal device, and the terminal device to which the one part is set, an acquisition unit for acquiring live performance data based on the performance sound obtained by actually performing the one part in accordance with the performance based on the minus one audio data; and the certain musical piece acquired from each of the plurality of terminal devices. and a storage processing unit for storing live performance data.
Other features of the present invention will be clarified by the description of the specification and drawings described later.

According to the present invention, online session performance can be performed using karaoke performance data used in karaoke performance.

1 is a diagram showing an outline of an online session system according to a first embodiment; FIG. 1 is a diagram showing an online conference server device according to a first embodiment; FIG. 4 is a flowchart showing processing of the online conference server device according to the first embodiment; 9 is a flowchart showing processing of an online conference server device according to the second embodiment;

<First Embodiment>
An online session server device according to the present embodiment will be described with reference to FIGS. 1 to 3. FIG.

==Online session system==
As shown in FIG. 1, an online session system 1 according to this embodiment includes an online conference system S1 and a communication karaoke system S2. Online conference system S1 and online karaoke system S2 are communicably connected as described in Patent Document 1, for example.

[Online meeting system]
The online conference system S1 is a system for conducting online conferences such as general web conferences and video conferences. The online conference system S1 includes an online conference server CS and a plurality of terminal devices. The online conference server CS and each of the plurality of terminal devices are communicably connected via a network.

The online conference server CS is a computer that manages various information and executes various processes. In this embodiment, the online conference server CS uses a general online conference mechanism to enable online session performances (details will be described later). That is, the online conference server device CS in this embodiment functions as an "online session server device".

A terminal device is, for example, a computer used by a user, such as a home PC, tablet terminal, or mobile terminal. FIG. 1 shows terminal devices T1 to T3. Terminal devices T1 to T3 are used by users U1 to U3, respectively. The number of terminal devices is not limited to the example shown in FIG. 1, as long as there are at least two.

Each terminal device includes a microphone for collecting the performance sound of the user's performance, a speaker for emitting the performance based on minus one audio data (details will be described later), a camera for photographing the user, It has a display for displaying images and various kinds of information, an operation means for performing various operation inputs, and the like. It should be noted that the performance in this embodiment includes performance of musical instruments and singing.

Furthermore, in this embodiment, each terminal device is pre-installed with an online session program. By executing the program on the terminal device, the user can perform an online session performance using the mechanism of the online conference.

[Communication karaoke system]
The communication karaoke system S2 is a system that distributes music for performing karaoke singing and performs karaoke performance. The communication karaoke system S2 includes a karaoke server device KS and a plurality of karaoke devices (not shown). The karaoke server device KS and each of the plurality of karaoke devices are communicably connected via a network.

The karaoke server device KS is a computer that manages karaoke performance data, various types of information, and performs processing for distributing songs.

The karaoke performance data is data used in karaoke performance. The karaoke performance data is provided with song identification information for specifying individual songs. The song identification information is information unique to each song, such as a song ID for identifying the song. In this embodiment, the karaoke performance data includes accompaniment data and guide vocal data.

The accompaniment data is data for karaoke performance of the music. Specifically, the accompaniment data includes data for each musical instrument part (eg, guitar part, bass part, drum part) used in the karaoke performance of the song. The accompaniment data is data in MIDI format.

The guide vocal data is data corresponding to singing voice for implementing the guide vocal function. The guide vocal function is a function that assists karaoke singing, and reproduces pre-registered guide vocals (sung by professional singers, etc.) together with karaoke performances of songs. The guide vocal data is data in audio format.

== Online Conference Server Device ==
FIG. 2 is a diagram showing the configuration of the online conference server CS. The online conference server device CS comprises storage means 10 , communication means 20 and control means 30 . Each configuration is connected to bus B via an interface (not shown).

[Storage means/Communication means]
The storage means 10 is a large-capacity storage device that stores various data. The communication means 20 provides an interface for communicating with each terminal device and the communication karaoke system S2.

[Control means]
The control means 30 performs various controls in the online conference server CS. The control means 30 includes a CPU and memory (both not shown). The CPU implements various functions by executing programs stored in the memory.

In this embodiment, the control means 30 functions as a setting unit 100, a generation unit 200, a transmission unit 300, an acquisition unit 400, and a storage processing unit 500 by executing programs stored in the memory by the CPU.

(setting part)
The setting unit 100 sets a performance part to be in charge of for each terminal device.

A user who desires an on-line session performance inputs the part to be played through the operating means of his/her own terminal device. The terminal device associates the input part information with its own terminal identification information and transmits the information to the online conference server CS. The terminal identification information is information unique to each terminal device, such as a terminal ID for identifying the terminal device.

The setting unit 100 sets one performance part for the terminal device based on the received information. In this embodiment, the performance part includes an instrumental part and a singing part.

(generator)
The generation unit 200 generates part-by-part audio data by processing karaoke performance data of a song used in karaoke performance. The part-by-part audio data is audio format data corresponding to each of the set parts.

The part-by-part audio data can be generated using a known rendering technology such as a DAW (for example, Japanese Patent Application Laid-Open No. 2017-097289).

One of the users who desire online session performance selects a piece of music desired for session performance through the operating means of his/her own terminal device. The terminal device transmits music identification information of the selected music to the online conference server CS.

The generation unit 200 transmits the received music identification information to the karaoke server device KS. The karaoke server device KS reads karaoke performance data based on the received music identification information from a storage means (not shown) and transmits the data to the online conference server device CS.

The generation unit 200 generates part-by-part audio data by rendering data for each part set by the setting unit 100 from the received karaoke performance data.

For example, the selected song X consists of a singing part, a guitar part, a bass part and a drum part. Then, assume that the setting unit 100 sets a singing part, a guitar part, a bass part, and a drum part for each terminal device.

In this case, the generation unit 200 renders the guitar part, the bass part, and the drum part from the accompaniment data included in the karaoke performance data of the song X, so that the per-part audio of the guitar part, the bass part, and the drum part is rendered. Generate data. Specifically, the generation unit 200 converts the set part information into tone color information in MIDI format data, and renders accompaniment data of one or more parts corresponding to the tone color information. For example, in the rendering of a guitar part, it is possible to generate tone signals for each of the six strings from accompaniment data for the six parts.

On the other hand, as described above, the guide vocal data in this embodiment is data in audio format. Therefore, the generation unit 200 uses the guide vocal data included in the karaoke performance data of the song X as part-specific audio data of the singing part.

Further, data corresponding to guide vocal data can be formed from accompaniment data by using a known voice synthesis engine or guide melody sound source. In this case, the generation unit 200 can generate the part-by-part audio data of the singing part by rendering the configured data. In this way, when the data corresponding to the guide vocal data is formed from the accompaniment data, the karaoke performance data may not include the guide vocal data.

Depending on the number of users participating in the session performance and the types of parts that the users are in charge of, not all the parts that make up a certain piece of music are set.

If there is a part that is not set by the setting unit 100 among a plurality of parts that constitute a piece of music, the generation unit 200 can generate part-specific audio data and accompaniment audio data. Accompaniment audio data is audio format data corresponding to all parts other than the set part.

Accompaniment audio data can be generated using a well-known rendering technique, similarly to part-specific audio data. The generation unit 200 generates accompaniment audio data by rendering data other than the part set by the setting unit 100 from the received karaoke performance data (details will be described later).

(Transmitter)
The transmission unit 300 transmits minus one audio data to the terminal device to which one part is set. Minus one audio data is part-by-part audio data corresponding to another part different from one part.

For example, assume that the setting unit 100 sets a singing part for one terminal device in the session performance of the song X described above.

In this case, the transmission unit 300 transmits parts corresponding to other parts (a guitar part, a bass part, and a drum part) different from the singing part as minus one audio data to one terminal device to which the singing part is set. Send another audio data. Note that the transmission unit 300 can refer to the terminal identification information linked to the information of the part to specify the terminal device that transmits the minus one audio data. Also, when the minus one audio data includes a plurality of part-specific audio data as in this example, the transmission unit 300 mixes each data and transmits the mixed data to one terminal device.

Similarly, the transmission unit 300 transmits parts corresponding to other parts (a singing part, a bass part, and a drum part) different from the guitar part as minus one audio data to one terminal device to which the guitar part is set. Send another audio data. In addition, the transmission unit 300 transmits, as minus one audio data, to one terminal device to which the bass part is set, part-by-part corresponding to other parts (a singing part, a guitar part, and a drum part) different from the bass part. Send audio data. In addition, the transmission unit 300 transmits, as minus one audio data, to one terminal device to which the drum part is set, part-by-part corresponding to other parts (a singing part, a guitar part, and a bass part) different from the drum part. Send audio data.

The transmission unit 300 simultaneously transmits minus one audio data to each terminal device.

As described above, the generation unit 200 may generate part-specific audio data and accompaniment audio data. In this case, the transmitting unit 300 transmits minus one audio data including accompaniment audio data to the terminal device to which one part is set (details will be described later).

In addition, the transmission unit 300 may transmit the count-in sound data together with the minus one audio data to the terminal device to which one part is set. The count-in sound is a sound indicating the timing of starting the actual performance in time with the performance based on the minus one audio data. The count-in sound data can be generated, for example, by the generation unit 200 based on the tempo information included in the karaoke performance data. By referring to the count-in sound emitted from the speaker of the terminal device, the user can easily grasp the timing of starting the performance even when performing a part without a prelude.

(acquisition part)
Acquisition unit 400 acquires live performance data from a terminal device in which one part is set. The live performance data is data based on the performance sound obtained by actually performing one part along with the performance based on the minus one audio data.

For example, in the session performance of the song X described above, the transmitting unit 300 sends part-specific audio data corresponding to other parts (guitar part, bass part, and drum part) to one terminal device in which singing parts are set. Suppose that minus one audio data including

In this case, one terminal device starts reproducing the received part-specific audio data. A user of one terminal device sings the song X while listening to performance sounds based on the part-by-part audio data. One terminal device links live performance data based on the user's singing voice with its own terminal identification information and transmits the data to the online conference server CS. The acquisition unit 400 acquires live performance data of singing. In addition, in this embodiment, the singing voice is an example of the "performance sound".

Similarly, the transmitting unit 300 transmits minus one audio data including part-specific audio data corresponding to other parts (bass part, drum part, and singing part) to one terminal device set with a guitar part. do. In this case, one terminal device starts reproducing the received part-specific audio data. A user of one terminal device plays a musical piece X on the guitar while listening to performance sounds based on the part-specific audio data. One terminal device links the live performance data based on the sound of the guitar performance with its own terminal identification information, and transmits the live performance data to the online conference server CS. Acquisition unit 400 acquires live performance data of a guitar performance. In addition, the transmitting unit 300 transmits minus one audio data including part-specific audio data corresponding to other parts (guitar part, drum part, and singing part) to one terminal device to which the bass part is set. . In this case, one terminal device starts reproducing the received part-specific audio data. A user of one terminal device performs a bass performance of the piece of music X while listening to the performance sound based on the part-by-part audio data. One terminal device links live performance data based on the bass performance sound with its own terminal identification information, and transmits the live performance data to the online conference server CS. The acquisition unit 400 acquires live performance data of a bass performance. In addition, the transmission unit 300 transmits minus one audio data including part-specific audio data corresponding to other parts (guitar part, bass part, and singing part) to one terminal device to which the drum part is set. . In this case, one terminal device starts reproducing the received part-specific audio data. A user of one terminal device plays the drums of the song X while listening to performance sounds based on the part-specific audio data. One terminal device links live performance data based on drum performance sounds with its own terminal identification information and transmits the live performance data to the online conference server CS. Acquisition unit 400 acquires live performance data of a drum performance.

As described above, even if the transmission unit 300 simultaneously transmits part-specific audio data to each terminal device, depending on the communication state, each terminal device receives the part-specific audio data at the timing Delays may occur. Similarly, there may be a delay in the timing at which the acquisition unit 400 acquires live performance data transmitted from each terminal device.

(memory processor)
The storage processing unit 500 causes the storage unit 10 to store live performance data of a piece of music acquired from each of a plurality of terminal devices.

For example, for song X, assume that the acquisition unit 400 acquires live performance data of singing, live performance data of guitar performance, live performance data of bass performance, and live performance data of drum performance.

In this case, the storage processing unit 500 causes the storage unit 10 to store each piece of live performance data each time it acquires it.

After each user's performance is finished, one user can instruct distribution of the session performance via his/her own terminal device. Based on the instruction, the online conference server CS reproduces the session performance in which all the live performance data stored in the storage means 10 are synchronized, and distributes it to each terminal device. Note that the online conference server CS can refer to the terminal identification information associated with the part to identify the terminal device that delivers the session performance. Further, the online conference server CS may automatically distribute the session performance when the transmission unit 300 receives the live performance data from all the terminal devices to which the minus one audio data has been transmitted.

Also, as described above, the generation unit 200 may generate part-specific audio data and accompaniment audio data. In this case, the storage processing unit 500 causes the storage unit 10 to store live performance data and accompaniment audio data of a certain piece of music (details will be described later).

== Operation of the online conference server device ==
Next, a specific example of the operation of the online conference server CS in this embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing an operation example of the online conference server CS. In this example, it is assumed that users U1 to U3 shown in FIG. 1 play an online session. Also, assume that a song Y consisting of a singing part, a guitar part, a bass part, a drum part, and a keyboard part is selected as a song for session performance.

Each user inputs the part he/she is in charge of performing through the operating means of his/her own terminal device. The terminal device associates the input part information with its own terminal identification information and transmits the information to the online conference server CS.

The setting unit 100 of the online conference server CS sets one part for the terminal devices T1 to T3 (one part is set for each terminal device, step 10).

For example, the user U1 inputs the bass part as the part to be played through the operating means of the terminal device T1. The terminal device T1 associates the input information of the bass part with the terminal identification information ID***T1 of the terminal device T1 and transmits it to the online conference server CS. The setting unit 100 sets the base part for the terminal device T1 based on the received information.

Similarly, the user U2 inputs a guitar part as a part to be played through the operating means of the terminal device T2. The terminal device T2 associates the input guitar part information with the terminal identification information ID***T2 of the terminal device T2, and transmits it to the online conference server CS. The setting unit 100 sets a guitar part for the terminal device T2 based on the received information. The user U3 inputs a singing part as a part to be played through the operating means of the terminal device T3. The terminal device T3 associates the input singing part information with the terminal identification information ID***T3 of the terminal device T3 and transmits it to the online conference server CS. The setting unit 100 sets a singing part for the terminal device T3 based on the received information.

The generation unit 200 renders the karaoke performance data of the selected song Y, and generates the part-specific audio data PA1 corresponding to the guitar part, the part-specific audio data PA2 corresponding to the bass part, and the singing part set in step 10. (Generate part-specific audio data corresponding to the set part, step 11).

In addition, the generation unit 200 renders the karaoke performance data of the song Y for the drum part and the keyboard part that are not set by the setting unit 100 among the parts that make up the song Y, and corresponds to the drum part and the keyboard part. Generate accompaniment audio data AC (generate accompaniment audio data, step 12).

The transmission unit 300 transmits, to each of the terminal devices T1 to T3, part-specific audio data corresponding to parts different from the set parts, and accompaniment audio data corresponding to all parts other than the set parts. Send minus one audio data including data (send minus one data to each terminal device, step 13).

For example, the transmitting unit 300 refers to the terminal identification information ID***T1, and transmits minus one audio data including the part-specific audio data PA1 and PA3 and the accompaniment audio data AC to the terminal device T1 in which the bass part is set. Send data MA1.

Similarly, the transmitting unit 300 refers to the terminal identification information ID***T2, and transmits minus one including the part-specific audio data PA2 and PA3 and the accompaniment audio data AC to the terminal device T2 in which the guitar part is set. Send audio data MA2. In addition, the transmitting unit 300 refers to the terminal identification information ID***T3, and transmits minus one audio data including the part-specific audio data PA1 and PA2 and the accompaniment audio data AC to the terminal device T3 in which the singing part is set. Send data MA3. In this example, transmitting section 300 starts transmitting minus one audio data MA1 to MA3 at the same timing t (t=0).

The acquisition unit 400 acquires live performance data from the terminal devices T1 to T3 (acquires live performance data from each terminal device; step 14).

For example, the terminal device T1 starts playing the minus one audio data MA1 received at the timing Dt1 from the online conference server CS. The user U1 plays the bass of the song Y while listening to the performance sound based on the minus one audio data MA1. The terminal device T1 transmits live performance data LM1 based on the bass performance sound to the online conference server CS in association with the terminal identification information ID***T1. The acquiring unit 400 starts acquiring live performance data LM1 at timing Dt2.

Similarly, the terminal device T2 starts playing the minus one audio data MA2 received from the online conference server CS at the timing Dt3. The user U2 plays the song Y on the guitar while listening to the performance sound based on the minus one audio data MA2. The terminal device T2 links the live performance data LM2 based on the sound of the guitar playing with the terminal identification information ID***T2 and transmits the live performance data LM2 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM2 at timing Dt4. Also, the terminal device T3 starts reproducing the minus one audio data MA3 received from the online conference server CS at the timing Dt5. The user U3 sings the song Y while listening to the performance sound based on the minus one audio data MA3. The terminal device T3 associates the live performance data LM3 based on the singing voice with the terminal identification information ID***T3 and transmits the live performance data LM3 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM3 at timing Dt6.

Timings Dt1 to Dt6 indicate time points later than timing t. The timings Dt1 to Dt6 change depending on the communication environment between the online conference server CS and each terminal device.

The storage processing unit 500 stores the live performance data of the song Y obtained in step 14 and the accompaniment audio data generated in step 12 in the storage means 10 (stores the live performance data and the accompaniment audio data; step 15).

For example, assume that the acquisition unit 400 acquires live performance data LM1 of the bass performance of the song Y, live performance data LM2 of the guitar performance of the song Y, and live performance data LM3 of the singing of the song Y.

In this case, the storage processing section 500 causes the storage means 10 to store the live performance data LM1 to LM3 and the accompaniment audio data AC.

After each user's performance is finished, the online conference server CS refers to the terminal identification information linked to the part, and sends the session performance based on the live performance data and the accompaniment audio data to the terminal devices T1 to T3. Distribute (Distribute the session performance to each terminal device. Step 16).

For example, the online conference server CS reproduces a session performance in which the live performance data LM1 to LM3 and the accompaniment audio data AC are synchronized, based on the terminal identification information ID***T1 to terminal identification information ID***T3. It is distributed to terminal devices T1 to T3.

As is clear from the above, the online conference server CS according to this embodiment is communicably connected to terminal devices used by a plurality of users. The online conference server device CS corresponds to each of the set parts by processing the karaoke performance data of a certain piece of music used in the karaoke performance, and the setting unit 100 that sets the performance part to be in charge for each terminal device. and minus-one audio data, which is part-specific audio data corresponding to another part different from the one part, to the terminal device to which one part is set. Acquire live performance data based on the performance sound obtained by actually performing the one part in accordance with the performance based on the minus one audio data from the transmitting unit 300 for transmission and the terminal device in which the one part is set. and a storage processing unit 500 for storing live performance data of a piece of music acquired from each of a plurality of terminal devices.

According to such an online conference server CS, it is possible to generate part-specific audio data using karaoke performance data and transmit minus one audio data to the terminal device in which the part is set. The minus one audio data transmitted to a certain terminal device includes part-by-part audio data corresponding to parts other than the part set to the certain terminal device. In other words, the performance sound based on the minus one audio data transmitted to a certain terminal device does not include the performance sound of the part set in the certain terminal device. Therefore, the user can easily perform his/her own part in accordance with the performance sound. That is, according to the online conference server device CS according to the present embodiment, an online session performance can be performed using the karaoke performance data used in the karaoke performance.

Further, in the online conference server device CS according to the present embodiment, if there is a part that is not set by the setting unit 100 among a plurality of parts that constitute a piece of music, the generation unit 200 generates the audio data for each part, Accompaniment audio data corresponding to all parts other than the set part is generated. The transmitting unit 300 transmits minus one audio data including accompaniment audio data to the terminal device to which one part is set. The storage processing unit 500 stores live performance data and accompaniment audio data of a certain piece of music. According to such an online conference server device CS, even if there is a performance part that is not set among a plurality of parts that make up a certain piece of music, minus one audio that is suitable for the part played by the user is provided. data can be provided.

<Second embodiment>
Next, an online session server device according to the second embodiment will be described with reference to FIG. In this embodiment, an example will be described in which part of the part-specific audio data included in the minus one audio data is replaced with live performance data. A detailed description of the configuration similar to that of the first embodiment will be omitted.

(Transmitter)
When the transmission unit 300 according to the present embodiment transmits minus one audio data to a terminal device to which one part is set, if live performance data is acquired from another terminal device, the other part minus one audio data obtained by replacing the part-specific audio data corresponding to , with the live performance data obtained from the terminal device in which the other part is set.

For example, the transmission unit 300 transmits other parts (a guitar part, a bass part, and a drum part) different from the singing part as minus one audio data to a certain terminal device in which the singing part of the song X of the first embodiment is set. start transmitting part-specific audio data corresponding to part). A certain terminal device starts reproducing the received part-by-part audio data. A user of a certain terminal device sings a piece of music X while listening to performance sounds based on part-by-part audio data. A certain terminal device links live performance data based on singing voice with its own terminal identification information and transmits it to the online conference server CS. The acquisition unit 400 starts acquiring live performance data of singing.

When the acquisition unit 400 starts acquiring live performance data of a song, the transmission unit 300 transmits another part (singing part) different from the guitar part as minus one audio data to another terminal device in which the guitar part is set. , bass part, and drum part).

Here, the online conference server CS has already acquired live performance data of singing from a certain terminal device. In this case, the transmission unit 300 transmits minus one audio data obtained by replacing the part-specific audio data corresponding to the singing part with the acquired live performance data of the singing to another terminal device. In this example, another terminal device corresponds to "a terminal device in which one part is set", a certain terminal device corresponds to "another terminal device", and a singing part corresponds to "another part". .

== Operation of the online conference server device ==
Next, a specific example of the operation of the online conference server CS in this embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing an operation example of the online conference server CS. In this example, it is assumed that users U1 to U3 shown in FIG. 1 play an online session. Also, assume that a song Y consisting of a singing part, a guitar part, a bass part, a drum part, and a keyboard part is selected as a song for session performance.

The setting unit 100 of the online conference server CS sequentially sets one part for the terminal devices T1 to T3 (sets one part for each terminal device in order, step 20). For example, assume that user U1, user U2, and user U3 input the parts in charge of performance in this order. In this case, the setting unit 100 sets the parts in the order of user U1 (bass part), user U2 (guitar part), and user U3 (singing part). The method of setting parts for each terminal device is the same as in step 10 of the first embodiment.

Steps 21 and 22 are the same as steps 11 and 12 of the first embodiment.

The transmission unit 300 transmits the part-specific audio data corresponding to other parts different from the set part and all parts other than the set part to the terminal device T1 used by the user U1 who set the part first. (The transmission of minus one data to the terminal device T1 is started. Step 23).

Specifically, the transmitting unit 300 refers to the terminal identification information ID ***T1, and sends the minus part including the part-specific audio data PA1 and PA3 and the accompaniment audio data AC to the terminal device T1 in which the bass part is set. Transmission of one audio data MA1 is started at timing t (t=0).

The acquisition unit 400 starts acquiring live performance data from the terminal device T1 (starts acquiring live performance data from the terminal device T1, step 24).

The terminal device T1 starts playing the minus one audio data MA1 received from the online conference server CS at the timing Dt7. The user U1 plays the bass of the song Y while listening to the performance sound based on the minus one audio data MA1. The terminal device T1 transmits live performance data LM1 based on the bass performance sound to the online conference server CS in association with the terminal identification information ID***T1. The acquiring unit 400 starts acquiring the live performance data LM1 at timing Dt8.

The transmitting unit 300 starts transmitting the minus one audio data to the terminal device T2 used by the user U2 who set the part next to the user U1.

At this point, the online conference server CS has acquired live performance data obtained by actually playing the bass part from the terminal device T1. Therefore, the transmitting unit 300 starts transmitting the minus one audio data obtained by replacing the part-specific audio data corresponding to the bass part with the acquired live performance data of the bass part (to the terminal device T2, the part of the audio data). Transmission of minus one data in which the audio data for each part is replaced with live performance data is started (step 25).

Specifically, the transmission unit 300 refers to the terminal identification information ID***T2, and sends the live performance data PA3 and the live performance data PA1 replaced with the part-specific audio data PA3 and the part-specific audio data PA1 to the terminal device T2 in which the guitar part is set. Transmission of minus one audio data MA2 including data LM1 and accompaniment audio data AC is started at timing Dt8.

The acquisition unit 400 starts acquiring live performance data from the terminal device T2 (starts acquiring live performance data from the terminal device T2, step 26).

The terminal device T2 starts playing the minus one audio data MA2 received at the timing Dt9 from the online conference server device CS. The user U2 plays the song Y on the guitar while listening to the performance sound based on the minus one audio data MA2. The terminal device T2 links the live performance data LM2 based on the sound of the guitar playing with the terminal identification information ID***T2 and transmits the live performance data LM2 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM2 at timing Dt10.

The transmitting unit 300 starts transmitting the minus one audio data to the terminal device T3 used by the user U3 who set the part next to the user U2.

At this point, the online conference server CS obtains live performance data obtained by actually playing the bass part from the terminal device T1, and obtains live performance data obtained by actually playing the guitar part from the terminal device T2. Acquiring performance data. Therefore, the transmission unit 300 replaces the part-specific audio data corresponding to the bass part and the part-specific audio data corresponding to the guitar part with the acquired live performance data of the bass part and the acquired live performance data of the guitar part. Audio data transmission is started (transmission of minus one data in which part of the part-by-part audio data is replaced with live performance data is started to the terminal device T3. Step 27).

Specifically, the transmitting unit 300 refers to the terminal identification information ID***T3, and sends the live performance data LM1 replaced with the part-specific audio data PA1 to the terminal device T3 in which the singing part is set, and the part-specific audio data LM1. Transmission of live performance data LM2 replaced with data PA2 and minus one audio data MA3 including accompaniment audio data AC is started at timing Dt10.

The acquisition unit 400 starts acquiring live performance data from the terminal device T3 (Starts acquiring live performance data from the terminal device T3. Step 28).

The terminal device T3 starts reproducing the minus one audio data MA3 received at the timing Dt11 from the online conference server device CS. The user U3 sings the song Y while listening to the performance sound based on the minus one audio data MA3. The terminal device T3 associates the live performance data LM3 based on the singing voice with the terminal identification information ID***T3 and transmits the live performance data LM3 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM3 at timing Dt12.

Timings Dt7 to Dt12 indicate points later than timing t. Timing Dt12 is later than timing Dt11, timing Dt11 is later than timing Dt10, timing Dt10 is later than timing Dt9, and timing Dt9 is later than timing Dt9. , and timing Dt8, which is later than timing Dt7. Also, the timings Dt7 to Dt12 change depending on the communication environment between the online conference server CS and each terminal device.

The storage processing unit 500 causes the storage unit 10 to store the live performance data of the piece of music Y whose acquisition was started in steps 24, 26, and 28 and the accompaniment audio data generated in step 22 (the live performance data and the accompaniment audio data are stored in the storage means 10). Storage, step 29).

After each user's performance is finished, the online conference server CS refers to the terminal identification information linked to the part, and sends the session performance based on the live performance data and the accompaniment audio data to the terminal devices T1 to T3. Distribute (Distribute the session performance to each terminal device. Step 30).

As is clear from the above, when the transmission unit 300 in the online conference server CS according to the present embodiment transmits the minus one audio data to the terminal device to which one part is set, the other terminal device When live performance data is acquired, transmit minus one audio data obtained by replacing part-specific audio data corresponding to other parts with live performance data acquired from the terminal device in which the other parts are set. .

According to such an online conference server CS, minus one audio data including live performance data can be transmitted to some terminal devices. Therefore, the user can feel as if he/she is performing an actual session performance when performing his or her own part in accordance with the performance sound based on the minus one audio data.

<Modification of Second Embodiment>
The transmitting unit 300 sends all of the part-specific audio data corresponding to the other parts to the terminal device of the user who has selected a certain piece of music, and transmits the live performance data obtained from the terminal device in which the other parts are set. minus one audio data replaced by .

For example, assume that user U1 selects song Y in the example of the second embodiment.

In this case, the transmitting unit 300 first transmits the minus one audio data to the terminal devices T2 and T3 used by the users U2 and U3 who are different from the user U1.

For example, the transmission unit 300 refers to the terminal identification information ID***T2, and sends minus-one audio data including the part-specific audio data PA1 and PA3 and the accompaniment audio data AC to the terminal device T2 in which the guitar part is set. Start sending data MA2.

The terminal device T2 starts playing the minus one audio data MA2 received from the online conference server CS. The user U2 plays the song Y on the guitar while listening to the performance sound based on the minus one audio data MA2. The terminal device T2 links the live performance data LM2 based on the sound of the guitar playing with the terminal identification information ID***T2 and transmits the live performance data LM2 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM2.

The transmission unit 300 refers to the terminal identification information ID***T3, and sends the part-specific audio data PA1, the live performance data LM2 replaced with the part-specific audio data PA2, and Transmission of minus one audio data MA3 including accompaniment audio data AC is started.

The terminal device T3 starts playing the minus one audio data MA3 received from the online conference server CS. The user U3 sings the song Y while listening to the performance sound based on the minus one audio data MA3. The terminal device T3 associates the live performance data LM3 based on the singing voice with the terminal identification information ID***T3 and transmits the live performance data LM3 to the online conference server CS. The acquiring unit 400 starts acquiring the live performance data LM3. In this example, the example in which the minus one audio data is transmitted in the order of the terminal device T2 and the terminal device T3 has been described, but the present invention is not limited to this. The order in which the minus one audio data is transmitted is not particularly limited as long as the terminal device of the user who has selected the music piece comes last.

The transmitting unit 300 sends the live performance data LM2 replaced with the part-specific audio data PA2 and the part-specific audio data to the terminal device T1 set with the bass part (the terminal device T1 used by the user who selected the song Y). Transmission of live performance data LM3 replaced with PA3 and minus one audio data MA1 including accompaniment audio data AC is started.

The terminal device T1 starts playing the minus one audio data MA1 received from the online conference server CS. The user U1 plays the bass of the song Y while listening to the performance sound based on the minus one audio data MA1. The terminal device T1 transmits live performance data LM1 based on the bass performance sound to the online conference server CS in association with the terminal identification information ID***T1. The acquiring unit 400 starts acquiring the live performance data LM1.

A user who has selected a piece of music is highly likely to want to actively perform a session performance of the piece of music. On the other hand, according to such an online conference server device CS, live performance data obtained based on actual performances of other users is transmitted to the terminal device used by the user who has selected the music to be performed in the session. All inclusive minus one audio data can be sent. Therefore, a user who has selected a piece of music can feel more strongly as if he or she is actually performing an actual session performance.

<Others>
The acquisition unit 400 can acquire live performance data of a certain piece of music and video data of a user using the terminal device from the terminal device.

The video data is obtained, for example, by photographing the user during the performance with a camera of the terminal device. The terminal device transmits video data together with the live performance data to the online conference server CS. The acquisition unit 400 acquires live performance data and video data. The storage processing section 500 causes the storage means 10 to store live performance data and video data. The online conference server CS reproduces the video based on the stored video data together with the session performance synchronized with the live music data, and distributes it to each terminal device.

Further, in the above embodiment, all the users U1 to U3 who use the online conference system S1 perform some performance. On the other hand, there may be users who do not perform. Therefore, the online conference server CS distributes all part-specific audio data (or all part-specific audio data and accompaniment audio data) generated by the generating unit 200 to the terminal devices used by such users. be able to. By distributing such data, even when other users are performing session performances, users who do not play will not be bored.

Also, the karaoke server device KS may have a part of the configuration described in the above embodiment. For example, in the example of the first embodiment, the online conference server CS has the setting unit 100, the transmission unit 300, the acquisition unit 400, and the storage processing unit 500, and the karaoke server KS has the generation unit 200. good too. In these cases, the online conference server CS and the karaoke server KS function as "online session servers."

Furthermore, one server device having the functions of the online conference server device CS and the karaoke server device KS described in the above embodiments may be provided. The terminal device is configured to be able to directly communicate with the server device. In this case, the server device functions as an "online session server device".

The above embodiments are presented as examples and are not intended to limit the scope of the invention. The above configurations can be implemented in combination as appropriate, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. The above-described embodiments and modifications thereof are included in the invention described in the claims and their equivalents, as well as being included in the scope and gist of the invention.

1 online session system 100 setting unit 200 generation unit 300 transmission unit 400 acquisition unit 500 storage processing unit CS online conference server device KS karaoke server device S1 online conference system S2 communication karaoke system T1 to T3 terminal device

Claims (4)

An online session server device communicably connected to a terminal device used by each of a plurality of users,
a setting unit for setting a performance part to be in charge of for each of the terminal devices;
a generation unit that generates part-specific audio data corresponding to each of the set parts by processing karaoke performance data of a song used in karaoke performance;
a transmission unit configured to transmit minus one audio data, which is the part-specific audio data corresponding to another part different from the one part, to the terminal device to which one part is set;
an acquisition unit for acquiring live performance data based on a performance sound obtained by actually performing the one part in accordance with the performance based on the minus one audio data from the terminal device in which the one part is set;
a storage processing unit that stores live performance data of the music piece that is acquired from each of the plurality of terminal devices;
An online session server device having If there is a part that is not set by the setting unit among the plurality of parts that make up the music piece, the generation unit corresponds to the part-specific audio data and all parts other than the set part. generate accompaniment audio data that
The transmission unit transmits the minus one audio data including the accompaniment audio data to the terminal device to which one part is set,
2. The online session server apparatus according to claim 1, wherein the storage processing unit stores the live performance data of the certain piece of music and the accompaniment audio data. When the minus one audio data is transmitted to the terminal device to which one part is set, the live performance data obtained by actually playing the other part is obtained from another terminal device 3. The on-line system according to claim 1, wherein said transmission unit transmits said minus one audio data obtained by replacing said part-specific audio data corresponding to other parts with acquired live performance data. Session server device. The transmission unit transmits all of the part-specific audio data corresponding to other parts to the terminal device of the user who has selected the certain music piece, the raw data obtained from the terminal device in which the other parts are set. 4. The online session server apparatus according to claim 3, wherein said minus one audio data replaced with performance data is transmitted.

Images