JP2020102053A - Content distribution system, receiving device and program - Google Patents

Abstract

To provide viewers with a highly entertaining viewing experience.SOLUTION: The content distribution system has a distribution unit that distributes a content that presents characters placed in three-dimensional space to a receiving device. A viewing position for viewing the three-dimensional space is set, and the content is reproduced and presented at the viewing position. Here, the content includes at least: a first part that provides a performance that does not define the viewing position as a part that provides the performance; and a second part that provides the performance for the predetermined viewing positions. For the second part, a control is performed so that the performance for the set viewing position is provided.SELECTED DRAWING: Figure 8

Description

The present invention relates to a content distribution system, a receiving device, and a program, and particularly to a technology for distributing binocular stereoscopic content.

Devices that enable binocular stereoscopic vision, such as television receivers and head mounted displays (HMDs), are generally distributed, and users can easily enjoy binocular stereoscopic content by using such devices. Is becoming possible. Patent Document 1 discloses a content distribution system that distributes three-dimensional image content that can be viewed with such a device.

JP, 2016-212622, A

By the way, in recent years, so-called live distribution, which distributes in real time a moving image captured by a camera and a moving image capturing a screen on which computer graphics (CG) such as a game is drawn, is performed using various platforms. ing. Among the contents provided by such live distribution, the facial expressions and behaviors of the performers are detected as motions and reflected in the CGs of the characters, etc., to provide a viewing experience as if the characters actually existed. Some do. In addition, the information transmitted in the live distribution is not limited to a mode in which the information is a moving image stream, and the motion and audio data applied to the CG is transmitted to the device (client) for viewing, and the client draws the CG for viewing. A similar viewing experience can also be provided in a mode in which a screen for use is generated and presented.

Here, for example, a three-dimensional space in which a character interlocking with the behavior of the performer is placed on the stage can be viewed live from any preset viewpoint position on the floor placed around the stage. Consider aspects. Further, it is assumed that the performance such as the performance performed by the character on the stage is configured to be equally viewable at any viewpoint position, for example, at the center of the stage.

In such an aspect, although there is a difference in the appearance of the effect depending on the viewpoint position, a substantially uniform viewing experience can be provided to the user (viewer) who uses the client. On the other hand, in order to raise the viewer's interest in the content, it is preferable to include not only the effect that a plurality of viewers can view uniformly, but also a special effect for a specific viewer. is there.

However, as described above, in a mode in which a plurality of viewers simultaneously view the character from various viewpoint positions, providing such a special effect may rather reduce the viewer's interest. there were. More specifically, for a viewer who views from a viewpoint position that is not the target of the effect, the effect may be difficult to see, as shown in FIG. 6A, for example. There was a possibility that it would give an unfair impression to and reduce interest in the content.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a content distribution system, a receiving device, and a program that suitably provide a viewer with a highly enjoyable viewing experience.

In order to achieve the above-mentioned object, a content delivery system according to the present invention delivers a content presenting a character arranged in a three-dimensional space, a delivery device for receiving the delivered content, and an experience of viewing the content. And a receiving unit that provides information, the receiving device including a first communication unit that communicates information with the distribution unit, and a setting unit that sets a viewing position for viewing the content in a three-dimensional space. Reproduction means for executing processing relating to reproduction of content, including drawing of a character based on the viewing position set by the setting means, production control means for controlling production of reproduced content, and presentation of reproduced content And a second communication means for communicating information with the receiving device, and the content is a first part for providing an effect that does not define a viewing position as a part for providing an effect. At least a part and a second part that provides an effect for a predetermined viewing position, and the reproduction means reproduces the part that provides the effect in response to a start instruction received from the distribution device. Is performed, and the effect control unit controls the second part to provide an effect for the viewing position set by the setting unit.

In order to achieve the above-mentioned object, another content distribution system according to the present invention distributes a content presenting a character arranged in a three-dimensional space, receives the distributed content, and distributes the content. A content distribution system having a receiving device for providing a viewing experience, wherein the receiving device sets a first communication means for communicating information with the distributing device and a setting means for setting a viewing position for viewing the content in a three-dimensional space. And a reproduction unit that executes a process relating to the reproduction of the content, including a drawing of the character based on the viewing position set by the setting unit, and a presentation unit that presents the reproduced content, and the distribution device includes: And a second communication unit that communicates information with the receiving device and that transmits at least information necessary for presenting the production related to the content, The content includes at least a first part that provides an effect that does not define a viewing position and a second part that provides an effect for a predetermined viewing position as a part that provides the effect, and the reproducing unit includes: In response to the reception of the start instruction from the distribution device, the first communication unit executes the process related to the reproduction of the part providing the effect, and the first communication unit executes the process related to the reproduction including the part providing the effect. Along with this, the viewing position information set by the setting means is transmitted to the distribution device, and the effect control means controls the second part to provide an effect for the viewing position set by the setting means. And

According to the present invention having such a configuration, it is possible to suitably provide a viewer with a viewing experience with a highly entertaining effect.

The figure which showed the structure of the content delivery system which concerns on embodiment of this invention. FIG. 3 is a block diagram showing the functional configurations of the distribution client 100 and the motion capture system 400 according to the embodiment of the present invention. FIG. 3 is a block diagram showing a functional configuration of the receiving client 200 according to the embodiment of the present invention. Block diagram showing the functional configuration of the connection server 300 according to an embodiment of the present invention The figure which illustrated the three-dimensional space which concerns on the content delivered by the content delivery system which concerns on embodiment of this invention. The figure for demonstrating the presentation content in the special effect part of the content delivered by the content delivery system which concerns on embodiment of this invention. The figure for demonstrating the production aspect in the special production part of the content delivered by the content delivery system which concerns on embodiment of this invention. The flowchart which illustrated the production|presentation control process performed with the receiving client 200 which concerns on Embodiment 1 of this invention. The flowchart which illustrated the production|presentation control process performed with the delivery client 100 which concerns on Embodiment 2 of this invention.

[Embodiment 1]
Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims, and all combinations of the features described in the embodiments are not necessarily essential to the invention. Two or more of the plurality of features described in the embodiments may be arbitrarily combined. Further, the same or similar configurations are denoted by the same reference numerals, and duplicated description will be omitted.

It should be noted that one embodiment described below is capable of performing live distribution of binocular stereoscopic content, which presents a main character that is interlocked with the performer's behavior, as an example of a distribution device that constitutes a content distribution system. An example will be described in which the present invention is applied to a distribution client and a reception client configured to be able to view the live-distributed binocular stereoscopic content as an example of a reception device. However, the present invention can be applied to any device group and system capable of providing a content viewing experience configured so that it can be presented to a large number of people for viewing at the same time.

<Structure of content distribution system>
FIG. 1 is a diagram showing a system configuration of a content distribution system according to an embodiment of the present invention. As shown in the figure, the content distribution system is configured such that the distribution client 100 and the connection server 300 can communicate with each other, and the connection server 300 and the plurality of reception clients 200 can communicate with each other via the network 500. The distribution client 100 is also configured to be able to acquire information from the motion capture system 400.

In the content distribution system, the distribution client 100 provides, as a live distribution, a content viewing experience including presentation of a main character and presentation of a voice uttered by the performer, which is linked to the behavior of the performer. The behavior of the performer and the acquisition of the voice are assumed to be performed by the motion capture system 400, and the distribution client 100 acquires the motion data regarding the behavior of the performer and the voice data regarding the voice from the motion capture system 400. ..

In the present embodiment, the performer whose behavior is being measured in the motion capture system 400 will be described as being in charge of both the behavior of the main character relating to the content to be distributed and the voice uttered, but these are different performers. May be done by. Alternatively, it is needless to say that the performer does not need to perform it in real time, but may use motion data or voice data that is preconfigured.

In the present embodiment, the reception client 200 is assumed to be a HMD (Head Mounted Display) type device configured so that the three-dimensional space in which the main character is placed can be viewed with stereoscopic vision, and the content delivered live is It is assumed that the content is binocular stereoscopic content in which the presentation mode changes in association with the head movement of the user (viewing user) who uses the reception client 200. For this reason, in the content distribution system of the present embodiment, in order to improve the processing efficiency and reduce the calculation load of the distribution client 100, the live distribution is not performed by the video streaming method, but the reception client 200 generates the frame and It is performed by a method of distributing information necessary for outputting audio (reproduction of content).

The information required to reproduce the content includes information that requires real-time processing (motion data, audio data, etc. caused by the performer) and other information that is stored in the reception client 200 in advance and used during reproduction. (3D model data, music data, performance data, fixed motion data in real time that is not attributed to the performer, etc.). In the content distribution system of the present embodiment, the latter information is downloaded to the receiving client 200 in advance in response to the viewing user using the receiving client 200 to perform a selection operation or a payment operation of the content desired to be viewed. Shall be stored.

A controller 240 is wirelessly connected to the reception client 200, and an operation input relating to such content selection, an operation of changing a viewing position (a viewpoint position used for drawing) in a three-dimensional space, and other operations by a viewing user. It is assumed that the operation input can be detected. The content provided by the distribution client 100 in the content distribution system of the present embodiment is configured so that various operation inputs can be performed using the controller 240 during the viewing experience. Is a manipulation input device that has a physical button 241 capable of detecting the. However, in the implementation of the present invention, the connection method between the reception client 200 and the controller 240 and the presence or absence of the controller 240 are not limited to the aspect of the present embodiment.

In the present embodiment, the receiving client 200 is described as a so-called eyeglass-type device including a display device (display unit 220) capable of providing a binocular stereoscopic viewing experience. It is not limited to this. That is, the receiving client 200 may be any device configured to execute processing related to the reproduction of the content and output a presentation screen or sound in order to present the viewing experience of the distributed content. The device for presenting these screens and sounds may be a device connected to the outside of the receiving client 200.

The connection server 300 is a device provided to provide stable live distribution of content to a plurality of connected reception clients 200. The connection server 300 is used to realize management of the receiving client 200 being connected, information transmission from the distribution client 100 to the receiving client 200, and management of information transmission from the receiving client 200 to the distribution client 100. Therefore, in the present embodiment, an example will be described in which the connection server 300 is introduced so as not to impose the processing load related to these processes on the distribution client 100, but the present invention is not limited to this. That is, the function performed by the connection server 300 may be performed by the distribution client 100, and in this case, the distribution client 100 and the reception client 200 are configured to directly exchange information. In other words, in the implementation of the present invention, whether or not the information communication between the distribution client 100 and the reception client 200 is performed via a device is not an essential item in the invention.

The network 500 may realize information communication between devices in a public communication network such as the Internet or a local area network (LAN). In this embodiment, the network 500 describes a mode in which information transmission/reception between the distribution client 100 and the connection server 300 and information transmission/reception between the reception client 200 and the connection server 300 are realized through one network 500. However, for example, the distribution client 100 and the connection server 300 are connected by a LAN, the reception client 200 and the connection server 300 are connected by a public communication network, and information communication is performed using different networks 500. May be.

<Functional configuration of distribution client>
Next, the functional configurations of the distribution client 100 and the motion capture system 400 according to the content distribution system of this embodiment will be described using the block diagram of FIG. In the present embodiment, the distribution client 100 will be described as having the configuration shown by the solid line in FIG. 2 and not having the configuration shown by the broken line.

The control unit 101 is, for example, a CPU, and controls the operation of each block included in the distribution client 100. Specifically, the control unit 101 reads out, for example, an operation program of each block recorded in the recording medium 102 or a program of an application (distribution application) that performs live distribution of content, expands it in the memory 103, and executes it. Controls the operation of each block.

The recording medium 102 is a recording device such as a non-volatile memory or an HDD that can hold data permanently. The recording medium 102 stores information such as parameters necessary for the operation of each block, in addition to the operation program of each block of the distribution client 100 and the program of the distribution application. The memory 103 is a storage device used for temporary data storage, such as a volatile memory. The memory 103 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The acquisition unit 104 acquires motion data and audio data related to the main character for the content for which the viewing experience is provided in the content distribution system of this embodiment. As described above, the motion data and voice data related to the main character are acquired by the motion capture system 400 and output to the distribution client 100 using a sensor or a microphone (not shown) worn by the performer. The functional configuration of the motion capture system 400 is briefly shown in FIG. 2, but includes a detection unit 401 that detects the behavior of the performer (change of each part of the performer's body) based on a sensor attached to the performer, and a microphone. The audio acquisition unit 402 for acquiring the audio produced by the performer collected by the above, and the conversion unit 403 for converting this information into a data format usable by the distribution client 100 and the reception client 200. The motion capture system 400 also includes an output unit 404 that outputs, to the distribution client 100, motion data and voice data that are generated by being converted by the conversion unit 403 and that correspond to the behavior of the performer and the uttered voice. In the present embodiment, for the sake of simplicity, the motion capture system 400 is assumed to function as an interface of the distribution client 100 and does not have a unique control device or the like. However, the motion capture system 400 has a control device and the like as an independent device, It goes without saying that the operation may be controlled.

The distribution unit 105 controls live distribution of content for which a viewing experience is provided in the content distribution system of this embodiment. More specifically, the distribution unit 105 appropriately configures and sends information to be transmitted to the reception client 200 connected to the connection server 300 by making a viewing request for the content in a time period during which live distribution is performed. Perform processing. Further, the distribution unit 105 controls to send a start command or the like at a necessary timing for processing that needs to be executed in the reception client 200 for viewing the content.

The communication unit 106 is a communication interface with the external device of the distribution client 100. The communication unit 106 can be connected to an external device via a network 500 (whether wired or wireless) and can transmit/receive data. The communication unit 106 converts, for example, information input as a transmission target into data in a predetermined format, and transmits the data to an external device such as the connection server 300 via the network 500. Further, when the communication unit 106 receives information from an external device via the network 500, for example, the communication unit 106 decodes the information and stores it in the memory 103.

<Functional configuration of receiving client>
Next, the functional configuration of the reception client 200 according to the content distribution system of the present embodiment will be described using the block diagram of FIG. In the description of the functional configuration of the receiving client 200, the configuration that realizes the same function as the configuration of the distribution client 100 is prefixed with “reception” to distinguish it from the configuration of the distribution client 100. Shall be indicated. Here, “reception” added as a prefix indicates that the reception client 200 is simply a component, and is not used for limiting the functions of the blocks to be described. Further, in the present embodiment, the receiving client 200 will be described as having both the configuration shown by the solid line and the configuration shown by the broken line in FIG.

The reception control unit 201 is, for example, a CPU, and controls the operation of each block included in the reception client 200. Specifically, the reception control unit 201 reads, for example, an operation program of each block recorded in the reception recording medium 202 or a program of an application (a viewing application) that provides a viewing experience of live-distributed content, and a reception memory The operation of each block is controlled by expanding it into 203 and executing it.

The reception recording medium 202 is a recording device capable of holding data permanently, such as a non-volatile memory or an HDD. The reception recording medium 202 stores, in addition to the operation program of each block and the viewing application program of the reception client 200, information such as parameters necessary for the operation of each block. The reception memory 203 is a storage device used for temporary data storage, such as a volatile memory. The reception memory 203 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The setting unit 204 sets a viewing position for viewing the corresponding three-dimensional space in viewing the live-delivered content. The viewing position corresponds to a viewpoint position that serves as a reference when an image group for binocular stereoscopic vision is generated by drawing the three-dimensional space. In the present embodiment, as shown in FIG. 5, a viewing user selects a desired viewing position from a plurality of predetermined viewing positions 501a to 501f in a three-dimensional space 510 and sets it. However, the implementation of the present invention is not limited to this. The viewing position does not have to be set according to the selection operation by the viewing user, and may be set and allocated by the setting unit 204, for example, on a first-come-first-served basis or by lottery when viewing the content. Further, in the present embodiment, description will be made assuming that one is selected from the plurality of viewing positions as shown in FIG. 5. However, in carrying out the present invention, the viewing position is, for example, a viewpoint moving operation or a body movement by a viewing user. It may be configured to be changeable to any position according to the operation.

The reproduction control unit 205 uses the information necessary for reproducing the content, including the information received from the distribution client 100 via the connection server 300, and executes the process related to the reproduction of the content. The process related to content reproduction includes generation of a binocular stereoscopic image group for one frame of live distribution and display on the display unit 220, reproduction of corresponding audio data, and output to the audio output unit 230. Including. The display unit 220 is a display device such as an LCD, for example, and the audio output unit 230 is an audio output device such as a speaker or earphone.

Here, the generation of the binocular stereoscopic image group (the image for the left eye and the image for the right eye) is performed by the drawing unit 206. The drawing unit 206 is, for example, a drawing device such as a GPU, and performs a predetermined drawing process to generate a left-eye image and a right-eye image to be displayed on the display unit 220. Specifically, when the viewing application starts providing the content viewing experience in the viewing application, the drawing unit 206 performs necessary calculations on the drawing objects (including the three-dimensional model) of the main character and the mob character to generate motion. By applying it and arranging it in a three-dimensional space, and drawing the three-dimensional space for the viewpoint for the left eye and the viewpoint for the right eye determined according to the set viewing position, Generate an image group.

At this time, the viewpoint for the left eye and the viewpoint for the right eye reflect the movement of the head of the viewing user who wears the receiving client 200, and thus are incorporated in the receiving client 200 such as an acceleration sensor or a gyro sensor. The sensor output from the viewpoint control sensor 210 is reflected. The viewing position set by the setting unit 204 includes a region in the three-dimensional space associated with the viewing position (a region where viewing is performed) and a binocular stereoscopic image group predetermined for the region. The reference viewpoint position and the line-of-sight direction at the time of generation are specified. In order to provide a viewing experience of binocular stereoscopic content with a high immersive feeling, the reference viewpoint position and the line-of-sight direction of the viewing user are set. It is preferable to change it according to the movement of the head. Therefore, after applying the translation and rotation based on the sensor output from the viewpoint control sensor 210 to the reference viewpoint position and the line-of-sight direction, the viewpoint position after application is set as the middle point and the eye-to-eye distances equivalent separation is performed. The two viewpoints having the line-of-sight direction after application are the viewpoint for the left eye and the viewpoint for the right eye.

For example, in a mode in which the display area of the display unit 220 is divided to display left-eye and right-eye images and the images are viewed through a predetermined optical system, stereoscopic vision is realized. 206 generates a final screen by combining the generated binocular stereoscopic image groups in parallel to form one screen. The reproduction control unit 205 realizes the presentation of the binocular stereoscopic content by sequentially displaying, on the display unit 220, the image of the screen corresponding to the frame generated in this way.

The operation detection unit 207 is a user interface included in the reception client 200, such as a power button and a menu button. When the operation detection unit 207 detects that an operation input has been made to the operation member, the operation detection unit 207 outputs a control signal corresponding to the operation input to the reception control unit 201. Further, in the present embodiment, the reception client 200 includes the controller 240 wirelessly connected, and the operation detection unit 207 also detects an operation input made to the physical button 241 which is a physical operation member of the controller 240. To do.

The effect control unit 208 controls the effect related to the content that is live-distributed by the content distribution system of this embodiment. Although details will be described later, the content delivered live includes a part that provides an effect sung by the main character, and the effect control unit 208 mainly performs effect control in the part.

The reception communication unit 209 is a communication interface with the external device of the reception client 200. The reception communication unit 209 can be connected to an external device via a network 500 (whether wired or wireless) to enable data transmission/reception. The reception communication unit 209 converts, for example, the information input as a transmission target into data in a predetermined format, and transmits the data to an external device such as the connection server 300 via the network 500. Further, when receiving information from an external device via the network 500, for example, the reception communication unit 209 decodes the information and stores it in the reception memory 203.

<Functional configuration of connection server>
Next, the functional configuration of the connection server 300 according to the content distribution system of the present embodiment will be described using the block diagram of FIG. In the description of the functional configuration of the connection server 300, a configuration that realizes the same function as the configuration of the distribution client 100 and the reception client 200 has a prefix “connection” in order to distinguish it from the configuration of these devices. It shall be shown with letters. Here, the "connection" added as a prefix indicates merely one configuration of the connection server 300, and is not used for limiting the functions of the blocks to be described.

The connection control unit 301 is, for example, a CPU and controls the operation of each block included in the connection server 300. Specifically, the connection control unit 301 controls the operation of each block, for example, by reading out the operation program of each block recorded in the connection recording medium 302, expanding it in the connection memory 303, and executing it.

The connection recording medium 302 is a recording device capable of holding data permanently, such as a non-volatile memory or an HDD. The connection recording medium 302 stores information such as parameters necessary for the operation of each block, in addition to the operation program of each block included in the connection server 300. The connection memory 303 is a storage device used for temporary data storage, such as a volatile memory. The connection memory 303 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The communication control unit 304 controls to which device the information received from the external device is transmitted via the connection communication unit 305, which will be described later, and management of the receiving client 200 currently viewing the content. In order to provide a content viewing experience in the content distribution system, the necessary processing related to communication is performed. Note that, for the sake of simplicity, in the present embodiment, description will be made assuming that there is only one type of content distributed by the content distribution system, but the present invention is not limited to this, and is provided via the connection server 300. There may be multiple types of content. In this case, a combination of the distribution client 100 and the motion capture system 400 may be provided for each content.

The connection communication unit 305 is a communication interface with an external device included in the connection server 300. The connection communication unit 305 can connect to an external device via a network 500 (whether wired or wireless) and can transmit/receive data. The connection communication unit 305 converts, for example, information input as a transmission target into data in a predetermined format, and transmits the data to an external device such as the distribution client 100 and the reception client 200 via the network 500. Further, when the connection communication unit 305 receives information from an external device via the network 500, for example, the connection communication unit 305 decodes the information and stores it in the connection memory 303.

<Summary of provided contents>
Next, the experience of viewing the content provided by the receiving client 200 in the content distribution system of this embodiment having such a configuration will be specifically described.

In the content described as an example in the present embodiment, as shown in FIG. 5, a main character 502 that is interlocked with a performer's behavior and voice is placed mainly on a stage 503 in a three-dimensional space, and a microphone performance and a singing performance are performed. This is a binocular stereoscopic content that enables the viewing user to view the content. The viewing position 501 assigned to the receiving client 200 is provided on the floor (passenger seat) 504 provided around the stage 503 as shown in the figure, and the viewing user is basically more than the stage 503. The content is configured so that the main character 502 can be viewed from a lower position.

That is, in the three-dimensional space configured for the contents, various drawing objects are arranged in a modeled space such as a so-called live house or concert hall as shown in the figure, and the viewing user can see the stereoscopic image with both eyes. By viewing the three-dimensional space, it is possible to obtain a viewing experience as if one were visiting a concert live in which the main character 502 appears. In the content of the present embodiment, a mob (crowd) character 505 indicating that a concert live is being similarly viewed is arranged on the floor 504 in order to further enhance the realism of the viewing experience and as a lively element of the content. To be done.

The content is divided into an MC part in which the main character 502 performs a microphone performance and a live part in which the main character 502 performs a singing performance. In this embodiment, the motion data and the voice data acquired by the motion capture system 400 are distributed so that the voice and the behavior of the performer in real time are reflected only for the MC part, and the drawing is performed by the receiving client 200. It will be described as one used for control. On the other hand, the live part includes motion data and pre-configured motion data that are highly entertaining, such as those that are difficult to achieve only by the behavior actually performed by the performer, such as effects that float in the air and effects that move momentarily. It will be described as being provided by applying voice data. That is, the live part in the content of the present embodiment is not used in the processing related to the reproduction of the motion data and the audio data obtained in real time by the motion capture system 400 for the performer, and the content is watched before the content or before the live part. It is configured such that it can be presented using motion data and voice data stored in advance in the reception memory 203 or the like. Here, the motion data and the audio data used for the processing related to the reproduction of the live part may be configured by processing what the actor's behavior and audio are actually measured and recorded in advance.

As described above, in the present embodiment, in order to simplify the explanation, the behavior and the voice itself of the actually measured performer are reflected or presented, or the presenter is configured not based on the actual measurement, or by the actual measurement of the performer. The content is divided into an MC part and a live part, and the latter is a part for providing the effect according to the present invention, from the viewpoint of reflecting the behavior or the sound constituted by processing the obtained one and presenting the same. It will be described as a thing.

Further, the MC part and the live part may alternately appear a plurality of times during the period of providing the content viewing experience. In the content distribution system according to the present embodiment, in the MC part, in response to the fact that the motion data and the audio data actually measured by the performer are received from the distribution client 100, the reproduction control unit 205 performs the processing related to the reproduction. However, with regard to the live part, it is possible to complete the processing relating to the reproduction in the receiving client 200 using the data stored in advance, without the need to acquire the actual measurement information relating to the performer from the distribution client 100. .. On the other hand, in providing a live distribution for viewing by a large number of people, it is preferable that the progress statuses are synchronized between the receiving clients 200 and the viewing user's sense of unity is secured. At the time of switching, it is assumed that the distribution client 100 gives an instruction to start each live part to each reception client 200 via the connection server 300.

In the present embodiment, the live part does not use the motion data and audio data acquired by the motion capture system 400 for the performer in real time. However, the present invention is not limited to this. It's easy to understand. For example, the motion data applied in the live part may be stored in advance, and the voice data may be acquired in real time. In this case, for example, the state control of the lip portion of the model of the main character 502 may be performed based on the voice data.

<Direction of live part>
Further, the live part of the content for which the viewing experience is provided in the system of the present embodiment includes at least two types of effect parts in order to raise the interest of the viewing user for the content.

The first is a normal production part as the first part according to the present invention, and among the productions using the main character 502 during a singing performance, a production in which a preferable viewing position is not particularly defined, in other words, This is a part that provides a performance in which fairness is ensured among viewing users even when viewed from the viewing position. In the present embodiment, the normal effect part provides an effect in which the main character 502 performs a performance on the entire floor 504 on the stage 503. For example, the main character 502 places an effect such that the main character 502 is placed near the center of the stage 503 that is easy to view from any viewing position 501, or the position on the floor 504 that is easy to view from each viewing position 501. Includes effects that move evenly.

The second is a special effect part as the second part according to the present invention. Among the effects using the main character 502 during the singing performance, an effect in which a preferable viewing position is determined, in other words, other It will be a part that provides effects that can provide an unfavorable viewing experience from the viewing position. The special effect part, unlike the normal effect part, provides an effect in which the main character 502 limits the target to a specific viewing position 501 on the stage 503 or on the floor 504 and performs. For example, the main character 502 includes an effect in which the main character 502 moves to a position on the stage 503 that is closest to the target viewing position 501 and performs a performance toward the viewing position 501. In addition, for example, the main character 502 moves from the stage 503 to the target viewing position 501 on the floor 504, and the performance is performed while approaching the viewing position 501 direction.

By the way, if the viewing user who has set the viewing at the preferred viewing position 501, the performance provided by such a special performance part is displayed to the viewing user as shown in FIG. 6B. Although it is possible to obtain a viewing experience as if a performance is being performed only, the viewing user who has set to view at another viewing position 501, as shown in FIG. You may not be able to get the experience. In the content distribution system of the present embodiment, in such a special effect part, the effect control unit 208 performs effect control so that there is no difference in the viewing experience between the viewing users due to the difference in the viewing position that has been set.

Here, in the special effect part, as shown in FIG. 7A, the main character 502 appears at the position 702 in the area 701 corresponding to the viewing position 501a on the floor 504, and the performance while approaching the viewing position 501a. Consider an example in which it is determined that a performance for performing is provided. At this time, for example, the viewing user who has set the viewing position 501c for viewing cannot enjoy the effect, and may have an unfair impression in the content viewing experience.

Therefore, the effect control unit 208 of the receiving client 200, when such a special effect part is started, the viewing position (viewing position 501c) set in the receiving client 200 is a suitable target for providing the effect. It is determined whether or not it matches the viewing position (viewing position 501a). Then, if this is different, the effect control unit 208 performs control to provide the above-mentioned effect originally provided for the viewing position 501a to the viewing position 501c. Specifically, as shown in FIG. 7B, the effect control unit 208 changes the arrangement position of the main character 502 to the position 704 in the area 703 corresponding to the viewing position 501c, and then, to FIG. The control is performed so as to apply the motion data that provides the same effect as that shown (the effect of performing a performance while approaching the viewing position 501c).

By doing so, in the content distribution system of the present embodiment, when the special effect part is reached, a unified viewing experience is provided to each viewing user viewing at a different viewing position. Can be In other words, in the content distribution system of the present embodiment, in the special effect part, the content of the effect provided by each of the receiving clients 200 that are currently viewing the content is the same, but according to the set viewing position. As a result, a situation may occur in which a three-dimensional space having a different distribution of drawing objects is a drawing target.

It should be noted that, in the present embodiment, since the live part uses motion data and audio data stored in advance in the receiving memory 203 or the like to perform a process related to reproduction and provide a content viewing experience, a special effect is provided. All motion data related to the part is also stored in advance. That is, it is assumed that the motion data related to the special effect part is provided in advance for each of all the viewing positions 501 and stored in the memory 103 or the like as preliminary data. When the special effect part is reached, the effect control unit 208 responds to the viewing position set for viewing if it is not appropriate to provide the effect related to the viewing position originally defined for the part. The motion data is read out from the preliminary data and is controlled so as to substitute for the process related to the reproduction.

<Direction control processing>
Hereinafter, with regard to the effect control process executed in the receiving client 200 of the content distribution system of the present embodiment and performed while providing the experience of viewing the content of the live part, a specific process will be described with reference to the flowchart of FIG. 8. .. The process corresponding to the flowchart is realized by the reception control unit 201 by reading out a corresponding process program of the viewing application stored in the reception recording medium 202, loading it into the reception memory 203, and executing it. be able to. It should be noted that the present production control process will be described as being started, for example, when a live part start request transmitted from the distribution client 100 is received. Here, as for the content viewing experience, the receiving client 200 executes the viewing application during a predetermined period provided immediately before the live distribution by the distribution application, or the target content. It is assumed that the viewing request is provided during the live distribution period, provided that the viewing request is transmitted from the receiving client 200 to the connection server 300. At this time, the communication control unit 304 adds the identification information (and destination information) of the receiving client 200 that has made the viewing request to the list of the receiving clients 200 that are viewing the corresponding content.

In step S<b>801, the reception control unit 201 determines whether the content providing the viewing experience has reached the sequence of providing the effect related to the special effect part. The information on the section of the normal effect part and the special effect part in the content viewing related to the live part is included in the timeline information designed in advance for the live part, and the reception control unit 201 refers to the timeline information. Then, the determination in this step may be performed. When the reception control unit 201 determines that the sequence for providing the effect related to the special effect part has been reached, the process proceeds to S802, and if not, that is, if the sequence for providing the effect related to the normal effect part is determined. Moves the process to S806.

In step S802, the setting unit 204 performs state control so as not to accept an operation input related to the viewing position change. As described above, the effect related to the special effect part is controlled to be provided to the currently set viewing position (set viewing position) in the present embodiment. That is, if it is possible to change the setting viewing position in the middle of such an effect, the arrangement position of the main character 502 is changed along with the change of the viewing position, and as a result, continuity of the effects between frames is secured. Instead, it may hinder a good viewing experience. Therefore, in the effect control process of the present embodiment, at least in the sequence for providing the effect related to the special effect part, control is performed so that the set viewing position is not changed.

In S803, the effect control unit 208, under the control of the reception control unit 201, determines whether or not the set viewing position matches the viewing position of the provided target (target viewing position) defined for the current special effect part. Rendering control unit 208 moves the process to S804 if it determines that the set viewing position matches the target viewing position, and moves the process to S805 if it determines that the set viewing position does not match.

In S804, the effect control unit 208 transmits information indicating motion data and audio data that are predetermined for the current special effect part to the reproduction control unit 205 as information necessary for presenting the effect. The information necessary for presenting the effect thus transmitted is used for drawing processing by the drawing unit 206 and audio output to the audio output unit 230 in the process related to the reproduction of the presented content.

On the other hand, if it is determined in S803 that the set viewing position does not match the target viewing position, the effect control unit 208 determines in S805 the motion configured for the set viewing position in the preliminary data provided for the current special effect part. Information indicating the data and the voice data is transmitted to the reproduction control unit 205 as information necessary for presenting the effect. That is, in the example in which the information necessary for presenting the effect selected in S804 indicates the effect mode as shown in FIG. 7A, the information necessary for presenting the effect selected in this step is This corresponds to the one that shows the effect mode as shown in FIG. 7B (the setting viewing position is the viewing position 501c).

In addition, when it is determined in S801 that the sequence is to provide the effect related to the normal effect part, the effect control unit 208 determines in S806 the information indicating the motion data and the audio data that are predetermined for the current normal effect part, The information is transmitted to the reproduction control unit 205 as information necessary for presenting the effect. In the case of a sequence for providing an effect related to the normal effect part, the setting unit 204 performs a state control so as to be able to receive an operation input related to the viewing position change.

In step S807, the reception control unit 201 determines whether the live part has ended. The determination as to whether or not the live part has ended may be performed, for example, by referring to the timeline information related to the live part and based on whether or not all the prescribed presentation of presentation has been completed. However, it may be performed based on the presence/absence of reception of information related to the start of the MC part, which is transmitted from the distribution client 100 after the end of a predetermined period for presenting the live part. If the reception control unit 201 determines that the live part has ended, the production control process is completed, and if it determines that the live part has not ended, the process returns to S801.

By doing so, according to the content distribution system of the present embodiment, it is possible to suitably provide the viewer with a viewing experience with a highly entertaining effect. More specifically, the presentation experience can be included in the viewing experience as if the performance was being performed individually for the viewing user, and such presentation can be provided in a suitable manner at any viewing position. By controlling the performance as much as possible, it is possible to provide a highly entertaining viewing experience.

[Modification]
In the above-described embodiment, regarding the effect specified for the special effect part, the information necessary for presenting the effect for all the viewing positions is provided in advance so that the effect can be similarly provided at all the viewing positions. Although described as being provided, the implementation of the present invention is not limited to this.

For example, the information necessary for presenting the effect related to the special effect part is provided only for the appreciation position of the provision target (target appreciation position) defined for the special effect part, and the currently set appreciation position (setting When the viewing position) and the target viewing position are different, the effect control unit 208 changes the motion data provided for the target viewing position so as to provide the same effect for the set viewing position. You may make it produce|generate the information required for presentation of an effect.

[Embodiment 2]
In the above-described first embodiment and the modified example, the information necessary for presenting the production related to the live part is stored in advance in the reception client 200, and the production control unit 208 selectively selects suitable information according to the set viewing position. It has been described as being read out. However, the implementation of the present invention is not limited to this, and the information necessary for presenting the production related to the live part, or at least the presentation of the production related to the special production part dynamically changed according to the set viewing position. The information necessary for the above may be provided to each receiving client 200 from the distribution client 100 via the connection server 300, similarly to the MC part.

In the present embodiment, a mode in which effect control is performed on the distribution client 100 side in this way will be described. In the following description, the same components as those of the content distribution system according to the first embodiment described above are designated by the same reference numerals, and the description thereof will be omitted.

<<Functional configuration of distribution client>>
First, the functional configurations of the distribution client 100 and the motion capture system 400 according to the content distribution system of this embodiment will be described with reference to the block diagram of FIG. In the present embodiment, unlike the first embodiment, the distribution client 100 will be described as having both the configuration shown by the solid line and the configuration shown by the broken line in FIG.

The effect control unit 110 controls the effect related to the content delivered live by the content delivery system of the present embodiment. As described above, the live-delivered content includes a live part, and the effect control unit 110 performs effect control when the content related to the part is being viewed.

<<Functional configuration of receiving client>>
Unlike the first embodiment, the reception client 200 of the present embodiment has only the configuration shown by the solid line in FIG. 3 and does not have the configuration shown by the broken line.

Therefore, the content distribution system according to the present embodiment is different from the content distribution system according to the first embodiment in that the configuration for performing the effect control is provided not at the receiving client 200 but at the distribution client 100.

<Direction control processing>
Hereinafter, with regard to the effect control processing executed in the distribution client 100 of the content distribution system of the present embodiment and performed while providing the viewing experience of the live part content, specific processing will be described with reference to the flowchart of FIG. 9. .. The process corresponding to the flowchart can be realized by the control unit 101 reading out a corresponding process program of the distribution application stored in the recording medium 102, loading the program in the memory 103, and executing the program. .. In the production control process, for example, the distribution client 100 determines to shift from the MC part to the live part, and the distribution client 100 issues an instruction regarding the start of the live part to the reception client 200 currently providing the content. Description will be given assuming that it is started when it is transmitted to the destination.

Further, in the content distribution system of the present embodiment, from the receiving client 200 performing content viewing, the viewing position information set by the setting unit 204 in the receiving client 200 is associated with the device ID for identifying the receiving client 200. And is transmitted to the distribution client 100. The information is transmitted at the time of starting the content provision and when the setting viewing position is changed, and is stored in the memory 103 of the distribution client 100, for example, about the reception client 200 currently providing the content. It shall be stored in the location list.

In step S<b>901, the control unit 101 determines whether the content providing the viewing experience has reached the sequence of providing the effect related to the special effect part. The information on the section of the normal effect part and the special effect part in the content viewing related to the live part is included in the timeline information designed in advance for the live part, and the control unit 101 refers to the timeline information. The determination in this step may be made. If the control unit 101 determines that the sequence for providing the effect related to the special effect part has been reached, the control unit 101 shifts the processing to S902, and if not, that is, if it is the sequence for providing the effect related to the normal effect part, The process moves to S904.

In step S902, the control unit 101 sends an instruction to invalidate the operation input related to the viewing position change to the receiving client 200 currently viewing the content. When the receiving client 200 receives the instruction, the setting unit 204 may perform state control so as not to accept an operation input for changing the viewing position.

In S903, the effect control unit 110, under the control of the control unit 101, for each of the receiving clients 200 included in the viewing position list, the set viewing position is the viewing position of the provision target (target viewing position) determined for the current special effect part. Position)). Then, for the reception client 200 that has determined that the set viewing position matches the target viewing position, the performance control unit 110 transmits, to the communication unit 106, information indicating predetermined motion data and audio data for the current special performance part. , Is transmitted to the receiving client 200 as the information necessary for presenting the effect. For the reception client 200 that the setting control unit 110 has determined that the set viewing position does not match the target viewing position, the motion data configured for the setting viewing position among the preliminary data provided for the current special effect part. And information indicating voice data are transmitted to the communication unit 106, and are transmitted to the receiving client 200 as information necessary for presenting the effect.

Note that, in the present embodiment, for simplification of description, it is assumed that motion data and audio data necessary for presenting a live part effect are prepared in advance and stored in the recording medium 102, as in the first embodiment. As described, the implementation of the present invention is not limited to this. That is, as in the present embodiment, even in the sequence related to the live part, as long as the information necessary for presentation of the effect is transmitted from the distribution client 100 to each reception client 200, like the MC part, Motion data and audio data obtained based on actual measurement of the performer by the motion capture system 400 may be used. In this case, regarding the special effect part, the behavior performed by the performer is configured as motion data for one viewing position (target viewing position) defined for the part, and is provided to different setting viewing positions. The motion data based on the real-time behavior may be changed.

Further, when it is determined in S901 that the sequence is to provide an effect related to the normal effect part, the effect control unit 110 communicates information indicating motion data and voice data that are predetermined for the current normal effect part in S904. The information is transmitted to the unit 106, and is transmitted to the receiving client 200 currently viewing the content as information necessary for presenting the effect. In the case of the sequence for providing the effect related to the normal effect part, since the operation input related to the viewing position change may be accepted, the operation related to the viewing position change is addressed to the receiving client 200 currently viewing the content. An instruction to validate the operation input may be sent. When the receiving client 200 receives the instruction, the setting unit 204 may perform the state control so as to receive the operation input regarding the viewing position change.

In step S905, the control unit 101 determines whether the live part has ended. The determination as to whether or not the live part has ended may be performed, for example, by referring to the timeline information related to the live part and based on whether or not all the prescribed presentation of presentation has been completed. .. When it is determined that the live part is finished, the control unit 101 completes the main production control process, and when it is determined that the live part is not finished, the process is returned to S901.

By doing so, the effect control unit 110 of the distribution client 100 can be the main constituent so that each receiving client 200 can be provided with a suitable content viewing experience.

[Other Embodiments]
The present invention is not limited to the above-mentioned embodiments, and various modifications and variations can be made within the scope of the gist of the present invention. The content distribution system according to the present invention can also be realized by a program that causes one or more computers to function as a distribution device and a reception device of the system. The program can be provided/distributed by being recorded in a computer-readable recording medium or through an electric communication line.

100: distribution client, 101: control unit, 102: recording medium, 103: memory, 104: acquisition unit, 105: distribution unit, 106: communication unit, 110: performance control unit, 200: reception client, 201: reception control unit , 202: reception recording medium, 203: reception memory, 204: setting unit, 205: reproduction control unit, 206: drawing unit, 207: operation detection unit, 208: effect control unit, 209: reception communication unit, 210: viewpoint control Sensor, 220: display unit, 230: audio output unit, 240: controller, 241: physical button, 300: connection server, 301: connection control unit, 302: connection recording medium, 303: connection memory, 304: communication control unit , 305: connection communication unit, 400: motion capture system, 401: detection unit, 402: voice acquisition unit, 403: conversion unit, 404: output unit, 500: network

In order to achieve the above-mentioned object, a content distribution system according to the present invention is a distribution device that distributes content that presents characters arranged in a three-dimensional space, and a distribution experience that receives the distributed content. And a receiver for providing a content distribution system, the receiver including first communication means for communicating information with the distributor, setting means for setting a viewing position for viewing the content in a three-dimensional space, Reproducing means for executing processing relating to content reproduction, including drawing of a character based on the viewing position set by the setting means, effect control means for controlling effect of the reproduced content, and presenting the reproduced content. And a second communication means for communicating information with the receiving device, and the content is a first part that provides an effect that does not define a viewing position as a part that provides an effect. At least a part and a second part that provides an effect for a predetermined viewing position, and the reproduction means reproduces the part that provides the effect in response to the reception of the start instruction from the distribution device. The effect control unit controls the second part to provide an effect for the viewing position set by the setting unit, and the viewing position set by the setting unit is set to a plurality of preset viewing positions. Is selected from among the viewing positions, the information related to the second part is provided with information necessary for presenting the rendering for each of the plurality of viewing positions, and the reproduction means is the second. In the part (1), the processing relating to the reproduction of the content is executed based on the information necessary for presenting the effect provided for the viewing position set by the setting means .

In order to achieve the above-mentioned object, another content distribution system according to the present invention distributes a content presenting a character arranged in a three-dimensional space, receives the distributed content, and distributes the content. A content distribution system having a receiving device for providing a viewing experience, wherein the receiving device sets a first communication means for communicating information with the distributing device and a setting means for setting a viewing position for viewing the content in a three-dimensional space. And a reproduction unit that executes a process relating to the reproduction of the content, including a drawing of the character based on the viewing position set by the setting unit, and a presentation unit that presents the reproduced content, and the distribution device includes: And a second communication unit that communicates information with the receiving device and that transmits at least information necessary for presenting the production related to the content, The content includes at least a first part that provides an effect that does not define a viewing position and a second part that provides an effect for a predetermined viewing position as a part that provides the effect, and the reproducing unit includes: In response to the reception of the start instruction from the distribution device, the first communication unit executes the process related to the reproduction of the part providing the effect, and the first communication unit executes the process related to the reproduction including the part providing the effect. Along with this, the information on the viewing position set by the setting means is transmitted to the distribution device, and the effect control means controls the second part so as to provide the effect for the viewing position set by the setting means . The effect relating to the part is provided with information necessary for presenting the effect for one predetermined viewing position, and the effect control means sets the viewing position set by the predetermined viewing position and the setting means. When the position is different, the information necessary for presenting the effect related to the second part is changed so that the effect corresponds to the viewing position set by the setting means .

Claims (11)

What is claimed is: 1. A content distribution system, comprising: a distribution device that distributes content that presents characters arranged in a three-dimensional space; and a reception device that receives the distributed content and provides a viewing experience of the content.
The receiving device,
First communication means for communicating information with the distribution device;
Setting means for setting a viewing position for viewing the three-dimensional space for the content;
Reproduction means for executing processing relating to reproduction of the content, including drawing of the character based on the viewing position set by the setting means,
Production control means for controlling the production of the content to be reproduced,
Presentation means for presenting the reproduced content;
Have
The distribution device has a second communication means for communicating information with the reception device,
The content includes at least a first part that provides an effect that does not define a viewing position as a part that provides an effect, and a second part that provides an effect for a predetermined viewing position,
The reproduction means executes processing relating to the reproduction of the part providing the effect, in response to receiving a start instruction from the distribution device,
The content distribution system, wherein the effect control unit controls the second part to provide an effect for the viewing position set by the setting unit. What is claimed is: 1. A content distribution system, comprising: a distribution device that distributes content that presents characters arranged in a three-dimensional space; and a reception device that receives the distributed content and provides a viewing experience of the content.
The receiving device,
First communication means for communicating information with the distribution device;
Setting means for setting a viewing position for viewing the three-dimensional space for the content;
Reproduction means for executing processing relating to reproduction of the content, including drawing of the character based on the viewing position set by the setting means,
Presentation means for presenting the reproduced content;
Have
The delivery device is
Production control means for controlling the production of the content reproduced in the receiving device;
Second communication means for communicating information with the receiving device, and transmitting at least information necessary for presenting an effect related to the content;
Have
The content includes at least a first part that provides an effect that does not define a viewing position as a part that provides an effect, and a second part that provides an effect for a predetermined viewing position,
The reproduction means executes processing relating to the reproduction of the part providing the effect, in response to receiving a start instruction from the distribution device,
The first communication unit transmits the information of the viewing position set by the setting unit to the distribution device in accordance with the execution of the process related to the reproduction including the part that provides the effect,
The content distribution system, wherein the effect control unit controls the second part to provide an effect for the viewing position set by the setting unit. The viewing position set by the setting means is selected from a plurality of viewing positions provided in advance,
The production related to the second part is provided with information necessary for presenting the production for each of the plurality of viewing positions,
In the second part, the reproduction means executes processing relating to reproduction of the content based on information necessary for presenting the effect provided for the viewing position set by the setting means. The content distribution system according to claim 1 or 2. The production related to the second part is provided with information necessary for presenting the production for one of the predetermined viewing positions.
When the predetermined viewing position and the viewing position set by the setting unit are different from each other, the rendering control unit creates a rendering for the viewing position set by the setting unit when the viewing position is set by the setting unit. The content distribution system according to claim 1 or 2, wherein information necessary for presenting the effect according to is changed. The information necessary for presenting the effect includes information on behavior control of the character,
When the predetermined viewing position and the viewing position set by the setting unit are different from each other, the effect control unit changes the behavior control information of the character so that the viewing set by the setting unit is changed. The content distribution system according to claim 4, wherein the information necessary for presenting an effect for a position is configured. The content distribution system according to any one of claims 1 to 5, wherein the setting unit does not change the viewing position at least while the effect related to the second part is presented. The effect related to the first part is an effect in which the presentation mode of the character is different according to the viewing position set by the setting unit,
7. The effect according to the second part is an effect in which the presentation mode of the character is unified regardless of the viewing position set by the setting unit. Content distribution system described in. The content distribution system according to any one of claims 1 to 7, wherein the content is binocular stereoscopic content. A receiving device for providing a viewing experience of contents presented by a character arranged in a three-dimensional space, which is distributed by a distribution device,
Communication means for communicating information with the distribution device;
Setting means for setting a viewing position for viewing the three-dimensional space for the content;
Reproduction means for executing processing relating to reproduction of the content, including drawing of the character based on the viewing position set by the setting means,
Production control means for controlling the production of the content to be reproduced,
Presentation means for presenting the reproduced content;
Have
The content includes at least a first part that provides an effect that does not define a viewing position as a part that provides an effect, and a second part that provides an effect for a predetermined viewing position,
The reproduction means executes processing relating to the reproduction of the part providing the effect, in response to receiving a start instruction from the distribution device,
The reception device, wherein the effect control unit controls the second part to provide an effect for the viewing position set by the setting unit. A receiving device for providing a viewing experience of contents presented by a character arranged in a three-dimensional space, which is distributed by a distribution device,
Communication means for communicating information with the distribution device;
Setting means for setting a viewing position for viewing the three-dimensional space for the content;
Reproduction means for executing processing relating to reproduction of the content, including drawing of the character based on the viewing position set by the setting means,
Presentation means for presenting the reproduced content;
Have
The content includes at least a first part that provides an effect that does not define a viewing position as a part that provides an effect, and a second part that provides an effect for a predetermined viewing position,
The reproduction means executes processing relating to the reproduction of the part providing the effect, in response to receiving a start instruction from the distribution device,
The communication means is
With the execution of the process related to the reproduction including the part that provides the effect, the viewing position information set by the setting unit is transmitted to the distribution device,
With respect to the second part, information necessary for rendering at the viewing position set by the setting means is received,
The receiving device, wherein the reproducing means executes a process relating to the reproduction in the second part, based on the received information necessary for the effect. A program for causing a computer to function as each unit of the receiving device according to claim 9.

Images