JP7398655B2 - Computer program, server device, terminal device, and method - Google Patents

Description

The technology disclosed in this application relates to a computer program, a server device, a terminal device, and a method.

In recent years, technology using avatars played by broadcasters has been developed. In this type of technology, viewers can view an avatar played by a distributor via a distribution service.

Additionally, in relation to services using avatars, a service called "Custom Cast" is known as a service that uses technology to control the facial expressions and actions of avatar objects based on the actions of the performer, etc. Non-patent document 1). With this service, performers can pre-assign one of a number of prepared facial expressions or movements to each of multiple flick directions on the smartphone screen, and select the desired expression or movement. By flicking the screen of the smartphone along the corresponding direction, the performer can cause the avatar object displayed in the video to express its facial expression or action.

"Custom Cast", [online], Custom Cast Inc., [searched on December 10, 2019], Internet (URL: https://customcast.jp/)

However, in the technique disclosed in Non-Patent Document 1, the performer has to flick the screen of the smartphone while speaking, making it difficult for the performer to perform the flick operation.

Note that the above Non-Patent Document 1 is incorporated herein by reference in its entirety.

Some embodiments disclosed in this application provide a computer program, a server device, a terminal device, and a method that can more appropriately express an avatar.

A computer program according to one aspect acquires information regarding a gift of an object provided from a viewer to a distributor by being executed by one or more processors, and the information regarding the acquired gift is It is determined whether or not a predetermined condition is satisfied using the information related to the gift, and if it is determined that the information related to the acquired gift satisfies the predetermined condition, the facial expression or pose related to the distributor is Information for changing the facial expression or pose of the avatar related to the broadcaster, which has been generated based on the above, to a predetermined facial expression or pose corresponding to the predetermined condition.

In one aspect of the computer program, the predetermined condition includes a condition that is satisfied at a timing when the avatar wears the gift.

In one aspect of the computer program, the predetermined facial expression is a facial expression with the same motif as the attached gift.

In one aspect of the computer program, the part where the predetermined facial expression is activated is the same part as the part to which the gift is attached.

In one aspect of the computer program, the part where the predetermined facial expression is activated is different from the part where the gift is attached.

In the computer program according to one embodiment, the predetermined condition includes that a position related to the avatar and a position related to the gift are within a predetermined distance.

In the computer program according to one aspect, the position related to the avatar is any one of a position of the avatar in a virtual space, a part that activates the predetermined facial expression, a predetermined part of the avatar, or a collider of the avatar. is, is something.

In the computer program according to one embodiment, the predetermined condition uses priority related to the gift.

In the computer program according to one aspect, the priority regarding the gift is determined based on the price of the gift, the displayed size of the gift, the level of the gift, and/or information regarding the provider of the gift. It is something that can be done.

In the computer program according to one embodiment, the predetermined condition is to use a type to which the avatar belongs, which is one of a plurality of types to which the avatar belongs.

In the computer program according to one aspect, the predetermined facial expression is obtained by using predetermined facial expression parameters or facial expression parameters that are based on a broadcaster's facial expression and are corrected based on a predetermined rule. It is the expression or thing that is displayed.

In one aspect of the computer program, the predetermined facial expression changes in conjunction with the direction in which the gift is displayed.

In the computer program according to one aspect, the information related to the gift includes information on the start of display of the gift, information on a change in display of the gift, information that the gift has as an attribute, and/or information on end of display of the gift. , including.

In the computer program according to one embodiment, the information related to the gift includes information related to the first gift and information related to the second gift, and the predetermined condition includes the information related to the first gift and the information related to the second gift. and information regarding the combination, order, and/or time between the first gift and the second gift.

In the computer program according to one aspect, the information for making the predetermined facial expression or pose is generated without acquiring information for operating the avatar from the distributor terminal.

In one aspect of the computer program, information related to the gift is acquired from the viewer terminal.

In one aspect of the computer program, the one or more processors are included in a viewer terminal.

In the computer program according to one aspect, the one or more processors are provided in a distributor terminal,
The generated information is used to set the facial expression or pose of the avatar.

In the computer program according to one aspect, the one or more processors are provided in a distributor terminal,
The generated information is used to set the facial expression or pose of the avatar to generate an animation.

In one aspect of the computer program, the one or more processors are provided in a server, and use the generated information to set the facial expression or pose of the avatar.

In the computer program according to one aspect, the one or more processors are provided in a server and use the generated information to set the facial expression or pose of the avatar and generate a video. .

In the computer program according to one embodiment, the plurality of processors are included in some or all of a server, a viewer terminal, and a distributor terminal.

In one aspect of the computer program product, the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU).

A server device according to one embodiment includes one or more processors, and the one or more processors execute computer-readable instructions to generate objects that are provided from a viewer to a distributor. information related to the gift is acquired, it is determined whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and the information related to the acquired gift satisfies the predetermined condition. When it is determined that the condition is satisfied, the expression or pose of the avatar associated with the distributor, which has been generated based on the expression or pose associated with the distributor, is changed to a predetermined expression or pose corresponding to the predetermined condition. It is something that generates information.

In the server device according to one embodiment, the predetermined condition includes a condition that is satisfied at a timing when the avatar wears the gift.

A method according to one embodiment is a method executed by one or more processors that execute computer-readable instructions, and the processor executes the instructions to send a message from a viewer to a broadcaster. Information related to the gift of the provided object is acquired, it is determined whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and the information related to the acquired gift is When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar related to the broadcaster that has been generated based on the facial expression or pose related to the broadcaster is changed to a predetermined facial expression corresponding to the predetermined condition. Or, it generates information for posing.

In the method according to one aspect, the predetermined condition includes a condition that is satisfied at the timing when the avatar wears the gift.

FIG. 1 is a block diagram showing an example of the configuration of a system according to an embodiment. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the server device 20 (terminal device 30) shown in FIG. FIG. 3 is a block diagram schematically showing an example of the functions of the system according to one embodiment. FIG. 4 is an example of data possessed by the system according to one embodiment. FIG. 5 is an example of data possessed by the system according to one embodiment. FIG. 6 is an example of data possessed by the system according to one embodiment. FIG. 7 is an example of a flow processed by the system according to one embodiment. FIG. 8 is an example of a flow processed by the system according to one embodiment. FIG. 9 is an example of a flow processed by the system according to one embodiment.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that components depicted in one drawing may be omitted from another drawing for convenience of explanation. Furthermore, it should be noted that the accompanying drawings, while disclosing one embodiment of the present invention, are not necessarily drawn to scale. In addition, in the present document, the object indicated by the term application may be something called software or a program, as long as it is an instruction to a computer and is combined so as to obtain a result.

1. System Example FIG. 1 is a block diagram showing an example of a system configuration according to an embodiment. As shown in FIG. 1, the system 1 may include one or more server devices 20 connected to the communication network 10 and one or more terminal devices 30 connected to the communication network 10. Note that in FIG. 1, three server devices 20A to 20C are illustrated as examples of the server device 20, and three terminal devices 30A to 30C are illustrated as an example of the terminal device 30. , one or more server devices 20 other than these may be connected to the communication network 10, and one or more terminal devices 30 other than these may be connected to the communication network 10 as the terminal device 30. Note that in the present application documents, the term "system" may include both a server and a terminal device, and may also be used to indicate only a server or only a terminal device. That is, the system may be a server only, a terminal device only, or both a server and a terminal device.

Further, the system may be an information processing device on a cloud. Further, the system constitutes a virtual information processing device, and may be logically configured as one information processing device. Additionally, the system owner and administrator may be different.

The communication network 10 may be, but is not limited to, a mobile phone network, a wireless LAN, a fixed telephone network, the Internet, an intranet, an Ethernet (registered trademark), and/or a combination thereof. In the communication network 10, various data disclosed in the present document may be communicated.

The server device 20 executes the installed specific application to acquire information regarding the object gift provided from the viewer to the distributor,
Determining whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and when it is determined that the information related to the acquired gift satisfies the predetermined condition, An operation of generating information for changing the facial expression or pose of the avatar related to the broadcaster, which has been generated based on the facial expression or pose related to the broadcaster, to a predetermined facial expression or pose corresponding to the predetermined condition. It is good to be able to execute the following.

Alternatively, the terminal device 30 acquires information regarding the object gift provided from the viewer to the broadcaster by executing an installed web browser or a specific application,
Determining whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and when it is determined that the information related to the acquired gift satisfies the predetermined condition, An operation of generating information for changing the facial expression or pose of the avatar related to the broadcaster, which has been generated based on the facial expression or pose related to the broadcaster, to a predetermined facial expression or pose corresponding to the predetermined condition. It is good to be able to execute the following.

The terminal device 30 is any terminal device capable of performing such operations, and may be, but is not limited to, a smartphone, a tablet, a mobile phone (feature phone), a personal computer, or the like.

2. Hardware Configuration of Each Device Next, an example of the hardware configuration of the server device 20 will be described.

2-1. Hardware Configuration of Server Device 20 An example of the hardware configuration of the server device 20 will be described with reference to FIG. 2. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the server device 20 (terminal device 30) shown in FIG. (This is written in relation to each terminal device 30.)

As shown in FIG. 2, the server device 20 can mainly include a calculation device 21, a main storage device 22, and an input/output interface device 23. Server device 20 can further include an input device 24 and an auxiliary output device 26. These devices may be connected by a data bus and/or a control bus.

The arithmetic unit 21 performs arithmetic operations using instructions and data stored in the main memory device 22 and stores the results of the arithmetic operations in the main memory device 22 . Furthermore, the arithmetic device 21 can control the input device 24, the auxiliary storage device 25, the output device 26, etc. via the input/output interface device 23. The server device 20 may include one or more computing devices 21. Computing device 21 may include one or more central processing units (CPUs), microprocessors, and/or graphics processing units (GPUs).

The main storage device 22 has a storage function, and stores commands and data received from the input device 24, the auxiliary storage device 25, the communication network 10, etc. (server device 20, etc.) via the input/output interface device 23, and It stores the calculation results of the calculation device 21. The main storage device 22 can include, but is not limited to, RAM (random access memory), ROM (read only memory), and/or flash memory.

The auxiliary storage device 25 is a storage device. Instructions and data (computer programs) constituting the above-mentioned specific applications, web browsers, etc. may be stored, and these instructions and data (computer programs) may be stored in the input/output interface device 23 by being controlled by the arithmetic unit 21. may be loaded into the main storage device 22 via the . The auxiliary storage device 25 may be a magnetic disk device, an optical disk device, a file server, etc., but is not limited to these.

The input device 24 is a device that takes in data from the outside, and may be a touch panel, a button, a keyboard, a mouse, a sensor, or the like.

The output device 26 may include, but is not limited to, a display device, a touch panel, a printer device, and the like. Further, the input device 24 and the output device 26 may be integrated.

In such a hardware configuration, the arithmetic unit 21 sequentially loads instructions and data (computer program) constituting a specific application stored in the auxiliary storage device 25 into the main storage device 22, and loads the loaded instructions into the main storage device 22. and data, the output device 26 is controlled via the input/output interface device 23, or other devices (for example, the server device 20 and It may be possible to transmit and receive various information with other terminal devices 30, etc.).

The server device 20 has such a configuration, and by executing a specific installed application, acquires information related to a gift of an object provided from a viewer to a distributor, defined below, and performs the acquisition. It is determined whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and if it is determined that the information related to the acquired gift satisfies the predetermined condition, the distribution the operation of generating information for changing the facial expression or pose of the avatar related to the broadcaster, which has been generated based on the facial expression or pose related to the broadcaster, to a predetermined facial expression or pose corresponding to the predetermined condition, etc. (including various operations detailed below). Further, such operations may be performed by the user giving instructions to the system of the example of the invention disclosed in the present document using the input device 24 or the input device 34 related to the terminal device 30 described later. Further, when the program is executed on the computing device 21, it may be displayed on the output device 26 of the server device 20 as a system used by the user, or on the output device 26 of the terminal device 30 as the system used by the user. 36 may be displayed.

2-2. Hardware Configuration of Terminal Device 30 An example of the hardware configuration of the terminal device 30 will be described with reference to FIG. 2 as well. As the hardware configuration of each terminal device 30, it is possible to use, for example, the same hardware configuration as that of each server device 20 described above. Therefore, reference numerals for components included in each terminal device 30 are shown in parentheses in FIG. 2.

As shown in FIG. 2, each terminal device 30 mainly includes a calculation device 31, a main storage device 32, an input/output interface device 33, an input device 34, an auxiliary storage device 35, and an output device 36. , can be included. These devices are connected to each other by a data bus and/or a control bus.

The arithmetic device 31, the main storage device 32, the input/output interface device 33, the input device 34, the auxiliary storage device 35, and the output device 36 are the arithmetic device 21 and the main storage device 22 included in each of the above-mentioned server devices 20, respectively. , the input/output interface device 23, the input device 24, the auxiliary storage device 25, and the output device 26, or they may be substantially the same or different. Furthermore, the capacities and capabilities of the computing devices and storage devices may be different. In particular, input device 34 and output device 36 will be described in detail below.

In such a hardware configuration, the arithmetic unit 31 sequentially loads instructions and data (computer program) constituting a specific application stored in the auxiliary storage device 35 into the main storage device 32, and loads the loaded instructions into the main storage device 32. and data to control the output device 36 via the input/output interface device 33, or to control the output device 36 via the input/output interface device 33 and the communication network 10 to other devices (for example, each server device 20). etc.), various information can be sent and received between them.

The terminal device 30 is provided with such a configuration, and the position of at least one gift is moved by a transparent object in a virtual space between a subject and a camera that images the subject, and the position of at least one gift is moved by the camera. It may be possible to perform some or all of the following operations (including various operations described in detail later), such as increasing the display area of the subject to be imaged. Furthermore, by executing the installed web browser, it is possible to receive and display web pages from the server device 20 and perform similar operations. Further, such operations may be performed by the user using the input device 34 to give instructions to the system of the example of the invention disclosed in the present document. Furthermore, when the program is being executed on the arithmetic device 31, it may be configured to be displayed on the output device 36 of the terminal device 30 as a system used by the user.

In the present document, there may be two types of terminal devices: a "broadcaster terminal" and a "viewer terminal." These may be names given to the role of the terminal, and one terminal device may be a viewer terminal, a distributor terminal, or a viewer terminal and a distributor at the same time. The terminal may be a personal terminal. The distributor terminal may be a terminal used by a distributor who appears as a subject such as an avatar in the distributed content. On the other hand, the viewer terminal may be a terminal mainly used by a viewer who views what is being distributed. Furthermore, viewers can also exert a certain amount of influence on what is being broadcast, such as by giving a gift to a subject such as an avatar, so that the gift appears in the broadcast. Good too. Similarly, viewers may have some influence on what is delivered in the form of audio, comments, and the like. The content to be distributed may be a video, a video, or an animation, as long as the subject, such as an avatar played by the broadcaster, appears in a part of it. . Note that the above description is merely an example to which one embodiment of the present invention is applied, and each embodiment of the present invention may be applied even in a virtual space where a subject appears without such a premise.

As described above, the viewer terminal may be configured to allow voice input, comment input, gifting, and the like. For example, the user terminal may include, as the input device 34, a keyboard, a touch panel, etc., which can input comments and can acquire comments as text information. Further, the user terminal may have a function of generating certain comments using the computing device 34. These obtained comments may similarly be transmitted to the communication network 10 via the input/output interface device 23.

Note that the comments according to the present invention may be automatically generated by the system, in addition to comments input by the user terminal. Such comments may be transmitted to the user terminal and displayed on the user terminal.

Further, the terminal may include a microphone capable of acquiring sound as the input device 34. Note that the human voice may be acquired by a microphone as the input device 34 and treated as sound, or it can be acquired by the microphone as the input device 34 and converted into text by voice recognition by the arithmetic device 31 and then treated as a comment. You can handle it. Note that the user terminal may generate sound using the arithmetic device 31. These captured sounds or comments may similarly be transmitted to the communication network 10 via the input/output interface device 23.

On the other hand, the distributor terminal may include, as the input device 34, a motion sensor capable of detecting human movement. The motion sensor may have a function of being attached to the body of the broadcaster and capable of detecting movements of various parts of the body. Various parts of the body include the head, face, upper arm, forearm, hand, thigh, lower leg, foot, trunk, etc.; It may be something that can be attached to a position where it can be detected. Such a sensor may be able to track the broadcaster's facial expressions and poses and generate information. The motion sensor may include an acceleration sensor, a gyro sensor, a geomagnetic sensor, and the like. These acquired information may similarly be transmitted to the communication network 10 via the input/output interface device 23.

Further, the distributor terminal may include, as the input device 34, a camera capable of capturing moving images. The camera may image the head, face, eyes, part of the body, whole body, etc. of the broadcaster. When a camera images the head or face, it is possible to detect the direction of the face, when the camera images the face, it is possible to detect facial expressions, and when the camera images the eyes, it is possible to detect the line of sight. If a part of the body or the whole body is to be imaged, it may be possible to detect the movement of the body together with or in place of the above-mentioned motion sensor. Information acquired regarding the facial direction, line of sight, facial expression, and body movement based on these imaging information may similarly be transmitted to the communication network 10 via the input/output interface device 23.

Further, the output device 36 in the viewer terminal and the distributor terminal may be a display device or a sound generating device. The display device may be any type of device as long as it has a display function, such as a liquid crystal display, plasma display, organic EL display, micro LED display, head-mounted display, etc., but is not limited to these. It may be a device. The sound generating device may be a speaker, headphones, earphones, etc. The information output by the output device 36 may be information generated by the arithmetic device 31 or may be information received from the communication network 10.

3. Functions of Each Device Next, an example of the functions each of the server device 20 and the terminal device 30 have will be described with reference to FIG. 3. FIG. 3 is a block diagram schematically showing an example of the functions of the system shown in FIG. As shown in FIG. 3, an example system may include an acquisition unit that acquires information that can be used to generate an object, and a generation unit that generates an object using the acquired information.

2.1. Acquisition Unit The acquisition unit has a function of acquiring information that can be used to determine whether or not a predetermined condition, which will be described later, is applicable.

The information acquired by the acquisition unit may be information related to one or more gifts of objects provided from viewers to distributors, and/or information related to avatars.

The information related to the gift may include priority related to the gift, location related to the gift, attributes of the gift, information specifying the avatar to whom the gift was provided, and/or information related to multiple gifts.

Furthermore, as described above, the information related to the avatar may include the position related to the avatar, the type of avatar, facial expression parameters related to the avatar, actions related to the avatar, and/or situations related to the avatar. Here, if the avatar to whom the gift was provided is specified, the information related to the avatar may be information related to the avatar specified by the gift. On the other hand, if the avatar to whom the gift is provided is not specified, it may be all one or more avatars in the virtual space, or one of the avatars appearing in the virtual space. Or it may be multiple avatars.

The specific contents of the gift-related information and avatar-related information will be explained in the condition application section.

Furthermore, when a gift is provided to a specific avatar, the acquisition unit may acquire information that identifies the avatar to whom the gift was provided.

Note that the information processing device from which the acquisition unit acquires information may be the same as or different from the information processing device having the function of the acquisition unit. Furthermore, the term "acquisition" may be a general concept of receiving information from another information processing device, or may be a general concept of reading information from a storage device, and may also be a general concept of receiving information from another information processing device. It may be a general concept of receiving information from a program via a storage device.

2.2. Condition Application Unit The condition application unit determines whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift, and determines whether the information related to the acquired gift satisfies the predetermined condition. When it is determined that the condition is satisfied, the expression or pose of the avatar associated with the distributor, which has been generated based on the expression or pose associated with the distributor, is changed to a predetermined expression or pose corresponding to the predetermined condition. It has the function of generating information.

Here, if information is generated for changing the facial expression or pose of the avatar related to the broadcaster, which has been generated based on the facial expression or pose related to the broadcaster, to a predetermined facial expression or pose corresponding to the predetermined condition. is for changing the facial expression of the avatar related to the broadcaster that has been generated based on the facial expression related to the broadcaster to a predetermined facial expression corresponding to the predetermined condition when the predetermined condition is satisfied. Only information may be generated, or only information for changing the pose of the avatar related to the streamer that has been generated based on the pose related to the streamer to a predetermined pose corresponding to the predetermined condition. or to change the facial expressions and poses of avatars related to the broadcaster, which have been generated based on the facial expressions and poses related to the broadcaster, to predetermined facial expressions and poses corresponding to the predetermined conditions. information may be generated. In addition, in the application documents, a predetermined facial expression and/or pose that is specified in response to a predetermined condition is sometimes referred to as a predetermined facial expression, etc., and a facial expression and/or pose is sometimes referred to as a facial expression, etc. .

Further, the predetermined facial expression, etc. is a facial expression displayed using predetermined facial expression parameters, or facial expression parameters based on the broadcaster's facial expression that are corrected based on predetermined rules. It may be a facial expression displayed based on a predetermined shape or movement setting, or a predetermined pose parameter or a pose parameter based on the pose of the broadcaster according to a predetermined rule. The pose may be displayed using pose parameters corrected based on the pose parameters, or the pose may be displayed using predetermined shape or movement settings. Here, the predetermined expression parameters refer to a plurality of parameters that influence the movement of each part of the face, and by setting each of the plurality of parameters to predetermined values, one expression is indicated. It may be. The one or more parameters may be parameters that affect the movement of each part of the face, and each parameter may be related to a specific part of the face or may be related to the entire face (or part of the face). However, the shape of the entire face (or part of the face) may differ depending on the parameter. The latter is a technique in which a plurality of objects with the same area but different shapes are prepared and a single facial expression is expressed by combining some or more of them. An example of the latter is brand shape, in which a plurality of different overall face shapes are prepared in advance and a single facial expression is generated by combining the plurality of shapes. . Note that in the present technology, in addition to normal facial elements, a special facial expression may be created by preparing elements of a predetermined shape in advance. For example, if you prepare in advance a shape that shows the vertical lines of the face, you can create the vertical lines of the face, and if you prepare the elements of the eyes with crosses in advance, you can It becomes possible to produce a cross. In this case, since the parameters are set in advance, the facial expression, etc. that will be reproduced will be the same regardless of the broadcaster's facial expression, etc., so if the appropriate parameter indicating the facial expression, etc. is selected, the facial expression, etc. that is easy to understand will always be reproduced. It has the advantage of being reproducible. Note that even in this case, the predetermined facial expressions etc. may be set to be different depending on the avatar. That is, the predetermined facial expressions and the like that are applied when the same predetermined conditions are satisfied for avatars A and avatars B may be different. In this case, it may be set for each avatar or may be set depending on the type of avatar.

On the other hand, if the predetermined facial expression, etc. is a facial expression, etc. that has been corrected based on a predetermined rule based on the facial expression, etc., based on the broadcaster's facial expression, etc., then the predetermined facial expression, etc. It may be a facial expression based on the expression. For example, let's say that the distributor makes a surprised expression in response to some kind of gift. At this time, in response to the detection of the broadcaster's facial expression, the avatar's facial expression is set to an expression corresponding to surprise using one or more parameters, but multiple parameters related to surprise are set as a predetermined expression corresponding to a predetermined condition. A case is shown in which the expression of surprise is further emphasized by increasing or decreasing the value of by a predetermined value. While the above technology is based on absolute values in that one or more parameters are determined as a predetermined facial expression, this technology is based on relative values to information obtained from the current broadcaster's facial expression. It may be possible to use various parameters. Therefore, even if the same predetermined conditions are applied to the same avatar played by the same broadcaster, the parameters such as facial expressions based on the facial expressions, etc. that the broadcaster himself/herself has at that time may be the same. If there is, there is a possibility that the expression may be different, and therefore, a facial expression or the like corresponding to such potentially different parameters may become a predetermined facial expression or the like. Therefore, it may be possible to provide more variations in facial expressions, etc.

Furthermore, the facial expression displayed by setting a predetermined shape or movement may be a facial expression that uses a predetermined shape itself, such as a mask, as a predetermined facial expression. .

Note that although the above description has been made regarding predetermined facial expressions, the corresponding technique may be similarly used for predetermined poses.

Further, the predetermined expression or the like may be an expression or the like related to the gift to be provided. In this case, there is an advantage that the viewer can easily understand the reason why a predetermined facial expression or the like is expressed in association with the gift.

The related expression etc. may be, for example, an expression with the same motif as the gift. For example, if you wear a dog panel with a face showing on your face, your mouth will become a dog's mouth within a predetermined time, or if you put on cat ears, your mouth will become a cat's mouth within a predetermined time. The facial expressions with the same motif may include facial expressions that make the given gift more effective by making a predetermined facial expression or the like in response to the gift. Note that if the facial expression has the same motif as the gift, the avatar's facial expression may change to the predetermined facial expression within a predetermined time after the gift is worn. That is, the predetermined condition may include a condition that is satisfied at the timing when the avatar wears the gift. Note that the above-mentioned predetermined time period may be simultaneous or almost simultaneous, or may change to a predetermined facial expression as the gift is worn. Further, "almost simultaneously" may include a time difference such that from the viewer's perspective, it is felt that the facial expression has been changed to a predetermined expression at the same timing as when the gift is being put on.

In this case, the predetermined condition may be a rule that a given gift has a predetermined facial expression with the same motif as a predetermined gift. The condition application unit may use the predetermined rule to apply the rule to the given gift and specify a predetermined facial expression or the like. Note that this rule may be applied to each gift, or may be applied to the type of gift to which each gift belongs. Although the description will be omitted hereafter, information specifying the corresponding facial expression etc. may be generated by similarly using predetermined rules for the following various conditions.

Further, the related facial expression may be, for example, a predetermined and harmonious facial expression when a gift is given. Predetermined facial expressions, etc. may include facial expressions that occur in situations such as anime and manga. For example, regarding a gift of meat, a facial expression such as drooling may appear within a specified time when the gift of meat approaches the mouth, or an expression of drooling within a specified period of time when the gift of meat is attempted to be picked up. etc., or the person may start drooling as soon as the meat gift appears in front of them. The predetermined time may be the same as described above.

On the other hand, the predetermined facial expression or the like may be a facial expression that is completely opposite to the motif of the gift being provided. In this case, the viewer may feel a sense of discomfort, but this sense of discomfort has the advantage of providing new stimulation.

Further, the predetermined facial expression etc. may act on the parameters of all parts (parts) that make up the facial expression etc., or may act on some of the parameters. For example, the predetermined facial expression or the like may affect only the parameters regarding the mouth, and the parameters regarding parts other than the mouth may be parameters linked to the broadcaster's facial expression.

Further, the part where the predetermined facial expression or the like is activated may be the same part as the part where the gift is attached. For example, for a gift of a balloon that is attached to the mouth, the expression may be such that the mouth is pursed. In this case, since the expression of the mouth is appropriate for inflating the balloon, there is an advantage that the viewer does not feel uncomfortable about the gift of the balloon.

Further, the part where the predetermined facial expression or the like is activated may be different from the part where the gift is attached. For example, regarding a gift of a hot spring towel, at the same time when the gift is worn by placing the hot spring towel on the head, the eyes, which are a different part from the head, may have an expression such as the eyes becoming horizontal lines. Further, regarding a gift of a ring to be worn on a finger, as the ring is worn, the eyes, which are a different part from the finger, may have an expression such as the eyes becoming hearts. In addition, regarding the Pocky gift, when the Pocky is placed in the mouth in the state of eating, the eyes and cheeks, which are different parts from the mouth, may have an expression such as closing the eyes or blushing the cheeks.

Further, the predetermined expression or the like may change in conjunction with the direction in which the gift is displayed. The distributor may not know where the gift will appear in the virtual space. At this time, even if a gift is provided, the distributor may not be aware of it, and as a result, the avatar associated with the distributor may not be able to react to the gift. Therefore, by changing the facial expression of the avatar in conjunction with the direction in which the gift is displayed, the viewer who gave the gift understands that the avatar has responded, and this has the advantage of making communication more active. Here, there may be various ways of setting a predetermined facial expression that changes in conjunction with the direction in which the gift is displayed. Further, the number of parts within the predetermined expression etc. that changes in conjunction with the direction in which the gift is displayed may be one or more. For example, the predetermined facial expression etc. may include eyes that look in the direction in which the gift is displayed, may include a manner in which the mouth or eyebrows change with respect to the direction in which the gift is displayed, or may include a manner in which the mouth or eyebrows change in relation to the direction in which the gift is You can turn your face in the direction where is displayed, or you can pose with your palms in front of your eyebrows to look into the distance. As another example, when a gift of meat is provided, the predetermined facial expression may be a facial expression in which the person's mouth is mumbled toward the display side of the gift of meat. Furthermore, when a tower gift is provided, the user may face the direction in which the tower gift is displayed and look in surprise as the predetermined expression. Note that in such a case, the information regarding the position of the gift may be the rotation information of the gift.

The predetermined pose may be various poses. The pose may be of only a part of the avatar's body or may be of the entire avatar's body. For example, it may be a pose of only the head area such as a pose of tilting the head, a pose of only the upper body such as a fist pump, or a pose of the entire body such as a pose of holding a gun.

In addition, if the information related to the gift includes information that specifies the avatar to whom the gift was provided, the avatar that makes a predetermined facial expression etc. when the condition is applied is not processed for the specified avatar. It's fine.

The predetermined conditions may include information related to gifts and/or information related to avatars.

As mentioned above, information related to a gift includes gift attributes, gift priority, gift location, gift display start, gift display change, gift display end, and/or information related to multiple gifts. may include.

As described above, the information related to the avatar may include the location related to the avatar, the type of avatar, facial expression parameters related to the avatar, actions related to the avatar, and/or situations related to the avatar.

Priority regarding gifts is defined as the priority of gifts when multiple gifts are provided within a predetermined time and when a predetermined facial expression etc. is applied using some of the gifts. It may be used to set a predetermined facial expression or the like based on some gifts. In this case, information related to one gift or information related to a plurality of gifts may be used as some of the gifts. By setting priorities regarding gifts, there is an advantage that gifts provided within a predetermined time can be appropriately processed.

For example, if five gifts are provided within a predetermined time, predetermined conditions may be applied to information related to only the gift with the highest priority among the five gifts. However, predetermined conditions may be applied to information related to only the gift with the highest priority and other gifts with the next highest priority, or other gifts whose priorities are within a predetermined order. Predetermined conditions may be applied to information regarding a gift. In this case, the priority related to the gift may function as a precondition for application of the predetermined condition in the sense of selecting an object to which the predetermined condition is applied.

In addition, when the priority level related to gifts is not used as the predetermined condition, the predetermined condition may be applied to gifts for which information related to gifts was acquired in the order of time when information related to gifts was acquired. Here, the acquired time order may be the time order of the acquisition of information related to the gift that was finally acquired depending on the congestion level of the network, the progress of information processing, etc., or the time order of acquisition of information related to the gift that was finally acquired, or The order may be in the chronological order in which the gift provision instructions were acquired. For the latter, for example, the chronological order may be determined by associating the instructions with time information in the chronological order in which the instructions were acquired at the terminal or server. In this case, when information related to the next gift is obtained, the settings for the predetermined facial expressions, etc. based on the information related to the previous gift are replaced with the predetermined facial expressions, etc. based on the information related to the next gift. Information such as facial expressions may be deleted, or the previous predetermined information such as facial expressions may be continued and the predetermined facial expressions etc. for the information regarding the next gift may not be set. In the former case, there is an advantage that a predetermined facial expression or the like corresponding to the given gift can be displayed on the avatar. On the other hand, in the latter case, even if there are predetermined facial expressions that are not set, there is an advantage that the predetermined facial expressions that have been set can be continuously displayed so that the viewer can understand them. In addition, in such a case, the predetermined condition is determined by using information regarding the time between the first gift and the second gift, which is the time when the first gift was provided and the time interval between when the second gift was provided. You may specify facial expressions, etc.

The priority of a gift may include the price of the gift, the displayed size of the gift, the level of the gift, the location where the gift appears in the virtual space, the type of gift, information about the gift provider, and/or the gift's display size. It may be generated using the provided timing, etc.

Regarding gift price (points consumed), gift display size, and gift level, gifts with larger price, size, and number of levels have higher priority than gifts with smaller price, size, and number of levels. The degree may be set. In this case, if a gift with a large price, size, and number of levels is provided, it may have a function of influencing and activating a predetermined facial expression. Alternatively, the setting may be reversed.

As for the appearance location of the gift in the virtual space, for example, a gift that appears in the center of the virtual space may be given a higher priority than a gift that appears at the left and right edges of the virtual space. If a gift that appears in a conspicuous place in the virtual space is given a high priority, the viewer can see the gift that has appeared in a conspicuous place, so the viewer can understand the meaning of a predetermined facial expression, etc. It has the advantage of being easier to understand.

Furthermore, the priority regarding a gift may be static information that is preset for the gift, or may be dynamic information that is set for the first time when the gift is provided. The former may include, for example, the price of the gift, the displayed size of the gift, the type of gift, etc. The latter may include the appearance location of the gift in the virtual space, the gift giver, the timing at which the gift was given, and the like. Furthermore, the price of the gift may be dynamically set information such as in an auction format. When the priorities related to gifts are dynamically set, there is an advantage that gifts to which predetermined facial expressions and the like are set can be selected more appropriately and dynamically.

The type of gift may be the specific content of the gift, and may be of various types. Regarding the timing at which gifts are provided, a gift that is provided first may be given a higher priority than a gift that is provided later.

Further, as an example of using information related to the gift provider as the priority related to the gift, it is conceivable to set the priority related to the gift differently depending on the provider of the gift. For example, if a viewer has a particular status in a given stream, gifts from those who do not have that status may be given higher priority than gifts from those who do not have that status. It's okay. Such status may be based on whether the viewer is a specific member of the distribution, or based on a specific preferential status. In addition, the frequency of viewing the distribution and the total viewing time of the distribution for the gift giver, the frequency of viewing the distribution and the total viewing time of the avatar to whom the gift is provided, and the gift being provided. It is set based on the viewing frequency and total viewing time of the broadcaster's distribution related to avatars, the number of gifts provided in the currently viewed distribution, the total amount of gifts, and/or the frequency of gifts, etc. It can be anything. In addition, if there is information related to the support ranking between the gift giver and the avatar's distributor, or if the gift giver is a mutual follower of the avatar's distributor, we will respond based on such information. You can set the priority to The information related to the support ranking may include, for example, information on the ranking calculated from information related to the viewer's activities. Information related to viewer activities includes, for example, the total number of gifts provided by viewers, the total price of gifts, the total value of gift levels, evaluation values using information related to gifts, and comments from viewers. It may include the number of likes, the number of likes by viewers, or information using these.

Regarding the delivery of a specific broadcaster's avatar, if it is set that the more enthusiastic the viewer is, the higher the possibility that a predetermined expression, etc. can be applied to the gift provided by the viewer, the viewer's enthusiasm level will be improved. There is a possibility that it will work.

Note that the above-mentioned priority regarding gifts may be used to select some gifts from among multiple gifts as part of a predetermined condition, or a predetermined facial expression, etc. corresponding to the priority may be used. It may be used to select some gifts from a plurality of gifts, and may be used to specify a predetermined facial expression, etc. that corresponds to such priority. It's okay. For example, a predetermined expression, etc. that is applied when information related to a gift whose level is 5 is acquired, and a predetermined expression, etc. that is applied when information related to the same gift whose level is 3 is acquired. may be different. For example, for a certain gift, if the gift is level 3, the predetermined facial expression etc. is set as a predetermined facial expression that specifies only the mouth and mustache, whereas if the gift is level 5, the predetermined facial expression etc. is set. may be set with a predetermined facial expression that defines eyebrows in addition to the mouth and mustache. In the latter case, the predetermined facial expressions, etc. that specify the appearance of the mustache, eyebrows, and eyes are set (there are no settings for the mouth). It may be a facial expression including facial expressions. In addition, as another aspect, for example, for a certain gift, if the gift is level 3, a predetermined expression etc. that defines only the mouth and mustache is set, whereas if the gift is level 5 For example, the predetermined facial expression, etc. in the case of a gift with a high level is a predetermined facial expression, etc. that does not specify the mouth, but specifies the mustache and eyebrows. It is not necessary to include a predetermined facial expression etc. when the level is low. Like the facial expressions corresponding to levels 3 and 5 of the gifts mentioned above, the predetermined conditions include graded expressions, and the corresponding predetermined facial expressions, etc. are also graded. Each predetermined facial expression etc. may be different.

Furthermore, the predetermined conditions may use both information related to gifts and information related to avatars.

For example, the predetermined condition may include that a position related to the avatar and a position related to the gift are within a predetermined distance. Here, the position related to the gift may be a preset position on the gift, a position related to the outer circle or polygon when the gift is displayed, or a position related to the center of gravity when the gift is displayed. It can also be the location. The position related to the outer circle may be a position specified using the center of the outer circle or a tangent to the outer circle. Further, the position related to the polygon may be a position specified using the center of the polygon, the apex or side of the polygon, or the like. The position related to the avatar may be any one of a position of the avatar in a virtual space, a part that activates the predetermined facial expression, a predetermined part of the avatar, or a collider of the avatar.

By using the fact that the position related to the avatar and the position related to the gift are within a predetermined distance under a predetermined condition, the viewer can understand the meaning of a predetermined facial expression, etc. It has the advantage of being easy to understand.

In addition, contrary to the above, it may include that the position related to the avatar and the position related to the gift are outside the range of a predetermined distance under a predetermined condition. In particular, the position related to the avatar and the position related to the gift may be separated by a predetermined distance within a predetermined time. In this case, a situation is shown in which the gift and the avatar move relative to each other in the direction of separation, but for example, an expression such as tears coming out of the eyes may be set as the predetermined expression of the avatar.

Furthermore, the type to which the avatar belongs may be used as information related to the avatar. For example, the predetermined condition may be the type to which the avatar belongs, which is one of a plurality of types to which the avatar belongs. This may be such that when information related to a certain gift is acquired, the first avatar type has a predetermined facial expression, but the second avatar type does not have a predetermined facial expression. For example, a certain type of avatar may be set to have a timid personality, so that when a ghost gift is provided, the avatar is more likely to show a frightened expression. Further, a certain type of avatar may be set to have a cheerful personality, and the conditions for a certain type of avatar to have a facial expression including a smile as a predetermined facial expression etc. may be set to be less strict than for other avatars. In this case, the avatar may be provided with personality characteristics such as timid, cheerful, bullish, competitive, etc., and the satisfaction of the predetermined condition may be determined using such personality characteristics. Note that the personality may be set according to the type to which the avatar belongs, or may be set for each avatar separately from the type to which the avatar belongs.

Note that the predetermined condition may include probability. That is, the predetermined condition may include a pseudo-random number function or the like, and when the function value satisfies the predetermined condition, a predetermined facial expression or the like may be generated. When such a probability is included, a predetermined facial expression or the like is not always generated, so it is possible to continuously attract the viewer's interest. Further, for example, if the probability value is set lower than a predetermined value, and if the expression is a rare expression, it is possible to attract the viewer's interest in that the viewer can see a valuable expression.

Further, the information related to the avatar may include the actions of the avatar. For example, if information regarding a gift of a handgun and an avatar's action of shooting a gun are made, the avatar may display a brave face aiming a handgun. Note that in such a case, the facial expression may differ depending on the type of gun. For example, a bazooka, a sniper rifle, a handgun, and a machine gun may have the same or different facial expressions in some combinations.

Further, the information related to the avatar may include the situation related to the avatar. For example, if the avatar falls into a specific hole and is unable to get out, and a gift of a rope ladder is provided, the predetermined expression may be set as a predetermined expression showing a big smile. .

In addition, as a predetermined condition, the timing of the display of each gift, such as information on the start of gift display, information on gift display change, and/or information on the end of gift display, is used to check that the predetermined condition is satisfied. You may judge. In this case, the expression and pose of the avatar change in response to the display of the gift, such as the start, change, and end of the display of the gift. This has the advantage of making it easier to understand by relating it to

Furthermore, information that the gift has as an attribute may be used as the predetermined condition. The gift attributes may or may not include characteristics of the gift, such as the type of gift and the level of the gift. Also, depending on the specific characteristics of the gift, predetermined conditions may be set so that the predetermined conditions are satisfied, and a corresponding predetermined facial expression or the like may be set.

The type of gift may include, for example, information on when the gift is provided, information on the classification of the gift, and the part on which the gift is worn. The timing information on which a gift is provided may be a newly released gift, a gift within a predetermined period of time after release, a gift corresponding to a predetermined seasonal event such as Halloween, etc. Information on gift classification may include free gifts, accessories, support goods, and the like. Further, the part to which the gift is attached may be the eyes, mouth, nose, forehead, ears, hands, feet, etc., or a combination thereof.

The level of a gift may be increased by combining it with other gifts of the same or different nature. Even if they are the same gift, if you obtain a first level gift and a second level gift, the higher level gift will have a more luxurious predetermined expression, etc. good. For example, if the first predetermined facial expression etc. is more gorgeous than the second predetermined facial expression etc., for example, if there are many parameters set as the predetermined facial expression etc., it may be influenced by the predetermined facial expression etc. There are many parts, etc.

Further, the information related to the gift includes information related to the first gift and information related to the second gift, and the predetermined condition is the information related to the first gift and the information related to the second gift. and/or information regarding the time of the first gift and the second gift.

For example, when an avatar is given a predetermined combination of gifts, the avatar's expression may be a predetermined expression. More specifically, for example, if an avatar is provided with three types of gifts in an RPG: a hero's shield, a hero's sword, and a hero's cloak, the avatar's facial expression will be the hero's brave face. May be set. Note that the predetermined expression etc. set in such a case may be suggested from the combination of gifts given. For example, if you have a frog shield, a frog case, or a frog cloak, you can have a frog-like face.

Further, a predetermined condition may be set to be satisfied only when information related to gifts is acquired in a predetermined order. For example, if you are given a headband as a gift, your eyes will look as if they are on fire, and if you are given sunglasses as a gift, your eyes will be shiny even if you are wearing sunglasses. To the extent that the situation can be seen, an expression may be set in which the eyes are brighter than the bright eyes when only the headband is worn. Such a facial expression with big eyes flashing may not be applied when a headband is given after sunglasses, or the facial expression may be set to be the same.

The condition application unit may set the facial expression specified by satisfaction of a predetermined condition as the facial expression of the avatar to whom the gift was provided. More specifically, the information indicating the facial expression specified by satisfying a predetermined condition may be an ID indicating the specified facial expression, or one or more parameters of the facial expression indicating the specified facial expression, and The avatar's facial expression may be set using information indicating the specified facial expression so that the specified facial expression can be used as the facial expression of the avatar. After the information on the ID and parameters is specified or generated in the condition application section, the information may be transmitted to a server or terminal to generate a video or animation, and may be used for a predetermined period of time. Here, the condition application unit may store information indicating the specified facial expression in association with each predetermined condition, and when each of the predetermined conditions is satisfied, information indicating the corresponding specified facial expression. may be configured so that it can be specified.

In such processing, if the first information processing device on which the condition application section is executed and the second information processing device that performs the processing including setting the facial expression of the avatar are the same, the condition application section may Processing may be performed to set the facial expression of the avatar using the facial expression specified by satisfying the condition.

In addition, if the first information processing device on which the condition application unit is executed and the second information processing device that performs the process including setting the facial expression of the avatar are different, the condition application unit may perform the processing specified by the fulfillment of a predetermined condition. The first information processing device may transmit the facial expression to a second information processing device, and the second information processing device may process the facial expression to set it as the facial expression of the avatar using the specified facial expression.

Here, if the first information processing device is a server and the second information processing device is a server, the specified facial expressions, etc. are sent from the server of the first information processing device to the server of the second information processing device. Information indicating the information may be transmitted. Further, if the first information processing device is a server and the second information processing device is a viewer terminal, such specified information is sent from the server of the first information processing device to the viewer terminal of the second information processing device. Information indicating a facial expression, etc. may be transmitted. Further, if the first information processing device is a server and the second information processing device is a distributor terminal, such specified information is sent from the server of the first information processing device to the distributor terminal of the second information processing device. Information indicating a facial expression, etc. may be transmitted.

Similarly, if the first information processing device is a viewer terminal and the second information processing device is a server, the specified information is sent from the viewer terminal of the first information processing device to the server of the second information processing device. Information indicating the facial expression and the like may be transmitted. In addition, if the first information processing device is a viewer terminal and the second information processing device is a viewer terminal, the viewer terminal of the first information processing device may be connected to the viewer terminal of the second information processing device. , information indicating such specified facial expressions, etc. may be transmitted. In addition, if the first information processing device is a viewer terminal and the second information processing device is a distributor terminal, the viewer terminal of the first information processing device is connected to the distributor terminal of the second information processing device. , information indicating such specified facial expressions, etc. may be transmitted.

Similarly, if the first information processing device is a distributor terminal and the second information processing device is a server, the specific information is sent from the distributor terminal of the first information processing device to the server of the second information processing device. Information indicating the facial expression and the like may be transmitted. If the first information processing device is a distributor terminal and the second information processing device is a distributor terminal, the following information is transmitted from the distributor terminal of the first information processing device to the distributor terminal of the second information processing device. Information indicating the specified facial expression etc. may be transmitted. In addition, if the first information processing device is a distributor terminal and the second information processing device is a viewer terminal, from the distributor terminal of the first information processing device to the viewer terminal of the second information processing device. , information indicating such specified facial expressions, etc. may be transmitted.

Although the condition application unit is executed in a single information processing device in the above example, the condition application unit may be distributed and processed in a plurality of information processing devices.

Furthermore, the predetermined conditions may include timing at which a predetermined facial expression or the like is set. Under the various conditions described above, a predetermined facial expression, etc. is set depending on the timing when the position related to the avatar and the position related to the gift are within a predetermined distance, and information including information such as the predetermined facial expression, etc. The information may be transmitted to and displayed on the information processing device to be displayed. The timing shift in this case may include a time lag in communication and processing.

On the other hand, the timing at which a predetermined facial expression or the like is set after a predetermined condition is satisfied may be set separately. For example, after a predetermined condition is set, a predetermined facial expression or the like may be displayed two seconds later. For example, a predetermined expression of surprise may be set at an intentionally delayed timing of two seconds after a situation in which the avatar is surprised occurs.

5 and 6 are examples of rules with predetermined conditions. By using only the information related to the gift and/or the information related to the avatar as a predetermined condition, a corresponding predetermined facial expression etc. can be specified.

As mentioned above, the function of the condition application section allows a corresponding predetermined facial expression etc. to be realized when a predetermined condition is satisfied, which has the advantage of allowing variation in the facial expressions and poses of the avatar in the virtual space. There is. Furthermore, even if performers are able to specify Butter's facial expressions using a tablet or the like, when a wearable gift is worn by an operator during studio distribution, the performer may not notice that it has been worn. In such a case, there is an advantage that the facial expression or pose corresponding to the gift can be applied to the avatar without any operation by the performer. That is, information for making a predetermined facial expression or pose may be generated without any operation other than providing a gift from the distributor.

In addition, as mentioned above, the function of the condition application section makes it possible to automatically realize a corresponding predetermined facial expression, etc. when a predetermined condition is satisfied, so that by only providing a gift from the viewer terminal, other There is an advantage in that the user can enjoy the reaction that results in the avatar's facial expression changing without any operation from the viewer's terminal.

4. Embodiment
4-1. Embodiment 1
Embodiment 1 is an example in which a condition application unit operates in a viewer terminal. In this case, the acquisition unit may operate on the viewer terminal. This example will be explained using FIG. 7. Note that since the condition application unit operates on the viewer terminal, it may be possible to change the avatar's expression etc. for each viewer terminal by including information based on the viewer terminal as a predetermined condition. .

Step 1
The acquisition unit acquires information related to the gift (step 001). Information related to such a gift may be input on the viewer terminal where the condition application section operates, or information related to the gift input on another viewer terminal may be transmitted to the condition application section via a server or the like. It may also be acquired by an operating viewer terminal. Note that the acquisition unit may acquire information regarding other avatars.

Step 2
Next, the condition application unit determines a predetermined condition using at least the information related to the gift, and when determining that the predetermined condition is satisfied, specifies an expression etc. corresponding to the satisfied predetermined condition (step 002 ). For this purpose, the condition application unit may have information indicating at least some predetermined conditions and a predetermined facial expression etc. when the conditions are satisfied.

Step 3
The specified facial expression etc. are set as the facial expression etc. related to the avatar used to satisfy the predetermined condition on the viewer terminal (step 003). The viewer has the advantage of being able to watch the reaction of the avatar's facial expression etc. changing in response to the provision of the gift.

4-2. Embodiment 2
Embodiment 2 is an example in which a condition application unit operates in a server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 8 on the premise that the animation is generated on the viewer terminal.

Step 1
The acquisition unit acquires information related to the gift (step 001). Note that the acquisition unit may acquire information regarding other avatars.

Step 2
Next, the condition application unit determines a predetermined condition using at least the information related to the gift, and when determining that the predetermined condition is satisfied, specifies an expression etc. corresponding to the satisfied predetermined condition (step 002 ).

Step 3
The server transmits information indicating the specified facial expression to each viewer terminal viewing the distribution of the avatar (step 003).

Step 4
At each viewer terminal, the avatar's facial expressions and the like are set using the information indicating the facial expressions and the like obtained from the server (step 004). Here, an animation may be generated at each viewer terminal using the set avatar's facial expression and the like.

Step 5
On each viewer terminal, the avatar's facial expression, etc., using the information indicating the facial expression, etc. obtained from the server, is displayed for a predetermined period of time, and then the facial expression is set to the facial expression, etc. based on the broadcaster's facial expression, etc. related to the original avatar. (Step 005). In addition, when animations are generated, such generated facial expressions, etc. are used and displayed within each animation for a predetermined period, and after that, gains are made by tracking based on the broadcaster's facial expressions related to the avatar. The facial expression etc. that are displayed may be used as the facial expression etc. of the avatar.

4-3. Embodiment 3
Embodiment 3 is an example in which a condition application unit operates in a server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 9 on the premise that a video including an avatar's facial expression is generated on the server.

Step 1
The acquisition unit acquires information related to the gift (step 001). Note that the acquisition unit may acquire information regarding other avatars.

Step 2
Next, the condition application unit determines a predetermined condition using at least the information related to the gift, and when determining that the predetermined condition is satisfied, specifies an expression etc. corresponding to the satisfied predetermined condition (step 002 ).

Step 3
The server generates a video regarding the delivery of the avatar using information indicating the specified facial expression, etc. (step 003). Here, if the server that generates the video for avatar distribution and the server that executes the condition application section are the same server, the server that executes the condition application section generates the video for avatar distribution. It's fine. On the other hand, if the server that generates the video regarding avatar distribution and the server that executes the condition application section are different servers, the server that executes the condition application section transfers the specified Information indicating the facial expression is transmitted, and the information indicating the specified facial expression may be used in the server that generates the image. Similarly, when generating animation on such a server, an animation for distributing such an avatar may be generated together with or in place of the video, and in this case, the animation may be used together with or in place of the video. It's okay to be.

Step 4
An image of the avatar generated using the specified facial expression based on the information indicating the specified facial expression is transmitted to each viewer terminal (step 004).

Step 5
On each viewer terminal, a video using information indicating facial expressions obtained from the server is displayed for a predetermined period of time, and then an avatar video based on the facial expressions, etc. based on the broadcaster's facial expressions related to the original avatar is displayed. is displayed (step 005).

Furthermore, the systems and information processing described in this document may be implemented by software, hardware, or a combination thereof, and some or all of such systems and information processing may be implemented by implementing the processing/procedures as computer programs. However, these computer programs may be executed by various computers, and these computer programs may be stored in a storage medium. Additionally, these programs may be stored on non-transitory or temporary storage media.

What is explained in the application documents is not limited to what is explained in the application documents, but may be applied to various examples within the scope of various technical ideas having various technical advantages and configurations explained in the application documents. It goes without saying that it can be done.

1 System 10 Communication network 20 (20A to 20C) Server device 30 (30A to 30C) Terminal device 21 (31) Arithmetic device 22 (32) Main storage device 23 (33) Input/output interface 24 (34) Input device 25 ( 35) Auxiliary storage device 26 (36) Output device 41 Acquisition unit 42 Condition application unit

Claims (27)

A computer program,
By being executed by one or more processors,
Obtain information related to object gifts provided by viewers to broadcasters,
Determining whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift,
When it is determined that the information related to the acquired gift satisfies the predetermined condition, the expression or pose of the avatar related to the distributor that has been generated based on the expression or pose related to the distributor is changed to the predetermined condition. generate information for a predetermined facial expression or pose that corresponds to the conditions;
acquiring information related to the next gift, and continuously setting a predetermined facial expression or pose related to the gift using the information related to the gift and the information related to the time included in the information related to the next gift;
The information related to the gift includes at least one of information on the start of display of the gift, information on change in display of the gift, or information on end of display of the gift,
The predetermined condition uses timing related to the display of the gift included in the information related to the gift,
A computer program that causes the processor to function as described above. The information related to the gift includes information on the start of display of the gift, information on change in display of the gift, and information on end of display of the gift.
The computer program according to claim 1. The predetermined condition uses the priority related to the gift,
The priority related to the gift uses information related to a specific status of the gift provider,
The computer program according to claim 1 or 2. The part where the predetermined facial expression is activated is the same part as the part where the gift is attached.
A computer program according to any one of claims 1 to 3. The part where the predetermined facial expression is activated is a part different from the part where the gift is attached.
A computer program according to any one of claims 1 to 3. The predetermined condition includes that a position related to the avatar and a position related to the gift are within a predetermined distance.
A computer program according to any one of claims 1 to 5. The position related to the avatar is either a position of the avatar in a virtual space, a part that activates the predetermined facial expression, a predetermined part of the avatar, or a collider of the avatar.
The computer program according to claim 6. The predetermined condition uses a type to which the avatar belongs, which is one of a plurality of types to which the avatar belongs;
A computer program according to any one of claims 1 to 7. the predetermined facial expression changes in conjunction with the direction in which the gift is displayed;
A computer program according to any one of claims 1 to 8. The information for making the predetermined facial expression or pose is generated without acquiring information for operating the avatar from a terminal related to the distributor,
A computer program according to any one of claims 1 to 9. The information related to the gift includes the type of gift,
generating information for a predetermined facial expression or pose related to a predetermined activation part according to the type of gift;
A computer program according to any one of claims 1 to 10. The predetermined condition is a condition using probability;
A computer program according to any one of claims 1 to 11. The predetermined condition is a condition that uses both information related to the gift and the behavior of the avatar.
A computer program according to any one of claims 1 to 12. The predetermined condition is a condition that uses both information related to the gift and the situation related to the avatar.
A computer program according to any one of claims 1 to 13. The predetermined condition is applied in the chronological order in which the information regarding the gift is acquired.
A computer program according to any one of claims 1 to 14. the processor is a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU);
A computer program according to any one of claims 1 to 15. the processor has an arithmetic function;
A computer program according to any one of claims 1 to 16. comprising one or more processors,
the one or more processors executing computer readable instructions,
Obtain information related to object gifts provided by viewers to broadcasters,
Determining whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift,
When it is determined that the information related to the acquired gift satisfies the predetermined condition, the expression or pose of the avatar related to the distributor that has been generated based on the expression or pose related to the distributor is changed to the predetermined condition. generate information for a predetermined facial expression or pose that corresponds to the conditions;
acquiring information related to the next gift, and continuously setting a predetermined facial expression or pose related to the gift using the information related to the gift and the information related to the time included in the information related to the next gift;
The information related to the gift includes at least one of information on the start of display of the gift, information on change in display of the gift, or information on end of display of the gift,
The predetermined condition uses timing related to the display of the gift included in the information related to the gift,
A server device characterized by: The information related to the gift includes information on the start of display of the gift, information on change in display of the gift, and information on end of display of the gift.
The server device according to claim 18. The predetermined condition uses the priority related to the gift,
The priority related to the gift uses information related to a specific status of the gift provider,
The server device according to claim 18 or 19. the processor is a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU);
The server device according to any one of claims 18 to 20. the processor has an arithmetic function;
The server device according to any one of claims 18 to 21. A method performed by one or more processors executing computer readable instructions, the method comprising:
By the processor executing the instructions,
Obtain information related to object gifts provided by viewers to broadcasters,
Determining whether the information related to the acquired gift satisfies a predetermined condition using the information related to the gift,
When it is determined that the information related to the acquired gift satisfies the predetermined condition, the expression or pose of the avatar related to the distributor that has been generated based on the expression or pose related to the distributor is changed to the predetermined condition. generate information for a predetermined facial expression or pose that corresponds to the conditions;
acquiring information related to the next gift, and continuously setting a predetermined facial expression or pose related to the gift using the information related to the gift and the information related to the time included in the information related to the next gift;
The information related to the gift includes at least one of information on the start of display of the gift, information on change in display of the gift, or information on end of display of the gift,
The predetermined condition uses timing related to the display of the gift included in the information related to the gift,
A method characterized by: The information related to the gift includes information on the start of display of the gift, information on change in display of the gift, and information on end of display of the gift.
24. The method according to claim 23. The predetermined condition uses the priority related to the gift,
The priority related to the gift uses information related to a specific status of the gift provider,
25. The method according to claim 23 or 24. the processor is a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU);
26. A method according to any one of claims 23 to 25. the processor has an arithmetic function;
27. A method according to any one of claims 23 to 26.

Images