JP7329217B2 - Computer program, server device, terminal device, and method - Google Patents

Description

The technology disclosed in the present application relates to computer programs, server devices, terminal devices and methods.

In recent years, technology using avatars played by distributors has been developed. In this type of technology, viewers can view avatars played by distributors via distribution services.

In addition, in relation to services using avatars, a service called "custom cast" is known as a service that uses technology to control the facial expressions and actions of avatar objects based on the actions of performers, etc. ( Non-Patent Document 1). In this service, the performer pre-assigns one of a large number of prepared facial expressions and actions to each of a plurality of flicking directions on the screen of the smartphone, and selects the desired facial expression or action. By flicking the screen of the smartphone by the performer along the corresponding direction, the avatar object displayed in the moving image can express the facial expression or action.

"Custom Cast", [online], Custom Cast Inc., [searched December 10, 2019], Internet (URL: https://customcast.jp/)

However, in the technology disclosed in Non-Patent Document 1, the performer has to flick the screen of the smartphone while speaking, and it is difficult for the performer to perform the flick operation.

The above non-patent document 1 is incorporated herein by reference in its entirety.

Some embodiments disclosed in this application provide a computer program, a server device, a terminal device, and a method that can express avatars more appropriately.

A computer program according to one aspect is executed by one or more processors to perform predetermined actions of an avatar performed by a distributor, the expression of the avatar, the complexion of the avatar, the voice of the avatar, or acquiring information relating to any one or more of a predetermined situation occurring regarding the avatar, determining whether or not the acquired information satisfies a predetermined condition, and determining whether or not the acquired information satisfies a predetermined condition. When it is determined that the conditions are satisfied, the facial expression or pose of the avatar of the distributor generated based on the facial expression or pose of the distributor, the predetermined facial expression or pose corresponding to the predetermined condition, Information for the expression of the avatar is generated, and the avatar to which the predetermined expression or pose is applied is displayed.

A computer program according to one aspect is executed by one or more processors to perform predetermined actions of an avatar performed by a distributor, the expression of the avatar, the complexion of the avatar, the voice of the avatar, Alternatively, acquiring information related to any one or more of predetermined situations occurring regarding the avatar, determining whether or not the acquired information satisfies a predetermined condition, and determining that the predetermined condition is satisfied. information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition. and transmitting information for making the predetermined facial expression or pose.

A server device according to one aspect includes one or more processors, and the processors execute computer-readable instructions to perform predetermined actions of an avatar played by a distributor, Acquiring information related to any one or more of facial expression, complexion of the avatar, voice of the avatar, or a predetermined situation occurring with respect to the avatar, and whether or not the acquired information satisfies a predetermined condition and, if it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor corresponds to the predetermined condition. Information for making a predetermined facial expression or pose is generated, and the information for making the predetermined facial expression or pose is transmitted.

A method according to one aspect is a method executed by one or more processors executing computer readable instructions, wherein, in a virtual space, a given action of an avatar performed by a broadcaster, a facial expression of the avatar, Acquiring information related to any one or more of the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring regarding the avatar, and determining whether or not the acquired information satisfies a predetermined condition. and, if it is determined that a predetermined condition related to the acquired information is satisfied, the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, is changed to the predetermined condition. Information for making a corresponding predetermined facial expression or pose the facial expression of the avatar is generated, and the avatar to which the predetermined facial expression or pose is applied is displayed.

A method according to one aspect is a method executed by one or more processors executing computer readable instructions, wherein, in a virtual space, a given action of an avatar performed by a broadcaster, a facial expression of the avatar, Acquiring information related to any one or more of the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring regarding the avatar, and determining whether or not the acquired information satisfies a predetermined condition. and, when it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, is changed to a predetermined facial expression corresponding to the predetermined condition. Alternatively, information for making a pose is generated, and the information for making the predetermined expression or pose is transmitted.

A viewer terminal device according to one aspect includes one or more processors, and the processor executes a computer-readable instruction to perform a predetermined action of an avatar played by a distributor in a virtual space, Acquiring information related to any one or more of an avatar's facial expression, the avatar's complexion, the avatar's voice, or a predetermined situation occurring with respect to the avatar, and whether the obtained information satisfies a predetermined condition determining whether or not, and if it is determined that the predetermined condition related to the acquired information is satisfied, the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, Information for making a predetermined facial expression or pose corresponding to the predetermined condition the facial expression of the avatar is generated, and the avatar to which the predetermined facial expression or pose is applied is displayed.

A distributor terminal device according to one aspect includes one or more processors, and the processor executes a computer-readable instruction to perform a predetermined action of an avatar played by a distributor in a virtual space, Acquiring information related to any one or more of an avatar's facial expression, the avatar's complexion, the avatar's voice, or a predetermined situation occurring with respect to the avatar, and whether the obtained information satisfies a predetermined condition If it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar associated with the distributor, which has been generated based on the facial expression or pose associated with the distributor, is changed to satisfy the predetermined condition. Information for making a corresponding predetermined facial expression or pose is generated, and the information for making the predetermined facial expression or pose is transmitted.

FIG. 1 is a block diagram showing an example of the configuration of a system according to one embodiment. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the server device 20 (terminal device 30) shown in FIG. FIG. 3 is a block diagram that schematically shows an example of the functions of the system according to one embodiment. FIG. 4 is an example of data possessed by the system according to one embodiment. FIG. 5 is an example of data possessed by the system according to one embodiment. FIG. 6 is an example of a flow processed by a system according to one embodiment. FIG. 7 is an example of a flow processed by a system according to one embodiment. FIG. 8 is an example of a flow processed by a system according to one embodiment.

Various embodiments of the present invention will now be described with reference to the accompanying drawings. Note that components depicted in one drawing may be omitted in another drawing for convenience of explanation. Furthermore, it should be noted that the attached drawings, while disclosing one embodiment of the present invention, are not necessarily drawn to scale. Also, in this document, the object indicated by the term application may be referred to as software or a program, and may be instructions to a computer that are combined so as to obtain a result.

1. System Example FIG. 1 is a block diagram showing an example of a system configuration according to an embodiment. As shown in FIG. 1 , system 1 may include one or more server devices 20 connected to communication network 10 and one or more terminal devices 30 connected to communication network 10 . In FIG. 1, three server devices 20A to 20C are illustrated as examples of the server device 20, and three terminal devices 30A to 30C are illustrated as examples of the terminal device 30. , one or more server devices 20 other than these may be connected to the communication network 10 , and one or more terminal devices 30 other than these may be connected to the communication network 10 as terminal devices 30 . In this application document, the term "system" may include both a server and a terminal device, and may also be used as a term indicating only a server or only a terminal device. In other words, the system may be in any form of a server only, a terminal device only, or both a server and a terminal device.

Also, the system may be an information processing device on the cloud. Further, the system constitutes a virtual information processing device, and may be logically configured with one information processing device. Also, the system owner and administrator may be different.

The communication network 10 may be, but is not limited to, a mobile phone network, a wireless LAN, a fixed phone network, the Internet, an intranet, Ethernet, and/or combinations thereof. In communication network 10, various data disclosed in this document may be communicated.

By executing a specific installed application, the server device 20 performs a predetermined action of the avatar performed by the distributor, the facial expression of the avatar, the complexion of the avatar, the voice of the avatar, or the voice of the avatar in the virtual space. a predetermined situation occurring in relation to the avatar, and determining whether or not the obtained information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied , generating information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition;
and the like.

Alternatively, by executing an installed web browser or a specific application, the terminal device 30 can perform a predetermined action of the avatar performed by the distributor, the expression of the avatar, the complexion of the avatar, the avatar, voice, or a predetermined situation that occurs with respect to the avatar, obtain information related to one or more, determine whether the obtained information satisfies a predetermined condition, and determine whether the predetermined condition is for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition when it is determined that the condition is satisfied It may be able to perform an operation such as generating information.

The terminal device 30 is any terminal device capable of executing such operations, and may be, but is not limited to, a smart phone, tablet, mobile phone (feature phone), personal computer, or the like.

2. Hardware Configuration of Each Device Next, an example of the hardware configuration of the server device 20 will be described.

2-1. Hardware Configuration of Server Device 20 An example of hardware configuration of the server device 20 will be described with reference to FIG. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the server device 20 (terminal device 30) shown in FIG. It is described in relation to each terminal device 30.)

As shown in FIG. 2 , the server device 20 can mainly include an arithmetic device 21 , a main storage device 22 and an input/output interface device 23 . Server device 20 may further include input devices 24 and auxiliary output devices 26 . These devices may be connected by a data bus and/or a control bus.

The arithmetic device 21 performs arithmetic operations using instructions and data stored in the main memory device 22 and causes the main memory device 22 to store the results of the arithmetic operations. Further, the computing device 21 can control an input device 24, an auxiliary storage device 25, an output device 26 and the like via an input/output interface device 23. FIG. The server device 20 may include one or more computing devices 21 . Computing device 21 may include one or more central processing units (CPUs), microprocessors, and/or graphics processing units (GPUs).

The main storage device 22 has a storage function, and receives instructions and data from the input device 24, the auxiliary storage device 25, the communication network 10, etc. (server device 20, etc.) via the input/output interface device 23, and It stores the calculation result of the calculation device 21 . Main memory 22 may include, but is not limited to, RAM (random access memory), ROM (read only memory), and/or flash memory.

The auxiliary storage device 25 is a storage device. Instructions and data (computer programs) constituting the specific application, web browser, etc. may be stored. may be loaded into the main memory 22 via the . The auxiliary storage device 25 may be a magnetic disk device and/or an optical disk device, a file server, etc., but is not limited to these.

The input device 24 is a device that takes in data from the outside, and may be a touch panel, buttons, keyboard, mouse and/or sensor.

Output devices 26 may include, but are not limited to, display devices, touch panels, and/or printer devices. Alternatively, the input device 24 and the output device 26 may be integrated.

In such a hardware configuration, the arithmetic unit 21 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 25 into the main storage device 22, and loads the loaded instructions. and data to control the output device 26 via the input/output interface device 23, or to control other devices (for example, the server device 20 and It may be possible to transmit and receive various information to and from other terminal devices 30, etc.).

The server device 20 is provided with such a configuration, and by executing a specific installed application, predetermined actions of the avatar played by the distributor, the expression of the avatar, the expression of the avatar, and the Acquiring information related to any one or more of complexion, voice of the avatar, or a predetermined situation occurring regarding the avatar, determining whether the acquired information satisfies a predetermined condition, When it is determined that a predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, is changed to a predetermined facial expression or pose corresponding to the predetermined condition. It may be possible to perform some or all of operations such as generating information for . Further, such operations may be performed by the user using the input device 24 or an input device 34 associated with the terminal device 30, which will be described later, to give instructions to the system of the example of the invention disclosed in this document. Further, when the program is executed on the arithmetic device 21, it may be displayed by the output device 26 of the server device 20 as the system used by the user, or the output device of the terminal device 30 as the system used by the user. 36 may be used.

2-2. Hardware Configuration of Terminal Device 30 An example of hardware configuration of the terminal device 30 will be described with reference to FIG. As the hardware configuration of each terminal device 30, for example, the same hardware configuration as that of each server device 20 described above can be used. Therefore, the reference numerals for the components of each terminal device 30 are shown in parentheses in FIG.

As shown in FIG. 2, each terminal device 30 mainly includes an arithmetic device 31, a main storage device 32, an input/output interface device 33, an input device 34, an auxiliary storage device 35, and an output device 36. , can be included. These devices are connected to each other by a data bus and/or a control bus.

Arithmetic device 31, main storage device 32, input/output interface device 33, input device 34, auxiliary storage device 35, and output device 36 are included in each server device 20 described above, respectively. , the input/output interface device 23, the input device 24, the auxiliary storage device 25, and the output device 26, or may be different. Also, the capacities and capabilities of the computing devices and storage devices may be different. Input device 34 and output device 36, in particular, are described in greater detail below.

In such a hardware configuration, the arithmetic unit 31 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 35 into the main storage device 32, and loads the loaded instructions. and data to control the output device 36 via the input/output interface device 33, or to control other devices (for example, each server device 20) via the input/output interface device 33 and the communication network 10 etc.).

The terminal device 30 has such a configuration, and in the virtual space, a predetermined action of the avatar played by the distributor, the facial expression of the avatar, the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring in relation to the avatar. acquire information related to any one or more of them, determine whether the acquired information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, express or An operation such as generating information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the pose, to a predetermined facial expression or pose corresponding to the predetermined condition (various actions) may be performed in part or in whole. Also, by executing the installed web browser, it is possible to receive and display web pages from the server device 20 and perform similar operations. Also, such operations may be performed by the user using the input device 34 to give instructions to the system of the example of the invention disclosed in this document. Further, when the program is executed on the arithmetic device 31, it may be configured to be displayed on the output device 36 of the terminal device 30 as a system used by the user.

In the present document, there may be two types of terminal devices, a "distributor terminal" and a "viewer terminal". These may be names given to the roles of terminals, and one terminal device may be a viewer terminal or a distributor terminal, and may be a viewer terminal and a distributor terminal at the same time. may be a user terminal. The distributor terminal may be a terminal used by a distributor who appears as a subject such as an avatar in what is distributed. On the other hand, the viewer terminal may be a terminal primarily used by viewers who watch what is being distributed. In addition, viewers can also give a certain amount of influence to what is distributed by giving gifts to subjects such as avatars in such a way that gifts appear in what is distributed. good too. Similarly, viewers may influence what is delivered in the form of audio, comments, and the like. The content to be distributed may be a video, a video, or an animation, as long as the subject such as an avatar played by the distributor appears in a part of them. . Note that the above description is merely an example to which one embodiment of the present invention is applied, and each embodiment of the present invention may be applied even in a virtual space in which a subject without such assumptions appears.

The viewer terminal may be configured to allow voice input, comment input, gifting, etc., as described above. For example, the user terminal may have, as the input device 34, a keyboard, a touch panel, or the like that can input comments and obtain comments that are text information. Also, the user terminal may have a function of generating a certain comment by means of the computing device 34 . These acquired comments may be similarly transmitted to the communication network 10 via the input/output interface device 23 .

Note that the comments according to the present invention may be automatically generated by the system in addition to the comments input by the user terminal. Such comments may be sent to the user terminal and displayed at the user terminal.

The terminal may also include a microphone capable of acquiring sound as the input device 34 . The human voice may be acquired by the microphone as the input device 34 and treated as sound, or after being acquired by the microphone as the input device 34, converted into text by speech recognition by the arithmetic unit 31 and used as a comment. can handle. It should be noted that the user terminal may generate sound using the computing device 31 . These acquired sounds or comments may be similarly transmitted to the communication network 10 via the input/output interface device 23 .

On the other hand, the distributor terminal may include a motion sensor capable of detecting human movement as the input device 34 . The motion sensor may be attached to the body of the broadcaster and have a function capable of detecting movements of each part of the body. Each part of the body can include various parts such as the head, face, upper arm, forearm, hand, thigh, lower leg, foot, trunk, etc., but not limited to some or all of these. It may be something that can be attached to a detectable position. Such sensors may be able to track the facial expressions and poses of the broadcaster and generate them as information. Motion sensors may include acceleration sensors, gyro sensors, geomagnetic sensors, and the like. These acquired information may be similarly transmitted to the communication network 10 via the input/output interface device 23 .

Also, the distributor terminal may include a camera capable of capturing moving images as the input device 34 . The camera may capture the head, face, eyes, body parts, whole body, etc. of the broadcaster. When the camera captures an image of the head or face, it can detect the direction of the face. When capturing the image of the face, it can detect the facial expression. When capturing the image of the eye, the camera can detect the line of sight. Yes, it may be possible to detect body movement in conjunction with or in place of the motion sensors described above when imaging a portion of the body or the entire body. Information about facial direction, line of sight, facial expression, and body movement based on these imaging information may be similarly transmitted to the communication network 10 via the input/output interface device 23 .

Also, the output device 36 in the viewer terminal and distributor terminal may be a display device or a sound generator. The display device may be a variety of devices as long as it has a displayable function, and examples thereof include liquid crystal displays, plasma displays, organic EL displays, micro LED displays, head-mounted displays, etc. It can be a device. The sound-producing device may be a speaker, headphones, earphones, or the like. The information output by the output device 36 may be information generated by the arithmetic device 31 or information received from the communication network 10 .

3. Functions of Each Device Next, an example of the functions of each of the server device 20 and the terminal device 30 will be described with reference to FIG. FIG. 3 is a block diagram schematically showing an example of functions of the system shown in FIG. As shown in FIG. 3, an example system may include an acquisition unit that acquires information that can be used to generate an object, and a generation unit that uses the acquired information to generate the object.

2.1. Acquisition Unit The acquisition unit has a function of acquiring information that can be used to determine whether or not a predetermined condition, which will be described later, is applicable.

The information acquired by the acquisition unit is any of a predetermined action of the avatar played by the distributor, the facial expression of the avatar, the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring in relation to the avatar in the virtual space. It may include information (in this document, it may also be referred to as a predetermined operation, etc.) related to one or more of them.

Concrete contents of the information acquired by the acquisition unit will be explained in the condition application unit. Note that FIG. 4 shows an example of part of the information acquired by these acquisition units.

Moreover, when a gift is provided to a specific avatar, the obtaining unit may obtain information specifying the avatar to whom the gift has been provided.

The information processing device from which the obtaining unit obtains information may be the same as or different from the information processing device having the function of the obtaining unit. The term "acquisition" may be a broader concept of receiving information from another information processing device, or may be a broader concept of reading information from a storage device. It may be a broader concept of the meaning of receiving information from a program via a storage device.

2.2. The condition application unit The condition application unit determines whether the acquired information satisfies a predetermined condition, and if it determines that the acquired information satisfies the predetermined condition, the facial expression or It has a function of generating information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the pose, to a predetermined facial expression or pose corresponding to the predetermined condition.

Here, generating information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition. is for changing the facial expression of the avatar of the distributor, which has been generated based on the facial expression of the distributor, to a predetermined facial expression corresponding to the predetermined condition when such a predetermined condition is satisfied; Only information may be generated, or only information for making the pose of the avatar associated with the distributor, which has been generated based on the pose associated with the distributor, a predetermined pose corresponding to the predetermined condition. Alternatively, the expression and pose of the avatar of the distributor, which have been generated based on the expression and pose of the distributor, are set to predetermined expressions and poses corresponding to the predetermined conditions. information may be generated. In the documents of the present application, a predetermined facial expression and/or pose specified in response to a predetermined condition may be referred to as a predetermined facial expression, etc., and a facial expression and/or pose may be referred to as a facial expression, etc. .

Further, the predetermined facial expression or the like is a facial expression displayed using a predetermined facial expression parameter or a facial expression parameter obtained by correcting the facial expression parameter based on the distributor's facial expression based on a predetermined rule. It may be an expression displayed by setting a predetermined shape or movement, a predetermined pose parameter, or a pose parameter based on the pose of the broadcaster according to a predetermined rule. It may be a pose displayed using the parameters of the pose corrected based on the pose, or a pose displayed by setting a predetermined shape or movement. Here, the predetermined facial expression parameter indicates a plurality of parameters that affect the movement of each part of the face, and by setting each of the plurality of parameters to a predetermined value, one facial expression is indicated. can be Such one or more parameters may be parameters that affect the movement of each part of the face, and each parameter may be for a specific part of the face, or for the entire face (or part of the face) However, the shape of the whole face (or part of the face) may be different for each parameter. The latter is a technique in which a plurality of objects having the same area but different shapes are prepared, and one facial expression is expressed by synthesizing a part or a plurality of them. An example of the latter is a blend shape, in which a plurality of different overall face shapes are prepared in advance and a single facial expression is generated by synthesizing the plurality of shapes. . Note that, in the present technology, in addition to normal facial elements, a special facial expression may be given by preparing a predetermined shape in advance. For example, if a shape indicating the vertical lines of the face is prepared in advance, it is possible to produce the vertical lines of the face. It is possible to produce a cross. In this case, since the parameters are set in advance, the facial expression, etc. will be reproduced regardless of the facial expression, etc. of the broadcaster. It has the advantage of being reproducible. Also in this case, the predetermined facial expression may be set differently depending on the avatar. That is, for avatar A and avatar B, the predetermined facial expression applied when the same predetermined condition is satisfied may be different between avatar A and avatar B. In this case, it may be set for each avatar, or may be set according to the type of avatar.

On the other hand, if the predetermined facial expression, etc. is a facial expression parameter, etc., that is obtained by correcting the facial expression, etc. parameter based on the facial expression, etc. of the distributor based on a predetermined rule, the predetermined facial expression, etc. does not correspond to the facial expression, etc. of the distributor. It may be a facial expression etc. based on. For example, assume that the distributor has a surprised expression for some kind of gift. At this time, the facial expression of the avatar is set to a facial expression corresponding to surprise by one or a plurality of parameters corresponding to the detection of the facial expression of the broadcaster. is increased or decreased by a predetermined value to emphasize the expression of surprise. While the above-described technology is absolute in that one or more parameters are determined with a predetermined facial expression, the present technology can be said to be relative to information obtained from the current facial expression of the distributor parameters may be used. Therefore, even if the same predetermined conditions are applied to the same avatar played by the same distributor, the parameters such as facial expressions based on the facial expressions that the distributor himself has at that time may be the same. If there is, there is a possibility that it will be different, so there are cases where the facial expression etc. corresponding to such different parameters may become the predetermined facial expression etc. Therefore, there is a possibility that it is possible to have more variations such as facial expressions.

In addition, the facial expression displayed by setting a predetermined shape or movement indicates that a predetermined shape itself, such as a mask, is used as a predetermined expression as a facial expression, and may be such. .

In the above description, a predetermined facial expression has been described, but similarly, a corresponding technique may be used for a predetermined pose.

Also, the predetermined facial expression or the like may be a facial expression or the like associated with a predetermined action or the like that occurs. In this case, there is an advantage that the viewer can easily understand the reason why a predetermined expression or the like is expressed in association with a predetermined action or the like.

Moreover, a predetermined facial expression or the like may act on parameters for all parts constituting the facial expression or the like, or may act on a part of the parameters. For example, a predetermined facial expression or the like affects only the parameters for the mouth, and the parameters for parts other than the mouth are parameters that are linked to the facial expression of the distributor, and correspond to the movement of the facial expression of the distributor from the distributor terminal. It may be a parameter based on the information obtained. Also, the predetermined facial expression or the like may include the appearance of the avatar. For example, the predetermined facial expression or the like may include an effect in which the avatar is charred.

Also, the predetermined facial expression or the like may change in conjunction with the direction of a predetermined action or the like. In the virtual space, there are times when the distributor does not know the place where things such as things described later appear. At this time, the distributor may not be aware of where the object appears, and as a result, the avatar associated with the distributor may not be able to react to the object. For example, when a ghost character appears or a basin falls from overhead, the broadcaster may not notice. Therefore, there is an advantage that the viewer can be more interested in the behavior of the avatar by changing the predetermined facial expression etc. of the avatar in conjunction with the direction in which the predetermined action etc. is displayed. Here, there may be various modes for changing in conjunction with the direction of the predetermined motion or the like. For example, a predetermined facial expression or the like may be set to face the direction in which the ghost character has come into contact or the direction in which the zombie character has approached. When a predetermined facial expression or the like is directed, a part of the avatar may be changed in conjunction, or a plurality of parts of the avatar may be changed in conjunction. For example, when a predetermined facial expression or the like is directed, the entire body may be directed in the direction in which the expression is directed, only the head may be directed in the direction in which the expression is directed, or only a part of the face such as only the eyes may be directed in the direction in which the facial expression is directed. may

The predetermined pose may be various poses. A pose may be only a part of the avatar's body or may be the entire avatar's body. For example, poses with only the head as a pose to tilt the neck, poses with only the hands including a production that the hands are enlarged or swollen red, poses with only the upper body such as guts poses, gun poses, etc. It can be a pose that holds up or a whole pose that jumps with the whole body.

The predetermined condition is information related to any one or more of a predetermined action of the avatar performed by the distributor, the facial expression of the avatar, the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring regarding the avatar. , can be used.

The predetermined action of the avatar performed by the distributor may be a predetermined action that the avatar perceives with the five senses if the avatar is a living creature. Also, the predetermined motion of the avatar may include a motion of looking at a predetermined target. For example, when the avatar sees something and is surprised, a surprised expression may be set as the predetermined expression. In this case, an object that is natural for the avatar to make a surprised expression may be set in advance, and it may be determined that the avatar has seen the object when the object is on the extension line of the line of sight. That is, when the predetermined action includes a motion of looking at the predetermined target, the predetermined condition is to move the line of sight of the avatar so that the predetermined target is in the line-of-sight direction of the avatar, or It may include that the predetermined target exists on the line of sight of the avatar. In the former case, when the avatar changes its line of sight by actively moving the direction in which it looks, and if the surprising object is positioned on the extension line of the line of sight, a surprised expression may be set as a predetermined expression or the like. The latter may be the case where a predetermined target exists, and when the avatar is looking in a predetermined direction, the predetermined target may appear and be displayed in that direction.

In the above description, an object that gives the avatar a feeling of surprise has been described, but the object is not limited to the feeling of surprise, and may be an object that gives another feeling or sensation. For example, when a person looks at an object, certain emotions and sensations arise, such as objects that evoke feelings of fear such as ghosts, objects that evoke feelings of liking such as steak, and objects that evoke feelings of dislike such as natto. It can be a thing. Also, a predetermined facial expression or the like representing a face that a person generally makes when viewing such an object may be set as a predetermined facial expression or the like when viewing such an object. The above-mentioned things that evoke feelings of fear, things that evoke feelings of liking, and things that evoke feelings of dislike can be correspondingly horrified expressions, smiles, expressions of disgust, and the like.

In the above description, the object to be seen is described as an object, but the object to be seen may be other characters or other avatars. That is, when the predetermined target is another avatar or character, the predetermined action of the avatar may include matching the line of sight of the avatar with the line of sight of the predetermined target. In this way, when the line of sight matches that of another character or other avatar, a predetermined facial expression or the like may be set. When the line of sight of the avatars matches, a predetermined facial expression or the like may be set for only one of the avatars, or a predetermined facial expression or the like may be set for both avatars. As for the latter, for example, it is conceivable that both avatars' facial expressions are heart-pounding, such as their faces turning red when their eyes are on each other. In addition, in the case where the predetermined facial expressions etc. are set for both of the latter avatars, the facial expressions etc. to be set may be the same predetermined facial expressions etc., or may be different predetermined facial expressions etc. . The reason for such a different predetermined facial expression may be a predetermined facial expression that corresponds to the type to which the avatar belongs, or a predetermined facial expression that corresponds to the avatar's preference or nature. A predetermined facial expression or the like selected stochastically may be used. In addition, as an example of a case where the avatar meets the line of sight of another character, when the line of sight of the ghost character and the avatar meet, the avatar's face may turn pale as a predetermined facial expression of the avatar. The sufficiency of such a predetermined condition may be determined by determining whether or not there is an intersection between the line-of-sight direction of the avatar and an object such as a ghost character.

In the above description, the object to be viewed is described as an object, but the object to be viewed may be some event. The event may have various scales and targets, and is not limited. For example, it may be a scary event such as a rushing swarm of crickets, being swept away by a flooded river, or a gust of wind, or a painful event such as falling or bloodshed of another character or avatar. It may be so. Even in the case of such an event, similarly, the corresponding predetermined facial expression or the like may be set for the avatar who has seen such an event.

Also, the predetermined motion of the avatar may include a motion of touching a predetermined target. When the predetermined motion includes a motion of contacting the predetermined target, the predetermined condition is that the distance between the position of the avatar and the position of the predetermined target is within a predetermined distance. may include being. Here, the position related to the avatar is any one of the position of the avatar in the virtual space, the part that activates the predetermined facial expression, the predetermined part of the avatar, or the position of the collider of the avatar. good.

Here, the action of contact means a mode of contact from the avatar, regardless of whether the avatar is active or passive, but does not mean a mode of contact from a third party. The mode of being contacted by a third party, regardless of whether it is subjective or passive, is included in the "predetermined situations that occur with respect to the avatar." Note that "passive" includes, for example, a case where an avatar or a third party objectively moves and is contacted regardless of the intention of the avatar or the third party.

For example, when the object touched by the avatar is hot, the avatar may make a surprised expression, or a predetermined expression such as swelling of the touched part due to heat. may be More specifically, for example, when the hand touches something hot, it may be a predetermined facial expression such that the hand becomes larger and becomes red and swollen.

The predetermined action of the avatar includes eating and drinking action of a predetermined target. Here, the predetermined eating action may be a predetermined eating action of the avatar, and may be detected by an ID or the like indicating the eating action. The ID indicating the eating action may be an ID given to a specific movement predetermined as the eating action, or an ID given to a predetermined movement based on one or a plurality of motion data. may In the present invention, the technical significance of incorporating an eating action as a predetermined action is that a predetermined facial expression or the like is set by combining it with other information described later. For example, even if the action of eating and drinking is the same (for example, the action of eating meat), when a certain avatar A eats meat, a predetermined facial expression expressing an elegant way of eating is set, while other avatars When B eats meat, it is conceivable that a predetermined facial expression or the like expressing wild eating is set. Here, factors that cause different effects for avatar A and avatar B include, for example, if avatar A and avatar B are avatars of different types, or if they are avatars of the same type, the set preferences Predetermined facial expressions and the like expressing different ways of eating may be set according to taste, preference, and the like. Such an avatar type may be the personality set when the avatar is set, or may be determined from the parts or clothes that the avatar is equipped with when a predetermined condition is determined.

Also, the predetermined action of the avatar may include an odor or a predetermined gesture with respect to the odor. Here, examples of the predetermined gesture include a gesture of sniffing the nose, a gesture of moving the air in front of the nose left and right with a hand, and the like. Such gestures may be detected, for example, by using optical sensors, inertial sensors, or the like, for example, nose movements or hand movements such as right or left hand. That is, the predetermined condition may use detection of the predetermined gesture of the avatar. As the corresponding predetermined facial expression, etc., there are a predetermined facial expression that produces an ecstatic feeling, a predetermined facial expression that agonizes, and the like. As a result, there is an advantage that the viewer can more clearly understand the meaning of the action by the distributor when he or she senses a certain smell or smell. By sharing such smells and specific gestures corresponding to such smells between the distributor and the system in advance, it is possible to more effectively produce a predetermined expression or the like corresponding to such gestures.

Also, the predetermined action of the avatar may include a predetermined gesture of hearing/listening to a sound. As a predetermined gesture for listening/listening to a sound, for example, a motion of putting a hand on an ear or a gesture of moving one ear in a certain direction can be considered. As a predetermined facial expression corresponding to such a gesture, it is conceivable that the hand becomes larger in the case of the gesture of placing the hand on the ear, and that the ear becomes larger and moves in the case of the gesture of moving one ear in a certain direction. can be considered.

The predetermined condition including the facial expression of the avatar played by the distributor may be, for example, a predetermined facial expression when the facial expression of the avatar is one or more specific parameters. Moreover, even if the predetermined facial expression is not always obtained, it is conceivable that the predetermined facial expression or the like is used when other predetermined conditions are satisfied. For example, even if a predetermined expression, etc. is not normally set as the face of the broadcaster with a surprised expression, if a specific event occurs and the broadcaster shows a surprised face, the predetermined expression may be set.

The complexion of the avatar played by the broadcaster may include complexion changes. For example, when the facial color of the avatar changes from normal, the facial expression or the like of the avatar may be set to a different predetermined facial expression or the like in accordance with the facial color.

The voice of the avatar performed by the distributor may include that the sound quality of the avatar's voice changes by a predetermined range or more, or that the volume of the avatar's voice changes by a predetermined amount or more. For example, the avatar's voice may have an unusual voice quality due to tension, anger, or the like. In the case of such voice, a predetermined facial expression or the like corresponding to such voice may be set.

The predetermined situation occurring with respect to the avatar may include the occurrence of a predetermined sound within the virtual space. The predetermined sound may be, for example, a loud sound. For such a loud sound, a surprised expression may be given as a predetermined facial expression, or a pose expressing surprise with the whole body, such as jumping up, may be used.

A predetermined situation that occurs with respect to an avatar may include a distance between a position of a predetermined object and a position of the avatar being within a predetermined range. Here, the position related to the predetermined object is, for example, the center of gravity of the predetermined object, the center of the predetermined object, the center of the circumscribed circle of the predetermined object, or the inscribed circle of the predetermined object. It may be the center, the center of the circumscribed circle on the representation of the given object, the center of the inscribed circle on the representation of the given object, or the like. Alternatively, the position of the predetermined object may be the intersection of the circle or polygon on the tangent of the straight line distance between the circle or polygon of the predetermined object and the position of the avatar. Further, as for the position related to a predetermined object, a collider may be set for the predetermined object, a collider may be set for the avatar, and contact determination may be performed using the collider. The position associated with the avatar may be as described above. Note that when the position related to the avatar is the position of the collider, it may be determined that the avatar is in contact with a predetermined object.

The predetermined entity may be any of the various entities described above. For example, when a basin falls from overhead and touches the head, the eyes may be crossed out (X) as a predetermined facial expression. Also, when lightning strikes (contacts) the avatar, the avatar may be set to have a charred-black appearance as a predetermined facial expression.

Also, instead of the predetermined object, another object such as a character or another avatar may be used. As an example in the case of a character, the predetermined situation that occurs with respect to the avatar may include that the distance between the position of the zombie character and the position of the avatar falls within a predetermined range. In this case, a vertical line may be drawn on the avatar's face when the distance is within a predetermined distance. Also, the predetermined facial expression or the like may change according to the extent of the distance. For example, when the distance changes to a first distance and a second distance shorter than the first distance, a predetermined facial expression at the first distance and a predetermined facial expression at the second distance and may be different. For example, the predetermined facial expression at the first distance has 10 vertical lines in the facial expression, while the predetermined facial expression at the second distance has 20 vertical lines in the facial expression. And the width of such vertical lines may be doubled, and so on. Contrary to the above, it may include that the position related to the avatar and the position related to the character or avatar are outside the range of the predetermined distance under a predetermined condition. In this case, a situation in which the avatar or the character relatively moves away from the avatar is shown.

For example, a given situation that occurs with respect to an avatar may include a case where a given relationship occurs between a first avatar and a second avatar. For example, when the first avatar strokes the head of the second avatar, the eyes of the second avatar may be set to hearts as a predetermined facial expression or the like. In this case, a collider may be set for each of the first avatar and the second avatar, and determination may be made based on the contact determination.

Further, the predetermined situation occurring regarding the avatar may include a predetermined combination of a predetermined target and a predetermined pose of the avatar. Here, the predetermined target may be an object, a predetermined situation, or a predetermined event. Objects may also include gifts. For example, when the avatar wears a gift of a pistol as an object and makes a shooting pose, a brave expression may be set as a predetermined expression or the like. Here, the determination of the shooting pose may be made by setting a collider from the center position of the avatar and determining the collision of each position of the body such as the right hand, left hand, right foot, left foot, and head. Here, the positions of each distributor's right hand, left, right foot, left foot, head, etc. may be detected by an optical sensor or an inertial sensor. It should be noted that the predetermined expression or the like set in such a case may be suggested by the combination of gifts given. For example, a frog shield, a frog case, or a frog cloak can have a frog-like face.

Further, among the plurality of conditions according to the present invention, a plurality of conditions may be satisfied. For example, when the line of sight of the avatar matches that of the ghost character, and at the same time, the distance between the position of the avatar and the position of the ghost character is within a predetermined distance range. In such a case, if predetermined facial expressions and the like corresponding to each condition that the lines of sight match and that the distance is within a range are different, processing when a plurality of predetermined conditions are satisfied may be defined. For example, when a first predetermined condition is satisfied and a second predetermined condition is satisfied within a predetermined time, priority is given to the first predetermined condition or the second predetermined condition. A predetermined facial expression or pose corresponding to the satisfaction of a predetermined condition set to a high degree may be applied. Here, the priority may be stored in association with each predetermined condition, or may be set as a rule when a plurality of predetermined conditions are satisfied. For example, when there are A1, A2, A3, A4, etc. as the predetermined conditions, it is sufficient that the predetermined conditions that are given priority can be selected after comparing the respective predetermined conditions. The function using such priority has the advantage of being able to perform appropriate processing even when a plurality of predetermined conditions are satisfied, particularly when a plurality of predetermined conditions are satisfied almost simultaneously. Moreover, probability may be used for such priority. For example, when the probability is determined by a pseudo-random number function or the like, and when the first predetermined condition and the second predetermined condition are satisfied, the pseudo-random number function value is the predetermined value. , a predetermined facial expression corresponding to the first predetermined condition is applied, and in the case of another predetermined value, a predetermined facial expression corresponding to the second predetermined condition is applied. you can In this case, there is a possibility that the variation of production will increase, and the viewer's interest will be more attractive.

In addition, in the judgment of multiple conditions that can be satisfied, since the judgment is made after waiting for the passage of the predetermined time, if the predetermined time is set to a certain time or more, the predetermined expression etc. will be produced from the satisfaction of the predetermined condition A time lag may occur. Therefore, without setting such a priority, a predetermined facial expression or the like corresponding to a predetermined condition satisfied earlier in time may be set when the above-described plurality of conditions are satisfied. Here, "preceding in terms of time" means that even when a predetermined condition is substantially simultaneously established as a result of a network, information processing, etc., the processing unit first determines whether the predetermined condition is satisfied. good. In this case, there is an advantage that a predetermined facial expression or the like corresponding to the previously satisfied predetermined condition can be set without waiting for the passage of the predetermined time. In such a case, a predetermined facial expression or the like corresponding to a predetermined condition satisfied later may be set, or may not be set. When setting, the predetermined facial expression corresponding to the predetermined condition satisfied earlier is overwritten, but there is an advantage that the predetermined facial expression corresponding to the predetermined condition satisfied later can also be presented to the viewer. On the other hand, if it is not set, there is an advantage that the viewer can fully enjoy the preceding predetermined facial expressions. In this way, when both the first predetermined condition and the second predetermined condition are satisfied, the timing when the first predetermined condition is satisfied and the timing when the second predetermined condition is satisfied A predetermined facial expression or the like may be set accordingly. For example, if the time between the first timing when the first predetermined condition is satisfied and the second timing when the second predetermined condition is satisfied is shorter than the predetermined time, the first predetermined condition is met. The processing may be such that only the corresponding first predetermined facial expression or the like is produced, and the second predetermined facial expression or the like corresponding to the second prescribed condition is not produced. In this case, there is an advantage that viewers can enjoy the production of the first predetermined facial expression without changing it in the middle. On the other hand, if the time between the first timing and the second timing is longer than the predetermined time, the second predetermined condition is met at the timing when the second predetermined condition is satisfied. A predetermined facial expression or the like may be set. In this case, the viewer has the advantage of being able to enjoy the effects of both the first predetermined facial expression and the like, followed by the second predetermined facial expression and the like.

Note that the above-described predetermined time period for determining the time period between the first timing and the second timing may be set according to a set of predetermined conditions to be satisfied. For example, when a first predetermined condition is satisfied, a first predetermined facial expression or the like is produced, when a second predetermined condition is satisfied, a second predetermined facial expression or the like is produced, and a third predetermined facial expression or the like is produced. Suppose that a third predetermined facial expression or the like is produced when the condition of (1) is satisfied. Also assume that the first predetermined condition is satisfied at the first timing, the second predetermined condition is satisfied at the second timing, and the third predetermined condition is satisfied at the third timing. At this time, if the time between the first timing and the second timing is shorter than the twelfth predetermined time, the second predetermined expression or the like is not produced, and if the time is longer than the twelfth predetermined time. It is assumed that a second predetermined facial expression or the like is produced at . Further, when the time between the first timing and the third timing is shorter than the thirteenth predetermined time, the third predetermined expression or the like is not produced, and when the time is longer than the thirteenth predetermined time. Assume that a third predetermined facial expression or the like is produced. At this time, the twelfth predetermined time and the thirteenth predetermined time may be the same or different. If they are the same, after satisfying the first predetermined condition, even if the second predetermined condition is satisfied or the third predetermined condition is satisfied, the first predetermined facial expression or the like is produced. It means that time does not change. On the other hand, if different, after the first predetermined condition is satisfied, the first predetermined facial expression, etc., when the second predetermined condition is satisfied and when the third predetermined condition is satisfied. The performance time of will change. In this case, there is a possibility that the variation of production will increase, and the viewer's interest will be more attractive.

In the above description, priority is associated with a predetermined condition, or an example of a rule for a predetermined condition that is satisfied has been described. , status, etc. For example, when a gift is used under predetermined conditions, the gift may be associated with a priority.

The priority of things, events, and situations (sometimes referred to as "things, etc." in this application document) related to predetermined conditions is the size of things, etc. displayed, the place where things, etc. appear, the timing of the appearance of things, etc. , or the like.

For example, regarding the displayed size of an object, if the displayed size of a second object is larger than the size of the first object, such a second object may be A higher priority may be set than the first thing and the like. This is because, in this case, the larger displayed image attracts the viewer's attention more, and therefore, it is considered that prioritization is in line with the viewer's intention. On the other hand, the setting may be reversed. Conversely, if it is set, there is an advantage that unexpectedness appears.

In addition to the size of the item, the item may be the price if the item has a price, or the consumption point if the item has a consumption point. For example, if the price of the second item is higher than the price of the first item, etc., the second item is given higher priority than the first item. good. In this case, the higher the price, the higher the viewer's attention. On the other hand, the setting may be reversed. Conversely, if it is set, there is an advantage that unexpectedness appears. Similarly, with respect to consumption points, for example, if the consumption points of a second item or the like are higher than the consumption points of a first item or the like, the consumption points of the first item or the like are A higher priority may be set than the above. This is because, in this case, it is considered that priority is in line with viewers' intentions, since higher consumption points attract more attention from viewers. On the other hand, the setting may be reversed. Conversely, if it is set, there is an advantage that unexpectedness appears.

As for the place where objects appear in the virtual space, for example, objects appearing in the center of the virtual space may be given higher priority than objects appearing at the left and right ends of the virtual space. . When objects appearing in conspicuous places in the virtual space are given a higher priority, the viewer can see the objects appearing in conspicuous places. This has the advantage of making the meaning easier to understand.

In addition, the priority of an item may be static information that is preset for the item, or dynamic information that is set only when the item is provided. good. The latter may be, for example, information set according to the position where the object or the like appears. For example, the priority may be set according to the relationship between the position where the object appears in the virtual space and the position with respect to the avatar. In this case, the distance itself may play the role of priority, or the distance may be used as one piece of information for setting priority. For example, when the distance between the avatar and the position of the first object that appears is the first distance, and the distance between the avatar and the position of the second object that has appeared is the second distance, the first distance is the second distance. If it is greater than the distance, the first object may be given higher priority over the second object. In addition, when the first distance between the screen center and the position related to the first object is shorter than the second distance between the screen center and the position related to the second object, the first object Since it appears more in the center of the screen, the priority of the first object may be set higher than that of the second object. Dynamically setting the priority of objects has the advantage of being able to more appropriately and dynamically select a gift for which a predetermined facial expression or the like is set.

Also, the type to which the avatar belongs may be used as the information related to the avatar. For example, the predetermined condition may use the type to which the avatar belongs, which is one of a plurality of types to which the avatar belongs. This may be such that when certain information is acquired, the first avatar type will have a predetermined facial expression or the like, but the second avatar type will not have the predetermined facial expression or the like. For example, a certain type of avatar may be set to have a timid personality, and if the distance between the ghost character and the avatar is within the first distance range, a frightened expression may be likely to appear. Also, a certain type of avatar may be set to have a cheerful personality, and may be set such that the conditions for a facial expression including a smile are looser than other avatars as a predetermined facial expression. In this case, the avatar may be provided with character traits such as timid, cheerful, bullish, and a sore loser, and these character traits may be used to determine whether the predetermined condition is satisfied. The personality may be set according to the type to which the avatar belongs, or may be set for each avatar separately from the type to which the avatar belongs.

Note that the predetermined condition may include probability. That is, the predetermined condition may include a pseudo-random number function or the like, and a predetermined facial expression or the like may be generated when the function value satisfies the predetermined condition. When such a probability is included, a predetermined facial expression or the like is not always generated, so the viewer's interest can be continuously attracted. Also, for example, when the probability value is set to be lower than a predetermined value, a rare facial expression can attract the viewer's interest in that the viewer can see the valuable facial expression.

Further, as a predetermined condition, using the display timing of each object such as information on the start of display of the object, information on the change in the display of the object, and/or information on the end of the display of the object, etc., Satisfaction of conditions may be determined. In this case, since the expression and pose of the avatar change in response to the display of the object, such as the start, change, and end of the display of the object, etc., the viewer cannot understand the meaning of the change in the expression and pose of the avatar. There is an advantage that it becomes easier to understand by associating with the situation of things, etc.

Also, information that an object or the like has as an attribute may be used as the predetermined condition. The attributes of an entity may or may not include characteristics of the entity, such as the type of the entity and the level of the entity. Further, a predetermined condition may be set so that the predetermined condition is satisfied, and a corresponding predetermined facial expression or the like may be set according to a specific feature of the object or the like.

Also, the predetermined condition may define the order or timing of the acquired information.

For example, with respect to a predetermined facial expression when predetermined conditions and satisfaction are satisfied, a predetermined facial expression A is set when a bat character appears on the screen within a predetermined time period after a ghost character appears on the screen. A predetermined facial expression B may be set when the ghost character appears on the screen within a predetermined time after it appears on the screen.

The condition application unit may use the facial expression or the like specified by satisfying the predetermined condition as the facial expression or the like of the avatar to whom the gift is provided. More specifically, the information indicating the facial expression or the like specified by satisfying a predetermined condition may be an ID indicating the specified facial expression or the like, or one or more parameters of the facial expression indicating the specified facial expression or the like. The facial expression or the like of the avatar may be set using information indicating the specified facial expression or the like so that the specified facial expression or the like can be used as the facial expression of the avatar. After the ID and parameter information are specified or generated by the condition application unit, the information may be transmitted to the server or the terminal for the generation of video or animation, and may be used for a predetermined period of time. Here, the condition application unit may store information indicating a specified facial expression or the like in association with each predetermined condition, and when each predetermined condition is satisfied, the corresponding specified facial expression or the like may be stored. It may be configured such that the information to be indicated can be specified. Note that after a predetermined facial expression or the like applied by satisfying a predetermined condition is set to the avatar for a predetermined period of time, the original action of the avatar may be restored. That is, since the avatar is played by the distributor, the facial expression and pose of the avatar may be based on information on the distributor's facial expression and pose after a predetermined period of time. Here, "based on information on the facial expression and pose of the broadcaster" may mean that the action of the avatar is defined by information obtained by detecting the facial expression and pose of the broadcaster by an optical sensor or an inertial sensor.

In this process, if the first information processing device that executes the condition application unit is the same as the second information processing device that performs processing including setting of the facial expression of the avatar, the condition application unit performs A facial expression specified by satisfying a predetermined condition may be used to set the facial expression of the avatar.

In addition, when the first information processing device on which the condition application unit is executed is different from the second information processing device that performs processing including setting of the facial expression of the avatar, the condition application unit specified by satisfying the predetermined condition The first information processing device may transmit the facial expression or the like to the second information processing device, and the second information processing device may use the identified facial expression or the like to set the facial expression or the like of the avatar.

Here, when the first information processing device is a server and the second information processing device is a server, the server of the first information processing device transmits the specified facial expression, etc. to the server of the second information processing device. may be transmitted. Further, when the first information processing device is a server and the second information processing device is a viewer terminal, the server of the first information processing device sends the specified information to the viewer terminal of the second information processing device. Information indicating facial expressions and the like may be transmitted. Further, when the first information processing device is a server and the second information processing device is a distributor terminal, the server of the first information processing device sends the distributor terminal of the second information processing device to the specified Information indicating facial expressions and the like may be transmitted.

Similarly, when the first information processing device is a viewer terminal and the second information processing device is a server, the viewer terminal of the first information processing device sends the server of the second information processing device a specific Information indicating the facial expression or the like may be transmitted. Further, when the first information processing device is the viewer terminal and the second information processing device is the viewer terminal, the viewer terminal of the first information processing device is connected to the viewer terminal of the second information processing device. , information indicating the identified facial expression or the like may be transmitted. Further, when the first information processing device is the viewer terminal and the second information processing device is the distributor terminal, the viewer terminal of the first information processing device is sent to the distributor terminal of the second information processing device. , information indicating the identified facial expression or the like may be transmitted.

Similarly, when the first information processing device is a distributor terminal and the second information processing device is a server, the distributor terminal of the first information processing device sends the server of the second information processing device a specific Information indicating the facial expression or the like may be transmitted. Further, when the first information processing device is the distributor terminal and the second information processing device is the distributor terminal, from the distributor terminal of the first information processing device to the distributor terminal of the second information processing device , information indicating the identified facial expression or the like may be transmitted. Further, when the first information processing device is the distributor terminal and the second information processing device is the viewer terminal, from the distributor terminal of the first information processing device to the viewer terminal of the second information processing device, , information indicating the identified facial expression or the like may be transmitted.

In the above description, the condition application unit is executed in a single information processing device, but the condition application unit may be distributed among a plurality of information processing devices.

Also, the condition application unit may set the predetermined facial expression or the like using the timing at which the predetermined facial expression or the like is set. The timing may be, for example, the timing at which the line of sight of the avatar and the avatar or the character meet, the timing at which the avatar contacts an object, etc., as described above. In this case, information including information indicating such a predetermined facial expression may be transmitted to the displayed information processing device and displayed. The timing lag in this case may include a time lag in communication or processing.

On the other hand, when a predetermined condition is satisfied, the timing for setting a predetermined facial expression or the like may be set separately. For example, it may be possible that a predetermined facial expression or the like appears two seconds after a predetermined condition is set. As an example, it is possible to set a predetermined expression of surprise at a timing that is intentionally delayed two seconds after the occurrence of a situation that surprises the avatar.

FIG. 5 is an example showing the relationship between predetermined conditions and predetermined facial expressions. By using the acquired information as a predetermined condition, a corresponding predetermined facial expression or the like is specified.

As described above, the function of the condition application unit enables a corresponding predetermined facial expression, etc., to be realized when a predetermined condition is satisfied, which is an advantage in that variations can be given to the facial expressions and poses of the avatar in the virtual space. There is In addition, even if the performer can specify the facial expression of the avatar by himself using a tablet or the like, when the operator wears the attached gift in the studio distribution, the performer may not be aware that it has been attached. In such a case, there is an advantage that the expression or pose corresponding to the gift can be applied to the avatar without any operation by the performer. That is, the information for making a predetermined facial expression or pose may be generated without any operation other than providing the gift from the distributor.

In addition, as described above, when a predetermined condition is satisfied, the function of the condition application unit can automatically realize a corresponding predetermined facial expression, etc., and the facial expression of the avatar can be changed without any operation from the viewer terminal. It has the advantage of being able to enjoy the reaction that results in doing.

4. embodiment
4-1. Embodiment 1
Embodiment 1 is an example in which a condition application unit operates in a viewer terminal. In this case, the acquisition unit may operate in the viewer terminal. This example will be described with reference to FIG. In addition, since the condition application unit operates in the viewer terminal, it may be possible to change the expression of the avatar for each viewer terminal by including information based on the viewer terminal as a predetermined condition. .

step one
An acquisition unit acquires information such as a predetermined action (step 001). Information such as the predetermined action may be input at the viewer terminal in which the condition application unit operates, or information input at another viewer terminal or information generated in the server as an effect may be sent to the server. The viewer terminal in which the condition application unit operates may acquire the information via, for example.

step 2
Next, the condition applying unit determines a predetermined condition using at least information such as a predetermined motion, and if it determines that the predetermined condition is satisfied, specifies the facial expression or the like corresponding to the satisfied predetermined condition ( step 002). For this reason, the condition applying section may have information indicating at least a part of the predetermined condition and a predetermined facial expression or the like when the condition is satisfied.

step 3
The specified facial expression or the like is set as the facial expression or the like of the avatar used to satisfy the predetermined condition at the viewer terminal (step 003). The viewer has the advantage of being able to view the reaction of the avatar's facial expression or the like being changed in response to a predetermined action or the like.

step 4
After the predetermined facial expression etc. of the avatar is displayed for a predetermined period on each viewer terminal, the facial expression etc. of the avatar returns to the original facial expression etc. based on the distributor's facial expression etc. related to the avatar (step 004).

4-2. Embodiment 2
Embodiment 2 is an example in which a condition application unit operates in a server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 7 on the premise that animation is generated on the viewer terminal.

step one
An acquisition unit acquires information such as a predetermined action (step 001).

step 2
Next, the condition applying unit determines a predetermined condition using at least information such as a predetermined motion, and if it determines that the predetermined condition is satisfied, specifies the facial expression or the like corresponding to the satisfied predetermined condition ( step 002).

step 3
The server transmits information indicating the specified facial expression and the like to each viewer terminal viewing the distribution of the avatar (step 003).

step 4
At each viewer terminal, the information indicating the facial expression obtained from the server is used to set the facial expression of the avatar (step 004). Here, in each viewer terminal, an animation may be generated using the set facial expression of the avatar.

step 5
At each viewer terminal, the avatar's facial expression, etc., which uses information indicating facial expressions, etc. obtained from the server, is displayed for a predetermined period, and then the facial expression is set based on the original facial expression, etc. of the broadcaster associated with the avatar. (Step 005). In addition, when an animation is generated, it will be displayed using the generated facial expression, etc. for a predetermined period in each animation, and after that, it will be obtained by tracking based on the facial expression, etc. of the broadcaster related to the avatar. The facial expression or the like obtained by the user may be used as the facial expression or the like of the avatar.

4-3. Embodiment 3
Embodiment 3 is an example in which a condition application unit operates in a server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 8 on the premise that an image including facial expressions of the avatar is generated on the server.

step one
An acquisition unit acquires information such as a predetermined action (step 001).

step 2
Next, the condition applying unit determines a predetermined condition using at least information such as a predetermined motion, and if it determines that the predetermined condition is satisfied, specifies the facial expression or the like corresponding to the satisfied predetermined condition ( step 002).

step 3
The server uses the specified information indicating the facial expression and the like to generate a video regarding the distribution of the avatar (step 003). Here, if the server that generates the video for avatar distribution and the server that executes the condition application unit are the same server, the server that executes the condition application unit generates the video for avatar distribution. you can On the other hand, if the server that generates the video for the distribution of the avatar and the server that executes the condition application unit are different servers, the server that executes the condition application unit transfers the specified video to the server that generates the video. Information indicating the facial expression or the like may be transmitted, and the specified information indicating the facial expression or the like may be used in the server that generates the video. Similarly, when animation is generated on such a server, animation for distribution of such avatars may be generated together with or in place of the video, and in this case, the animation will be used together with or in place of the video thereafter. may be

step 4
An image of the avatar generated using the facial expression specified based on the information indicating the specified facial expression is transmitted to each viewer terminal (step 004).

step 5
On each viewer terminal, a video using information indicating facial expressions, etc. obtained from the server is displayed for a predetermined period, after which the video of the avatar based on the facial expressions, etc. of the distributor related to the original avatar is displayed. displayed (step 005).

4-4. Various Embodiments A computer program according to the first aspect is defined as "a computer program, which is executed by one or more processors to perform a predetermined action of an avatar performed by a distributor in a virtual space, the avatar the facial expression of the avatar, the facial color of the avatar, the voice of the avatar, or a predetermined situation that occurs with respect to the avatar, and obtains information on any one or more of them, and whether the obtained information satisfies a predetermined condition and if it is determined that the predetermined condition related to the acquired information is satisfied, the facial expression or pose of the avatar of the distributor generated based on the facial expression or pose of the distributor is changed to the generating information for making a predetermined facial expression or pose corresponding to predetermined conditions the facial expression of the avatar; and displaying the avatar to which the predetermined facial expression or pose is applied.

A computer program according to a second aspect is the computer program according to the first aspect, wherein "the predetermined action of the avatar is a predetermined action perceived by the avatar with the five senses if the avatar is a living creature." .

A computer program according to a third aspect is the computer program according to the first aspect or the second aspect, wherein "the predetermined action of the avatar includes an action of looking at a predetermined target".

A computer program according to a fourth aspect is a computer program according to any one of the first to third aspects, wherein "when the predetermined target is another avatar or character, the predetermined action of the avatar is the line of sight of the avatar. and the line of sight of the predetermined object."

A computer program according to a fifth aspect is a computer program according to any one of the first to fourth aspects, wherein "when the predetermined action includes an action of looking at the predetermined target, the predetermined condition is the predetermined condition. moving the line of sight of the avatar so that the target is in the direction of the line of sight of the avatar, or the predetermined target exists on the line of sight of the avatar.

A computer program according to a sixth aspect is the computer program according to any one of the first to fifth aspects, wherein "the predetermined action of the avatar includes an action of contacting a predetermined target."

A computer program according to a seventh aspect is a computer program according to any one of the first to sixth aspects, wherein "when the predetermined action includes an action of contacting the predetermined target, the predetermined condition is the avatar and that the distance between the position related to the object and the position related to the predetermined object is within a predetermined distance.

A computer program according to an eighth aspect is a computer program according to any one of the first to seventh aspects, wherein "the position related to the avatar includes the position of the avatar in the virtual space, the part that activates the predetermined facial expression, the It is either a predetermined part of the avatar or the position of the collider of said avatar.

A computer program according to a ninth aspect is the computer program according to any one of the first to eighth aspects, wherein "the predetermined action of the avatar includes an eating action of a predetermined target."

A computer program according to a tenth aspect is the computer program according to any one of the first to ninth aspects, wherein "the predetermined action of the avatar includes an odor or a predetermined gesture with respect to the odor."

A computer program according to an eleventh aspect is the computer program according to any one of the first to tenth aspects, wherein "the predetermined action of the avatar listens to a sound and/or includes a predetermined gesture of listening." .

A computer program according to a twelfth aspect is the computer program according to any one of the first to eleventh aspects, wherein "the predetermined condition uses detection of the predetermined gesture of the avatar."

A computer program according to a thirteenth aspect is the computer program according to any one of the first to twelfth aspects, wherein "the predetermined situation occurring with respect to the avatar includes occurrence of a predetermined sound in the virtual space."

A computer program according to a fourteenth aspect is a computer program according to any one of the first to thirteenth aspects, wherein "the predetermined situation occurring with respect to the avatar is the distance between the position of a predetermined object and the position of the avatar. is within a predetermined range."

A computer program according to a fifteenth aspect is a computer program according to any one of the first to fourteenth aspects, wherein "the predetermined situation occurring with respect to the avatar is a predetermined combination of a predetermined target and a predetermined pose of the avatar. “including becoming”.

A computer program according to a sixteenth aspect is the computer program according to any one of the first to fifteenth aspects, wherein "the facial color of the avatar includes a facial color change."

A computer program according to a seventeenth aspect is a computer program according to any one of the first to sixteenth aspects, wherein "the voice of the avatar changes the sound quality of the voice of the avatar by a predetermined range or more, or "Voice volume changes by more than a predetermined amount."

The computer program according to the eighteenth aspect is a computer program according to any one of the first to seventeenth aspects, wherein "within a predetermined time, the first predetermined condition is satisfied and the second predetermined condition is satisfied If the first predetermined condition or the second predetermined condition, a predetermined condition set with a higher priority is satisfied, a predetermined facial expression or pose corresponding to the condition is satisfied." be.

A computer program according to a nineteenth aspect is a computer program according to any one of the first to eighteenth aspects, wherein "the information for making the predetermined facial expression or pose is acquired from the distributor terminal for operating the avatar. is generated without

A computer program according to the twentieth aspect is a "computer program that, when executed by one or more processors, causes a given action of an avatar performed by a broadcaster, an expression of the avatar, an expression of the avatar, acquires information related to any one or more of the complexion of the avatar, the voice of the avatar, or a predetermined situation that occurs regarding the avatar, and determines whether the acquired information satisfies a predetermined condition; When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. generating information for making a pose, and transmitting the information for making the predetermined facial expression or pose."

A computer program according to a twenty-first aspect is the computer program according to the twentieth aspect, wherein "the predetermined action of the avatar is a predetermined action perceived by the avatar with the five senses if the avatar is a living creature." .

The server device according to the twenty-second aspect is "comprising one or more processors, the processors executing computer-readable instructions to perform a predetermined action of an avatar played by a distributor in a virtual space, Acquiring information related to any one or more of the facial expression of the avatar, the complexion of the avatar, the voice of the avatar, or a predetermined situation occurring with respect to the avatar, and the obtained information satisfies a predetermined condition. If it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar associated with the distributor, which has been generated based on the facial expression or pose associated with the distributor, is changed to the predetermined condition generates information for making a predetermined expression or pose corresponding to , and transmits the information for making the predetermined expression or pose.

A server apparatus according to a twenty-third aspect is the one according to the twenty-second aspect, wherein "the predetermined action of the avatar is a predetermined action that is perceived by the avatar with the five senses if the avatar is a living creature." .

A method according to the twenty-fourth aspect is defined as "a method executed by one or more processors executing computer-readable instructions, wherein, in a virtual space, a predetermined action of an avatar performed by a broadcaster, Acquiring information related to any one or more of facial expression, complexion of the avatar, voice of the avatar, or a predetermined situation occurring with respect to the avatar, and whether or not the acquired information satisfies a predetermined condition is determined, and if it is determined that a predetermined condition related to the acquired information is satisfied, the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, is changed to the predetermined generating information for making a predetermined facial expression or pose corresponding to the conditions of (1) to be the facial expression of the avatar, and displaying the avatar to which the predetermined facial expression or pose is applied.

A method according to a twenty-fifth aspect is the method according to the twenty-fourth aspect, wherein "the predetermined motion of the avatar is a predetermined motion perceived by the avatar with the five senses if the avatar is a living creature."

A method according to the twenty-sixth aspect is defined as "a method executed by one or more processors executing computer-readable instructions, wherein in a virtual space a predetermined action of an avatar performed by a broadcaster, Acquiring information related to any one or more of facial expression, complexion of the avatar, voice of the avatar, or a predetermined situation occurring with respect to the avatar, and whether or not the acquired information satisfies a predetermined condition and, if it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor corresponds to the predetermined condition. It generates information for making a predetermined facial expression or pose, and transmits the information for making the predetermined facial expression or pose.

A method according to a twenty-seventh aspect is the method according to the twenty-sixth aspect, wherein "the predetermined motion of the avatar is a predetermined motion perceived by the avatar with the five senses if the avatar is a living creature."

The viewer terminal device according to the twenty-eighth aspect is "comprising one or more processors, the processors executing computer-readable instructions to create a predetermined image of an avatar played by a broadcaster in a virtual space. acquiring information related to any one or more of a motion, facial expression of the avatar, complexion of the avatar, voice of the avatar, or a predetermined situation occurring with respect to the avatar, wherein the acquired information satisfies a predetermined condition; It is determined whether or not it is satisfied, and if it is determined that the predetermined condition related to the acquired information is satisfied, the facial expression or facial expression of the avatar of the distributor generated based on the facial expression or pose of the distributor generate information for making a pose a predetermined facial expression or pose corresponding to the predetermined condition as the facial expression of the avatar, and display the avatar to which the predetermined facial expression or pose is applied. .

A viewer terminal device according to a twenty-ninth aspect is the viewer terminal device according to the twenty-eighth aspect, wherein "the predetermined action of the avatar is a predetermined action that is perceived by the avatar with the five senses if the avatar is a living creature." is.

The distributor terminal device according to the thirtieth aspect "includes one or more processors, and the processor executes computer-readable instructions to create a predetermined image of an avatar played by the distributor in a virtual space. acquiring information related to any one or more of a motion, facial expression of the avatar, complexion of the avatar, voice of the avatar, or a predetermined situation occurring with respect to the avatar, wherein the acquired information satisfies a predetermined condition; If it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor generated based on the facial expression or pose of the distributor is changed to the predetermined condition. generates information for making a predetermined facial expression or pose corresponding to the conditions of (1), and transmits the information for making the predetermined facial expression or pose.

A distributor terminal device according to a thirty-first aspect is the distributor terminal device according to the thirtieth aspect, wherein "the predetermined action of the avatar is a predetermined action that is perceived by the avatar with the five senses if the avatar is a living creature." is.

Further, the systems and information processing described in this document may be implemented by software, hardware, or a combination thereof, and some or all of such systems and information processing may implement processes and procedures as computer programs. These computer programs may be executed by various computers, and may be stored in a storage medium. Also, these programs may be stored in a non-transitory or temporary storage medium.

What is described in this document is not limited to what is described in this document, and can be applied to various examples within the scope of various technical ideas having various technical advantages and configurations described in this document. It goes without saying that we can.

1 system 10 communication network 20 (20A to 20C) server device 30 (30A to 30C) terminal device 21 (31) arithmetic device 22 (32) main storage device 23 (33) input/output interface 24 (34) input device 25 ( 35) Auxiliary storage device 26 (36) Output device 41 Acquisition unit 42 Condition application unit

Claims (27)

A computer program,
by being executed by one or more processors,
In the virtual space, obtain information related to the predetermined action of the avatar played by the distributor,
Determining whether the acquired information satisfies a predetermined condition,
When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. Generate information for posing,
displaying the avatar with the predetermined facial expression or pose applied;
The predetermined action of the avatar is
an action in which a predetermined target enters the line-of-sight direction of the avatar;
Smells or predetermined gestures for odors, or
Predetermined gestures about hearing or listening to sounds,
including at least one of
A computer program that causes the one or more processors to function as a computer program. When the predetermined target is another avatar or character,
The predetermined action of the avatar includes matching the line of sight of the avatar with the line of sight of the predetermined target,
2. A computer program as claimed in claim 1 . When the predetermined action includes the action of looking at the predetermined target,
The predetermined condition includes moving the line of sight of the avatar so that the predetermined target is in the line-of-sight direction of the avatar, or that the predetermined target exists on the line of sight of the avatar.
2. A computer program as claimed in claim 1 . the predetermined action includes an action of contacting a predetermined object;
The predetermined condition includes that the distance between the position of the avatar and the position of the predetermined target is within a predetermined distance,
2. A computer program as claimed in claim 1 . The position related to the avatar is either the position of the avatar in the virtual space, the part that activates the predetermined expression, the predetermined part of the avatar, or the position of the collider of the avatar.
5. Computer program according to claim 4 . the predetermined condition uses detection of the predetermined gesture of the avatar;
2. A computer program as claimed in claim 1 . When a first predetermined condition is satisfied and a second predetermined condition is satisfied within a predetermined time, the priority of the first predetermined condition or the second predetermined condition is applying a predetermined facial expression or pose corresponding to the satisfaction of a predetermined condition set high;
7. A computer program according to any one of claims 1-6 . The information for making the predetermined facial expression or pose is generated without obtaining information for operating the avatar from the terminal used by the distributor,
A computer program according to any one of claims 1-7 . A memory that stores the operation result of the processor is used,
9. A computer program according to any one of claims 1-8. wherein the processor is a central processing unit (CPU), microprocessor or graphics processing unit (GPU);
10. Computer program according to any one of claims 1-9. the processor has computing capabilities,
Computer program according to any one of claims 1 to 10. A computer program,
by being executed by one or more processors,
In the virtual space, obtain information related to the predetermined action of the avatar played by the distributor,
Determining whether the acquired information satisfies a predetermined condition,
When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. Generate information for posing,
Sending information for making the predetermined facial expression or pose,
The predetermined action of the avatar is
an action in which a predetermined target enters the line-of-sight direction of the avatar;
Smells or predetermined gestures for odors, or
Predetermined gestures about hearing or listening to sounds,
including at least one of
A computer program that causes the one or more processors to function as a computer program. A memory that stores the operation result of the processor is used,
13. Computer program according to claim 12. wherein the processor is a central processing unit (CPU), microprocessor or graphics processing unit (GPU);
14. Computer program according to claim 12 or 13. the processor has computing capabilities,
15. A computer program according to any one of claims 12-14. comprising one or more processors;
The processor executing computer readable instructions to
In the virtual space, obtain information related to the predetermined action of the avatar played by the distributor,
Determining whether the acquired information satisfies a predetermined condition,
When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. Generate information for posing,
Sending information for making the predetermined facial expression or pose,
The predetermined action of the avatar is
an action in which a predetermined target enters the line-of-sight direction of the avatar;
Smells or predetermined gestures for odors, or
Predetermined gestures about hearing or listening to sounds,
including at least one of
A server device characterized by: A memory that stores the result of computation by the processor is used,
The server device according to claim 16. wherein the processor is a central processing unit (CPU), microprocessor or graphics processing unit (GPU);
The server device according to claim 16 or 17. the processor has computing capabilities,
The server device according to any one of claims 16 to 18. 1. A method performed by one or more processors executing computer readable instructions, comprising:
In the virtual space, obtain information related to the predetermined action of the avatar played by the distributor,
Determining whether the acquired information satisfies a predetermined condition,
When it is determined that a predetermined condition related to the acquired information is satisfied, the facial expression or pose of the avatar of the distributor generated based on the facial expression or pose of the distributor corresponds to the predetermined condition. generating information for making a predetermined facial expression or pose to be the facial expression of the avatar;
displaying the avatar with the predetermined facial expression or pose applied;
The predetermined action of the avatar is
an action in which a predetermined target enters the line-of-sight direction of the avatar;
Smells or predetermined gestures for odors, or
Predetermined gestures about hearing or listening to sounds,
including at least one of
A method characterized by: A memory that stores the operation result of the processor is used,
21. The method of claim 20. wherein the processor is a central processing unit (CPU), microprocessor or graphics processing unit (GPU);
22. A method according to claim 20 or 21. the processor has computing capabilities,
23. A method according to any one of claims 20-22. 1. A method performed by one or more processors executing computer readable instructions, comprising:
In the virtual space, obtain information related to the predetermined action of the avatar played by the distributor,
Determining whether the acquired information satisfies a predetermined condition,
When it is determined that the predetermined condition is satisfied, the facial expression or pose of the avatar of the distributor that has been generated based on the facial expression or pose of the distributor is changed to a predetermined facial expression or pose corresponding to the predetermined condition. Generate information for posing,
Sending information for making the predetermined facial expression or pose,
The predetermined action of the avatar is
an action in which a predetermined target enters the line-of-sight direction of the avatar;
Smells or predetermined gestures for odors, or
Predetermined gestures about hearing or listening to sounds,
including at least one of
A method characterized by: A memory that stores the operation result of the processor is used,
25. The method of claim 24. wherein the processor is a central processing unit (CPU), microprocessor or graphics processing unit (GPU);
26. A method according to claim 24 or 25. the processor has computing capabilities,
27. The method of any one of claims 24-26.