JP7202935B2 - Attention level calculation device, attention level calculation method, and attention level calculation program - Google Patents

Description

The present invention relates to a technique for calculating the degree of attention of a video viewer.

Conventionally, a camera capable of live streaming of omnidirectional video that allows the user to view the surroundings in 360 degrees is known. Live streaming of omnidirectional video is expected to be utilized in various fields such as so-called telexistence.

Conventionally, a function is known that allows multiple users to share an AR (Augmented Reality) experience. Patent Literature 1 discloses a system in which a plurality of users can share MR (Mixed Reality) images and perform collaborative work.

Further, Patent Literature 2 discloses that the line of sight of the user wearing the HMD and the point of interest of the user beyond the line of sight are specified from the position and orientation of the HMD (Head Mounted Display) in the real space. It is

Furthermore, in Patent Document 3, the degree of attention between presented images is measured based on the contraction of the subject's pupil, etc., and this degree of attention is learned as microscopic model information such as a neural network, and this model Based on the information, creating marketing content for viewers of predetermined attributes is described.

JP 2006-293604 A JP 2017-33299 A JP 2011-118558 A

An object of the present invention is to calculate the degree of attention of a guest who shares a video shot by a host.

A attention level calculation device according to a first aspect of the present invention includes an acquisition unit, a specification unit, and a calculation unit. The acquisition unit acquires the viewing behavior data of the guest based on the input acquired by the terminal of the guest sharing the video data of the exhibition captured by the host. The specifying unit specifies, based on the viewing behavior data, an object of interest of the guest among the plurality of exhibition objects at the exhibition. The calculation unit calculates the attention levels of the plurality of display objects so that the attention level of the display object specified as the target of the guest's attention is relatively increased.

Since the attention degree calculation device according to the first aspect calculates the attention degree of the exhibition target from the guests who virtually participate in the exhibition, it is an approach different from the questionnaire to the actual participants of the exhibition. , data related to interest in the exhibition subject can be collected.

In the attention level calculation device according to the first aspect, the image data may represent a panoramic image. The viewing behavior data may include viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time. The identification unit extracts the visual field image of the guest at the first reproduction time from the video data based on the reproduction time data and the visual field direction data, and identifies the object of the guest's attention at the first reproduction time from the subject of the visual field image. you can According to this attention level calculation device, it is possible to extract the visual field image of the guest at the reproduction time using the visual field direction data associated with the reproduction time, and specify the object that the guest paid attention to from the subject of the visual field image. can. In addition, since the visual field direction data is generally smaller in capacity than the image data such as the captured image, according to this attention degree calculation device, the amount of memory used and the amount of data communication related to the attention degree calculation can be saved. It is possible.

In the attention level calculation device according to the first aspect, the image data may represent a panoramic image. The viewing behavior data includes playback time data indicating a first playback time of the video data corresponding to when the guest's terminal captured the video data, and view direction data indicating the viewing direction of the guest at the first playback time. OK. The identification unit extracts the visual field image of the guest at the first reproduction time from the video data based on the reproduction time data and the visual field direction data, and identifies the object of the guest's attention at the first reproduction time from the subject of the visual field image. you can According to this attention level calculation device, the visual field image of the guest at the reproduction time is extracted by using the visual field direction data associated with the reproduction time when the guest performed the capture operation, and the subject of the visual field image is extracted. It is possible to specify the object that the guest has paid attention to. Since it can be estimated that there is a high possibility that the object that the guest has paid attention to is included in the visual field image of the guest when the guest performs the capture operation, according to this attention degree calculation device, it is possible to obtain a highly reliable degree of attention. can be calculated. In addition, since the visual field direction data is generally smaller in capacity than the image data such as the captured image, according to this attention degree calculation device, the amount of memory used and the amount of data communication related to the attention degree calculation can be saved. It is possible.

In the attention level calculation device according to the first aspect, the viewing behavior data includes a captured image of video data generated by a guest's terminal in response to a capture operation, and a first image of video data corresponding to when the capture operation was performed. and playback time data indicating the playback time. The specifying unit may specify the object that the guest paid attention to at the first reproduction time from the objects of the captured image. According to this attention level calculation device, it is possible to identify the target of the guest's attention from the subject of the image captured by the guest. Since it can be estimated that such a captured image is likely to include an object that the guest paid attention to, according to this attention degree calculation device, it is possible to calculate a highly reliable degree of attention.

In the attention level calculation device according to the first aspect, the viewing behavior data is a GUI (Graphical User Interface) part superimposed and displayed on the video represented by the video data on the terminal of the guest at the first reproduction time of the video data. It may include GUI selection data indicating the first GUI component selected by the guest, and playback time data indicating the first playback time. The identification unit may extract the visual field image of the guest from the video data based on at least the reproduction time data, and identify the target of the guest's attention at the first reproduction time based on the GUI selection data from the subject of the visual field image. According to this degree-of-attention calculation device, it is possible to extract the visual field image of the guest based on at least the reproduction time data, and to specify the target of the guest's attention from the subjects of the visual field image based on the GUI selection data. The GUI selection data indicates the GUI parts actively selected by the guest, and whether or not the guest is really interested in some or all of the exhibition objects included in the visual field image from the selected GUI parts. , or whether they want to see another exhibit as soon as possible. In other words, by specifying the target that the guest paid attention to in consideration of the GUI selection data in this way, it is possible to calculate the degree of attention with high reliability.

In the attention level calculation device according to the first aspect, the viewing behavior data includes viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time. OK. The specifying unit determines the visual field range of the guest at the first reproduction time based on the reference position/orientation data and the viewing direction data indicating the position of the host and the reference angle of the video data at the first reproduction time, and determines the determined visual field. The object of interest of the guest at the first playback time may be identified based on the range and map data that associates each of the plurality of objects on display with the position of the object on display.

According to this attention level calculation device, the visual field direction data associated with the reproduction time and the reference position/orientation data associated with the same reproduction time are used to determine the visual field range of the guest at the reproduction time. The target of the guest's attention can be identified based on the range and the map data. Note that since the reference position/orientation data and the map data are generally smaller in capacity than the video data, this attention level calculation device may save memory usage related to attention level calculation. .

In the attention level calculation device according to the first aspect, the image data may represent a panoramic image. The viewing behavior data includes playback time data indicating a first playback time of the video data corresponding to when the guest's terminal captured the video data, and view direction data indicating the viewing direction of the guest at the first playback time. OK. The specifying unit determines the visual field range of the guest at the first reproduction time based on the reference position/orientation data and the viewing direction data indicating the position of the host and the reference angle of the video data at the first reproduction time, and determines the determined visual field. The object of interest of the guest at the first playback time may be identified based on the range and map data that associates each of the plurality of objects on display with the position of the object on display.

According to this attention degree calculation device, by using the viewing direction data associated with the playback time when the guest performed the capture operation and the reference position/orientation data associated with the same playback time, the guest at the playback time is captured. , and based on this visual field range and the map data, the object focused on by the guest can be identified. Since it can be estimated that there is a high possibility that the target focused on by the guest is included in the visual field range of the guest when the guest performs the capture operation, according to this attention level calculation device, it is possible to obtain a highly reliable level of attention. can be calculated. Note that since the reference position/orientation data and the map data are generally smaller in capacity than the video data, this attention level calculation device may save memory usage related to attention level calculation. .

In the attention degree calculation device according to the first aspect, the calculation unit causes the degree of attention of the exhibition object specified as the object to which the guest has paid attention to increase according to the duration of the attention of the guest to the exhibition object. You may calculate the degree of attention of a plurality of display objects. According to this attention level calculation device, the longer the time that a guest gazes at an exhibition object, the more the attention degree of the exhibition object increases. It is possible to calculate the degree of attention by properly distinguishing the exhibition object that has been deprived of the for a long time.

A method for calculating the degree of attention according to a second aspect of the present invention comprises: obtaining, by a computer, guest viewing behavior data based on an input obtained at a terminal of a guest sharing video data of an exhibition captured by a host; , a computer identifying an object of interest of a guest among a plurality of exhibition objects in an exhibition based on viewing behavior data; and calculating attention levels of the plurality of display objects so as to increase exponentially.

This method of calculating the degree of attention to the exhibition target from the guests who virtually participate in the exhibition calculates the degree of attention to the exhibition target. Data related to interests can be collected.

A program for calculating the degree of attention according to a third aspect of the present invention comprises means for acquiring viewing behavior data of a guest based on an input acquired at a terminal of a guest who shares video data of an exhibition captured by a host by a computer, Means for identifying, based on the viewing behavior data, the target of the guest's attention among the plurality of exhibition targets at the exhibition; It functions as a means for calculating the degree of attention of the display object.

This attention level calculation program calculates the degree of attention to the exhibition target from guests who virtually participate in the exhibition. Data related to interests can be collected.

According to the present invention, it is possible to calculate the degree of attention of a guest who shares a video shot by a host.

1 is a block diagram illustrating a video sharing system including an attention level calculation server according to an embodiment; FIG. 2 is a block diagram illustrating a guest terminal in FIG. 1; FIG. 4 is a block diagram illustrating an attention level calculation server according to the embodiment; FIG. An explanatory view of a guest's field of view. The figure which illustrates a guest's field-of-view image. Explanatory drawing of the partition of a picture. FIG. 4 is a diagram illustrating GUI components displayed superimposed on a video; Explanatory drawing of attention degree calculation using map data. 4 is a flowchart illustrating the operation of the attention level calculation server of FIG. 3;

An embodiment will be described below with reference to the drawings. Elements that are the same as or similar to elements that have already been explained are denoted by the same or similar reference numerals, and overlapping explanations are basically omitted. For example, when there are a plurality of identical or similar elements, common reference numerals may be used to describe each element without distinction, and the common reference numerals may be used to distinguish and describe each element. In addition, branch numbers are sometimes used.

(embodiment)
In the attention degree calculation server according to the embodiment, a guest terminal (hereinafter referred to as a guest terminal) shares video data captured and posted by a host terminal (hereinafter referred to as a host terminal) via a video distribution server. can be incorporated into a video sharing system that FIG. 1 illustrates a video sharing system including an attention level calculation server 40 according to the embodiment.

The video sharing system of FIG. 1 includes a host terminal 10, a video distribution server 20, guest terminals 30-1, 30-2, . The video distribution server 20 in FIG. 1 is connected to some or all of the host terminal 10, the guest terminal 30, and the attention level calculation server 40 via a network such as the Internet, and can exchange data with each other. Also, the attention level calculation server 40 is connected to some or all of the host terminal 10, the video distribution server 20, and the guest terminals 30 via a network, and can exchange data with each other.

The host terminal 10, video distribution server 20, and guest terminal 30 have basic functions for exchanging video sharing services, as described below. In the following description, it is assumed that the video sharing system in FIG. 1 shares live video, that is, live streaming. It is possible to calculate degrees. Therefore, the embodiment can be applied not only to live video but also to video in general. In addition, it is assumed that the images are shot by the host at the exhibition. In other words, the guests share the video data of the exhibition captured by the host. Here, the exhibition is typically a trade fair for industry, but is not limited to this, artificial objects (for example, products, works, buildings, (virtual) objects, etc.), natural products (landscapes, human (including creatures, weather), performances of singing, dancing, playing, sports, etc. by people, or a combination of these. Specifically, exhibitions can include concerts, plays, sporting events, and the like.

The host terminal 10 sequentially encodes live video/audio data generated by, for example, a camera/microphone connected to the host terminal 10, and transmits (uploads) the encoded (video or multimedia) data to the video distribution server 20. )do. Here, the camera may be an omnidirectional or semispherical 360-degree camera or other panorama camera, or may be an ordinary monocular camera.

The host terminal 10 may be an electronic device such as a computer, for example, a mobile terminal (for example, a smartphone, a tablet, a laptop, etc.), a PC (Personal Computer), or the like, or may be a VR (Virtual Reality) having a communication function. )/AR/MR terminals such as, but not limited to, HMDs. When a head-mounted device such as an HMD is used as the host terminal 10, a monocular or transmissive device can be used to secure the host's field of view to some extent.

The video distribution server 20 receives the encoded data from the host terminal 10 and processes the data, such as re-encoding, as necessary. The video distribution server 20 distributes the (processed) encoded data to the guest terminal 30 .

The guest terminal 30 receives encoded data from the video distribution server 20, decodes it, and reproduces live video (which may include audio). It should be noted that, when reproducing a panoramic video including a 360-degree video, the guest terminal 30 detects, for example, the direction of the guest's head and/or line of sight, or the direction of the guest terminal 30, and reproduces based on this. The viewpoint of the video may be switched. Similarly, when reproducing a panoramic video, the guest terminal 30 detects, for example, an operation for switching viewpoints by the guest, such as a flick, stroke, touch, tap, swipe, drag, click, etc. You may switch the viewpoint of the image|video reproduced based on.

The guest terminal 30 is typically a combination of a display device such as a VR/AR/MR terminal such as an HMD with excellent immersive feeling, and a control device thereof such as a mobile terminal, a PC, a game machine, or the like. good. It should be noted that the display device and its control device need not be separated and may be integrated.

The guest can have an experience as if he/she is participating in the exhibition through the video imaged by the host in the remote place and reproduced by the guest terminal 30. - 特許庁Then, the attention degree calculation server 40 according to the embodiment collects the guest's viewing behavior data based on the input acquired by the guest terminal 30 during video reproduction, and based on this viewing behavior data, the plurality of exhibition objects in the exhibition is collected. Identify the target of the guest's attention. Further, the attention degree calculation server 40 calculates the degree of attention of these exhibition objects based on the object that the guest paid attention to.

In this way, the attention level calculation server 40 calculates the attention level of the guests who virtually participate in the exhibition. • Can collect data related to interests. It should be noted that this degree of attention can be expected to be utilized in various ways in combination with, for example, analysis results of conventional questionnaires, or in place of this. For example, the degree of attention is the most It can be utilized for, for example, requesting the host to photograph an exhibition object (object to be photographed) that the host may desire. For example, at a concert where people appreciate singing by multiple people, ranking these people based on the degree of attention of the guests, or ranking the people whose interests are strongly attracted to individual guests can also be identified based on In addition, as an example of utilization of such degree of attention, an individual guest may send a message from a person whose interest is strongly attracted to the guest.

It should be noted that the number of each device shown in FIG. 1 is merely an example. For example, the number of guest terminals 30 changes from moment to moment, so it can be 0, or it can be hundreds or thousands. Moreover, since the video sharing system of FIG. 1 can distribute a plurality of live videos in parallel, the number of host terminals 10 can be two or more.

The configuration of the guest terminal 30 will be described in detail below. The guest terminal 30 includes a communication I/F 100, a processor 110, a memory 120, an output device 130, and an input device 140, as illustrated in FIG.

The communication I/F 100 is a module for communicating with external devices such as the video distribution server 20 and the attention level calculation server 40 via a network. , a LAN (Local Area Network) terminal, and the like. The communication I/F can be, for example, a module for wide area communication such as mobile communication, a module for wireless/wired LAN, a module for Bluetooth (registered trademark), or the like.

Processor 110 is typically a CPU (Central Processing Unit) and/or GPU (Graphics Processing Unit), but may also be a microcomputer, FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), or other general-purpose or dedicated processor. processor or the like. The processor 110 performs, for example, input/output control, communication control, image/audio processing, generation of viewing behavior data to be described later, and the like.

Memory 120 temporarily stores programs executed by processor 110 and data used by processor 110, such as input data, video data, viewing behavior data, etc., for processor 110 to implement each process. do. The memory 120 can include a RAM (Random Access Memory) having a work area in which such programs/data are developed.

The output device 130 may include, for example, a display device that outputs video reproduced by the processor, a speaker that outputs audio reproduced by the processor, and the like. Output device 130 may be some or all of the elements of the HMD.

Input device 140 may include, for example, some or all of a sensor, touch screen, microphone, keyboard, mouse, and the like.

Here, the sensors may include, for example, motion sensors such as accelerometers, gyroscopes, magnetic sensors, and the like. Sensor data detected by the motion sensor can be used to estimate the guest's head and/or the orientation (eg, 3-axis tilt) of the guest terminal 30 . Specifically, at the time of playback of the panoramic video, the Yaw angle, Roll angle, and Pitch angle, which are the three-dimensional rotation angles of the guest's head and/or the guest terminal 30, are estimated based on this sensor data, Accordingly, the viewpoint of the video displayed on the guest terminal 30 can be controlled.

Sensors may also include so-called eye-tracking devices. The eye tracking device captures the movement of the guest's eyeballs and generates input data representing the guest's gaze direction. Such input data can also be used to control the viewpoint of the video displayed on the guest terminal 30 .

Processor 110 executes a program stored in memory 120 to obtain received data acquisition unit 111, decoder 112, video output unit 113, input data acquisition unit 114, viewing behavior data generation unit 115, and viewing behavior data generation unit 115 shown in FIG. It can function as the reporting unit 116 . Also, the memory 120 may include the video data storage unit 121, the viewing behavior data storage unit 122, and the input data storage unit 123 of FIG.

Communication I/F 100 includes a receiving unit 101 that receives various data from an external device via the network, and a transmitting unit 102 that transmits various data to the external device via the network. The receiving unit 101 receives the above-described encoded data from the video distribution server 20 and sends it to the received data acquisition unit 111 . The transmitting unit 102 also receives the viewing behavior data from the viewing behavior reporting unit 116 and transmits it to the attention level calculation server 40 .

The received data acquisition unit 111 acquires received data from the receiving unit 101 and writes it to the memory 120 . As an example, the reception data acquisition unit 111 acquires encoded data from the reception unit 101 and writes it to the video data storage unit 121 .

The video data storage unit 121 stores the encoded data acquired by the received data acquisition unit 111 . This encoded data includes at least encoded video data and may include encoded audio data. The encoded video data stored in the video data storage unit 121 can be read by various functional units of the processor 110 , such as the decoder 112 .

The decoder 112 reads the encoded video data from the video data storage unit 121 . Decoder 112 decodes the encoded video data according to an appropriate codec to reproduce the video data. Note that the decoder 112 may include an audio decoder in addition to the video decoder. The decoder 112 sends the reproduced video data to the video output unit 113 . Also, although decoder 112 is depicted in FIG. 2 as being part of the functionality of processor 110, it may be a dedicated circuit.

The video output unit 113 receives the video data from the decoder 112 timely, that is, before the output timing corresponding to the video data arrives, and sends it to the output device 130 . Note that the video output unit 113 may control the output of not only video data but also audio data.

If the video data represents a panoramic video, the video output unit 113 may control the viewpoint of the reproduced video. For example, the video output unit 113 outputs input data representing the direction of the guest's head and/or line of sight stored in the input data storage unit 123, or the posture of the guest terminal 30, and/or the input data for switching the viewpoint by the guest. The viewpoint of the video to be played may be determined based on the input data representing the manipulation. Then, the video output unit 113 may generate video data corresponding to the determined viewpoint based on the video data from the decoder 112 and send it to the output device 130 .

In addition, the video output unit 113 adds GUI (Graphical User Interface) parts and/or displays for clearly indicating partitions (for example, partition boundaries and/or partition labels) to the video data as necessary. may be added before being sent to the output device 130 .

The output device 130 receives video data from the video output unit 113 and displays it. Note that the output device 130 may output audio data in addition to displaying video data.

The input device 140 generates input data based on the input at the guest terminal 30 and sends it to the input data acquisition section 114 . Here, the input data is, for example, data representing the direction of the guest's head and/or line of sight, or the direction of the guest terminal 30, and the guest's operation on the displayed image of the guest terminal 30 (for example, capture operation, viewpoint switching, etc.). operation, operation to select a GUI component superimposed on the video, operation to select a partition of the video, etc.), and playback indicating the playback time of the video data corresponding to the generation of such data. time data.

Note that the guest terminal 30 may optionally implement a capture function. When the guest terminal 30 implements a capture function, the processor 110 captures an image displayed on a display device included in the output device 130 according to input data representing, for example, a capture operation, and generates a capture image. It can function as a capture unit (not shown) for

The input data acquisition unit 114 receives various input data from the input device 140 and writes them to the input data storage unit 123 .

The input data storage unit 123 stores the input data acquired by the input data acquisition unit 114 . The input data stored in the input data storage unit 123 are read by various functional units of the processor 110 , such as the viewing behavior data generation unit 115 .

The viewing behavior data generation unit 115 reads the input data from the input data storage unit 123, and based on this, generates viewing behavior data representing the conscious or unconscious behavior of the guest when viewing the video. The viewing behavior data generation unit 115 writes the generated viewing behavior data to the viewing behavior data storage unit 122 .

Specifically, when the video data is a panoramic video, that is, a video in which the viewpoint can be switched, the viewing behavior data generation unit 115 generates viewing direction data representing the guest's viewing direction at a certain point in time and Viewing behavior data including playback time data indicating the playback time of the video data to be played may be generated. Here, the certain point in time may be, for example, a predetermined timing, or may be, for example, a timing at which the guest performs a predetermined input such as a capture operation.

The viewing behavior data generation unit 115 can generate view direction data based on data representing the direction of the guest's head and/or line of sight, or the direction of the guest terminal 30, which can be included in the input data, for example. As an example, the viewing behavior data generation unit 115 generates the visual field of the guest as illustrated in FIG. B) is divided into the four viewing areas, it is determined which viewing area the direction of the guest's head and/or line of sight or the direction of the guest terminal 30 corresponds to, and the index representing the determined viewing area is assigned to the viewing area. You may generate|occur|produce as direction data. Note that FIG. 4 is merely an example, and the guest's field of view may be divided into three or less visual field areas, or may be divided into five or more visual field areas. Also, the field of view area may or may not match the unit for switching the video viewpoint.

As will be described later, by combining the viewing direction data, the playback time data, and the video data, it is possible to reproduce the guest's view image at the playback time indicated by the playback time data to some extent. This field-of-view image may include the subject of the guest's attention. For example, the field-of-view image illustrated in FIG. You may be interested in some or all of them.

Further, when the video data is a panoramic video or a normal video, that is, a video in which viewpoint switching is impossible, the viewing behavior data generation unit 115 generates a captured image generated according to a capture operation by the guest, Viewing behavior data including reproduction time data indicating the reproduction time of video data corresponding to when the capture operation was performed may be generated. Although the captured image has a larger data volume than the viewing direction data, it is the guest's visual field image itself when the capture operation is performed, so processing to reproduce the visual field image is unnecessary, and the visual field resulting from such processing is unnecessary. There is an advantage that no error occurs. In addition, it can be estimated that there is a high possibility that the captured image that the guest actively performs the capture operation and saves includes the target of the guest's attention.

Note that the video may be divided into a plurality of partitions and displayed, as shown in FIG. 6, for example. In the example of FIG. 6, the video is partitioned into three partitions L (left), C (middle) and R (right). If the guest performs an operation to select one of the partitions, it can be inferred that there is a high possibility that the target of interest of the guest is included in that partition. For example, in the example of FIG. 6, if the guest selects partition L, it can be estimated that the guest is interested in exhibit 301-1 and/or booth 301, and if the guest selects partition C, the guest is interested in exhibit 301-2, exhibit 301-2 and/or booth 301. It can be inferred that there is an interest in the subject. However, FIG. 6 is merely an example, and the video may be divided into two or less partitions, or four or more partitions. Also, for example, a partition corresponding to each object may be determined using an object recognition technique or the like.

Therefore, the viewing behavior data generation unit 115 generates viewing direction data and/or a captured image when a guest selects a partition, partition selection data indicating the selected partition, and partition selection data corresponding to when the partition is selected. Viewing behavior data including reproduction time data indicating the reproduction time of video data may be generated.

Furthermore, as shown in FIG. 7, for example, GUI parts can be superimposed and displayed on the video. A guest who has selected the GUI component 401 in FIG. 7 is interested in at least part of the booth 301 and the exhibits 301-1, 301-2, and 301-3, which are the subjects of the currently displayed video. can be estimated. On the other hand, the guest who selected the GUI component 402 in FIG. 7 is interested in at least part of the booth 301 and the exhibits 301-1, 301-2, and 301-3, which are the subjects of the currently displayed video. or it can be presumed that they are interested in other booths and/or exhibits.

Therefore, the viewing behavior data generation unit 115 indicates GUI selection data indicating the GUI component selected by the guest from among the GUI components superimposed on the video and the playback time of the video data corresponding to the selection. viewing behavior data including playback time data may be generated. Note that when the video data is a panoramic video, the viewing behavior data generation unit 115 generates the viewing direction data and/or the captured image when the GUI component is selected so that the guest's viewing image can be reproduced. Must be included in viewing behavior data.

The viewing behavior data storage unit 122 stores the viewing behavior data generated by the viewing behavior data generation unit 115 . The viewing behavior data stored in the viewing behavior data storage unit 122 is read by various functional units of the processor 110 , such as the viewing behavior reporting unit 116 .

The viewing behavior reporting unit 116 reads the viewing behavior data from the viewing behavior data storage unit 122 and reports it to the attention level calculation server 40 . Specifically, the viewing behavior reporting unit 116 sends viewing behavior data to the transmitting unit 102, and the transmitting unit 102 transmits this viewing behavior data to the attention level calculation server 40 via the network.

Next, the configuration of the attention level calculation server 40 will be described in detail. The attention level calculation server 40 includes a communication I/F 200, a processor 210, and a memory 220, as illustrated in FIG.

The communication I/F 200 is a module for communicating with external devices such as the host terminal 10, the video distribution server 20, and the guest terminal 30 via a network. , optical connectors, and the like. Communication I/F 200 may be, for example, an optical communication module.

Processor 210 may be a general purpose or special purpose processor, or the like. The processor 210 performs, for example, input/output control, communication control, identification of a target of interest and calculation of the degree of attention, which will be described later.

The memory 220 temporarily stores programs executed by the processor 210 and data used by the processor 210, such as viewing behavior data, target data, attention level, etc., so that the processor 210 realizes each process. Store. Additionally, memory 220 may store video data and/or store reference pose data and map data, described below, to reproduce the guest's view image associated with the viewing behavior data. Memory 220 may include RAM having a work area in which such programs/data are deployed.

By executing the programs stored in memory 220, processor 210 can function as received data acquisition unit 211, attention target identification unit 212, attention level calculation unit 213, and attention level reporting unit 214 in FIG. The memory 220 may also include the viewing behavior data storage unit 221, the attention level storage unit 222, and the target data storage unit 223 of FIG.

Communication I/F 200 includes a receiving section 201 that receives various data from an external device via the network, and a transmitting section 202 that transmits various data to the external device via the network. The reception unit 201 receives viewing behavior data from the guest terminal 30 and sends it to the reception data acquisition unit 211 . The receiving unit 201 can receive (encoded) video data and/or reference position/orientation data from the host terminal 10 and/or the video distribution server 20 and send it to the received data acquisition unit 211 . Here, the reference position/orientation data indicates the position of the host and the reference angle of the video data at a given time. Further, the transmission unit 202 receives attention levels from the attention level reporting unit 214 and transmits them to, for example, a not-shown attention level utilization server.

Here, the attention degree utilization server receives the attention degree from the attention degree calculation server 40 via the network and utilizes it. Specifically, the attention level utilization server provides the feedback information created based on the attention level to exhibitors of the exhibition or the organizer of the exhibition, and provides feedback information to guests or other (potential) consumers. It supports marketing based on the degree of attention, or decides the object to be photographed based on the degree of attention and requests the host to photograph the object to be photographed.

The received data acquisition unit 211 acquires the received data from the receiving unit 201 and writes it to the memory 220 . As an example, the received data acquisition unit 211 acquires viewing behavior data from the receiving unit 201 and writes it to the viewing behavior data storage unit 221 . Also, the received data acquisition unit 211 can acquire (encoded) video data and/or reference position/orientation data from the reception unit 201 and write it to the memory 120 .

The viewing behavior data storage unit 221 stores the viewing behavior data acquired by the received data acquisition unit 211 . The viewing behavior data stored in the viewing behavior data storage unit 221 can be read by various functional units of the processor 210 , for example, the attention target identification unit 212 .

The viewing behavior data may be accumulated without distinguishing between the host and the guest, or may be accumulated by distinguishing between the host and the guest. That is, the viewing behavior data generated when viewing a video transmitted by a certain host terminal 10 and the viewing behavior data generated when viewing a video transmitted by another host terminal 10 may be distinguished. In this case, the viewing behavior data may include a host identifier indicating from which host terminal 10 the viewing behavior data was generated when the video was viewed. Also, the viewing behavior data generated by the guest terminal 30 of a certain guest and the viewing behavior data generated by the guest terminal 30 of another guest may be distinguished. In this case, the viewing behavior data may include a guest identifier indicating which guest terminal 30 generated the viewing behavior data.

The object-of-interest specifying unit 212 reads viewing behavior data from the viewing behavior data storage unit 221 and reads reference data of a plurality of display objects in the exhibition from the memory 220 . Here, the reference data may include, for example, the appearance of the exhibition object (which may include the logo and/or trademark of the exhibition object and/or the exhibitor to which it belongs, the face of the person as the exhibition object, etc.), the exhibition A code assigned to the object (e.g., one-dimensional code, two-dimensional code, etc.), text associated with the object to be exhibited (e.g., name, nickname, identification number, description, etc.), position of the object to be exhibited, etc. . Note that the reference data indicating the positions of a plurality of exhibits can also be called map data. Based on the viewing behavior data and the reference data, the target-of-interest identifying unit 212 identifies the target of the guest's attention at the playback time of the video data indicated by the playback time data included in the viewing behavior data. Then, the target-of-interest specifying unit 212 generates, for example, target-of-interest data including this reproduction time data and an exhibition target identifier for identifying the specified target, and writes this to the target-of-interest data storage unit 223 .

It should be noted that if the viewing behavior data includes the aforementioned host identifier and/or guest identifier, the interest data may also include this. Also, the host identifier and/or guest identifier may each be translated into attribute identifiers that identify one or more attributes, such as gender, age, preferences, skills, and the like. Note that the rules for converting host identifiers and/or guest identifiers into attribute identifiers may be created by, for example, a server that manages user information in the video sharing system of FIG. For example, a guest identifier may be converted to attribute identifiers that identify attributes such as "male", "30's", "early adopter", "expert", and included in the data of interest.

Specifically, the target-of-interest identifying unit 212 can identify the target of the guest's attention by various techniques as listed below.

(1) Suppose that the viewing behavior data includes the viewing direction data and the playback time data described above, and video data (representing a panoramic video) is stored in the memory 220 . In this case, the target-of-interest identification unit 212 extracts the visual field image of the guest at the reproduction time indicated by the reproduction time data from the video data based on the reproduction time data and the visual field direction data. For example, the target-of-interest identification unit 212 extracts one panorama image corresponding to the reproduction time indicated by the reproduction time data from the video data, and extracts one viewpoint corresponding to the viewing direction indicated by the viewing direction data from this panoramic image. can be extracted as the guest's view image.

Then, the target-of-interest specifying unit 212 specifies the target of the guest's attention at the playback time indicated by the playback time data from the subject of the field-of-view image. Specifically, the target-of-interest identifying unit 212 compares the appearance of the subject with the appearance of a plurality of exhibits included in the reference data, extracts a text area or code area included in the subject, and refers to the extracted area. Determine whether the subject included in the view image corresponds to any of the multiple exhibition objects by comparing the text associated with the multiple exhibition objects included in the data or the code assigned to the exhibition object. can.

(2) Assume that the viewing behavior data includes the aforementioned captured image and playback time data. In this case, the target-of-interest specifying unit 212 specifies the target of the guest's attention at the playback time indicated by the playback time data, from the subject of the captured image. Specifically, the subject-of-interest specifying unit 212 can determine whether or not the subject included in the captured image corresponds to any one of the plurality of exhibits, using the same technique as in (1) above.

(3) Assume that the viewing behavior data includes the aforementioned GUI selection data and reproduction time data. Note that when the viewing behavior data further includes a captured image, or when video data (representing a normal video) is stored in the memory 220, the target-of-interest identifying unit 212 selects the captured image or the video. Of the data, one image corresponding to the reproduction time indicated by the reproduction time data can be treated as the guest's visual field image. On the other hand, when the viewing behavior data further includes view direction data and video data (representing a panoramic video) is stored in the memory 220, the target-of-interest identifying unit 212 performs the same operation as in (1) above. Techniques can extract a guest's view image.

Then, the object-of-interest specifying unit 212 extracts one of the plurality of display objects from the subject of the field-of-view image by the same technique as in (1) above, and further extracts the extracted display object based on the GUI selection data. By narrowing it down with the key, the target that the guest paid attention to at the playback time indicated by the playback time data is specified. For example, when the GUI selection data indicates that a GUI component such as "right exhibit" has been selected, the target-of-interest identifying unit 212 selects the exhibit on the right side of the field-of-view image as the target of the guest's attention. can be identified as Also, when the GUI selection data indicates that the GUI component 401 of FIG. 7 is selected, for example, the target-of-interest identifying unit 212 can specify the exhibition target included in the field-of-view image as the target of the guest's attention. On the other hand, if the GUI selection data indicates that, for example, the GUI component 402 of FIG. 7 has been selected, the target-of-interest identifying unit 212 can exclude the exhibit target included in the view image from the target of the guest's attention.

(4) Assume that the viewing behavior data includes the partition selection data and playback time data described above. Note that when the viewing behavior data further includes a captured image, or when video data (representing a normal video) is stored in the memory 220, the target-of-interest identifying unit 212 selects the captured image or the video. Of the data, one image corresponding to the reproduction time indicated by the reproduction time data can be treated as the guest's visual field image. On the other hand, when the viewing behavior data further includes view direction data and video data (representing a panoramic video) is stored in the memory 220, the target-of-interest identifying unit 212 performs the same operation as in (1) above. Techniques can extract a guest's view image.

Then, the target-of-interest identifying unit 212 further narrows down the field-of-view image based on the partition selection data. For example, if the partition selection data indicates that the partition L in FIG. 6 has been selected, the target-of-interest identifying unit 212 can trim the portion corresponding to this partition L from the VF image. Then, the subject-of-interest identification unit 212 can determine whether or not the subject included in the field-of-view image after narrowing down corresponds to any one of the plurality of display subjects by the same technique as in (1) above.

(5) Assume that the viewing behavior data includes the viewing direction data and playback time data described above, and that the memory 220 stores map data and reference position/posture data. Based on the reproduction time data and the reference position/orientation data, the attention target identification unit 212 determines the position of the host and the reference angle of the video data at the reproduction time indicated by the reproduction time data. Furthermore, the attention target specifying unit 212 rotates the determined reference angle of the video data according to the viewing direction indicated by the viewing direction data, and estimates the viewing range of the guest at the playback time indicated by the playback time data. The map data associates multiple exhibit objects in the exhibition with their locations. Then, the target-of-interest identifying unit 212 identifies the exhibition target exhibited within the estimated visual field range of the guest as the target that the guest paid attention to at the reproduction time indicated by the reproduction time data. Specifically, as exemplified in FIG. 8, the target-of-interest identification unit 212 estimates a guest's visual field range 502 centering on the host's position 501, and the booth A01 exhibited within this visual field range 502. , and/or the exhibit within the booth A01 may be identified as the subject of the guest's attention.

According to the technique of (5) above, the usage of the memory 220 can be reduced compared to the techniques of (1), (3) to (4) above, which require a large amount of video data to be stored in the memory 220. potential savings. In addition, according to the technique (5) above, by using the same map data when calculating attention levels related to videos of other hosts participating in the same exhibition, the usage of the memory 220 can be further reduced. . Alternatively, in the above techniques (1), (3) to (4), the attention level calculation server 40 sends an image corresponding to the reproduction time indicated by the reproduction time data of the video data to the host terminal 10 or the video distribution server 20. You may request it.

The attention target data storage unit 223 stores the attention target data generated by the attention target specifying unit 212 . The attention target data stored in the attention target data storage unit 223 is read by various functional units of the processor 210 , for example, the attention degree calculation unit 213 . Note that the data of interest may include, for example, playback time, guest identifier, host identifier, and/or attribute identifier as keys, and may be selectively readable that matches a desired key. .

The attention degree calculation unit 213 reads attention object data from the attention object data storage unit 223 and calculates the attention degree of a plurality of exhibition objects based on the attention object data. Then, the attention degree calculation unit 213 writes the calculated attention degree to the attention degree storage unit 222 .

The attention level calculation unit 213 can calculate the attention levels of a plurality of display objects such that the attention levels of the objects indicated by the attention object data are relatively increased. Specifically, the attention level calculation unit 213 calculates the attention level of the object indicated by the attention object data obtained for a certain guest so that the attention degree of the object is increased according to the duration of the attention of the guest to the object. Attention levels of a plurality of display objects may be calculated. For example, the attention level calculation unit 213 counts the number of appearances of the display target in the attention target data for each display target over a plurality of playback times, and calculates the attention level of the display target based on this tally result. do. In addition, the attention level calculation unit 213 may discard the series of attention target data when the duration of attention to a certain object indicated by the series of attention target data obtained for a certain guest is less than a threshold. . In other words, when the duration of attention to an object indicated by a series of attention target data obtained for a certain guest exceeds a threshold, the attention level calculation unit 213 calculates the attention level based on the series of attention target data. may be calculated.

The attention level calculation unit 213 may calculate the attention level for each guest/attribute. That is, the attention degree calculation unit 213 reads attention object data including a certain guest identifier/attribute identifier from the attention object data storage unit 223, and for each exhibition object, calculates the number of appearances of the exhibition object in the read attention object data as a plurality of numbers. It is also possible to calculate the degree of attention of the guest indicated by the guest identifier/attribute value indicated by the attribute identifier to the exhibition object based on the tabulated result.

Also, the attention level calculation unit 213 may calculate the attention level for each host. That is, the attention level calculation unit 213 reads attention object data including a certain host identifier from the attention object data storage unit 223, and calculates the number of appearances of the exhibition object in the read attention object data at a plurality of reproduction times for each exhibition object. It is also possible to calculate the degree of attention of the guest who views the image taken by the host indicated by the host identifier to the exhibition object based on the result of the aggregation.

Furthermore, the attention level calculation unit 213 may calculate the overall attention level in the exhibition. That is, the attention degree calculation unit 213 reads out all the attention object data from the attention object data storage unit 223, and aggregates the number of appearances of the exhibition object in the read attention object data over a plurality of reproduction times for each exhibition object. Then, the degree of attention of all the guests who virtually participate in the exhibition to the exhibition object may be calculated based on the total result.

The attention level storage unit 222 stores the attention level calculated by the attention level calculation unit 213 . The prominences stored in prominence storage unit 222 are read by various functional units of processor 210 , such as prominence reporting unit 214 . Note that the degree of attention may include, for example, the desired playback time, guest identifier, host identifier, and/or attribute identifier as keys, and may selectively read those that match the desired key. good.

The attention degree reporting unit 214 reads the attention degree from the attention degree storage unit 222 and reports it to, for example, an attention degree utilization server (not shown). Specifically, the attention degree reporting unit 214 sends the attention degree to the transmission unit 202, and the transmission unit 202 transmits this attention degree to the attention degree utilization server via the network.

An operation example of the attention level calculation server 40 will be described below with reference to FIG. The operation of FIG. 9 starts at step S601.
In step S<b>601 , the receiving unit 201 waits to receive viewing behavior data from the guest terminal 30 . When the reception unit 201 receives the viewing behavior data, the process proceeds to step S602. On the other hand, if the attention degree calculation timing arrives before the receiving unit 201 receives the viewing behavior data, the process proceeds to step S606 (step S605).

Here, the attention degree calculation timing can be determined arbitrarily. For example, when the attention level is calculated in real time during an exhibition, the attention level calculation timing may be determined, for example, every minute, every 10 minutes, every hour, or the like. Alternatively, the attention level calculation timing may be set after the end of a half-day or one-day schedule of the exhibition, or after the end of the entire schedule of the exhibition, for example.

In step S<b>602 , the received data acquisition unit 211 acquires the viewing behavior data received by the receiving unit 201 and writes it to the viewing behavior data storage unit 221 . Next, based on the viewing behavior data acquired in step S602, the target-of-interest identifying unit 212 identifies the target that the guest paid attention to at the playback time indicated by the playback time data included in the viewing behavior data. is generated (step S603).

Note that steps S602 and S603 do not necessarily have to be performed continuously, and step S603 may be performed, for example, each time step S602 is performed a plurality of times or at predetermined intervals. In this case, after step S602, the process may return to step S601 instead of step S603, or may proceed to step S605.

The target specifying unit 212 writes the target data generated in step S603 to the target data storage unit 223 (step S604), and the process proceeds to step S605. After step S604, the process may return to step S601 instead of step S605.

In step S<b>606 , the attention level calculation unit 213 calculates the attention levels of the plurality of exhibition objects based on the attention object data stored in the attention object data storage unit 223 in step S<b>604 , and stores them in the attention degree storage unit 222 . .

Then, the attention degree reporting unit 214 reports the attention degree calculated in step S606 to, for example, an attention degree utilization server (not shown) (step S607), and the operation in FIG. 9 ends.

Note that steps S606 and S607 do not necessarily have to be performed continuously, and step S607 may be performed, for example, each time step S606 is performed a plurality of times or at a predetermined cycle. In this case, after step S606, the process may return to step S601 instead of step S607, or may return to step S605.

As described above, the attention degree calculation server according to the embodiment acquires viewing behavior data based on the input acquired in the terminal of the guest sharing the video data of the exhibition shot by the host, Identify the target of the guest's attention among the multiple display targets at the exhibition. Then, this attention level calculation server calculates the attention levels of the plurality of exhibition objects so that the attention degree of the exhibition object specified as the object of the guest's attention is relatively increased. Therefore, since this attention level calculation server calculates the degree of attention to the exhibition target from the guests who virtually participate in the exhibition, it is possible to use an approach different from a questionnaire to the actual participants of the exhibition. It is possible to collect data related to interest in

(Modification)
In the description of the embodiment, the attention level calculation server calculates the attention level, but the attention level may be calculated by a guest terminal, another computer, or the like. A device that calculates the degree of attention, such as an attention degree calculation server and a guest terminal, can be called an attention degree calculation device. Also, the calculation of the degree of attention and the identification of the target of interest may be performed by different devices.

The above-described embodiments merely show specific examples to aid understanding of the concept of the present invention, and are not intended to limit the scope of the present invention. Embodiments can add, delete, or convert various components without departing from the gist of the present invention.

Although some functional units have been described in the above-described embodiments, these are only examples of implementation of each functional unit. For example, a plurality of functional units described as being implemented in one device may be implemented across a plurality of separate devices, and conversely, may be implemented across a plurality of separate devices. It is also conceivable that the functional units described may be implemented in a single device.

Various functional units described in the above embodiments may be realized by using circuits. A circuit may be a dedicated circuit that implements a specific function, or it may be a general-purpose circuit such as a processor.

At least part of the processing of each of the above-described embodiments can also be realized by using a processor installed in a general-purpose computer as basic hardware. A program that implements the above process may be provided by being stored in a computer-readable recording medium. The program is stored in the recording medium as an installable format file or an executable format file. Recording media include magnetic disks, optical disks (CD-ROM, CD-R, DVD, etc.), magneto-optical disks (MO, etc.), semiconductor memories, and the like. Any recording medium may be used as long as it can store the program and is readable by a computer. Alternatively, the program that implements the above processing may be stored on a computer (server) connected to a network such as the Internet, and downloaded to the computer (client) via the network.

DESCRIPTION OF SYMBOLS 10... Host terminal 20... Video distribution server 30, 30-1, 30-2... Guest terminal 40... Attention degree calculation server 100, 200... Communication I/F
101, 201... Receiving unit 102, 202... Transmitting unit 110, 210... Processor 111, 211... Received data acquisition unit 112... Decoder 113... Video output unit 114... Input Data acquisition unit 115 Viewing behavior data generation unit 116 Viewing behavior reporting unit 120, 220 Memory 121 Video data storage unit 122, 221 Viewing behavior data storage unit 123 Input data storage unit 130 Output device 140 Input device 212 Attention target identification unit 213 Attention degree calculation unit 214 Attention degree reporting unit 222 Attention degree storage unit 223. .. Target data storage unit 301, 302 .. Booth 301-1, 301-2, 301-3, 302-1 .・・・Guest's field of view

Claims (11)

an acquisition unit that acquires the viewing behavior data of the guest based on the input acquired in the terminal of the guest sharing the video data of the exhibition captured by the host;
an identification unit that identifies, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects in the exhibition;
a calculation unit that calculates the degree of attention of the plurality of exhibition objects so that the degree of attention of the exhibition object specified as the object of attention of the guest increases relatively,
The viewing behavior data includes a first GUI component selected by the guest from GUI (Graphical User Interface) components superimposed and displayed on the video represented by the video data at the first playback time of the video data. and playback time data indicating the first playback time,
The specifying unit extracts the visual field image of the guest at the first reproduction time from at least the reproduction time data from the video data, and extracts the first visual field image from the subject of the visual field image based on the GUI selection data. Identifying the target of the guest's attention at the playback time;
Attention degree calculation device. an acquisition unit that acquires the viewing behavior data of the guest based on the input acquired in the terminal of the guest sharing the video data of the exhibition captured by the host;
an identification unit that identifies, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects in the exhibition;
a calculation unit configured to calculate the degree of attention of the plurality of exhibition objects so that the degree of attention of the exhibition object specified as the object of interest of the guest increases relatively;
and
The video data represents a panoramic video,
The viewing behavior data includes viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time,
The specifying unit is configured to perform a visual field range of the guest at the first reproduction time based on reference position/orientation data indicating a position of the host and a reference angle of the video data at the first reproduction time and the viewing direction data. is determined, and based on the determined viewing range and map data that associates each of the plurality of exhibition objects with the position of the exhibition object, the object that the guest paid attention to at the first reproduction time is specified.
Attention degree calculation device. The video data represents a panoramic video,
The viewing behavior data includes viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time,
The specifying unit extracts the visual field image of the guest at the first reproduction time from the video data based on the reproduction time data and the visual field direction data, and extracts the visual field image of the guest at the first reproduction time from the subject of the visual field image. identifying an object of attention of the guest;
The attention level calculation device according to claim 1 . The video data represents a panoramic video,
The viewing behavior data includes playback time data indicating a first playback time of the video data corresponding to when the video data was captured by the terminal of the guest, and viewing direction of the guest at the first playback time. line of sight data;
The specifying unit extracts the visual field image of the guest at the first reproduction time from the video data based on the reproduction time data and the visual field direction data, and extracts the visual field image of the guest at the first reproduction time from the subject of the visual field image. identifying an object of attention of the guest;
The attention level calculation device according to claim 1 . The viewing behavior data includes a captured image of the video data generated in response to the capture operation by the terminal of the guest, and playback time data indicating a first playback time of the video data corresponding to when the capture operation was performed. and
The identifying unit identifies an object that the guest paid attention to at the first reproduction time from the subjects of the captured image.
The attention level calculation device according to claim 1 . The video data represents a panoramic video,
The viewing behavior data includes playback time data indicating a first playback time of the video data corresponding to when the video data was captured by the terminal of the guest, and viewing direction of the guest at the first playback time. line of sight data;
The specifying unit is configured to perform a visual field range of the guest at the first reproduction time based on reference position/orientation data indicating a position of the host at the first reproduction time and a reference angle of the video data and the viewing direction data. is determined, and based on the determined viewing range and map data that associates each of the plurality of exhibition objects with the position of the exhibition object, the object that the guest paid attention to at the first reproduction time is specified.
The attention level calculation device according to claim 1 . The calculation unit draws the attention of the plurality of exhibition objects so that the degree of attention of the exhibition object specified as the object of the guest's attention increases according to the duration of the attention of the guest to the exhibition object. 7. The attention level calculation device according to any one of claims 1 to 6 , which calculates a degree of attention. a computer acquiring the viewing behavior data of the guest based on the input acquired at the terminal of the guest sharing the video data of the exhibition captured by the host;
a computer identifying, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects at the exhibition;
A computer calculates the degree of attention of the plurality of exhibition objects so that the degree of attention of the exhibition object identified as the object of attention of the guest increases relatively,
The viewing behavior data includes a first GUI component selected by the guest from GUI (Graphical User Interface) components superimposed and displayed on the video represented by the video data at the first playback time of the video data. and playback time data indicating the first playback time,
The specifying includes extracting the visual field image of the guest at the first reproduction time from the video data based on at least the reproduction time data, and extracting the visual field image of the guest at the first reproduction time from the video data based on the GUI selection data. Identifying the target of the guest's attention at the playback time of
Attention calculation method. a computer acquiring the viewing behavior data of the guest based on the input acquired at the terminal of the guest sharing the video data of the exhibition captured by the host;
a computer identifying, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects at the exhibition;
calculating, by a computer, the degree of attention of the plurality of exhibition objects so that the degree of attention of the exhibition object identified as the object of attention of the guest increases relatively;
and
The video data represents a panoramic video,
The viewing behavior data includes viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time,
The step of specifying the visual field of the guest at the first reproduction time based on reference position/orientation data indicating the position of the host at the first reproduction time and the reference angle of the video data and the viewing direction data. determining a range, and identifying the target of the guest's attention at the first reproduction time based on the determined viewing range and map data that associates each of the plurality of display targets with the position of the display target;
Attention calculation method. the computer,
Means for acquiring the viewing behavior data of the guest based on the input acquired at the terminal of the guest sharing the video data of the exhibition shot by the host;
means for identifying, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects at the exhibition;
Functioning as a means for calculating the degree of attention of the plurality of exhibition objects so that the degree of attention of the exhibition object specified as the object of attention of the guest increases relatively ,
The viewing behavior data includes a first GUI component selected by the guest from GUI (Graphical User Interface) components superimposed and displayed on the video represented by the video data at the first playback time of the video data. and playback time data indicating the first playback time,
The identifying means extracts the visual field image of the guest at the first reproduction time from the video data based on at least the reproduction time data, and extracts the visual field image of the guest at the first reproduction time from the video data based on the GUI selection data. Identifying the target of the guest's attention at the playback time of
Attention calculation program. the computer,
Means for acquiring the viewing behavior data of the guest based on the input acquired at the terminal of the guest sharing the video data of the exhibition shot by the host;
means for identifying, based on the viewing behavior data, an object of interest of the guest among a plurality of exhibition objects at the exhibition;
Means for calculating the degree of attention of the plurality of exhibition objects so as to relatively increase the degree of attention of the exhibition object specified as the object of attention of the guest
function as
The video data represents a panoramic video,
The viewing behavior data includes viewing direction data indicating the viewing direction of the guest at the first playback time of the video data, and playback time data indicating the first playback time,
The identifying means is configured to perform a visual field of the guest at the first reproduction time based on reference position/orientation data indicating a position of the host at the first reproduction time and a reference angle of the video data and the viewing direction data. determining a range, and identifying the target of the guest's attention at the first reproduction time based on the determined viewing range and map data that associates each of the plurality of display targets with the position of the display target;
Attention calculation program.

Images