JP2021090549A - Determination apparatus, determination method and determination program - Google Patents

Abstract

To provide a determination apparatus and the like which can determine whether or not a user has a specific cognitive ability.SOLUTION: A determination apparatus comprises: an acquisition unit which acquires a brain image of a user; a calculation unit which calculates the gray matter volume of the brain of the user on the basis of the brain image; and a determination unit which determines whether or not the user has a specific cognitive ability on the basis of the calculated gray matter volume.SELECTED DRAWING: Figure 3

Description

The present invention relates to a determination device, a determination method, and a determination program.

In recent years, various actions have been virtually performed by a user by perceiving a three-dimensional (3D) image using a predetermined display device (for example, a head mounted display (HMD)). Experienced xR (Virtual Reality (VR), Augmented Reality (AR)), Mixed Reality (Mixed Reality (MR), etc.) are used in various fields. For example, VR is used not only for entertainment such as games, but also for sports, training such as aircraft operation and automobile driving, rehabilitation, and surgical support robots (for example, da Vinci (registered trademark)).

On the other hand, a user who perceives a 3D image using a predetermined display device may experience a predetermined biological reaction (for example, nausea, eyestrain, disorientation, etc.). Is known (for example, Patent Document 1). The biological reaction is also called video sickness, 3D sickness, VR sickness, or the like. Hereinafter, for convenience of explanation, the biological reaction will be referred to as "image sickness".

Japanese Unexamined Patent Publication No. 2019-184883

There are individual differences in human cognitive ability (for example, receptivity to 3D images), and the implementation of various actions related to the cognitive ability does not always work appropriately for all users. For example, when a user performs training or rehabilitation using a 3D image, the user may not be able to sufficiently obtain the effect of the training or the rehabilitation due to image sickness. Therefore, in order to appropriately obtain the effect of the user performing a specific action (for example, an action using 3D images), it is determined whether or not the user has a specific cognitive ability (for example, receptivity to 3D images). It is hoped that it will be possible.

Therefore, one of the objects of the present invention is to provide a determination device, a determination method, and a determination program capable of determining whether or not a person has a specific cognitive ability (for example, receptivity to a 3D image).

The determination device according to one aspect of the present invention includes an acquisition unit that acquires a user's brain image, an arithmetic unit that calculates the gray matter volume of the user's brain based on the brain image, and the calculated gray matter. A determination unit for determining whether or not the user has a specific cognitive ability based on the volume is provided.

According to this aspect, it can be determined whether or not the user has a specific cognitive ability based on the gray matter volume. Therefore, by carrying out a specific action in consideration of the determination result, the effect of carrying out the action can be appropriately obtained.

In the above embodiment, a predictor generated based on machine learning using the gray matter volume of the predetermined region calculated based on the brain image of each user and the performance of a predetermined task related to the specific cognitive ability of each user. Further, the determination unit determines whether or not the user has the specific cognitive ability based on the predicted value of the performance generated by inputting the calculated gray matter volume into the predictor. You may judge.

According to this aspect, since a predictor generated based on machine learning using the gray matter volume and the results is used, it is possible to improve the accuracy of determining whether or not the person has the specific cognitive ability. it can.

In the above aspect, in addition to the gray matter volume and the performance, the subjective index value regarding the specific cognitive ability of each user is used for the machine learning, and the brain image is acquired by the acquisition unit. The subjective index value of the user may be acquired, and the predicted value may be generated by inputting the calculated gray matter volume and the acquired subjective index value into the predictor.

According to this aspect, since the predictor learned by machine learning using the above subjective index value in addition to the above gray matter volume and the above results is used, the accuracy of determining whether or not the person has the above specific cognitive ability. Can be improved.

In the above aspect, the predetermined region may be a region corresponding to at least a part of the superior parietal lobule and the caudate nucleus. According to this configuration, it is determined whether or not the user has the specific cognitive ability based on the gray matter volume of the superior parietal lobule and the caudate nucleus, so that the determination accuracy of the acceptability can be improved. ..

In the above aspect, the particular cognitive ability may include receptivity to 3D imaging. According to this aspect, by performing an act using the three-dimensional image in consideration of the determination result of whether or not the user has receptivity to the three-dimensional image, the execution effect of the act can be appropriately obtained. Can be done.

In the above aspect, the three-dimensional image is perceived by the user using a predetermined display device, and the predetermined display device is for the first display unit and the left eye for displaying the two-dimensional image for the right eye of the user. A second display unit for displaying a two-dimensional image may be provided.

In the above embodiment, a head-mounted display device worn on the user's head, or a non-wearable display in which the user looks into the first display unit and the second display unit with his / her right eye and left eye, respectively. It may be a device.

The determination method according to another aspect of the present invention includes a step of acquiring a user's brain image, a step of calculating the gray matter volume of a predetermined region of the user's brain based on the brain image, and the above-mentioned calculation. It includes a step of determining whether or not the user has a specific cognitive ability based on the gray matter volume.

In the determination program according to another aspect of the present invention, the calculation unit provided in the determination device is an acquisition unit that acquires a user's brain image, and the gray matter volume of a predetermined region of the user's brain based on the brain image. It functions as a calculation unit for calculating the above and a determination unit for determining whether or not the user has a specific cognitive ability based on the calculated gray matter volume.

According to the present invention, it is possible to provide a determination device, a determination method, and a determination program capable of determining whether or not a person has a specific cognitive ability (for example, receptivity to a 3D image).

It is a figure which shows an example of the act using the 3D image which concerns on embodiment of this invention. It is a figure which shows an example of the correlation between the acceptability to the 3D image which concerns on embodiment of this invention, and the gray matter volume calculated with a predetermined region of a brain image as a region of interest. It is a figure which shows an example of the functional structure of the determination system 1 which concerns on embodiment of this invention. It is a figure which shows an example of the hardware composition of each apparatus which comprises the determination system 1 which concerns on embodiment of this invention. It is a flowchart which shows an example of the learning operation which concerns on embodiment of this invention. It is a flowchart which shows an example of the determination operation which concerns on embodiment of this invention. It is a figure which shows an example of the relationship between the performance of the perceptual task which concerns on embodiment of this invention, and the acceptability to a 3D image. It is a figure which shows an example of the effect of the determination of the acceptability to the 3D image which concerns on embodiment of this invention.

Embodiments of the present invention will be described with reference to the accompanying drawings. In each figure, those having the same reference numerals have the same or similar configurations.

In this embodiment, acceptability to a three-dimensional (3D) image is illustrated as an example of a specific cognitive ability determined based on the volume of gray matter (gray matter volume) in a predetermined region of the brain. The specific cognitive ability is not limited to the receptivity to 3D images, and may be any ability that correlates with the gray matter volume of a predetermined region of the brain.

In the present embodiment, the user perceives (also referred to as visual recognition, recognition, etc.) a three-dimensional (3D) image using a predetermined display device. Specifically, the predetermined display device displays a first display unit that displays a two-dimensional (2D) image for the user's right eye (image for the right eye) and a second display unit that displays a 2D image for the left eye (image for the left eye). It is provided with two display units. Here, the 3D image is an image that can be seen stereoscopically by the user, and is also called a stereoscopic image or 3D stereoscopic vision. The 3D image can be said to be an image in which the user can perceive the depth. For example, displaying to the user an image having parallax in the left and right eyes (that is, an image for the right eye and an image for the left eye) (also referred to as binocular parallax), and / or the user's head. The user may perceive the 3D image by changing the appearance of the image according to the movement (also referred to as motion parallax). The 3D image is not limited to one that utilizes binocular parallax, and may be a hologram, a volume scanning type, or the like. Further, the 2D image is an image that can be seen in a plane by the user, and is also called a flat image or the like.

Here, the right-eye image and the left-eye image for perceiving a 3D image using binocular parallax are images in which the same object is captured at different angles. The user synthesizes the image for the right eye displayed on the first display unit for the right eye and the image for the left eye displayed on the second display unit for the left eye into one in the brain, and stereoscopically combines the object. Perceived (that is, perceived as a 3D image).

The predetermined display device including the first display unit and the second display unit may be a wearable display device mounted on a predetermined portion of the user, or may be a non-wearable display device. The wearable display device may be, for example, a head mounted display (HMD) mounted on the user's head, a VR headset, goggles, or the like. Non-wearable display devices include, for example, a console with an immersive display such as the Cave Automatic Virtual Environment (CAVE), a stereoscopic television, a first display for the right eye and a second display for the left eye. It may be a surgery support robot or the like.

FIG. 1 is a diagram showing an example of an act using a 3D image according to the present embodiment. In FIG. 1, the HMD 100 worn on the user's head is exemplified as the predetermined display device, but as described above, the predetermined display device is not limited to this. Further, in FIG. 1, sports (here, tennis) is shown as an example of an act using a 3D image, but the act is an act in a field other than sports (for example, maneuvering an aircraft, driving a vehicle, etc.). Rehabilitation, patient surgery), etc. may be used. Further, in FIG. 1, the 3D image realizes virtual reality (VR), but may also realize augmented reality (AR) or mixed reality (MR).

In FIG. 1, the user perceives the 3D image V by the parallax of the right-eye image and the left-eye image displayed on the first display unit 101 and the second display unit 102 of the HMD 100, respectively. For example, in FIG. 1, a user is training tennis based on a 3D image V perceived using an HMD100 worn on the head.

Further, the HMD 100 includes a sensor 103 that detects the motion of a predetermined part (for example, head, eyes, hands, feet, etc.) of the user, and based on the analysis result of the movement, the right eye image and the left eye image are displayed. It may have a tracking function to control. The tracking function is also called head tracking, eye tracking, motion tracking, or the like. The sensor that detects the movement may be, for example, a magnetometer that detects a direction, a gyroscope that detects an angle or an angular velocity, an accelerometer that detects acceleration, an infrared sensor, or the like.

The tracking function changes the 3D image perceived by the user according to a predetermined movement of the user. For example, in FIG. 1, the movement, position, and the like of the racket and the ball perceived in the 3D image V change according to the movement of the hand of the user who operates the racket, the direction of the eyes, and the movement of the head. Therefore, it is possible to improve the user's sense of presence and immersiveness.

By the way, there are users who experience a predetermined biological reaction (for example, feeling unpleasant, eyestrain, light-headedness, etc.) called "image sickness" or the like triggered by the visual movement in the 3D image as described above. It is known.

Therefore, for a predetermined number of subjects, a perception task (for example, Multiple Object Tracking (MOT) using 2D / 3D video) for each of 2D video and 3D video (hereinafter abbreviated as 2D / 3D video) is performed. When the task) was performed, a group of subjects who did not have a significant difference in the results of 2D / 3D images (first subject group) and a group of subjects whose results of 3D images were significantly lower than those of 2D images (first subject group). 2 subject groups) and. The MOT task is a predetermined task of tracking a plurality of objects in an image, and is used for evaluating the perceptual ability of a subject.

It can be said that the first subject group is more receptive to the 3D image than the second subject group because the perceptual task using the 3D image can produce the same results as when the 2D image is used. On the other hand, it can be said that the second subject group is less receptive to the 3D image than the first subject group because the performance of the perceptual task using the 3D image is significantly lower than that when the 2D image is used. Here, the receptivity to a 3D image means that even if the 3D image is perceived, the image sickness is unlikely to occur, and the effect of carrying out the act using the 3D image can be appropriately obtained.

FIG. 2 is a diagram showing an example of the correlation between the acceptability for a 3D image according to the present embodiment and the gray matter volume calculated by setting a predetermined region of a brain image as a region of interest (ROI). FIG. 2 shows a brain image I1 viewed from the side surface of the human brain and a brain image I2 showing a cross section of the human brain cut at the cut surface X of the brain image I1. The superior parietal lobule shown in brain image I1 is one of the gyruses on the lateral surface of the cerebrum. The superior parietal lobule has a role of recognizing space-related information such as the position, movement, and depth of an object. On the other hand, the caudate nucleus shown in the brain image I2 exists near the center of the brain and on both sides of the thalamus. The caudate nucleus is involved in motor control, learning, cognition, and so on.

As shown in FIG. 2, for the first subject group, which is relatively highly receptive to 3D images (that is, the difference in performance of perceptual tasks in 2D / 3D images is relatively small), the gray matter of the superior parietal lobule. The volume tends to be large and the gray matter volume of the caudate nucleus tends to be small. On the other hand, for the second group of subjects, who are relatively highly receptive to 3D images (ie, the results of 3D images are significantly lower than those of 2D images), the gray matter volume of the superior parietal lobule is small and caudate. The gray matter volume of the nucleus tends to be large.

Thus, there is a positive correlation between receptivity to 3D images and the gray matter volume of the superior parietal lobule. On the other hand, there is a negative correlation between receptivity to 3D images and gray matter volume of the caudate nucleus. That is, it is assumed that specific cognitive abilities such as receptivity to 3D images are correlated with the gray matter volume of a predetermined region of the brain. Therefore, the present inventors have focused on the correlation between a specific cognitive ability (for example, receptivity to 3D images) and the gray matter volume of a predetermined region of the brain, and are specific based on the gray matter volume of the predetermined region. The idea was to determine cognitive ability (eg, receptivity to 3D images).

In the following, it is assumed that a predetermined region of the brain used for determining a specific cognitive ability (for example, receptivity to 3D images) is at least one of the superior parietal lobule and the caudate nucleus. I can't. The predetermined region may be any region of the brain that correlates with the particular cognitive ability. The predetermined region may include, for example, at least one of the superior parietal lobule and the caudate nucleus, or in addition to the angular gyrus and the basal ganglia.

(Configuration of judgment system)
FIG. 3 is a diagram showing an example of the determination system 1 according to the embodiment of the present invention. The determination system 1 uses a predictor 14a to predict whether or not it has a specific cognitive ability (for example, receptivity to 3D images), and a learning process of a prediction model 22b mounted on the predictor 14a. 20 and a learning device 20 for performing the above. The predictor 14a that implements the prediction model 22b may be paraphrased as a prediction model, a prediction program, a prediction algorithm, or the like.

Further, in this example, the determination device 10 using the predictor 14a learned by machine learning by the learning device 20 will be described, but the predictor 14a is not necessarily limited to the one generated by machine learning and is a predetermined region of the brain. Judgment of a specific cognitive ability based on a theoretically derived relational expression or an empirically derived relational expression regarding the relationship between the gray matter volume and a specific cognitive ability (for example, receptivity to 3D images). It may be.

Further, in this example, the case where the determination device 10 and the learning device 20 are separate bodies is shown, but the determination device 10 and the learning device 20 may be configured as an integrated device. For example, the determination device 10 and the learning device 20 may be realized as different operations of the PLC (Programmable Logic Controller).

<Learning device>
As a functional configuration, the learning device 20 includes a generation unit 21 that generates learning data 22a, a storage unit 22 that stores learning data 22a and a prediction model 22b, and a prediction model 22b by machine learning using the learning data 22a. It has a learning unit 23 that executes the learning process of the above. The generation unit 21 includes a brain image acquisition unit 211, a gray matter volume calculation unit 212, a grade acquisition unit 213, and a subjective index value acquisition unit 214.

The brain image acquisition unit 211 acquires the brain image of each user. The brain image may be, for example, an image captured by a magnetic resonance imaging (MRI) device (for example, brain images I1, I2 in FIG. 2). The brain image may be called a brain anatomical image or the like. The MRI apparatus may be, for example, a 3 Tesla MRI apparatus.

The gray matter volume calculation unit 212 calculates the gray matter volume of a predetermined region of the user's brain based on the brain image acquired by the brain image acquisition unit 211. Specifically, the gray matter volume calculation unit 212 sets a predetermined region of the brain image as the region of interest and calculates the volume of gray matter in the region of interest. The predetermined region is, for example, a region corresponding to at least a part of the superior parietal lobule and the caudate nucleus, but is not limited to this, and may be a region corresponding to at least a part such as the angular gyrus and the basal ganglia. Good. The gray matter volume may be evaluated using, for example, a Voxel-based-Morp hometry (VBM).

The grade acquisition unit 213 acquires the grade (for example, correct answer rate or score) of a predetermined task (for example, a perceptual task using a 3D image) related to a specific cognitive ability of each user. The result of the perception task using the 3D image may be, for example, the result of the MOT task using the 3D image.

The subjective index value acquisition unit 214 acquires a subjective index value related to a specific cognitive ability of each user (for example, a subjective index value related to video sickness). The subjective index value related to the image sickness may be, for example, information indicating a Simulator Sickness Questionnaire (SSQ) value. In SSQ, the user answers a predetermined number of question items (for example, 16 question items) classified into feelings of discomfort, eyestrain, and light-headedness in predetermined stages (for example, 4 stages). The SSQ value may be a predetermined index value generated based on the answer. The SSQ value may be generated based on the user's response after performing the above perceptual task.

The storage unit 22 stores the learning data 22a generated by the generation unit 21 and the prediction model 22b generated by the learning process by the learning unit 23. The learning data 22a may be data in which the gray matter volume calculated by the gray matter volume calculation unit 212 and the grade acquired by the grade acquisition unit 213 are associated with each user. Further, the learning data 22a may be data in which the subjective index value acquired by the subjective index value acquisition unit 214 is associated with each user in addition to the gray matter volume and the grade.

The learning unit 23 executes the learning process of the prediction model 22b by machine learning using the learning data 22a stored in the storage unit 22. For example, the learning unit 23 learns the prediction model 22b so that the results of each user stored as the learning data 22a are output based on the gray matter volume of each user stored as the learning data 22a. May be done. Further, the learning unit 23 predicts that the results of each user stored as the learning data 22a are output based on the gray matter volume and the subjective index value of each user stored as the learning data 22a. The learning process of the model 22b may be performed.

The prediction model 22b machine-learned by the learning unit 23 may be composed of, for example, a neural network. When the prediction model 22b is composed of a neural network, the learning unit 23 may perform the learning process of the neural network by, for example, the backpropagation method.

<Judgment device>
The determination device 10 includes a brain image acquisition unit 11, a gray matter volume calculation unit 12, a subjective index value acquisition unit 13, and a determination unit 14. The brain image acquisition unit 11 acquires a brain image of a user to be determined for the presence or absence of a specific cognitive ability (for example, receptivity to a 3D image). The gray matter volume calculation unit 12 sets a predetermined region of the brain image acquired by the brain image acquisition unit 11 as the region of interest, and calculates the gray matter volume in the region of interest. The subjective index value acquisition unit 13 acquires a subjective index value related to a specific cognitive ability of the user (for example, a subjective index value related to video sickness). The details of the brain image acquisition unit 11, the gray matter volume calculation unit 12, and the subjective index value acquisition unit 13 will be described in the brain image acquisition unit 211, the gray matter volume calculation unit 212, and the subjective index value acquisition unit 214, respectively. That's right.

The determination unit 14 determines whether or not the user has a specific cognitive ability (for example, receptivity to a 3D image) based on the gray matter volume calculated by the gray matter volume calculation unit 12. Further, the determination unit 14 may include a predictor 14a generated by machine learning in the learning device 20. It should be noted that the predictor 14a may implement a new prediction model produced by distillation or produced by distillation. In distillation, a new prediction model may be generated using the input / output data to the prediction model 22b machine-learned by the learning device 20.

Specifically, the determination unit 14 inputs the gray matter volume calculated by the gray matter volume calculation unit 12 into the predictor 14a, and the user has a specific cognitive ability (for example, 3D) based on the predicted value generated. It may be determined whether or not the image has receptivity). Further, the determination unit 14 identifies the user based on the predicted value generated by inputting the subjective index value acquired by the subjective index value acquisition unit 13 into the predictor 14a in addition to the gray matter volume. It may be determined whether or not the cognitive ability (for example, receptivity to 3D images) is possessed. The predicted value may be a predicted value of the performance of a predetermined task related to a specific cognitive ability of each user (for example, a predicted value of a correct answer rate of a MOT task using a 3D image).

The determination result by the determination unit 14 may be output to the output unit 10e, which will be described later, and displayed to the user. Alternatively, the determination result may be transmitted to another device via the communication unit 10c.

<Hardware configuration>
Next, the hardware configuration of each device in the determination system 1 (for example, at least one of the determination device 10 and the learning device 20) will be described. As shown in FIG. 4, each device in the determination system 1 includes a CPU (Central Processing Unit) 10a corresponding to an arithmetic unit, a storage device 10b, a communication unit 10c, an input unit 10d, and an output unit 10e. Have. Each of these configurations is connected to each other via a bus so that data can be transmitted and received. In this example, the determination device 10 and the learning device 20 are composed of separate computers, but may be composed of one computer.

The CPU 10a is a control unit that controls execution of a program stored in the storage device 10b, calculates data, and processes data. The CPU 10a is an arithmetic unit (arithmetic unit) that executes a program (determination program) for determining a specific cognitive ability (for example, receptivity to 3D images) based on the gray matter volume calculated with a predetermined region of a brain image as an region of interest. ) May be. The CPU 10a receives various input data from the input unit 10d and / or the communication unit 10c, outputs (for example, displays) the calculation result of the input data to the output unit 10e, stores it in the storage device 10b, or communicates. It may be transmitted via unit 10c.

The storage device 10b is at least one of a memory, an HDD (Hard Disk Drive), and an SSD (Solid State Drive). The storage device 10b of the learning device 20 may constitute a storage unit 22. The storage device 10b of the determination device 10 may store the determination program executed by the CPU 10a.

The communication unit 10c is an interface for connecting each device in the determination system 1 to an external device. When the determination device 10 and the learning device 20 are configured as an integrated device, the communication unit 10c includes interprocess communication between the process operating as the determination device 10 and the process operating as the learning device 20. Good.

The input unit 10d receives data input from the user, and may include at least one of a keyboard, a mouse, a touch panel, and a microphone, for example.

The output unit 10e outputs the calculation result by the CPU 10a, and may be composed of at least one of a display such as an LCD (Liquid Crystal Display) and a speaker, for example.

The determination program may be stored in a storage medium readable by a computer such as the storage device 10b and provided, or may be provided via the network N connected by the communication unit 10c. The storage medium in which the determination program is stored may be a non-transitory computer readable medium. The non-temporary storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory, a CD-ROM, or a DVD.

In the determination device 10, the CPU 10a executes the determination program to realize the operations of the brain image acquisition unit 11, the gray matter volume calculation unit 12, the subjective index value acquisition unit 13, and the determination unit 14 described with reference to FIG. Will be done. It should be noted that these physical configurations are examples and do not necessarily have to be independent configurations. For example, the determination device 10 may include an LSI (Large-Scale Integration) in which the CPU 10a and the storage device 10b are integrated.

Further, the brain image acquisition units 11 and 211, the subjective index value acquisition units 13 and 214, and the grade acquisition unit 213, respectively, have predetermined tasks related to the brain image received by the communication unit 10c, the subjective index value, and the specific cognitive ability. (For example, a MOT task using a 3D image) may be obtained, or a brain image stored in a computer-readable storage medium such as a storage device 10b, the subjective index value, and the result. May be obtained.

(Operation of judgment system)
<Learning movement>
FIG. 5 is a flowchart showing an example of the learning operation according to the present embodiment. In addition, in FIG. 5, the learning operation of the prediction model 22b that assumes the acceptability to the 3D image as an example of the specific cognitive ability and predicts the acceptability to the 3D image will be described.

As shown in FIG. 5, in step S101, the learning device 20 acquires a brain image of each user. For example, the learning device 20 may acquire a brain image captured by an MRI device (for example, a 3 Tesla MRI device). In step S102, the learning device 20 sets a predetermined region of the brain image acquired in step S101 as the region of interest, and calculates the gray matter volume of the region of interest. The region of interest may be, for example, a region corresponding to at least a portion of the superior parietal lobule and the caudate nucleus.

In step S103, the learning device 20 acquires a subjective index value (for example, SSQ value) regarding the video sickness of each user. For example, an SSQ may be performed on each user after a perceptual task (for example, a MOT task using a 3D image), and an SSQ value determined based on the answer of the SSQ may be acquired as a subjective index value.

In step S104, the learning device 20 acquires the results of the perceptual task using the 3D image of each user. As described above, the perceptual performance information may be information (for example, correct answer rate) indicating the performance of the perceptual task (for example, the MOT task using 3D images).

In step S105, the learning device 20 uses the learning data 22a in which the gray matter volume calculated in step S102, the subjective index value acquired in step S103, and the perceptual performance information acquired in step S104 are associated with each other. The learning process of the prediction model 22b may be performed by the machine learning that has been performed.

In FIG. 5, the order of steps S101, S103, and S104 does not matter. Further, in FIG. 5, step S103 may be omitted. When step S103 is omitted, in step S105, the learning device 20 performs the prediction model 22b by machine learning using the learning data in which the gray matter volume calculated in step S102 is associated with the grade acquired in step S104. You may carry out the learning process of.

<Judgment operation>
FIG. 6 is a diagram showing an example of a determination operation according to the present embodiment. Note that FIG. 6 assumes receptivity to a 3D image as an example of a specific cognitive ability, and describes a determination operation for determining receptivity to the 3D image.

As shown in FIG. 6, in step S201, the determination device 10 acquires a brain image of the user to be determined for acceptability to the 3D image. In step S202, the determination device 10 sets a predetermined region of the brain image acquired in step S201 as the region of interest, and calculates the gray matter volume of the region of interest. In step S203, the determination device 10 acquires the subjective index value regarding the image sickness of the user. The details of steps S201, S202, and S203 are the same as those of steps S101, S102, and S103 of FIG.

In step S204, the determination device 10 inputs the gray matter volume calculated in step S202 and the subjective index value acquired in step S203 into the trained predictor 14a, and outputs a 3D image as the output of the predictor 14a. You may generate the predicted value (performance predicted value) of the performance of the perceptual task using. The performance prediction value may be, for example, a prediction value of the correct answer rate or the score of the MOT task.

In step S205, the determination device 10 determines whether or not the performance prediction value generated in step S204 satisfies a predetermined condition. When the performance prediction value satisfies a predetermined condition, in step S206, the determination device 10 determines that the user has receptivity to the 3D image. On the other hand, when the performance prediction value does not satisfy the predetermined condition, in step S207, the determination device 10 determines that the user does not have receptivity to the 3D image.

In FIG. 6, the order of steps S201 and S203 does not matter. Further, in FIG. 6, step S203 may be omitted. When step S203 is omitted, in step S204, the determination device 10 inputs the gray matter volume calculated in step S202 to the trained predictor 14a, and generates a performance prediction value as the output of the predictor 14a. You may.

FIG. 7 is a diagram showing an example of the relationship between the performance of the perceptual task according to the present embodiment and the receptivity to the 3D image. As shown in FIG. 7, the predetermined conditions used for determining the presence or absence of acceptability for the 3D image may be determined based on the performance distribution of the perceptual task for a predetermined number of subjects. FIG. 7 shows the performance distribution when a perceptual task (here, MOT task) using 3D images is performed on 78 subjects.

For example, as shown in FIG. 7, when the grades of the perceptual task are distributed, the predetermined condition for being judged to have receptivity to the 3D image is that the grades have a correct answer rate of 70% or more. Good.

For example, when the condition of FIG. 7 is used, if the grade prediction value output from the predictor 14a in step S204 of FIG. 6 shows a correct answer rate of 70% or more, the user corresponding to the grade prediction value refers to the 3D image. Judged to be receptive. On the other hand, when the grade prediction value output from the predictor 14a shows a correct answer rate of less than 70% in step S204 of FIG. 6, it is determined that the user corresponding to the grade prediction value does not have receptivity to the 3D image. To.

Note that FIG. 7 is only an example, and the predetermined condition for determining that the 3D image is acceptable is not limited to the correct answer rate of 70% or more. The predetermined condition may be any condition as long as it is determined based on the performance of the perceptual task (for example, the correct answer rate) and the predetermined threshold value.

FIG. 8 is a diagram showing an example of the effect of determining acceptability for a 3D image according to the present embodiment. FIG. 8 shows the distribution of the growth rate when a predetermined number of subjects (here, 48 subjects) are subjected to sports training using a 3D image for a predetermined period (for example, one month).

In FIG. 8, the growth rate of 22 out of 48 subjects is less than 10%. Based on the brain images of the 22 subjects, the region corresponding to the superior parietal lobule and the caudate nucleus was set as the region of interest, and the gray matter volume was calculated. When the gray matter volume was input to the trained predictor 14a and the acceptability of the 22 subjects to the 3D image was determined in advance, it was determined that the 19 subjects did not have the acceptability to the 3D image. It was.

Therefore, even if sports training using 3D images is performed on a user who is determined to be unacceptable to 3D images using the trained predictor 14a, the effect of implementing the training cannot be appropriately obtained. Probability is high. Therefore, by training a user who is determined to have receptivity to the 3D image using the 3D image, the effect of the training can be efficiently obtained.

Note that, in FIG. 8, sports training is shown as an example of an act using 3D images, and the act uses 3D images such as, for example, maneuvering an aircraft, driving a vehicle, rehabilitation, and surgery for a patient. It may be any action performed. When performing an action using a 3D image, the effect of performing the action can be obtained more efficiently by considering the determination result of acceptability for the 3D image using the learned predictor 14a.

As described above, according to the determination system according to the present embodiment, the acceptability of the user to the 3D image is determined based on the gray matter volume of the predetermined region of the user's brain calculated based on the user's brain image. Can be judged. Therefore, by performing an act using the 3D image for a user determined to have receptivity to the 3D image, the effect of performing the act can be efficiently obtained.

In the above embodiment, a specific cognitive ability (for example, receptivity to 3D images) is determined based on the gray matter volume of a predetermined region of the brain, and the parameters used for the determination are brain function and brain structure. The parameter is not limited to the gray matter volume as long as it is a parameter indicating at least one of. For example, instead of or in addition to the gray matter volume, the degree of diffusion anisotropy (Fractional Anisotropy (FA) value) of a predetermined region of the brain may be calculated. The degree of diffusion anisotropy may reflect qualitative and quantitative changes in white matter fibers of the brain. The diffusion anisotropy may be calculated based on the diffusion weighted image. The diffusion-weighted image may be acquired using, for example, a gradient magnetic field (Motion Probing gradient (MPG)) applied to detect the diffusion of hydrogen atoms by MRI. In addition, parameters determined based on at least one of resting brain function coupling, myelin map, and microstructure image may be used.

The embodiments described above are for facilitating the understanding of the present invention, and are not for limiting and interpreting the present invention. Each element included in the embodiment and its arrangement, material, condition, shape, size, and the like are not limited to those exemplified, and can be changed as appropriate. In addition, the configurations shown in different embodiments can be partially replaced or combined.

1 ... Judgment system, 10 ... Judgment device, 20 ... Learning device, 11 ... Brain image acquisition unit, 12 ... Gray matter volume calculation unit, 13 ... Subjective index value acquisition unit, 14 ... Judgment unit, 14a ... Predictor, 21 ... Generation unit, 22 ... Storage unit, 22a ... Learning data, 22b ... Prediction model, 23 ... Learning unit, 211 ... Brain image acquisition unit, 212 ... Gray matter volume calculation unit, 213 ... Grade acquisition unit, 214 ... Subjective Index value acquisition unit, 10a ... CPU, 10b ... Storage device, 10c ... Communication unit, 10d ... Input unit, 10e ... Output unit, I1, I2 ... Brain image, V ... 3D image, X ... Cut surface, 100 ... HMD, 101 ... 1st display, 102 ... 2nd display, 103 ... Sensor

Claims (9)

The acquisition unit that acquires the user's brain image,
An arithmetic unit that calculates the gray matter volume of a predetermined region of the user's brain based on the brain image,
A determination unit that determines whether or not the user has a specific cognitive ability based on the calculated gray matter volume.
Judgment device including. Further equipped with a predictor generated based on machine learning using the gray matter volume of the predetermined region calculated based on the brain image of each user and the performance of a predetermined task related to the specific cognitive ability of each user.
The determination unit determines whether or not the user has the specific cognitive ability based on the predicted value of the performance generated by inputting the calculated gray matter volume into the predictor.
The determination device according to claim 1. In the machine learning, in addition to the gray matter volume and the performance, subjective index values relating to the specific cognitive ability of each user are used.
The acquisition unit acquires a subjective index value relating to the specific cognitive ability of the user from whom the brain image has been acquired.
The predicted value is generated by inputting the calculated gray matter volume and the acquired subjective index value into the predictor.
The determination device according to claim 2. The predetermined region is a region corresponding to at least a part of the superior parietal lobule and the caudate nucleus.
The determination device according to any one of claims 1 to 3. The particular cognitive ability includes receptivity to 3D images.
The determination device according to any one of claims 1 to 4. The three-dimensional image is perceived by the user using a predetermined display device, and the three-dimensional image is perceived by the user.
The predetermined display device includes a first display unit that displays a two-dimensional image for the right eye of the user and a second display unit that displays a two-dimensional image for the left eye.
The determination device according to claim 5. The predetermined display device is a wearable display device that is attached to the predetermined portion of the user, or a non-wearable display device that is not attached to the predetermined portion of the user.
The determination device according to claim 6. The process of acquiring the user's brain image and
A step of calculating the gray matter volume of a predetermined region of the user's brain based on the brain image, and
A step of determining whether or not the user has a specific cognitive ability based on the calculated gray matter volume, and
Judgment method having. The arithmetic unit provided in the judgment device,
Acquisition unit that acquires the user's brain image,
A calculation unit that calculates the gray matter volume of a predetermined region of the user's brain based on the brain image, and
A determination unit that determines whether or not the user has a specific cognitive ability based on the calculated gray matter volume.
Judgment program to function as.

Images