JP6664579B2 - Learning device, learning method and learning program - Google Patents

Description

  The present invention relates to a learning device, a learning method, and a learning program.

  2. Description of the Related Art Conventionally, there has been known a technique of inputting an arbitrary image to a neural network and learning a subject or a phenomenon drawn in the image. For example, a deep learning technique for performing learning by inputting learning data to a neural network having a plurality of intermediate layers is known.

JP 2008-113071 A

  However, in the above-described conventional technology, learning by the learning device cannot always be easily performed. For example, in the above-described related art, learning is performed using a large amount of data in order to generate a learning device that outputs a desired result. For this reason, in the above-mentioned prior art, it takes time and effort to collect data for learning. For this reason, in the above-described related art, learning by the learning device cannot always be easily performed.

  The present application has been made in view of the above, and an object of the present invention is to provide a learning device, a learning method, and a learning program that can easily perform learning of a learning device.

  The learning device according to the present application restores a three-dimensional shape of the subject using an image in which the subject is drawn, and moves the subject from a viewpoint other than a predetermined viewpoint based on the restored three-dimensional shape of the subject. The image processing apparatus further includes a generation unit that generates the drawn image, and a learning unit that learns a learning device that extracts the feature of the subject using the image generated by the generation unit.

  According to an aspect of the embodiment, there is an effect that learning by a learning device can be easily performed.

FIG. 1 is an explanatory diagram illustrating an example of a learning process performed by the learning device according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the learning device according to the embodiment. FIG. 3 is a diagram illustrating an example of a learning data storage unit according to the embodiment. FIG. 4 is an explanatory diagram for describing complementary data according to the embodiment. FIG. 5 is a flowchart illustrating a learning processing procedure performed by the learning device according to the embodiment. FIG. 6 is an explanatory diagram for explaining complementary data according to the modification. FIG. 7 is an explanatory diagram for explaining complementary data according to the modification. FIG. 8 is an explanatory diagram for describing complementary data according to a modification. FIG. 9 is a hardware configuration diagram illustrating an example of a computer that implements the functions of the learning device.

  Hereinafter, a learning device, a learning method, and a mode for implementing a learning program according to the present application (hereinafter, referred to as “embodiments”) will be described in detail with reference to the drawings. Note that the learning device, the learning method, and the learning program according to the present application are not limited by this embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and overlapping description will be omitted.

[1. Learning process)
First, an example of a learning process according to the embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram illustrating an example of a learning process performed by the learning device according to the embodiment. The example of FIG. 1 illustrates an example in which the learning device 100 performs a learning process of learning a learning device that outputs a calculation result for input data.

  The learning device 100 is a server device that learns a learning device in which a predetermined node outputs a predetermined value when input data is input. More specifically, the learning device 100 includes a learning device Le that is a DNN (Deep Neural Network) that outputs a predetermined value at a node corresponding to a feature of a subject depicted in the image when the image is input as input data. Do the learning. This will be described in detail with reference to FIG.

  The learning device Le is a DNN that connects a plurality of nodes (for example, neurons) that output a calculation result with respect to the input data, and a node corresponding to a feature of the input data outputs a predetermined value. Specifically, when an image in which a subject is depicted is input as input data, a node corresponding to a feature of the subject outputs a predetermined value to the learning device Le.

  In the example of FIG. 1, the learning device Le includes an input layer, an intermediate layer formed by a multi-stage node group, and an output layer. Here, for example, the output layer is a node that is output when an image depicting “cat” is input, and has a node corresponding to the feature of “cat”.

  The learning device 100 holds, as learning data to be input to the learning device Le, images P11 to P15 of “cat” photographed from the front and images P21 to P22 of “cat” photographed from the side. Shall be. Although FIG. 1 shows an example in which the learning device 100 holds seven pieces of learning data for the sake of simplicity, in practice, a sufficient number of pieces of learning data for learning the learning device Le are stored. Hold.

  Here, the learning device 100 restores the three-dimensional shape of the subject using the image in which the subject is depicted, and depicts the subject from a viewpoint other than the predetermined viewpoint based on the restored three-dimensional shape of the subject. Generate an image. Specifically, the learning device 100 first restores the three-dimensional shape of the subject using the image in which the subject is drawn (step S1). For example, the learning device 100 restores the three-dimensional shape of the subject by various known techniques. Then, the learning device 100 generates an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint based on the restored three-dimensional shape of the subject (Step S2). For example, the learning device 100 first detects a bias of the learning data input to the learning device Le. As an example, the learning device 100 detects, as a bias of the learning data, a bias of a direction in which the subject depicted in the image is photographed. In the example of FIG. 1, the learning device 100 detects, as the bias of the direction in which the subject is photographed, that the number of images of the subject photographed from the side is relatively small.

  Thereafter, when the bias of the learning data is detected, the learning device 100 generates an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint. For example, the learning device 100 uses the learning data to generate complementary data that complements the bias of the learning data. This is because the learning device 100 can perform learning of the learning device with higher accuracy if there is complementary data that complements the bias of the learning data.

  As an example, the learning device 100 generates, as supplementary data, an image in which the subject is photographed from a relatively small direction in the image in which the subject is drawn. In the example of FIG. 1, the learning device 100 generates images P23 to P24 in which the subject is photographed from the side as supplementary data because the number of images of the subject photographed from the side is smaller than the image of the subject photographed from the front. . Note that FIG. 1 illustrates an example in which the learning device 100 generates two pieces of complementary data. However, actually, the learning device 100 generates a sufficient number of pieces of complementary data for learning the learning device Le.

  Subsequently, the learning device 100 performs learning of the learning device Le that extracts the feature of the subject using the generated image (step S3). Specifically, the learning device 100 performs learning of the learning device Le using the images P23 to P24, which are complementary data, and the images P11 to P15 and the images P21 to P22, which are learning data.

  In the example of FIG. 1, when the learning device 100 receives the images P23 to P24, the images P11 to P15, and the images P21 to P22 as input data, the node corresponding to the feature of “cat” outputs a predetermined value. Learning of the learning device Le is performed as described above. For example, the learning device 100 calculates a coupling coefficient between nodes at which a node corresponding to the feature of “cat” outputs a predetermined value using various methods such as a back propagation method and a supervised learning method. Then, the learning device 100 sets the calculated coupling coefficient between the nodes of the learning device Le to perform learning of the learning device Le in which the node corresponding to the feature of “cat” outputs a predetermined value.

  As described above, the learning device 100 according to the embodiment restores the three-dimensional shape of the subject using the image in which the subject is drawn, and determines the subject based on the restored three-dimensional shape of the subject. Generate an image drawn from a viewpoint other than the viewpoint. Further, the learning device 100 performs learning of a learning device that extracts the feature of the subject using the generated image.

  In addition, the learning device 100 according to the embodiment detects a bias of a subject or a phenomenon depicted in an image. Further, when the bias of the subject or the phenomenon is detected, the learning device 100 generates an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint.

  Accordingly, the learning device 100 can increase the number of learning data even when the number of learning data is not sufficient, and thus can easily learn the learning device. For example, since the learning device 100 can generate the supplementary data using the learning data, the labor for collecting the learning data can be reduced. Further, the learning device 100 can perform learning of the learning device even when the number of learning data is not sufficient.

  In addition, the learning device 100 can increase the amount of biased and relatively small data in the learning data, so that the output accuracy of the learning device with respect to the input data can be increased. For example, the learning device 100 can generate an image in which the subject is photographed from the side when the number of images of the subject photographed from the side is smaller than that of the image photographed from the front. The learning of the learning device captured with high accuracy can be performed. For this reason, the learning device 100 can increase the recognition accuracy of the learning device.

  In the example described above, the learning device 100 generates an image so as to complement the bias of the learning data, but the embodiment is not limited to this. For example, the learning device 100 may restore the three-dimensional shape from the image in which the subject is drawn and generate new learning data from the restored three-dimensional shape without detecting the bias of the learning data.

  In the example described above, the learning device 100 restores the three-dimensional shape of the subject (for example, a cat) depicted in the image, and generates an image of the learning device based on the restored three-dimensional shape. Here, the subject includes not only the object depicted in the image but also the scenery, background, scenery, and the like depicted in the image, and the three-dimensional shape of the scenery, background, and scenery may be restored. . That is, the learning device 100 may restore the three-dimensional shape of the object from the image depicting the outside of the object, or may restore the three-dimensional shape of the object from the image depicting the inside of the object. . As a specific example, the learning device 100 may restore the three-dimensional shape of the baseball stadium using an image shot from outside the baseball stadium, or use the image shot from inside the baseball stadium. Alternatively, the three-dimensional shape of the baseball field may be restored. That is, the learning apparatus 100 can restore the three-dimensional shape of the subject using an arbitrary image regardless of the relationship between the position of the subject and the position of the starting point when drawing the image.

[2. Configuration of learning device)
Next, the configuration of the learning device 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the learning device 100 according to the embodiment. As illustrated in FIG. 2, the learning device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The learning device 100 includes an input unit (for example, a keyboard and a mouse) for receiving various operations from an administrator or the like who uses the learning device 100, and a display unit (for example, a liquid crystal display) for displaying various information. May have.

(About the communication unit 110)
The communication unit 110 is realized by, for example, an NIC or the like. The communication unit 110 is connected to a network by wire or wirelessly, and transmits and receives information to and from various server devices and terminal devices via the network.

(About the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 includes a learning data storage unit 121 and a learning device information storage unit 122.

(About the learning data storage unit 121)
The learning data storage unit 121 stores information related to learning data. For example, the learning data storage unit 121 stores, for each image that is learning data, the direction in which the subject depicted in the image was captured. Here, FIG. 3 illustrates an example of the learning data storage unit 121 according to the embodiment. As shown in FIG. 3, the learning data storage unit 121 has items such as “data ID” and “category”.

  “Data ID” indicates identification information for identifying data. “Category” indicates the classification to which the data belongs. For example, the “category” indicates the direction in which the subject depicted in the image was photographed. That is, in FIG. 3, the image corresponding to the data ID “P11” shows an example in which the subject is photographed from “front”.

(About the learning device information storage unit 122)
The learning device information storage unit 122 stores information about a learning device (for example, DNN). Specifically, the learning device information storage unit 122 stores information on nodes of the input layer, the intermediate layer, and the output layer included in the DNN, and information on coupling coefficients between the nodes. For example, the learning device information storage unit 122 stores the learning device Le illustrated in FIG.

(About the control unit 130)
The control unit 130 stores various programs (corresponding to an example of a learning program) stored in a storage device inside the learning device 100 in a RAM by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. It is realized by being executed as. The control unit 130 is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

  As illustrated in FIG. 2, the control unit 130 includes a reception unit 131, a detection unit 132, a generation unit 133, and a learning unit 134, and implements or executes functions and operations of information processing described below. . Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 2 and may be another configuration as long as the configuration performs a learning process described later. Further, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship shown in FIG. 2 and may be another connection relationship.

(About the reception unit 131)
The receiving unit 131 receives input data. Specifically, the receiving unit 131 receives, as input data, learning data to be input to the learning device. For example, the receiving unit 131 receives, as learning data, an image in which a subject is drawn.

(About the detection unit 132)
The detection unit 132 detects a data bias. Specifically, the detecting unit 132 detects a bias of the learning data input to the learning device. For example, when performing the learning of the learning device, the detecting unit 132 first acquires the learning data stored in the learning data storage unit 121. Then, the detecting unit 132 detects the bias of the acquired learning data. As an example, the detection unit 132 detects a bias in the direction in which the subject depicted in the image is photographed. In the example of FIG. 3, the detection unit 132 determines, as the bias in the direction in which the subject is photographed, that the image photographed from the side of the subject is relatively less than the image photographed from the front. To detect.

(About the generation unit 133)
The generation unit 133 generates complementary data that complements the bias of the learning data. Specifically, the generation unit 133 restores the three-dimensional shape of the subject using the image in which the subject is drawn, and based on the restored three-dimensional shape of the subject, moves the subject from a viewpoint other than the predetermined viewpoint. Generate the rendered image. For example, when the detecting unit 132 detects the bias of the learning data, the generating unit 133 generates complementary data that complements the bias of the learning data using the learning data. This is because the learning device 100 can perform learning of the learning device with high accuracy if there is complementary data that complements the bias of the learning data. For example, the generation unit 133 generates, as supplementary data, an image in which the direction in which the subject is photographed is small among a plurality of images in which the subject is photographed from all directions.

  This will be described with reference to FIG. FIG. 4 is an explanatory diagram for explaining complementary data according to the embodiment. As illustrated in FIG. 4, the generation unit 133 preferentially generates, as supplementary data, an image in which the direction in which the subject is photographed is smaller among a plurality of images in which the subject is photographed. For example, the generation unit 133 generates the complementary data so that the images of the subject photographed from all directions of 360 ° are uniformly arranged. Then, the generation unit 133 stores the generated complementary data in the learning data storage unit 121. Accordingly, the learning device 100 can uniformly hold images of the subject photographed from all directions of 360 ° as learning data.

  In the example of FIG. 3, the generation unit 133 determines that the images P21 to P22 captured from the “horizontal” position are smaller than the images P11 to P15 captured from the “front”, The generated image is generated as complementary data.

(About the learning unit 134)
The learning unit 134 performs learning of a learning device. Specifically, the learning unit 134 learns a learning device that extracts a feature of the subject using the image generated by the generation unit 133. For example, the learning unit 134 performs learning of a learning device using the complementary data generated by the generation unit 133. As an example, the learning unit 134 first acquires the learning data and the complementary data stored in the learning data storage unit 121. Subsequently, the learning unit 134 inputs the acquired learning data and complementary data to the learning device. Then, the learning unit 134 performs learning so that the node corresponding to the predetermined classification outputs a predetermined value.

  As an example, the learning unit 134 calculates a coupling coefficient between nodes included in the DNN by a back propagation method or supervised learning. Then, the learning unit 134 performs learning by setting the calculated coupling coefficient between nodes included in the learning device.

  Thereafter, the learning unit 134 stores the learned learning device in the learning device information storage unit 122. Thereby, the learning device 100 can learn an image in which the subject is photographed from all directions, so that the subject can be identified regardless of the direction in which the subject depicted in the image input as the input data is photographed from any direction. Learning device can be generated.

[3. Learning processing procedure)
Next, a procedure of a learning process performed by the learning device 100 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart illustrating a learning processing procedure performed by the learning device 100 according to the embodiment.

  As shown in FIG. 5, the learning device 100 acquires learning data (step S101). For example, the learning device 100 acquires an image stored as learning data from the learning data storage unit 121.

  Subsequently, the learning device 100 restores the three-dimensional shape of the subject using the image in which the subject is depicted, and depicts the subject from a viewpoint other than the predetermined viewpoint based on the restored three-dimensional shape of the subject. Generate an image. For example, the learning device 100 first detects a bias of the learning data (step S102). As an example, the learning device 100 detects a bias in the direction in which the subject depicted in the image is photographed. When the learning device 100 does not detect the bias of the learning data (Step S102; No), the learning device 100 learns the learning device using the learning data (Step S105). The learning device 100 stores the learned learning device in the learning device information storage unit 122 (Step S106).

  On the other hand, when the learning device 100 detects the bias of the learning data (Step S102; Yes), the learning device 100 generates complementary data that complements the bias of the learning data using the learning data (Step S103). For example, the learning device 100 generates, as supplementary data, an image in which the direction in which the subject is photographed is small among a plurality of images in which the subject is photographed from all directions. After that, the learning device 100 stores the generated complementary data in the learning data storage unit 121 (Step S104).

  Thereafter, the learning device 100 performs learning of the learning device (Step S105). Specifically, the learning device 100 uses the generated image to perform learning of a learning device that extracts a feature of a subject. For example, the learning device 100 calculates a coupling coefficient between nodes that output a predetermined value from a node corresponding to a classification feature to which a subject belongs to an image that is input data. As an example, the learning device 100 calculates a coupling coefficient by a back propagation method.

  Then, the learning device 100 performs learning of the learning device by setting the calculated coupling coefficient as a coupling coefficient between nodes. Thereafter, the learning device 100 stores the learned learning device in the learning device information storage unit 122 (Step S106).

[4. Modification)
The learning device 100 according to the above-described embodiment may be implemented in various different forms other than the above-described embodiment. Therefore, hereinafter, another embodiment of the learning device 100 will be described.

[4-1. Optical flow (1)]
In the above-described embodiment, an example has been described in which the learning device 100 generates complementary data that complements the bias of the learning data using the learning data when the detection unit 132 detects the bias of the learning data. Here, the learning device 100 may generate complementary data in various forms.

  Specifically, the learning device 100 estimates and estimates an image obtained by observing a subject or a phenomenon depicted in an image from a plurality of viewpoints based on the motion of the subject depicted in a plurality of temporally continuous images. An image in which a subject is drawn from a viewpoint other than a predetermined viewpoint using the image is generated.

  For example, the learning device 100 first expresses, using a technique of optical flow, the motion of a subject depicted in a plurality of temporally continuous images as a vector. As an example, the learning device 100 expresses the movement of the cat's feet from the cat's feet depicted in a continuous image by a vector.

  Then, the learning device 100 estimates an image obtained by observing the subject from a plurality of viewpoints based on the motion of the subject represented by the vector. Then, the learning device 100 generates complementary data using the estimated image. Specifically, the learning device 100 preferentially generates, as supplementary data, an image in which the direction in which the subject is photographed is smaller, among a plurality of images in which the subject is photographed. For example, the learning device 100 generates supplemental data so that images of a subject taken from all directions of 360 ° are evenly arranged. Then, the learning device 100 stores the generated complementary data in the learning data storage unit 121.

  As described above, the learning device 100 estimates an image obtained by observing a subject or a phenomenon depicted in an image from a plurality of viewpoints based on the movement of the subject depicted in a plurality of temporally continuous images, and estimates the estimated image. Is used to generate an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint.

  Accordingly, the learning device 100 can generate images observed from a plurality of viewpoints as supplementary data, so that the learning of the learning device can be performed with high accuracy. For this reason, the learning device 100 can increase the recognition accuracy of the learning device.

  Further, the learning device 100 may generate the complementary data such that the number of various images is sufficient to perform learning by the learning device. This will be described with reference to FIGS. 6 to 8 are explanatory diagrams for explaining complementary data according to the modified example. As shown in FIG. 6, the learning device 100 generates complementary data so that the number of images obtained by moving the camera and photographing the subject from all directions is sufficient for learning. This is because the learning device 100 can improve the accuracy of the learning device when the number of images captured from all directions becomes sufficient.

  Further, as illustrated in FIG. 7, the learning device 100 may generate the complementary data so that images obtained by photographing the subject from all directions using a plurality of cameras are sufficient in number for learning. In addition, the learning device 100 may generate the complement data such that the number of images obtained by moving the light source in all directions and photographing the subject is sufficient for learning as shown in FIG. Thereby, the learning device 100 can uniformly hold images illuminated from each direction.

  The image in which the subject is depicted is not limited to the examples of the camera arrangement shown in FIGS. 6 to 8, and may be images obtained by photographing the subject in various arrangements. For example, the image in which the subject is depicted may be an image obtained by photographing the subject with the cameras arranged in an orthogonal trinocular arrangement, a serial arrangement, or a matrix arrangement.

  Accordingly, the learning device 100 can uniformly hold the learning data necessary for learning the learning device, so that the accuracy of the learning device can be improved. For example, the learning device 100 can generate a learning device that identifies a subject even in an image in which the subject is captured from any direction. In addition, the learning device 100 can generate a learning device that identifies a subject even if the image in which the subject is depicted has an uneven contrast.

[4-2. Optical flow (2)]
In the above modification, the example in which the learning device 100 generates the supplementary data by using the optical flow technique has been described. Here, the learning device 100 may estimate the shape of the subject using a known technique, and generate the supplementary data based on the estimated shape.

  Specifically, the learning device 100 estimates the shape of the subject depicted in the image based on the motion of the subject depicted in a plurality of temporally continuous images, and determines the subject based on the estimated shape of the subject. An image drawn from a viewpoint other than a predetermined viewpoint is generated.

  For example, the learning device 100 first expresses, using a technique of optical flow, the motion of a subject depicted in a plurality of temporally continuous images as a vector. As an example, the learning device 100 expresses the movement of the cat's feet from the cat's feet depicted in a continuous image by a vector.

  Then, the learning device 100 estimates the shape of the subject based on the motion of the subject represented by the vector. Specifically, the learning device 100 estimates the shape of the subject from the difference between the images of the captured subject. For example, the learning device 100 estimates the three-dimensional shape of the subject from a change in the imaging position of the same point on the subject that occurs when the camera is moved to photograph the subject. As an example, the learning device 100 estimates the shape of the subject by a factorization method or a stereo method. Further, the learning device 100 may estimate the three-dimensional shape of the subject from a change in the density of a shadow formed on the same object that occurs when the subject is photographed by moving the light source.

  Further, the learning device 100 may estimate the shape of the cat's feet from the movement of the cat's feet represented by the vector. As an example, the learning device 100 estimates the shape of the cat's feet by identifying the cat's feet from the movement of the cat's feet and removing the background. Note that the learning device 100 may estimate the shape of the subject by not only the above-described example but also various known technologies. Then, the learning device 100 converts the subject into a 3D model from the estimated shape.

  After that, the learning device 100 generates complementary data using the estimated shape. For example, the learning device 100 renders an image in which the subject is depicted using the shape of the subject that is 3D modeled. As an example, the learning device 100 preferentially generates, as supplementary data, an image in which the direction in which the subject is photographed is smaller, among a plurality of images in which the subject is photographed. In another example, the learning device 100 uses the 3D model of the subject to generate supplemental data by giving priority to an image in which the part of the subject is small among a plurality of images of the subject.

  As described above, the learning device 100 estimates the shape of the subject depicted in the images based on the movement of the subject depicted in a plurality of temporally continuous images, and determines the subject based on the estimated shape of the subject. Generate an image drawn from a viewpoint other than the viewpoint.

  Accordingly, the learning device 100 can generate relatively few images as supplementary data, and thus can learn the learning device with high accuracy. For this reason, the learning device 100 can increase the recognition accuracy of the learning device. Further, since the learning device 100 can make the shape of the subject into a 3D model, for example, it is possible to generate, as supplementary data, an image taken from a direction in which the subject is not taken. For this reason, the learning device 100 can improve the recognition accuracy of the learning device and can reduce the trouble of preparing an image of the learning data with various variations.

[4-3. Supplementary data)
In the above-described embodiment, the learning apparatus 100 has been described as an example in which the image of the subject is generated from the side when the number of images of the subject captured from the side is smaller than the image of the subject captured from the front. Here, the learning device 100 may generate various types of complementary data.

  Specifically, the learning device 100 generates an image of a relatively small number of portions of the captured subject portion. For example, the learning device 100 generates an image in which the “body” of the subject is captured when the number of images of the “body” of the subject is smaller than the number of images of the “face” captured.

  Further, the learning device 100 may generate an image in which the learning data is inverted, as the supplementary data. Further, the learning device 100 may generate, as supplementary data, an image obtained by converting the shading or the color of the image of the learning data.

  Thus, the learning device 100 generates various types of complementary data. Accordingly, the learning device 100 can perform learning of the learning device with high accuracy, and thus can improve the recognition accuracy of the learning device.

  In addition, in the example of FIG. 3, the direction in which the subject is photographed is classified into two categories of “front” and “horizontal” for the sake of simplicity. However, the present invention is not limited to this. May be. For example, the direction in which the subject was photographed may be classified into subdivided categories such as “front”, “right side”, “left side”, and “back”. In this case, since the learning device 100 can more accurately detect the bias of the image, the bias of the learning data can be reduced with high accuracy. Therefore, the learning device 100 can learn the learning device with high accuracy.

[4-4. Applicable)
In the above-described embodiment, an example has been described in which the learning device 100 detects the bias of a subject depicted in an image. Here, the learning apparatus 100 may detect not only the bias of the subject depicted in the image but also the bias of the phenomenon depicted in the image. Accordingly, the learning device 100 can generate complementary data that complements the bias of the phenomenon depicted in the image, and thus can perform a variety of learning.

[4-5. DNN]
In the above embodiment, an example has been described in which the learning device 100 generates a learning device including an input layer, a middle layer, and an output layer. Here, the learning device 100 may generate a learning device including an arbitrary number of node groups in various layers. For example, the learning device 100 generates a multi-stage learning device including a plurality of node groups in an intermediate layer. Further, the node group included in the learning device may be configured by an arbitrary number of nodes.

[4-6. Device configuration〕
In the above embodiment, the learning device 100 may be a learning device that does not perform the learning process of learning the learning device and performs only the generation process by the generation unit 133. In this case, the learning device does not include at least the learning unit 134. Then, the learning device having the learning unit 134 performs learning of the learning device using the complementary data generated by the learning device 100.

[4-7. Others)
Of the processes described in the above embodiment, all or some of the processes described as being performed automatically can be manually performed, or all or some of the processes described as being performed manually can be performed. Some of them can be performed automatically by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

  Each component of each device illustrated is a functional concept, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed / arbitrarily divided into arbitrary units according to various loads and usage conditions. Can be integrated and configured. For example, the detection unit 132 and the generation unit 133 illustrated in FIG. 2 may be integrated.

[4-8. Hardware configuration)
The learning device 100 according to the embodiment described above is realized by, for example, a computer 1000 having a configuration as illustrated in FIG. Hereinafter, the learning device 100 will be described as an example. FIG. 9 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the learning device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, a program that depends on hardware of the computer 1000, and the like.

  The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from another device via the communication network 500, sends the data to the CPU 1100, and transmits the data generated by the CPU 1100 to the other device via the communication network 500.

  The CPU 1100 controls output devices such as a display and a printer and input devices such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 obtains data from an input device via the input / output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the program or data to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. It is.

  For example, when the computer 1000 functions as the learning device 100 according to the embodiment, the CPU 1100 of the computer 1000 implements the function of the control unit 130 by executing a program loaded on the RAM 1200. The HDD 1400 stores data in the storage unit 120. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800. However, as another example, the CPU 1100 may acquire these programs from another device via the communication network 500.

[5. effect〕
As described above, the learning device 100 according to the embodiment includes the generation unit 133 and the learning unit 134. The generation unit 133 restores the three-dimensional shape of the subject using the image in which the subject is drawn, and generates an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint based on the restored three-dimensional shape of the subject. I do. The learning unit 134 learns a learning device that extracts a feature of a subject using the image generated by the generation unit 133.

  In the learning device 100 according to the embodiment, the detection unit 132 detects a bias of a subject or a phenomenon depicted in an image. The generation unit 133 generates an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint when the detection unit 132 detects the bias of the subject or the phenomenon.

  As a result, the learning device 100 according to the embodiment can increase the number of learning data even when the number of learning data is not sufficient, so that the learning of the learning device can be easily performed. For example, since the learning device 100 can generate the supplementary data using the learning data, the labor for collecting the learning data can be reduced. Further, the learning device 100 can perform learning of the learning device even when the number of learning data is not sufficient.

  In addition, the learning device 100 can increase the amount of biased and relatively small data in the learning data, so that the output accuracy of the learning device with respect to the input data can be increased. For example, the learning device 100 can generate an image in which the subject is photographed from the side when the number of images of the subject photographed from the side is smaller than that of the image photographed from the front. The learning of the learning device captured with high accuracy can be performed. For this reason, the learning device 100 can increase the recognition accuracy of the learning device.

  Further, in the learning device 100 according to the modification, the generation unit 133 gives priority to an image in which the direction in which the subject is photographed is smaller among a plurality of images in which the subject is photographed, as the image depicting the subject from a viewpoint other than the predetermined viewpoint. And generate.

  As a result, the learning device 100 according to the modification can increase the biased and relatively small amount of the learning data, thereby improving the output accuracy of the learning device with respect to the input data.

  In the learning device 100 according to the modified example, the generation unit 133 may be configured to generate an image obtained by observing a subject or a phenomenon depicted in an image from a plurality of viewpoints based on the movement of the subject depicted in a plurality of temporally continuous images. Is estimated, and an image in which the subject is drawn from a viewpoint other than the predetermined viewpoint is generated using the estimated image.

  Thereby, the learning device 100 according to the modified example can uniformly hold learning data necessary for learning the learning device, so that the accuracy of the learning device can be improved. For example, the learning device 100 can generate a learning device that identifies a subject even in an image in which the subject is captured from any direction. In addition, the learning device 100 can generate a learning device that identifies a subject, for example, even when an image in which the subject is drawn has an uneven contrast.

  Further, in the learning device 100 according to the modification, the generation unit 133 estimates the shape of the subject depicted in the image based on the movement of the subject depicted in a plurality of temporally continuous images, and An image in which the subject is drawn from a viewpoint other than a predetermined viewpoint is generated based on the shape.

  Accordingly, the learning device 100 according to the modified example can generate relatively few images as supplementary data, so that learning by the learning device can be performed with high accuracy. For this reason, the learning device 100 can increase the recognition accuracy of the learning device. Further, since the learning device 100 can model the shape of the subject as a 3D model, for example, it is possible to generate, as supplementary data, an image captured from a direction in which the subject is not captured. For this reason, the learning device 100 can improve the recognition accuracy of the learning device, and can reduce the trouble of preparing an image of learning data that is rich in variations.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, It is possible to implement the invention in other forms with improvements.

REFERENCE SIGNS LIST 100 learning device 121 learning data storage unit 122 learning device information storage unit 131 accepting unit 132 detecting unit 133 generating unit 134 learning unit

Claims (3)

An image in which the shape of the subject depicted in the image is estimated based on the movement of the subject depicted in a plurality of temporally continuous images, and the subject is depicted from a plurality of viewpoints based on the estimated shape of the subject. A generation unit that generates an image in which the direction in which the subject is photographed is relatively small among a plurality of images in which the subject is photographed,
A learning unit for learning a learning device that classifies the image into categories based on node values corresponding to the characteristics of the classification of the subject using the image generated by the generation unit. Neural Network) learning device. A learning method performed by the learning device,
An image in which the shape of the subject depicted in the image is estimated based on the movement of the subject depicted in a plurality of temporally continuous images, and the subject is depicted from a plurality of viewpoints based on the estimated shape of the subject. A generating step of generating an image in which the direction in which the subject is photographed is relatively small among a plurality of images in which the subject is photographed,
A learning step of learning a learning device that classifies the image into categories based on node values corresponding to the characteristics of the classification of the subject using the image generated in the generation step. Neural Network) learning method. An image in which the shape of the subject depicted in the image is estimated based on the movement of the subject depicted in a plurality of temporally continuous images, and the subject is depicted from a plurality of viewpoints based on the estimated shape of the subject. A generation procedure for generating an image in which the direction in which the subject is photographed is relatively small among a plurality of images in which the subject is photographed,
(Deep Neural Network) for causing a computer to execute a learning procedure of learning a learner that performs a category classification of an image from a value of a node corresponding to the feature of the classification of the subject using the image generated in the generation procedure. Network) learning program.

Images