JP7099506B2 - Detection device, detection method, information processing device, and processing program - Google Patents

Description

The present invention relates to a detection device, a detection method, an information processing device, and a processing program.

A technique has been proposed in which an object is detected by a plurality of image pickup devices, a plurality of obtained images are input to a computer, and a three-dimensional shape of the object is acquired (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2010-134546

In the above-mentioned techniques, it is desired that the three-dimensional shape of the object can be constructed with high accuracy from the result of detecting the object.

According to the aspect of the present invention, the detection unit, the first region that can be detected by the detection unit, and the second region that cannot be detected by the detection unit or the reliability of the detection result of the detection unit is lower than that of the first region. It is equipped with a calculation unit that calculates the information to be specified, the detection unit detects the object from one viewpoint, and the calculation unit calculates the accuracy information of the object in one viewpoint using the detection result of the detection unit. As accuracy information, the calculation unit calculates information that distinguishes between the first region that can be detected from one viewpoint and the second region that is behind the first region when viewed from one viewpoint, among the regions containing the object. Detection provided with an information calculation unit that determines a region farther than the point from one viewpoint on the line connecting the viewpoint and a point on the object as the second region, and calculates shape information of the object using the detection result and accuracy information. Equipment is provided.
Further, according to the aspect of the present invention, the plurality of detection devices include the plurality of detection devices and an information processing device for processing information output from the plurality of detection devices, and the plurality of detection devices include shape information of an object and a target. A detection system is provided that outputs model information including at least one of the texture information of an object, and the information processing apparatus includes a model integration unit that integrates information output from a plurality of detection devices.
Further, according to the aspect of the present invention, the object is detected from one viewpoint by the detection unit, the first region that can be detected by the detection unit, and the reliability of the detection result that cannot be detected by the detection unit or is detected by the detection unit. The information that identifies the second region, which is lower than the first region, is calculated, the accuracy information of the object in one viewpoint is calculated using the detection result of the detection unit, and the object is used as the accuracy information. Information that distinguishes the first region that can be detected from one viewpoint and the second region that is behind the first region when viewed from one viewpoint is calculated, and on the line connecting the one viewpoint and the point on the object. , A detection method including determining a region farther than a point from one viewpoint as a second region and calculating shape information of an object using the detection result and accuracy information is provided.
Further, according to the aspect of the present invention, the first model information including the shape information of the object in one viewpoint, the second detection result and the second, calculated by using the first detection result and the first accuracy information. A receiver that receives second model information including shape information of an object from a viewpoint different from one viewpoint, calculated using accuracy information, and a model that integrates the first model information and the second model information. The first accuracy information and the second accuracy information include an integrated unit, and the first accuracy information and the second accuracy information are information that distinguishes between a detectable first region and a second region that is undetectable or has a lower reliability of the detection result than the first region. The shape information of the object in one viewpoint is calculated as, the accuracy information of the object in one viewpoint is calculated using the detection result of the object detected by the detection unit from one viewpoint, and the object is used as the accuracy information. Information that distinguishes the first region that can be detected from one viewpoint and the second region that is behind the first region when viewed from one viewpoint is calculated, and on the line connecting the one viewpoint and the point on the object. Provided is an information processing apparatus that determines a region farther than a point from one viewpoint as a second region and calculates using the detection result and accuracy information.
Further, according to the aspect of the present invention, it is a processing program that processes the detection result in which the object is detected by the detection unit from one viewpoint, and is detected by the first region that can be detected by the detection unit and the detection unit by the computer. Calculate information that identifies the second region where the reliability of the detection result of the detection unit is lower than that of the first region, and calculate the accuracy information of the object from one viewpoint using the detection result of the detection unit. As accuracy information, information that distinguishes between the first region that can be detected from one viewpoint and the second region that is behind the first region when viewed from one viewpoint is calculated as accuracy information. On the line connecting the viewpoint and the point on the object, the region farther than the point from one viewpoint is determined as the second region, and the shape information of the object is calculated using the detection result and the accuracy information. A processing program to be executed is provided.
Further , according to the first aspect of the present invention, a detection device is provided. The detection device may include a detection unit. The detection device calculates information that identifies the first region that can be detected by the detection unit and the second region that cannot be detected by the detection unit or the reliability of the detection result of the detection unit is lower than that of the first region. It may be provided with a calculation unit. According to the second aspect of the present invention, a detection device is provided. The detection device may include a detection unit that detects an object. The detection device may include a calculation unit that calculates information for specifying an area between the object detected by the detection unit and the detection unit. According to the third aspect of the present invention, a detection system is provided. The detection system may include a plurality of detection devices as described above. The detection system may include an information processing device that processes information output from a plurality of detection devices. The plurality of detection devices may output model information including at least one of the shape information of the object and the texture information of the object. The information processing device may include a model integration unit that integrates information output from a plurality of detection devices. According to a fourth aspect of the present invention, a detection system is provided. The detection system may include the detection device as described above. The detection system may include an information processing device that processes information output from the detection device. According to the fifth aspect of the present invention, a detection method is provided. The detection method may include identifying a first region that can be detected by the detection unit and a second region that cannot be detected by the detection unit or whose reliability of the detection result of the detection unit is lower than that of the first region. According to the sixth aspect of the present invention, a detection method is provided. The detection method may include detecting an object by a detection unit. The detection method may include calculating information that identifies the area between the object detected by the detection unit and the detection unit. According to the seventh aspect of the present invention, an information processing apparatus is provided. The information processing device uses the first model information including the shape information of the object in one viewpoint, and the second detection result and the second accuracy information calculated by using the first detection result and the first accuracy information. It may be provided with a receiving unit for receiving the second model information including the shape information of the object at a viewpoint different from one viewpoint calculated by the above. The information processing device may include a model integration unit that integrates the first model information and the second model information. The first accuracy information and the second accuracy information may be calculated as information for distinguishing the first region that can be detected from the second region that cannot be detected or the reliability of the detection result is lower than that of the first region. According to the eighth aspect of the present invention, an information processing apparatus is provided. The information processing device uses the first model information including the shape information of the object in one viewpoint, and the second detection result and the second accuracy information calculated by using the first detection result and the first accuracy information. It may be provided with a receiving unit for receiving the second model information including the shape information of the object at a viewpoint different from one viewpoint calculated by the above. The information processing device may include a model integration unit that integrates the first model information and the second model information. The first accuracy information and the second accuracy information may be calculated as information for specifying an area between the object and one viewpoint. According to the ninth aspect of the present invention, a processing program is provided. The processing program may be a processing program that processes the detection result detected by the detection unit. The processing program executes the computer to identify the first region that can be detected by the detection unit and the second region that cannot be detected by the detection unit or the reliability of the detection result of the detection unit is lower than that of the first region. You may let me. According to the tenth aspect of the present invention, a processing program is provided. The processing program may be a processing program that processes the detection result obtained by detecting the object by the detection unit. The processing program may cause the computer to calculate information that identifies the area between the object detected by the detection unit and the detection unit.

It is a figure which shows an example of the detection apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the detection part which concerns on 1st Embodiment. It is explanatory drawing of the accuracy information based on the optical characteristic of the detection part which concerns on 1st Embodiment. It is a conceptual diagram which shows an example of the detection area of the detection apparatus which concerns on 1st Embodiment. It is explanatory drawing of the accuracy information based on the distance which concerns on 1st Embodiment. It is explanatory drawing of the accuracy information based on the distribution of the distance which concerns on 1st Embodiment. It is a flowchart which shows an example of the detection method which concerns on 1st Embodiment. It is a flowchart which shows an example of the process of FIG. It is a flowchart which shows the other example of the process of FIG. It is a figure which shows the detection system which concerns on 2nd Embodiment. It is a block diagram which shows the detection system which concerns on 2nd Embodiment. It is a figure which shows the detection system which concerns on 3rd Embodiment. It is a block diagram which shows the detection system which concerns on 3rd Embodiment. It is a conceptual diagram which shows an example of the model integration process which concerns on 3rd Embodiment.

[First Embodiment]
The first embodiment will be described. FIG. 1A is a conceptual diagram showing an example of the detection device 1 according to the present embodiment. The detection device 1 is, for example, an image pickup device, and detects an object OB in the detection area A1 (eg, visual field). The detection device 1 may be, for example, a fixed-point camera, a camera whose field of view can be changed manually or automatically, or a portable information terminal (eg, a smartphone, a tablet, a mobile phone with a camera). The detection device 1 performs arithmetic processing of information about the object OB by using the result of detecting the object OB. The detection device 1 models at least a part of the object OB by the arithmetic processing of the own terminal, and calculates the model information (model data). For example, the detection device 1 performs computer graphic processing (CG processing) on at least a part of the object OB by arithmetic processing, and calculates model information (eg, CG model data). The model information includes, for example, at least one of shape information indicating a three-dimensional shape of the object OB and texture information indicating a pattern on the surface of the object OB. Further, for example, the model information is a space of an image such as three-dimensional point coordinates, information related to the point coordinates, texture information of the surface defined by the point coordinates and the related information, lighting conditions of the entire image, and light source information. Includes at least one piece of information and polygon data as shape information. The texture information includes, for example, characters and figures on the surface of the object OB, patterns, information defining irregularities, a specific image, and at least one information of a color (eg, chromatic color, achromatic color). The detection device 1 calculates model information indicating an object OB viewed from, for example, a viewpoint Vp (eg, a predetermined viewpoint, a single viewpoint, a single viewpoint, one direction). Further, for example, the detection device 1 calculates model information indicating the object OB viewed from one viewpoint at a predetermined angle with respect to the object OB.

FIG. 1B is a block diagram showing an example of the configuration of the detection device 1. The detection device 1 includes, for example, a detection unit 2, an accuracy calculation unit 3, an information calculation unit 4, a storage unit 5, a communication unit 6, a control unit 7, and a main body unit 8. The main body 8 is, for example, a camera body, a case, a housing, or the like. The detection unit 2, the accuracy calculation unit 3, and the information calculation unit 4 are provided in, for example, the main body unit 8.

The detection unit 2 detects the object OB from the viewpoint Vp. The accuracy calculation unit 3 calculates the accuracy information of the object OB at the viewpoint Vp. The accuracy calculation unit 3 calculates, for example, the accuracy information of the object OB according to the viewpoint Vp. The accuracy information includes, for example, at least one of a probability, a probability distribution, and an expected value that an object exists at each position in a region including an object OB (eg, detection region A1 of detection unit 2, visual field). This probability distribution is, for example, information showing the relationship between the accuracy of a part of the detection region of the detection unit 2 and the accuracy of the surrounding region. The accuracy information includes, for example, the reliability of the detection result of the detection unit 2 with respect to each position of the region including the object OB (eg, the detection area of the detection unit 2, the visual field). For example, accuracy is a numerical value of the accuracy (accuracy, accuracy, reliability) or uncertainty (ambiguity, ambiguity) of data at each position, and accuracy information is accuracy at multiple positions. Includes the distribution of. For example, high accuracy corresponds to high accuracy or low uncertainty. For example, low accuracy corresponds to low accuracy or high uncertainty. Further, for example, the detection unit 2 transmits the detection result (eg, image data, etc.) of detecting the object OB to the information calculation unit 4, and the accuracy calculation unit 3 transmits the calculated accuracy information of the object OB to the information calculation unit 4. Send to. The information calculation unit 4 calculates at least one of the shape information and the texture information of the object OB at the viewpoint Vp by using the detection result and the accuracy information of the received detection unit 2. For example, the information calculation unit 4 calculates at least one of the shape information and the texture information of the object OB when viewed from the viewpoint Vp. For example, the accuracy calculation unit 3 calculates the accuracy information for each area of the object OB, and the information calculation unit 4 transfers the accuracy information calculated for each area of the object OB into at least one of the shape information and the texture information. To associate. Hereinafter, each part of the detection device 1 will be described.

FIG. 2 is a diagram showing an example of the detection unit 2. The detection unit 2 optically detects the object OB from the viewpoint Vp, for example. For example, at least one of an image of the object OB viewed from one viewpoint (viewpoint Vp) and a distance from one viewpoint (viewpoint Vp) to each point on the object OB is acquired. The detection unit 2 may detect the object OB at a predetermined viewing angle, for example. The detection unit 2 may detect the object OB with a predetermined line of sight (eg, a single line of sight), for example. The detection unit 2 includes, for example, an image pickup unit 11 and a distance measuring unit 12. The image pickup unit 11 takes an image of the object OB from the viewpoint Vp and outputs image data or the like of the object OB as a detection result. The detection result of the detection unit 2 includes, for example, image data of the detected object OB. The distance measuring unit 12 detects the distance from the viewpoint Vp (see FIG. 1A) to each point on the object OB. The detection unit 2 may not include the imaging unit 11 or the distance measuring unit 12.

The image pickup unit 11 includes an imaging optical system 13 and an image pickup element 14. The imaging optical system 13 forms an image of the object OB. The imaging optical system 13 is held in the lens barrel, for example, and is attached to the main body 8 (see FIG. 1) together with the lens barrel. The imaging optical system 13 and the lens barrel are, for example, interchangeable lenses, and are removable from the main body 8. The imaging optical system 13 and the lens barrel may be a built-in lens. For example, the lens barrel may be a part of the main body 8 or may not be removable from the main body 8.

The image pickup device 14 is, for example, a CMOS image sensor or a CCD image sensor in which a plurality of pixels are two-dimensionally arranged. The image pickup device 14 is housed in, for example, the main body 8. The image pickup device 14 captures an image formed by the imaging optical system 13. The image pickup result (detection result) of the image pickup element 14 includes, for example, information on the gradation value for each color of each pixel (eg, RGB data). The image sensor 14 outputs, for example, an image pickup result in a full-color image data format.

The ranging unit 12 detects the distance from each point on the surface of the object OB. The distance measuring unit 12 detects the distance by, for example, a TOF (time of flight) method. The distance measuring unit 12 may detect the distance by another method. For example, the distance measuring unit 12 may include a laser scanner and detect the distance by laser scanning. For example, the distance measuring unit 12 may project a predetermined pattern on the object OB and measure the distance based on the detection result of this pattern. Further, the distance measuring unit 12 may include a phase difference sensor and detect the distance by the phase difference method. Further, the distance measuring unit 12 may detect the distance by the DFD (depth from defocus) method. When the DFD method is used, the ranging unit 12 may use at least one of the imaging optical system 13 of the imaging unit 11 and the image pickup element 14.

The ranging unit 12 includes, for example, an irradiation unit 15, an imaging optical system 16, an image pickup element 17, and a controller 18. The irradiation unit 15 can irradiate the object OB with infrared light. The irradiation unit 15 is controlled by the controller 18. The controller 18 changes the intensity of the infrared light emitted from the irradiation unit 15 over time (eg, amplitude-modulates). The imaging optical system 16 forms an image of the object OB. Like the imaging optical system 13, the imaging optical system 16 may be at least a part of an interchangeable lens or at least a part of a built-in lens. The image pickup device 17 is, for example, a CMOS image sensor or a CCD image sensor. The image pickup device 17 has sensitivity at least in the wavelength band of the light irradiated by the irradiation unit 15. The image pickup device 17 is controlled by the controller 18, for example, and detects infrared light reflected and scattered by the object OB. For example, the image pickup device 17 captures an image formed by the imaging optical system 16.

The controller 18 detects the distance (depth) from each point on the surface of the object OB to the image sensor 17 by using the detection result of the image sensor 17. For example, the flight time of light incident on the image sensor 17 from a point on the surface of the object OB changes according to the depth of this point. The output of the image sensor 17 changes according to the flight time, and the controller 18 calculates the depth based on, for example, the output of the image sensor 17. For example, the controller 18 calculates the depth for each region (eg, one pixel, a plurality of pixels) of the image captured by the image sensor 17, and calculates the depth information by associating the position of this region with the depth. (Generate). The depth information includes, for example, information relating the position of a point on the surface of the object OB and the distance (depth, depth) from this point to the detection device 1. Depth information includes, for example, information (eg, depth image) indicating the distribution of depth (eg, depth map) in the object OB.

Next, the accuracy information calculated by the accuracy calculation unit 3 will be described. The accuracy calculation unit 3 (see FIG. 1B) calculates accuracy information based on, for example, the optical characteristics of the optical system provided in the detection unit 2 (eg, the imaging optical system 13 and the imaging optical system 16). do. FIG. 3 is an explanatory diagram of accuracy information based on the optical characteristics of the detection unit 2.

FIG. 3A is a conceptual diagram showing an example of the image Im1 captured by the image pickup unit 11. In the imaging optical system 13 (see FIG. 2), for example, the aberration increases as the distance from the optical axis 13a increases. For example, the optical axis 13a of the imaging optical system 13 corresponds to the visual field center 14a of the image pickup unit 11, and the image pickup unit 11 is a region (eg, one pixel, a plurality of pixels) away from the visual field center 14a in the image pickup element 14. The more, the reliability of the detection result using this region decreases. In FIG. 3A, the reference numeral Imc is a position on the captured image Im1 corresponding to the center of the visual field of the imaging unit 11. For example, the captured image Im1 is affected by aberrations as the distance from the position Imc increases, and the reproducibility of the subject (reliability of the detection result) decreases due to blurring or the like. The accuracy calculation unit 3 calculates accuracy information based on the distance from the position Imc, for example, for each region (eg, one pixel, a plurality of pixels) of the captured image Im1.

FIG. 3B is a conceptual diagram showing an example of the accuracy according to the distance from the center of the visual field (position Imc) of the image pickup unit 11 on the image pickup image Im1. In FIG. 3, the vertical axis is the accuracy P1 and the horizontal axis is the position on the captured image Im1. In FIG. 3B, the accuracy P1 has a maximum at position Imc and decreases as it moves away from position Imc. The relationship between the accuracy P1 and the distance of the image pickup unit 11 from the center of the field of view is determined, for example, based on the distribution of aberrations on the image pickup element 14 by the image pickup optical system 13. Here, the accuracy P1 changes non-linearly with respect to the distance from the position Imcc, but may change linearly with respect to the distance from the position Imc, or changes discontinuously (eg, stepwise). You may.

Information indicating the relationship between the accuracy P1 and the distance of the image pickup unit 11 on the image pickup image Im1 from the center of the field of view (hereinafter referred to as the first relationship information) is stored in advance in, for example, the storage unit 5 (see FIG. 1). .. The accuracy calculation unit 3 calculates the distance from the position Imc for each region (eg, one pixel, a plurality of pixels) of the captured image Im1. The accuracy calculation unit 3 collates the calculated distance with the above-mentioned first relational information, and calculates the accuracy P1 in this region. The accuracy calculation unit 3 calculates, for example, as accuracy information, information in which the position of each region and the accuracy P1 are associated with each other (hereinafter, referred to as accuracy P1 information).

The accuracy calculation unit 3 may calculate the accuracy information based on the aberration of the imaging optical system 13 (eg, the distribution of the aberration). Further, the accuracy calculation unit 3 may calculate the accuracy information based on the aberration of the imaging optical system 13 and the distance from the center of the visual field. Further, the accuracy calculation unit 3 calculates accuracy information based on at least one of the distance from the center of the visual field of the distance measuring unit 12 and the aberration of the imaging optical system 16 with respect to the detection result of the distance measuring unit 12. You may.

Further, the brightness of the captured image Im1 may be saturated in a part of the region due to the reflected light from the object OB, for example. Further, on the object OB, a dark part may be formed when viewed from the viewpoint Vp, for example, when there is a shadow of another object. The accuracy calculation unit 3 may calculate the accuracy information based on, for example, the brightness (brightness) in the captured image Im1. FIG. 3C is a conceptual diagram showing an example of the accuracy P2 according to the brightness in the captured image Im1. In FIG. 3C, the vertical axis is the accuracy P2, and the horizontal axis is the brightness (eg, the gradation value of the pixel). Here, the brightness is constant when the brightness is within the predetermined range BR, the accuracy P2 decreases as the brightness decreases (becomes darker) than the range BR, and the brightness increases above the range BR (). As it gets brighter), the accuracy P2 decreases. The relationship between brightness and accuracy is arbitrarily set based on, for example, an experiment or a simulation. For example, the accuracy P2 may change non-linearly with respect to brightness, may change linearly, or may change discontinuously (eg, stepwise).

Information indicating the relationship between brightness and accuracy (hereinafter referred to as second relationship information) is stored in advance in, for example, a storage unit 5 (see FIG. 1). The accuracy calculation unit 3 collates the brightness (example, gradation value) for each region (eg, one pixel, a plurality of pixels) of the captured image with the above-mentioned second relational information, and calculates the accuracy P2 in this region. The accuracy calculation unit 3 calculates, for example, as accuracy information, information in which the position of each region and the accuracy P2 are associated with each other (hereinafter, referred to as accuracy P2 information).

At least one of the above-mentioned information of accuracy P1 and information of accuracy P2 may be stored in the same file as the data of the captured image. For example, the data structure (data format, data format) of this information is, for example, the gradation value of the pixel (each gradation value of R, G, B) and the accuracy (eg, at least one of the accuracy P1 and the accuracy P2). It may be a structure in which and is combined. Further, at least one of the information of the accuracy P1 and the information of the accuracy P2 may be stored in a file different from the data of the captured image. For example, the data structure of the information of the accuracy P1 may be a structure in which the accuracy values of each pixel are arranged in correspondence with the pixel arrangement of the captured image. The accuracy calculation unit 3 does not have to calculate the accuracy information based on the optical characteristics of the optical system (eg, the imaging optical system 13 and the imaging optical system 16) provided in the detection unit 2. For example, the accuracy calculation unit 3 does not have to calculate at least one of the above-mentioned accuracy P1 and accuracy P2.

Next, the accuracy information of the region including the periphery of the object OB will be described. FIG. 4 is a conceptual diagram showing an example of the detection area A1 of the detection device 1 (detection unit 2). FIG. 4A shows a conceptual diagram of the object OB and the detection device 1 as viewed from the X direction (eg, sideways). FIG. 4B shows a conceptual diagram of the object OB and the detection device 1 as viewed from the Z direction (eg, from above).

For example, the accuracy calculation unit 3 uses the first region (eg, the object seen from the viewpoint Vp) that can be detected from the viewpoint Vp in the region including the object OB (eg, the detection area A1 of the detection unit 2) as the accuracy information. Information that distinguishes between AR1 (a region where the OB can be viewed in a plane) and AR2, which is a second region (eg, a region where the object OB cannot be viewed in a plane) which is behind the first region AR1 when viewed from the viewpoint Vp. calculate. The detection area A1 is, for example, the field of view of the detection device 1 (eg, the detection unit 2). The first region AR1 is, for example, the surface of an object (eg, an object OB, a background, a floor F in FIG. 4A, a wall W in FIG. 4B) detected by the detection unit 2 in the detection region A1. The area between the detection unit 2 and the detection unit 2 is included. In FIG. 4, the reference numeral OBx is an arbitrary point on the surface of an object (eg, an object) detected by the detection unit 2. The position of the point OBx is represented by, for example, three-dimensional coordinates with respect to the viewpoint Vp, and is obtained from the shape information (eg, point cloud data, surface information described later) calculated by the information calculation unit 4. The accuracy calculation unit 3 sets the region ARa between the viewpoint Vp and the point OBx as the first region AR1. The region ARa is, for example, a region where the probability of existence of an object that blocks light on the line connecting the viewpoint Vp and the point OBx is low. The first region AR1 is, for example, a set of regions ARa for each of a plurality of points OBx. For example, the first region AR1 is a void (gas space). The accuracy calculation unit 3 calculates the coordinates indicating the range of the first region AR1 based on the shape information calculated by the information calculation unit 4, for example. Further, the accuracy calculation unit 3 has a low probability that, for example, as accuracy information, a coordinate indicating the first region AR1 and an object that blocks light exists in the first region AR1 (an object that blocks light exists in the first region AR1). Calculate the information associated with the probability of not doing so).

Further, the accuracy calculation unit 3 determines, for example, the region ARb farther from the viewpoint Vp than the point OBx as the second region AR2 on the line connecting the viewpoint Vp and the point OBx on the object OB. The second region AR2 is, for example, a set of regions ARb for each of a plurality of points OBx. The second region AR2 is, for example, a region in which the reliability of the detection result of the detection unit 2 is lower than that of the first region AR1. For example, when the portion of the point OBx is glass or the like, the reliability of the detection result of the detection unit 2 regarding the second region AR2 is increased due to refraction, reflection, scattering, etc. of the light directed from the second region AR2 toward the viewpoint Vp at the point OBx. May decrease. Further, if the portion of the point OBx is made of a material that does not transmit light, the second region AR2 may not be detected from the viewpoint Vp. In such a case, the accuracy calculation unit 3 may calculate, as the accuracy information, information indicating that the second region AR2 is a region that cannot be detected from the viewpoint Vp.

In this way, the accuracy calculation unit 3 may calculate information on the area around the object OB (eg, accuracy information on the first area AR1 and accuracy information on the second area AR2). Further, the accuracy calculation unit 3 may calculate information on the space (eg, void) on the side of the object OB with respect to the viewpoint Vp. Reference numeral SPx in FIG. 4B is a point in the space on the side of the object OB. The distance between the point SPx and the viewpoint Vp is, for example, the same as the distance between the viewpoint Vp and the point OBx. The accuracy calculation unit 3 determines, for example, whether or not the point SPx belongs to the first region AR1 (a region closer to the viewpoint Vp than the surface of the object). When the accuracy calculation unit 3 determines that the point SPx belongs to the first region AR1, for example, the accuracy calculation unit 3 determines that there is a gap at the position of the point SPx. In this way, the accuracy calculation unit 3 may detect, for example, a gap on the side of the object OB. Further, the accuracy calculation unit 3 calculates, for example, as accuracy information, information indicating the position of the detected void and the high probability that the void exists at this position (the probability that an object that blocks light exists is low). You may. The accuracy calculation unit 3 does not have to calculate at least a part of the accuracy information described with reference to FIG.

Next, the accuracy information based on the distance (depth) detected by the detection unit 2 (eg, the distance measuring unit 12) will be described. The accuracy calculation unit 3 calculates accuracy information based on, for example, the distance detected by the distance measuring unit 12. FIG. 5 is an explanatory diagram of accuracy information based on the distance detected by the ranging unit 12. FIG. 5A is a conceptual diagram showing an example of depth information (depth image Im2). In FIG. 5A, the object OB is represented by a dotted line, and the distance for each region included in the detection region A1 of the detection unit 2 (eg, one pixel of the depth image Im2, a plurality of pixels) is represented by a gray scale. In the depth image Im2, the portion having a high gradation value (close to white) is close to the viewpoint Vp, and the portion having a low gradation value (close to black) is far from the viewpoint.

FIG. 5B is a diagram showing an example of the relationship between the distance and the accuracy P3. For example, the farther the distance measuring unit 12 is from the viewpoint Vp, the lower the reliability (eg, detection accuracy) of the detection result may be. In such a case, the accuracy calculation unit 3 calculates, for example, the accuracy P3 of a value having a negative correlation with respect to the distance as the accuracy information. In FIG. 5B, the accuracy P3 decreases as the distance from the viewpoint Vp increases. Information indicating the relationship between the accuracy P3 and the distance from the viewpoint Vp (hereinafter referred to as the third relationship information) is stored in advance in, for example, the storage unit 5 (see FIG. 1). The accuracy calculation unit 3 collates, for example, the distance to the viewpoint Vp in the region with the third relation information for each region (eg, one pixel, a plurality of pixels) on the depth image Im2 (see FIG. 5 (A)). , The accuracy P3 of this region is calculated. The accuracy calculation unit 3 calculates, for example, as accuracy information, information in which the position of each region and the accuracy P3 are associated with each other (hereinafter, referred to as accuracy P3 information).

FIG. 5C is a diagram showing another example of the relationship between the distance and the accuracy P3. The ranging unit 12 has, for example, a predetermined ranging range (range measuring range), and the reliability of the detection result (eg, detection accuracy) may decrease as the distance from the center of the ranging range increases. In such a case, the accuracy calculation unit 3 may calculate the accuracy information as the accuracy information, for example, based on the amount of deviation between the center of the ranging range and the distance from the viewpoint. In FIG. 5C, the accuracy P3 becomes maximum at the center of the range-finding range, and decreases as the distance deviates from the center of the range-finding range. As shown in FIGS. 5 (B) and 5 (C), the relationship between the distance and the accuracy P3 is appropriately set according to, for example, the characteristics of the distance measuring unit 12.

Further, the accuracy calculation unit 3 may calculate the accuracy information based on the distribution of the distance detected by the distance measuring unit 12. FIG. 6 is an explanatory diagram of accuracy information based on the distribution of distances. FIG. 6A is a conceptual diagram showing an example of depth information (depth image Im2) and an object OB. In FIG. 6, reference numerals X1 to X5 indicate positions in one direction on the depth image Im2. The position X1 is the position of one end of the depth image Im2, and the position X2 is the position of the other end of the depth image Im2. The section from the position X1 to the position X3 is a part of the background of the object OB (eg, the floor F and the wall W in FIG. 4). The section from the position X3 to the position X4 is a part of the front surface OB1 of the object OB. The section from the position X4 to the position X5 is a portion of the surface OB2 having a step with the front surface OB1. The section between the position X5 and the position X2 is a part of the background of the object OB (eg, the floor F and the wall W in FIG. 4). The position X3 is a portion of the side surface OB3 with respect to the front surface OB1, and the position X4 is a portion of the side surface OB4 between the front surface OB1 and the surface OB2. The position X5 is a portion of the side surface OB5 with respect to the surface OB2. The side surface OB3, the side surface OB4, and the side surface OB5 are, for example, planes that are almost parallel to the line of sight from the viewpoint Vp, and the reliability of the detection result may be low.

FIG. 6B is a conceptual diagram showing the distribution of distances on the line Imd of FIG. 6A. In FIG. 6B, the distance from the viewpoint Vp changes stepwise at each of the position X3, the position X4, and the position X5. The reliability of the detection result of the distance measuring unit 12 decreases, for example, at a position where the amount of change in distance is larger than the threshold value (eg, in the vicinity of a step). In such a case, the accuracy calculation unit 3 may calculate the accuracy according to the amount of change in the distance as the accuracy information, for example. For example, the accuracy calculation unit 3 may calculate the accuracy of the value of the negative correlation with respect to the amount of change in the distance as the accuracy information. For example, the accuracy calculation unit 3 may calculate the accuracy indicating that the reliability of the detection result of the distance measuring unit 12 is high in the region where the amount of change in the distance is relatively small. Further, for example, the accuracy calculation unit 3 may calculate the accuracy indicating that the reliability of the detection result of the distance measuring unit 12 is low in the region where the amount of change in the distance is relatively large.

FIG. 6C is a conceptual diagram showing an example of the distribution of the accuracy P4 based on the amount of change in the distance. Here, the distribution of the accuracy P4 according to the distribution of the distance in FIG. 6B is conceptually shown. The amount of change in the distance is, for example, an amount according to the inclination of the line of sight passing through the viewpoint of the detection unit 2 and the surface on the object OB. For example, when the amount of change in distance is large, the surface on the object OB is close to parallel to the line of sight, and when the amount of change in distance is small, the surface on the object OB is close to perpendicular to the line of sight. In FIG. 6C, the accuracy P4 is relatively low at a position where the amount of change in distance is large (eg, positions X3, X4, position X5). The accuracy calculation unit 3 calculates, for example, the amount of change in distance (for example, the difference from the depth in two adjacent regions) for each region of the depth image Im2, and compares the calculated amount of change with the threshold value. The accuracy calculation unit 3 sets the accuracy P4 in this region to a high level, for example, when the amount of change in distance is equal to or less than the threshold value. The accuracy calculation unit 3 sets the accuracy P4 in this region to a low level when the amount of change in distance is larger than the threshold value.

FIG. 6D is a conceptual diagram showing an example of the relationship between the amount of change in distance and the accuracy P4. In FIG. 6D, the accuracy P4 is constant in the range where the amount of change in distance is equal to or less than the threshold value Dth, and decreases in the range where the amount of change in distance exceeds the threshold value Dth. The accuracy P4 may change non-linearly, linearly, or discontinuously (eg, stepwise) with respect to the amount of change in distance. Information indicating the relationship between the accuracy P4 and the amount of change in the distance from the viewpoint Vp (hereinafter referred to as the fourth relational information) is stored in advance in the storage unit 5 (see FIG. 1), for example. The accuracy calculation unit 3 calculates, for example, the amount of change in the distance to the viewpoint Vp for each region (eg, one pixel, multiple pixels) on the depth image Im2 (see FIG. 6A), and calculates this amount of change. The accuracy P4 of each area is calculated by collating with the fourth relation information. The accuracy calculation unit 3 calculates, for example, as accuracy information, information in which the position of each region and the accuracy P4 are associated with each other (hereinafter, referred to as accuracy P4 information).

At least one of the above-mentioned information of accuracy P3 and information of accuracy P4 may be stored in the same file as the depth information. For example, the data structure (data format, data format) of this information may be, for example, a structure in which accuracy (at least one of accuracy P3 and accuracy P4) and depth are combined for each part of the detection area A1. good. For example, this information may be expressed in a format in which a depth value and an accuracy are paired for each region of the depth image Im2 (eg, one pixel, a plurality of pixels). Further, at least one of the information of the accuracy P3 and the information of the accuracy P4 may be stored in a file different from the depth information. For example, at least one of the data structure of the information of the accuracy P1 and the data structure of the information of the accuracy P2 may be a structure in which the accuracy values of each region are arranged so as to correspond to the data array of the distance of each region in the depth information.

The accuracy calculation unit 3 does not have to calculate at least one of the information of the accuracy P3 and the information of the accuracy P4. Further, the accuracy calculation unit 3 does not have to calculate the accuracy information based on the distance detected by the detection unit 2 (eg, the distance measuring unit 12). The accuracy calculation unit 3 may calculate the accuracy by combining two or more of the above-mentioned accuracy P1 to accuracy P4. For example, the accuracy calculation unit 3 may calculate the accuracy by performing weighting (calculation of a weighted average) using two or more of the above-mentioned accuracy P1 to accuracy P4. Further, the accuracy calculation unit 3 may calculate at least one of the arithmetic mean and the geometric mean of two or more of the accuracy P1 to the accuracy P4 as the accuracy.

Returning to the description of FIG. 1, the information calculation unit 4 includes, for example, a digital signal processor (DSP). The information calculation unit 4 calculates at least one of the shape information and the texture information of the object OB by using the detection result (eg, depth information) of the detection unit 2. In the above description, the distance measuring unit 12 generates the depth information, but the information calculation unit 4 may generate the depth information based on the depth detected by the distance measuring unit 12. Further, the information calculation unit 4 may generate information in which the accuracy information calculated by the accuracy calculation unit 3 based on the depth information is associated with the depth information.

For example, the information calculation unit 4 calculates point cloud data including the coordinates of a plurality of points on the object OB based on the detection result of the detection unit 2 as shape information. The information calculation unit 4 calculates point cloud data using the detection result (eg, depth information) of the distance measuring unit 12 (point cloud data processing). For example, the information calculation unit 4 calculates point cloud data by fluoroscopic conversion from a distance image (depth image) indicated by depth information to a plane image. When the field of view is different between the image pickup unit 11 and the distance measurement unit 12, the information calculation unit 4 obtains the detection result of the distance measurement unit 12 from the field of view of the image pickup unit 11 by fluoroscopic transformation (projection transformation) or the like. The OB may be converted into the detected result. Even if the information calculation unit 4 executes fluoroscopy conversion using parameters that depend on the positional relationship between the field of view of the image pickup unit 11 and the field of view of the distance measuring unit 12 (eg, the position of the viewpoint, the direction of the line of sight). good.

The information calculation unit 4 generates point cloud data using, for example, accuracy information. For example, the information calculation unit 4 may select a region having a relatively high accuracy in the depth image and perform fluoroscopic transformation from the depth image to the plane image. For example, the information calculation unit 4 may omit the fluoroscopic transformation of at least a part of the region where the accuracy is relatively low in the depth image. The information calculation unit 4 may generate point cloud data without using, for example, depth information having a relatively low accuracy. In this case, the processing load is reduced and the generated point cloud data is generated. The amount of data can be reduced. Further, the information calculation unit 4 may use, for example, a region having a relatively high accuracy in the depth image, interpolate the region having a relatively low accuracy, and perform fluoroscopic conversion from the depth image to the planar image.

The information calculation unit 4 stores, for example, the calculated point cloud data in the storage unit 5. The information calculation unit 4 generates, for example, information in which point cloud data and accuracy information are associated with each other. For example, the information calculation unit 4 calculates information in which the three-dimensional point coordinates included in the point cloud data and the accuracy of the points on the depth image corresponding to the points are associated with each other. The data structure of this information may be, for example, a structure in which three-dimensional point coordinates and accuracy are combined. The information calculation unit 4 stores, for example, information in which the point cloud data and the accuracy information are associated with each other in the storage unit 5.

The information calculation unit 4 may generate point cloud data without using the accuracy information. Further, the accuracy calculation unit 3 may calculate the accuracy information (accuracy information of the point cloud data) regarding the information of the points included in the point cloud data by using the point cloud data generated by the information calculation unit 4. For example, the accuracy calculation unit 3 may calculate the accuracy information based on the distance information between two predetermined points included in the point cloud data. For example, the accuracy calculation unit 3 may select two adjacent points from the point cloud data, compare the distance between these two points with the threshold value, and calculate the accuracy information. Further, the accuracy calculation unit 3 may calculate the accuracy information based on the spatial frequency information of a plurality of points included in the point cloud data (eg, the density of the spatial distribution of the points). Further, the accuracy calculation unit 3 may calculate the accuracy information based on the vector information connecting the two points included in the point cloud data. For example, the accuracy calculation unit 3 may select two adjacent points from the point cloud data and calculate the accuracy information by using the vector connecting these two points and the position information of the detection device 1. The position information of the detection device 1 includes, for example, the direction of the viewpoint Vp (detection direction, line of sight, direction of the optical axis of the optical system). The accuracy calculation unit 3 may calculate the accuracy information according to the angle between the vector connecting the two points included in the point cloud data and the direction of the viewpoint Vp, for example. For example, when the angle between the vector and the direction of the viewpoint Vp is close to 0 ° or 180 °, the accuracy calculation unit 3 may relatively lower the accuracy of at least one of the start point and the end point of the vector. good. Further, for example, the accuracy calculation unit 3 may relatively increase the accuracy when the angle between the above vector and the direction of the viewpoint Vp is close to 90 ° or 270 °.

The information used by the accuracy calculation unit 3 for calculating the accuracy information may be one type or two or more types of the above-mentioned distance information, spatial frequency information, vector information, and position information of the detection device 1. The point cloud data used by the accuracy calculation unit 3 for calculating the accuracy information may be generated by the information calculation unit 4 using the accuracy information or may be generated by the information calculation unit 4 without using the accuracy information. Further, the accuracy calculation unit 3 may store the accuracy information calculated using the point cloud data in the storage unit 5.

Further, the information calculation unit 4 generates surface information as shape information, including coordinates of a plurality of points on the object OB and connection information between the plurality of points, based on the detection result of the detection unit 2. The surface information is, for example, polygon data, vector data, draw data, or the like. The connection information includes, for example, information relating the points at both ends of the line corresponding to the ridgeline (eg, edge) of the object OB to each other, and information relating a plurality of lines corresponding to the contour of the surface of the object OB to each other. The information calculation unit 4 estimates, for example, a surface between a point selected from a plurality of points included in the point cloud data and a point in the vicinity thereof, and converts the point cloud data into polygon data having plane information between the points. Convert (surface processing). The information calculation unit 4 converts the point cloud data into polygon data by, for example, an algorithm using the least squares method. This algorithm may be, for example, an algorithm applied to the algorithm published in the point cloud processing library.

The information calculation unit 4 generates surface information using, for example, accuracy information. For example, the information calculation unit 4 selects nearby points in consideration of accuracy when estimating a surface between a point selected from a plurality of points included in the point cloud data and a point in the vicinity thereof. For example, the information calculation unit 4 selects a point having a relatively high accuracy as a nearby point. For example, the information calculation unit 4 may generate surface information without using information having a relatively low accuracy among the point cloud data. In this case, the processing load is reduced and the generated surface information data is used. The amount can be reduced. Further, when the accuracy of the points in the vicinity is lower than the threshold value, the information calculation unit 4 may perform interpolation using points having a relatively high accuracy among the surrounding points.

The information calculation unit 4 generates, for example, information in which surface information and accuracy information are associated with each other. For example, the information calculation unit 4 generates information in which the information of the line (or surface) included in the surface information is associated with the information of the point cloud accuracy of the point cloud data corresponding to the line (or surface). This information may be stored, for example, as attribute information of elements (eg, lines, faces) included in the surface information. The information calculation unit 4 stores, for example, information in which surface information and accuracy information are associated with each other in the storage unit 5.

The information calculation unit 4 does not have to use the accuracy information when generating the surface information. For example, the information calculation unit 4 does not have to use the accuracy information when generating the point cloud data using the accuracy information and generating the surface information based on the point cloud data. Further, the accuracy calculation unit 3 may calculate the accuracy information by using the surface information generated by the information calculation unit 4. For example, the surface information includes information on a line (graph) connecting two points, and the accuracy calculation unit 3 may calculate accuracy information on the line information. For example, similarly to the calculation of the accuracy information using the point group data described above, the accuracy calculation unit 3 uses at least one of the distance information, the vector information, and the position information of the detection device 1 to provide line information. You may calculate the accuracy information about.

Further, the surface information includes, for example, information on the surface of an object surrounded by three or more lines, and the information calculation unit 4 may generate surface information using the accuracy information regarding the line information. For example, a line having a relatively high accuracy among a plurality of candidates for the outer peripheral line of the surface may be adopted as the outer peripheral line of the surface. Further, the surface information includes, for example, information on a surface surrounded by three or more lines, and the accuracy calculation unit 3 may calculate accuracy information on the surface information. For example, the accuracy calculation unit 3 may calculate the accuracy information regarding the surface information by arithmetic mean, geometric mean, or weighted average using the accuracy information regarding the line information corresponding to the outer peripheral line of the surface. Further, the accuracy calculation unit 3 may calculate the accuracy information by using the normal direction of the surface and the position information of the detection device 1 (eg, the direction of the viewpoint Vp). For example, the accuracy calculation unit 3 may calculate the accuracy information according to the angle between the normal vector of the surface and the direction of the viewpoint Vp, for example. For example, the accuracy calculation unit 3 may relatively lower the accuracy of the surface when the angle between the normal vector of the surface and the direction of the viewpoint Vp is close to 0 ° or 180 °. Further, for example, the accuracy calculation unit 3 may relatively increase the accuracy of the surface when the angle between the normal vector of the surface and the direction of the viewpoint Vp is close to 90 ° or 270 °.

Further, the accuracy calculation unit 3 may calculate the accuracy information regarding the point cloud data by using the surface information generated by the information calculation unit 4. For example, the accuracy calculation unit 3 may calculate the accuracy information of the point information used for the surface information based on the distance between the surface and the point. For example, the accuracy calculation unit 3 may lower the accuracy of a point belonging to a surface when the distance from the surface is equal to or greater than the threshold value as compared with the case where the distance is less than the threshold value. The accuracy calculation unit 3 calculates the accuracy information about the point group data using the surface information, and even if the accuracy information is used to recalculate at least a part of the surface information (eg, line information, surface information). good. Further, the accuracy calculation unit 3 may store the accuracy information calculated using the surface information in the storage unit 5.

The information calculation unit 4 calculates texture information by, for example, an inverse rendering method. The texture information includes, for example, pattern information indicating a pattern on the surface of the object OB, light source information of light illuminating the object OB, and optical characteristics (eg, reflectance, scattering rate, transmittance) of the surface of the object OB. It contains information of at least one item of the optical characteristic information shown. The light source information includes, for example, information on at least one item of the position of the light source, the direction in which the light is emitted from the light source to the object, the wavelength of the light emitted from the light source, and the type of the light source.

The information calculation unit 4 calculates the light source information by using, for example, a model assuming Lambertian reflection, a model including albedo estimation, and the like. For example, the information calculation unit 4 estimates, among the pixel values of each pixel of the image captured by the image pickup unit 11, a component derived from the light diffused by the object OB and a component that is specularly reflected by the object OB. Further, the information calculation unit 4 calculates the direction in which light is incident from the light source to the object OB by using, for example, the estimation result of the component specularly reflected by the object OB and the shape information. The information calculation unit 4 estimates the reflection characteristic of the object OB using, for example, the calculated light source information and the shape information, and calculates the optical characteristic information including the estimation result of the reflection characteristic. Further, the information calculation unit 4 calculates pattern information by removing the influence of illumination light from the data of the visible light image by using, for example, the calculated light source information and optical characteristic information.

The accuracy calculation unit 3 may calculate the accuracy information regarding the texture information by using, for example, the image data which is the source of the texture information. For example, the accuracy calculation unit 3 uses at least one of the brightness (gradation value) of R, G, and B of each pixel included in the image data and the spatial frequency information of the texture calculated from the image data to obtain the accuracy. Information may be calculated. Further, the accuracy calculation unit 3 includes the characteristics of the optical system (eg, illuminating field, aberration) used for acquiring the above image data, the optical characteristics of the object (eg, reflectance, transmittance, absorptivity), and the target. Accuracy information may be calculated using at least a portion of the information from the light source of the light that illuminates the object. The information calculation unit 4 calculates texture information using, for example, accuracy information. For example, the information calculation unit 4 may selectively use a region having a relatively high accuracy from the detection results (captured images) of the detection unit 2 (eg, the imaging unit 11) to calculate the texture information. .. For example, the information calculation unit 4 does not have to use a region of the image captured by the imaging unit 11 having a relatively low accuracy for calculating the texture information. In this case, the processing load is reduced and the generated texture is generated. The amount of information data can be reduced. Further, the information calculation unit 4 may interpolate the information in the region having a relatively low accuracy by using the information in the region having a relatively high accuracy to calculate the texture information.

The accuracy calculation unit 3 calculates accuracy information for each region of the object OB (eg, each point of the point cloud data, an element of surface information). The accuracy information may include, for example, information weighted with respect to the detection result of the detection unit 2 in order to calculate the shape information or the texture information. The accuracy information may include, for example, information used for calculating the shape information or the texture information, which is the detection result of the detection unit 2 or information weighted with respect to the information generated from the detection result. The coefficient used for this weighting may be a value according to the accuracy included in the accuracy information acquired before the weighting process.

The information calculation unit 4 associates, for example, the accuracy information calculated for each region of the object OB with at least one of the shape information and the texture information. For example, the information calculation unit 4 is a data (example) in which the coordinates of each point of the point cloud data (eg, (Xi, Yi, Zi)) and the accuracy information of this point (eg, accuracy P = Pi) are combined. , (Xi, Yi, Zi, Pi)). The information calculation unit 4 associates each point of the point cloud data with the accuracy information by, for example, associating the identifier (eg, ID, number) of each point of the point cloud data with the accuracy information of this point. You may. For example, the information calculation unit 4 may generate data in which the identifier of this point is added to the accuracy information of each point in the point cloud data. Further, the information calculation unit 4 may generate data in which an identifier (ID, number) of the accuracy information of this point is added to the coordinates of each point of the point cloud data.

Further, for example, the information calculation unit 4 generates data in which elements of surface information (eg, lines, surfaces, mesh regions, voxels) and accuracy information of these elements are combined. For example, the information calculation unit 4 generates data in which an element defined in the surface information (eg, point connection information) and the accuracy information of this element are combined. Further, for example, the information calculation unit 4 generates data in which the texture information calculated for each region of the object OB (eg, one pixel of the captured image, a plurality of pixels) is combined with the accuracy information of this region.

The information calculation unit 4 may associate each element of the surface information with the accuracy information by associating the identifier (eg, ID, number) of each element of the surface information with the accuracy information of this element, for example. good. For example, the information calculation unit 4 may generate data in which the identifier of this element is added to the accuracy information of the element of the surface information. Further, the information calculation unit 4 may generate data in which an identifier (ID, number) of the accuracy information of this point is added to the information of the element of the surface information (eg, connection information of the point). The information calculation unit 4 generates model information in which header information (eg, identification information such as a number or code) is added to information including at least one of shape information and texture information. This header information includes identification information, the position (position information) of the detection device 1, the imaging timing by the imaging unit 11, the imaging time by the imaging unit 11, the optical characteristic information of the object OB, and the imaging environment information (eg, light source information, etc.). It includes at least one of the lighting conditions for the object OB and the like). In this way, the information calculation unit 4 generates model information having header information based on a predetermined data format.

The storage unit 5 is a non-volatile memory such as a USB memory or a memory card, and stores various information. The storage unit 5 may include a storage device built in the detection device 1, or may include a port to which a storage device that can be released from the detection device 1 can be connected.

The control unit 7 controls each unit of the detection device 1 by, for example, a command (control signal) from a user or an external device. For example, the control unit 7 causes the detection unit 2 to execute the above detection process. This detection process includes, for example, an image pickup process by the image pickup unit 11 and a distance detection process by the distance measuring unit 12. The control unit 7 stores, for example, at least a part of the detection result of the detection unit 2 in the storage unit 5. The control unit 7 causes, for example, the accuracy calculation unit 3 to calculate the accuracy information. For example, at least a part of the accuracy information calculated by the accuracy calculation unit 3 is stored in the storage unit 5. The control unit 7 causes, for example, the information calculation unit 4 to calculate model information. The control unit 7 stores, for example, at least a part of the model information calculated by the information calculation unit 4 in the storage unit 5.

The communication unit 6 includes, for example, an I / O port such as a USB port, and at least one of a communication device that performs radio wave or infrared wireless communication. The communication unit 6 is controlled by the control unit 7, reads out the information stored in the storage unit 5, and transmits the read information to the external device. For example, the communication unit 6 transfers at least a part of the calculation result (example, accuracy information) of the accuracy calculation unit 3 and the calculation result (example, model information) of the information calculation unit 4 to an external device (example, later in FIG. 9 and the like). It is transmitted to the information processing apparatus 51) shown. Further, the communication unit 6 receives, for example, information including a command from an external device. The communication unit 6 can store the received information in the storage unit 5 and supply the received information to the control unit 7.

Further, for example, the detection device 1 can output at least a part of the model information to a digital device capable of inputting / outputting digital information such as a barcode or a two-dimensional code. Such a digital device can display or print digital information including at least a part of model information on a display, paper, or the like. A reader device provided with a reader unit (eg, an optical reader) capable of reading displayed or printed digital information can input the digital information into a storage area or the like of the own device via the reader unit. Further, the reader device may further include a rendering processing unit described later. The detection system 50 may be configured to include a reader device including the digital device and the reader unit. Further, the detection device 1 may be configured to include the above-mentioned digital device and reader device. When the detection device 1 includes the above digital device, the communication unit 6 may transmit at least a part of the model information to the above digital device. Further, the above digital device may generate digital information based on the received model information and output the digital information to a medium such as paper.

Next, a detection method according to the present embodiment will be described based on the operation of the detection device 1 having the above configuration. FIG. 7 is a flowchart showing an example of the detection method according to the present embodiment. In step S1, the detection unit 2 detects the object OB from a predetermined one viewpoint (single viewpoint). For example, the image pickup unit 11 of the detection unit 2 images the object OB from the viewpoint Vp. Further, for example, the distance measuring unit 12 of the detection unit 2 detects the distance between the viewpoint Vp and each point on the surface of the object OB. Further, in step S2, the accuracy calculation unit 3 calculates the accuracy information of the object OB at a predetermined viewpoint. For example, the accuracy calculation unit 3 calculates the accuracy information by using at least a part of the detection result of the imaging unit 11 (eg, captured image) and the detection result of the ranging unit 12 (eg, depth information). For example, the accuracy calculation unit 3 calculates at least a part of the accuracy information described with reference to FIGS. 3 to 6. Further, in step S3, the information calculation unit 4 calculates model information including at least one of the shape information and the texture information of the object at the viewpoint Vp by using the detection result and the accuracy information of the detection unit 2.

FIG. 8 is a flowchart showing an example of the processing of step S2 and step S3 of FIG. In the present embodiment, the depth is detected by, for example, the ranging unit 12 prior to the calculation of the accuracy information. In step S11, the accuracy calculation unit 3 calculates the accuracy information based on the distance (depth) detected by the distance measuring unit 12, for example, as shown in FIGS. 4 to 6. Further, in step S13, the information calculation unit 4 calculates information in which the accuracy information calculated in step S11 is associated with the depth information generated in step S12. In step S14, the information calculation unit 4 calculates the point cloud data using the information calculated in step S13. In step S15, the information calculation unit 4 generates information in which the point cloud data and the accuracy information are associated with each other. In step S16, the information calculation unit 4 calculates surface information using the information calculated in step S15. In step S17, the information calculation unit 4 calculates information in which surface information and accuracy information are associated with each other.

FIG. 9 is a flowchart showing another example of the processing of step S2 and step S3 of FIG. In FIG. 9, the same processing as in FIG. 8 is designated by the same reference numerals, and the description thereof will be omitted or simplified as appropriate. In step S11, the detection device 1 (eg, controller 18) generates depth information. In step S14, the information calculation unit 4 calculates the point cloud data. In step S21, the accuracy calculation unit 3 generates the accuracy information of the point cloud data calculated by the information calculation unit 4 in step S14.

The accuracy calculation unit 3 stores, for example, the accuracy information of the generated point cloud data in the storage unit 5. In step S22, the information calculation unit 4 calculates line information among the surface information using the point cloud data and the accuracy information of the point cloud data generated by the accuracy calculation unit 3 in step S21. In step S23, the accuracy calculation unit 3 generates the accuracy information of the line information calculated by the information calculation unit 4 in step S22. The accuracy calculation unit 3 stores, for example, the accuracy information of the generated line information in the storage unit 5. In step S24, the information calculation unit 4 calculates the surface information of the surface information by using the line information and the accuracy information of the line information generated by the accuracy calculation unit 3 in step S23. In step S25, the accuracy calculation unit 3 generates accuracy information of the surface information calculated by the information calculation unit 4 in step S24. The accuracy calculation unit 3 stores, for example, the accuracy information of the generated surface information in the storage unit 5.

In step S26, the accuracy calculation unit 3 calculates the accuracy information of the point cloud data by using the surface information calculated by the information calculation unit 4 in step S24. The accuracy calculation unit 3 may update the accuracy information of the point cloud data generated in step S21 to the accuracy information of the point cloud data generated in step S26. Further, the accuracy calculation unit 3 integrates the accuracy information of the point group data generated in step S21 and the accuracy information of the point group data generated in step S26 by arithmetic mean, geometric mean, weighted average, or the like. May be good. Further, the accuracy calculation unit 3 may store the accuracy information of the point cloud data generated in step S26 in the storage unit 5 separately from the accuracy information of the point cloud data generated in step S21. In step S27, the detection device 1 (eg, the information calculation unit 4) determines whether or not to recalculate the surface information. For example, the information calculation unit 4 performs the determination process in step S27 according to the setting information for which whether or not to recalculate is determined in advance. When the information calculation unit 4 determines that the surface information is to be recalculated (step S27; Yes), the information calculation unit 4 returns to step S22 and uses at least a part of the accuracy information generated in the process of the previous step S22 to step S26. The processing after step S22 is repeated. When the information calculation unit 4 determines that the surface information is not recalculated (step S27; No), the information calculation unit 4 ends a series of processes.

The accuracy calculation unit 3 generates accuracy information of point group data in step S21, generates accuracy information of line information in step S23, generates accuracy information of surface information in step S25, and in step S26. It is not necessary to perform at least one generation process of generating the accuracy information of the point group data. In this case, the information calculation unit 4 may generate at least a part of the surface information (eg, line information, surface information) without using the accuracy information of the item for which the above generation process has not been performed. .. Further, the information calculation unit 4 may generate surface information (eg, line information, surface information) without using the accuracy information of the item for which the above generation processing has been performed. In this case, the accuracy information obtained from the above generation process may be used for a process other than the surface information calculation process, for example, a process using the surface information (eg, a rendering process). Further, the information calculation unit 4 does not have to perform the process of step S27, and does not have to recalculate the surface information using at least a part of the accuracy information.

In the above-described embodiment, the detection device 1 includes, for example, a computer (eg, a microcomputer). This computer reads out the processing program stored in the storage unit 5 and executes various processes according to the processing program. This processing program processes, for example, the detection result in which the object OB is optically detected by the detection unit 2 from a predetermined viewpoint (eg, viewpoint Vp). This processing program calculates, for example, the accuracy information of the object OB in a predetermined one viewpoint (eg, viewpoint) Vp on a computer, and uses the detection result and the accuracy information of the detection unit 2 to obtain a predetermined one viewpoint. (Example, generating at least one of the shape information and the texture information of the object OB in the viewpoint Vp) is executed. This processing program may be recorded and provided on a computer-readable storage medium.

[Second Embodiment]
The second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 10 is a diagram showing an example of the detection system 50 according to the present embodiment. The detection system 50 includes a detection device 1 and an information processing device 51 that processes information output from the detection device 1. The information processing device 51 is provided with, for example, an input device 52 and a display device 53.

The information processing device 51 acquires information from the detection device 1 by communication with the detection device 1. The information processing device 51 executes a rendering process using, for example, information (eg, model information, accuracy information) acquired from the detection device 1. For example, the information processing device 51 calculates the data of the estimated image of the object OB viewed from this viewpoint based on the setting information of the viewpoint input to the input device 52 by the user. The information processing device 51 supplies, for example, the data of the estimated image to the display device 53, and causes the display device 53 to display the estimated image.

The input device 52 includes, for example, at least one such as a keyboard, a mouse, a touch panel, a sensor such as an acceleration sensor, a voice input device, and a touch-type pen. The input device 52 is connected to the information processing device 51. The input device 52 receives, for example, input of information from a user, and supplies the input information to the information processing device 51. The display device 53 includes, for example, a liquid crystal display or a touch panel type display, and is connected to the information processing device 51. The display device 53 displays an image (eg, an estimated image by rendering processing) using image data supplied from the information processing device 51, for example.

FIG. 11 is a block diagram showing an example of the detection system 50 according to the present embodiment. The information processing device 51 includes a communication unit 55, a storage unit 56, a rendering processing unit 57, and a control unit 58. The communication unit 55 includes, for example, a USB port, a network card, and at least one of a communication device that performs radio wave or infrared wireless communication. The communication unit 55 can communicate with the communication unit 6 of the detection device 1.

The storage unit 56 includes, for example, a removable storage medium such as a USB memory, and a large-capacity storage device such as an external or internal hard disk. The storage unit 56 stores, for example, at least a part of the data of the information received via the communication unit 55, an image pickup control program for controlling the detection device 1, a processing program for executing each process of the information processing device 51, and the like. ..

The rendering processing unit 57 includes, for example, a graphics processing unit (GPU). The rendering processing unit 57 may have a mode in which the CPU and the memory execute each processing according to the image processing program. The rendering processing unit 57 performs at least one of drawing processing, texture mapping processing, and shading processing, for example.

In the drawing process, the rendering processing unit 57 can calculate, for example, an estimated image (eg, a reconstructed image) of the shape defined in the shape information of the model information viewed from an arbitrary viewpoint. In the following description, the shape indicated by the shape information is referred to as a model shape. The rendering processing unit 57 can reconstruct the model shape (eg, estimated image) from the model information (eg, shape information) by, for example, drawing processing. The rendering processing unit 57 stores, for example, the calculated estimated image data in the storage unit 56. Further, in the texture mapping process, the rendering processing unit 57 can calculate, for example, an estimated image in which the image indicated by the texture information of the model information is attached to the surface of the object on the estimated image. The rendering processing unit 57 can also calculate an estimated image in which a texture different from the object OB is attached to the surface of the object on the estimated image. In the shading process, the rendering processing unit 57 can calculate, for example, an estimated image in which a shadow formed by a light source indicated by the light source information of the model information is added to an object on the estimated image. Further, in the shading process, the rendering processing unit 57 can calculate, for example, an estimated image in which a shadow formed by an arbitrary light source is added to an object on the estimated image.

The rendering processing unit 57 performs rendering processing using, for example, the accuracy information generated by the detection device 1. For example, the rendering processing unit 57 may set the resolution of the region of the model shape having a relatively high accuracy higher than the resolution of the region having a relatively low accuracy. For example, the rendering processing unit 57 may generate an estimated image by reducing the resolution (eg, blurring) of a region of the model shape in which the accuracy is relatively low. For example, the rendering processing unit 57 may omit or simplify the rendering processing of the region of the model shape whose accuracy is relatively low. For example, the rendering processing unit 57 may perform rendering processing by interpolating a region of the model shape having a relatively low accuracy using a region having a relatively high accuracy.

The control unit 58 controls, for example, each unit of the information processing device 51, the detection device 1, the input device 52, and the display device 53. The control unit 58 controls, for example, the communication unit 55, and causes the detection device 1 to transmit a command (control signal) and setting information. The control unit 58 stores, for example, the information received from the detection device 1 by the communication unit 55 in the storage unit 56. The control unit 58 controls, for example, the rendering processing unit 57 to execute the rendering processing.

The detection system 50 does not have to include the input device 52. For example, the detection system 50 may be in a form in which various commands and information are input via the communication unit 6. Further, the detection system 50 does not have to include the display device 53. For example, the detection system 50 may output the data of the estimated image generated by the rendering process to an external display device, and this display device may display the estimated image.

[Third Embodiment]
The third embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 12 is a diagram showing a detection system 50 according to the present embodiment. The detection system 50 includes a plurality of detection devices (first detection device 1a, second detection device 1b) and an information processing device 51 that processes information output from the plurality of image pickup devices.

The information processing device 51 receives information (eg, model information, accuracy information) from the first detection device 1a arranged at the position of the first viewpoint with respect to the object OB by communication with the first detection device 1a. get. The information processing device 51 receives information (eg, model information, accuracy information) from the second detection device 1b by communication with the second detection device 1b arranged at the position of the second viewpoint with respect to the object OB. get. The information processing device 51 uses the information acquired from the first detection device 1a that detects the object OB from the first viewpoint and the information acquired from the second detection device 1b that detects the object OB from the second viewpoint. Information processing. For example, the first detection device 1a and the second detection device 1b each supply model information representing an object OB viewed from the viewpoint (one viewpoint, a single viewpoint, one direction) of the own device to the information processing device 51. do. The information processing device 51 includes first model information representing the object OB viewed from the viewpoint (first viewpoint) of the first detection device 1a and the viewpoint of the second detection device 1b (second viewpoint different from the first viewpoint). Performs model integration processing that integrates with the second model information that represents the object seen from above.

FIG. 13 is a block diagram showing a detection system 50 according to the present embodiment. Each of the first detection device 1a and the second detection device 1b has the same configuration as the detection device 1 shown in FIG. 1, for example. The information processing device 51 includes a receiving unit (communication unit 55). The communication unit 55 receives the first model information of the object OB and the second model information of the object OB. The first model information includes at least one of the shape information and the texture information of the object in one viewpoint (eg, the viewpoint Vp1 shown later in FIG. 14). The first model information is calculated by the first detection device 1a using, for example, the first detection result by the first detection device 1a and the first accuracy information, and the communication unit 55 is the first model from the first detection device 1a. Receive information. The second model information includes at least one of the shape information and the texture information of the object at a viewpoint different from the above one viewpoint (eg, the viewpoint Vp2 shown later in FIG. 14). The second model information is calculated by the second detection device 1b using, for example, the second detection result by the second detection device 1b and the second accuracy information, and the communication unit 55 is the second model from the second detection device 1b. Receive information.

The information processing device 51 includes a model integration unit 59 that performs model integration processing. The model integration unit 59 integrates the above-mentioned first model information and the second model information. The model integration unit 59, for example, extracts feature points from the shape indicated by the first model information from the first detection device 1a. Further, the model integration unit 59 extracts feature points from the shape indicated by the second model information from the second detection device 1b. The feature point is a part of the shape indicated by each model information that can be distinguished from other parts. For example, a portion defined as a surface in surface information can be distinguished from other surfaces by the shape of the outer circumference thereof. For example, the model integration unit 59 extracts feature points using at least one of the shape information and the texture information included in each model information. Further, the model integration unit 59, for example, matches the feature points with the first model information and the second model information, and detects the feature points common to the first model information and the second model information. Further, the model integration unit 59 uses the feature points common to the first model information and the second model information to determine the relative position and the relative posture of the shape indicated by the first model information and the shape indicated by the second model information. Calculate and integrate the first model information and the second model information.

The model integration unit 59 uses, for example, at least a part of the accuracy information (first accuracy information) supplied from the first detection device 1a and the accuracy information (second accuracy information) supplied from the second detection device 1b. And perform model integration processing. The model integration unit 59 selects information to be used for integration from the model information, for example, based on the accuracy information. For example, when the model integration unit 59 extracts the feature points from the shape indicated by the model information, the model integration unit 59 extracts the feature points from the region having a relatively high accuracy in the shape. Further, for example, when the shape indicated by the first model information and the shape indicated by the second model information have an overlapping portion, the model integrating unit 59 uses the first accuracy information and the second accuracy information to make a second in the overlapping portion. The accuracy of the 1st model information and the accuracy of the 2nd model information are compared, and the shape of the overlapping portion is represented by using the model information having the relatively higher accuracy. Further, the model integration unit 59 may integrate the shape information by weighting the first model information and the second model information (calculation of the weighted average) for the above-mentioned overlapping portion, for example. Weighted averages include, for example, arithmetic mean, geometric mean, or exponential average (exponential weighted moving average). The coefficient of this weighting is determined based on, for example, the accuracy of the first model information in the overlapping portion and the accuracy of the second model information in the overlapping portion.

FIG. 14 is a conceptual diagram showing an example of the model integration process. Reference numeral MF1 is a model shape generated based on the detection result from the viewpoint (first viewpoint) Vp1 of the first detection device 1a. Reference numeral MF2 is a model shape generated based on the detection result from the viewpoint (second viewpoint) Vp2 of the second detection device 1b. Reference numeral MF3 is an integrated model shape (integrated model shape).

In the model shape MF1 viewed from the viewpoint Vp1, for example, the surfaces Aa, Ba, and Ca have high reliability (accuracy) of the detection result from the viewpoint Vp1. In the model shape MF1, for example, the surface Da and the surface Ea have low reliability (accuracy) of the detection result from the viewpoint Vp1. Further, in the model shape MF2 viewed from the viewpoint Vp2 in a direction different from the viewpoint Vp1 with respect to the object OB, for example, the surface Cb, the surface Db, and the surface Eb have the reliability (accuracy) of the detection result from the viewpoint Vp2. high. Further, in the model shape MF2, for example, the surface Ab and the surface Bb have low reliability (accuracy) of the detection result from the viewpoint Vp2. In such a case, the model integration unit 59 uses, for example, the surfaces Aa and Ab of the model shape MF1 for generating the model shape MF3, and the surfaces Db and Eb of the model shape MF2 for generating the model shape MF3. Further, the model integration unit 59 weights the surface Ca of the model shape MF1 and the surface Cb of the model shape MF2 and uses them for the C of the model shape MF3, for example.

The method by which the model integration unit 59 integrates the first model information and the second model information is not limited to the above example. For example, the model integration unit 59 may perform the model integration process using the relative positions and relative postures of the first detection device 1a and the second detection device 1b. Further, the model integration unit 59 is a relative position between the viewpoint of the first detection device 1a and the viewpoint of the second detection device 1b, the direction (line of sight) of the viewpoint of the first detection device 1a, and the second detection device 1b. The model integration process may be performed using the relationship with the direction of the viewpoint (line of sight). Further, the information processing apparatus 51 includes a model integration unit 59 and does not have to include a rendering processing unit 57. For example, the information processing device 51 may output the result of the model integration process to an external device, and the rendering processing unit provided in the external device may execute the rendering process.

The model integration unit 59 may be provided in at least one of a plurality of detection devices (eg, first detection device 1a, second detection device 1b). For example, the first detection device 1a may include a model integration unit 59. In this case, for example, the first detection device 1a receives the second model information from the detection device (second detection device 1b) different from the first detection device 1a by the communication unit 6, and the model integration unit 59 receives the first model information. The first model information calculated by the information calculation unit 4 of the detection device 1a and the second model information may be integrated.

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. Further, the requirements described in the above-described embodiments and the like can be appropriately combined. In addition, to the extent permitted by law, the disclosure of all documents cited in the above-mentioned embodiments and the like shall be incorporated as part of the description in the main text.

1, 1a, 1b ... Detection device, 2 ... Detection unit, 3 ... Accuracy calculation unit, 4 ... Information calculation unit, 11 ... Imaging unit, 12 ... Distance measuring unit, 50 ... detection system, 51 ... information processing device, 57 ... rendering processing unit, 59 ... model integration unit, Vp ... viewpoint, AR1 ... first area, AR2 ... second region

Claims (39)

With the detector
A calculation unit that calculates information that identifies a first region that can be detected by the detection unit and a second region that cannot be detected by the detection unit or whose reliability of the detection result of the detection unit is lower than that of the first region. And with
The detection unit detects an object from one viewpoint and
The calculation unit calculates the accuracy information of the object at the one viewpoint by using the detection result of the detection unit.
As the accuracy information, the calculation unit distinguishes between a first region that can be detected from the one viewpoint and a second region that is behind the first region when viewed from the one viewpoint. Information is calculated, and a region farther from the one viewpoint than the point on the line connecting the one viewpoint and the point on the object is determined to be the second region.
A detection device including an information calculation unit that calculates shape information of the object using the detection result and the accuracy information . The calculation unit defines the region from the one viewpoint to the point on the line connecting the one viewpoint and the point on the object as the first region, and in the first region, the one viewpoint and the object. The detection device according to claim 1, wherein the information indicating that the existence probability of an object that blocks light between the points and the points is low is calculated. The calculation unit defines the region from the one viewpoint to the object point as the first region on the line connecting the one viewpoint and the point on the object, and in the first region, the one viewpoint and the object. The detection device according to claim 1 or 2, wherein the information indicating the existence of a gap is calculated between the points and the points. The detection device according to any one of claims 1 to 3 , wherein the information calculation unit generates model information of the object in the one viewpoint based on the shape information. From claim 1 , the information calculation unit calculates texture information of the object using the detection result, and generates model information of the object in the one viewpoint based on the shape information and the texture information. The detection device according to any one of claims 4 . The calculation unit calculates the accuracy information for each region of the object, and the calculation unit calculates the accuracy information.
The detection device according to claim 5 , wherein the information calculation unit associates the accuracy information calculated for each region of the object with the shape information or the texture information. The detection device according to any one of claims 1 to 6 , wherein the detection result of the detection unit includes image data of the detected object. The detection device according to claim 5 or 6 , wherein the accuracy information includes information weighted with respect to the detection result in order to calculate the shape information or the texture information. The detection device according to any one of claims 1 to 8 , wherein the calculation unit calculates information indicating that the second region is a region that cannot be detected from the one viewpoint as the accuracy information. .. The detection unit includes a distance measuring unit that detects a distance from the one viewpoint to each point on the object.
The detection device according to any one of claims 1 to 9 , wherein the calculation unit calculates the accuracy information based on the distance detected by the distance measuring unit. The detection according to any one of claims 1 to 10 , wherein the accuracy information includes at least one of a probability, a probability distribution, and an expected value that an object exists at each position in a region including the object. Device. Based on the detection result of the detection unit, depth information including the distance from the one viewpoint to each point on the object is generated.
The detection device according to any one of claims 1 to 6 , wherein the information calculation unit generates information in which the depth information and the accuracy information are associated with each other. The information calculation unit interpolates a region of a distance image indicated by the depth information having a relatively low accuracy by using a region having a relatively high accuracy, and performs fluoroscopic conversion from the distance image to a plane image. The detection device according to claim 12 . The information calculation unit calculates point group data including coordinates of a plurality of points on the object based on the detection result of the detection unit as the shape information, and obtains the point group data and the accuracy information. The detection device according to any one of claims 1 to 6 , which generates the associated information. The detection device according to claim 14 , wherein the calculation unit uses the point cloud data calculated by the information calculation unit to generate the accuracy information of the point cloud data. The calculation unit includes distance information between two points included in the point cloud data, spatial frequency information of a plurality of points included in the point cloud data, vector information connecting two points included in the point cloud data, and the above. The detection device according to claim 15 , wherein the accuracy information of the point cloud data is generated by using at least a part of the position information of one viewpoint. The information calculation unit uses the detection result of the detection unit and the accuracy information as the shape information to obtain surface information including coordinates of a plurality of points on the object and connection information between the plurality of points. The detection device according to any one of claims 1 to 6 , which is calculated. The surface information includes information on a line connecting two points on the object and information on a surface surrounded by three or more lines.
The detection device according to claim 17 , wherein the information calculation unit generates information on the surface by using the accuracy information of the information on the line. The calculation unit uses the surface information to generate accuracy information of point cloud data including the coordinates of a plurality of points on the object.
The detection device according to claim 17 , wherein the information calculation unit recalculates at least a part of the surface information by using the accuracy information of the point cloud data. A claim that includes a model integration unit that receives the model information and second model information including shape information of the object at a viewpoint different from the one viewpoint, and integrates the model information and the second model information. The detection device according to any one of items 4 to 6 . The model information and the second model information including at least one of the shape information and the texture information of the object at a viewpoint different from the one viewpoint are received, and the model information and the second model information are integrated. The detection device according to any one of claims 4 to 6 , further comprising a model integration unit. The detection device according to any one of claims 4 to 6 , which outputs digital information including at least a part of the model information. Using the detection result and the accuracy information, point cloud data indicating the coordinates of a plurality of points on the object is calculated, and the shape information of the object at the one viewpoint is calculated based on the point cloud data. The detection device according to any one of claims 1 to 3, further comprising an information calculation unit. The information calculation unit selects a point in the vicinity by using the accuracy information when estimating a surface between a point selected from a plurality of points in the point cloud data and a point in the vicinity thereof. Item 23. The detection device. A claim that the information calculation unit selects a point having relatively high accuracy information when estimating a surface between a point selected from a plurality of points in the point cloud data and a point in the vicinity thereof. 23 or the detection device according to claim 24 . The detection result includes a captured image of the detected object.
The detection device according to any one of claims 23 to 25 , wherein the calculation unit calculates information in which the accuracy is associated with each pixel of the captured image as the accuracy information. The detection device according to any one of claims 23 to 26 , which outputs digital information including the shape information. The detection unit captures an object from a single viewpoint.
The calculation unit calculates the accuracy information of the object at the one viewpoint using the captured image captured by the detection unit.
The detection device according to any one of claims 1 to 3 , further comprising an information calculation unit for calculating texture information of the object at the one viewpoint using the captured image and the accuracy information. The information calculation unit calculates point cloud data indicating coordinates between a plurality of points on the object using the captured image, and based on the point cloud data, shape information of the object at the one viewpoint. 28. The detection device according to claim 28 . 29. The detection device according to claim 29 , which outputs digital information including the texture information or the shape information. The detection unit includes an imaging unit that captures an image of the object.
The calculation unit describes, for each region on the object detected by the detection unit, the distance from the center of the field of view of the image pickup unit, the brightness of the image captured by the image pickup unit, the aberration of the optical system of the image pickup unit, and the above. Amount of change in distance from one viewpoint, difference between the normal vector of the object and the direction of the one viewpoint, distance information between two points on the object, spatial frequency information of a plurality of points on the object. The detection device according to any one of claims 1 to 30 , which calculates the accuracy information based on at least one of the vector information connecting the two points on the object. The plurality of detection devices according to any one of claims 1 to 31 , and the plurality of detection devices.
An information processing device that processes information output from the plurality of detection devices is provided.
The plurality of detection devices output model information including at least one of the shape information of the object and the texture information of the object, respectively.
The information processing device is a detection system including a model integration unit that integrates information output from the plurality of detection devices. The detection system according to claim 32 , wherein the model integration unit integrates information output from the plurality of detection devices using the accuracy information of the object in one viewpoint. The detection device according to any one of claims 1 to 31 and
A detection system including an information processing device that processes information output from the detection device. The detection system according to any one of claims 32 to 34 , wherein the information processing apparatus includes a rendering processing unit that performs rendering processing. The detection system according to claim 35 , wherein the rendering processing unit performs rendering processing using the accuracy information of the object in one viewpoint. Detecting an object from one viewpoint by the detection unit,
To calculate information that identifies a first region that can be detected by the detection unit and a second region that cannot be detected by the detection unit or whose reliability of the detection result of the detection unit is lower than that of the first region. ,
Using the detection result of the detection unit to calculate the accuracy information of the object at the one viewpoint,
As the accuracy information, information for distinguishing the first region that can be detected from the one viewpoint and the second region that is behind the first region when viewed from the one viewpoint is calculated among the regions including the object. Then, on the line connecting the one viewpoint and the point on the object, the region farther from the one viewpoint than the point is determined to be the second region.
A detection method including calculating shape information of the object using the detection result and the accuracy information . The first model information including the shape information of the object in one viewpoint, which was calculated by using the first detection result and the first accuracy information, and the second detection result and the second accuracy information, which were calculated by using the second detection result and the second accuracy information. A receiving unit that receives the second model information including the shape information of the object from a viewpoint different from the one viewpoint, and the receiving unit.
A model integration unit that integrates the first model information and the second model information is provided.
The first accuracy information and the second accuracy information are calculated as information for distinguishing a first region that can be detected from a second region that cannot be detected or whose reliability of the detection result is lower than that of the first region .
The shape information of the object in the one viewpoint is
Using the detection result of the object detected by the detection unit from the one viewpoint, the accuracy information of the object in the one viewpoint is calculated.
As the accuracy information, information for distinguishing the first region that can be detected from the one viewpoint and the second region that is behind the first region when viewed from the one viewpoint is calculated among the regions including the object. Then, on the line connecting the one viewpoint and the point on the object, the region farther from the one viewpoint than the point is determined to be the second region.
An information processing device that calculates shape information of the object using the detection result and the accuracy information . It is a processing program that processes the detection result of detecting an object by the detection unit from one viewpoint .
The computer calculates information that identifies a first region that can be detected by the detection unit and a second region that cannot be detected by the detection unit or whose reliability of the detection result of the detection unit is lower than that of the first region. To do and
Using the detection result of the detection unit to calculate the accuracy information of the object at the one viewpoint,
As the accuracy information, information for distinguishing the first region that can be detected from the one viewpoint and the second region that is behind the first region when viewed from the one viewpoint is calculated among the regions including the object. Then, on the line connecting the one viewpoint and the point on the object, the region farther from the one viewpoint than the point is determined to be the second region.
A processing program that calculates shape information of the object using the detection result and the accuracy information, and executes the calculation.

Images