CN106997586B - Object material determination method, device and system - Google Patents
Object material determination method, device and system Download PDFInfo
- Publication number
- CN106997586B CN106997586B CN201610048980.1A CN201610048980A CN106997586B CN 106997586 B CN106997586 B CN 106997586B CN 201610048980 A CN201610048980 A CN 201610048980A CN 106997586 B CN106997586 B CN 106997586B
- Authority
- CN
- China
- Prior art keywords
- target object
- infrared
- visible light
- image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000000463 material Substances 0.000 title claims abstract description 166
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000002310 reflectometry Methods 0.000 claims abstract description 187
- 238000005259 measurement Methods 0.000 claims abstract description 60
- 239000013598 vector Substances 0.000 claims description 32
- 239000000284 extract Substances 0.000 claims description 22
- 230000011218 segmentation Effects 0.000 claims description 20
- 230000003287 optical effect Effects 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The embodiment of the invention discloses a method, a device and a system for determining an object material. An object material determination method applied to a processor may include the steps of: receiving a visible light image of a target scene containing a target object sent by a visible light camera, and extracting visible light characteristics of the target object from the visible light image; receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera, and extracting the infrared reflectivity characteristics of the target object from the infrared reflectivity image; and comparing the extracted visible light characteristic and the infrared reflectivity characteristic with the corresponding characteristic of each material in a preset material characteristic library to determine the material of the target object. By applying the technical scheme provided by the embodiment of the invention, the accuracy of object material identification is improved.
Description
Technical Field
The invention relates to the technical field of video monitoring, in particular to a method, a device and a system for determining an object material.
Background
In video surveillance and the like, it may be desirable to identify the material of a target object in a video image.
At present, the material of the target object is mainly identified by the visible light features, that is, after the visible light image of the target object is obtained, the visible light features of the target object, such as texture features, gray scale features, color features, etc., are extracted from the visible light image. From these visible light characteristics, the material of the target object is determined.
In fact, the visible light characteristics of objects made of different materials may be the same, and the determination of the object material by applying the method in the prior art is difficult to identify different materials with the same visible light characteristics, and the identification accuracy is low.
Disclosure of Invention
The embodiment of the invention aims to provide a method, a device and a system for determining an object material so as to improve the identification accuracy of the object material. The technical scheme is as follows:
an object material determination method applied to a processor, the method comprising:
receiving a visible light image of a target scene containing a target object sent by a visible light camera, and extracting visible light characteristics of the target object from the visible light image;
receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera, and extracting the infrared reflectivity characteristics of the target object from the infrared reflectivity image;
and comparing the extracted visible light characteristic and the infrared reflectivity characteristic with the corresponding characteristic of each material in a preset material characteristic library to determine the material of the target object.
In an embodiment of the present invention, the extracting the visible light feature of the target object from the visible light image includes:
and carrying out target segmentation on the visible light image, determining a first target area containing a specific position of the target object, and extracting visible light characteristics of the target object from the first target area.
In an embodiment of the present invention, the extracting the ir-reflectivity characteristics of the target object from the ir-reflectivity image includes:
and carrying out target segmentation on the infrared reflectivity image, determining a second target area containing the specific position of the target object, and extracting the infrared reflectivity characteristics of the target object from the second target area.
In an embodiment of the present invention, the comparing the extracted visible light characteristic and the infrared reflectivity characteristic with a corresponding characteristic of each material in a preset material characteristic library to determine the material of the target object includes:
determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
An object material determination apparatus for use with a processor, the apparatus comprising:
the visible light feature extraction unit is used for receiving a visible light image of a target scene containing a target object and sent by a visible light camera, and extracting visible light features of the target object from the visible light image;
the infrared reflectivity characteristic extraction unit is used for receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera and extracting the infrared reflectivity characteristic of the target object from the infrared reflectivity image;
and the material determining unit is used for comparing the extracted visible light characteristics and the extracted infrared reflectivity characteristics with corresponding characteristics of each material in a preset material characteristic library to determine the material of the target object.
In an embodiment of the present invention, the visible light feature extraction unit is specifically configured to:
and carrying out target segmentation on the visible light image, determining a first target area containing a specific position of the target object, and extracting visible light characteristics of the target object from the first target area.
In an embodiment of the present invention, the infrared reflectivity feature extracting unit is specifically configured to:
and carrying out target segmentation on the infrared reflectivity image, determining a second target area containing the specific position of the target object, and extracting the infrared reflectivity characteristics of the target object from the second target area.
In an embodiment of the invention, the material determination unit is specifically configured to:
determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
An object material determination system comprising a visible light camera, an infrared reflectance measurement camera, a processor:
the visible light camera is used for obtaining a visible light image of a target scene containing a target object and sending the visible light image to the processor;
the infrared reflectivity measuring camera is used for transmitting infrared light to the target object, receiving reflected light reflected by the target object, obtaining reflected light energy of the reflected light, depth information of the target object and position information of the target object, determining the infrared reflectivity of the target object according to the obtained reflected light energy, depth information and position information, obtaining an infrared reflectivity image of the target scene, and sending the infrared reflectivity image to the processor;
the processor is configured to receive the visible light image sent by the visible light camera, extract the visible light feature of the target object from the visible light image, receive the infrared reflectance image sent by the infrared reflectance measurement camera, extract the infrared reflectance feature of the target object from the infrared reflectance image, compare the extracted visible light feature and infrared reflectance feature with corresponding features of each material in a preset material feature library, and determine the material of the target object.
In one embodiment of the present invention, the infrared reflectance measuring camera includes: infrared emission module, infrared receiving module and degree of depth measurement module:
the infrared emission module is used for emitting infrared light to the target object;
the depth measuring module is used for receiving reflected light reflected by the target object, obtaining depth information of the target object and position information of the target object, and sending the depth information and the position information to the infrared receiving module;
the infrared receiving module is used for receiving the reflected light reflected by the target object, obtaining the reflected light energy of the reflected light, determining the infrared reflectivity of the target object according to the obtained reflected light energy, depth information and position information, obtaining an infrared reflectivity image of the target scene, and sending the infrared reflectivity image to the processor.
In an embodiment of the present invention, the infrared reflectance measurement camera is specifically configured to:
according to the formulaDetermining an infrared reflectance ρ of the target object1；
Wherein,
after the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object to obtain the energy of the reflected light; r1After the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object, and determining the distance from the target object to the infrared reflectivity measurement camera according to the obtained depth information of the target object; theta1Is the angle between the reflected light reflected by the target object and the optical axis of the infrared reflectivity measuring camera;
R2、ρ2、θ2calibration parameters of a reference object obtained in advance for the infrared reflectance measurement camera;
is the energy of reflected light reflected by the reference object; r2Measuring a distance of the reference object to the infrared reflectance measurement camera; rho2Is the infrared reflectivity of the reference object; theta2The included angle between the reflected light reflected by the reference object and the optical axis of the infrared reflectivity measuring camera is obtained in advance.
In one embodiment of the invention, the infrared reflectance measurement camera is a time-of-flight ToF depth camera.
By applying the technical scheme provided by the embodiment of the invention, the processor obtains the visible light image of the target scene containing the target object through the visible light camera, obtains the infrared reflectivity image of the target scene through the infrared reflectivity measuring camera, respectively extracts the visible light characteristics of the target object from the visible light image, extracts the infrared reflectivity characteristics of the target object from the infrared reflectivity image, compares the extracted visible light characteristics and infrared reflectivity characteristics with the corresponding characteristics of each material in the preset material characteristic library to determine the material of the target object, and improves the accuracy of object material identification.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of an embodiment of a method for determining a material of an object according to the present invention;
FIG. 2 is a flow chart of another embodiment of a method for determining a material of an object according to the present invention;
FIG. 3 is a schematic diagram showing a structure of an object material determination apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of one configuration of an object material determination system in accordance with an embodiment of the present invention;
FIG. 5 is a schematic view of another configuration of an object material determination system according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of IR reflectance measurement parameters according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating included angle calculation parameters according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of an implementation of a method for determining a material of an object according to an embodiment of the present invention is provided, where the method is applied to a processor, and may include the following steps:
s110: receiving a visible light image of a target scene containing a target object sent by a visible light camera, and extracting visible light characteristics of the target object from the visible light image;
in embodiments of the present invention, the processor may be in communication with a visible light camera. After the visible light camera obtains a visible light image of a target scene containing a target object, the visible light image is sent to the processor, the processor receives the visible light image, and visible light characteristics of the target object can be extracted from the visible light image.
In embodiments of the present invention, the visible light features may include, but are not limited to, texture features, grayscale features, color features, and the like.
In one embodiment of the present invention, the processor may perform target segmentation on the visible light image, determine a first target region including a specific position of the target object, and extract visible light features of the target object from the first target region.
Target segmentation is the process of dividing an image into several image regions that have feature consistency and do not overlap each other. According to a preset segmentation algorithm, the processor can perform target segmentation on the received visible light image, determine a first target area containing a specific position of a target object, namely an interested target area, and extract visible light features in the segmented first target area.
S120: receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera, and extracting the infrared reflectivity characteristics of the target object from the infrared reflectivity image;
in embodiments of the present invention, the processor may be in communication with an infrared reflectance measurement camera. After the infrared reflectivity measurement camera obtains the infrared reflectivity image of the target scene, the infrared reflectivity image is sent to the processor, the processor receives the infrared reflectivity image, and the infrared reflectivity characteristics of the target object can be extracted from the infrared reflectivity image.
In the embodiment of the invention, the infrared reflectivity characteristic is the magnitude of the infrared reflectivity. The target scene corresponding to the infrared reflectivity image is consistent with the target scene corresponding to the visible light image, and both the target scene and the visible light image contain target objects.
In practical applications, the infrared reflectivity measurement camera may include an infrared emission module, an infrared reception module, and a depth measurement module. The infrared transmitting module can transmit infrared light to a target object, the infrared light can be subjected to diffuse reflection when encountering the target object, the infrared receiving module can receive reflected light reflected by the target object and obtain reflected light energy, the depth measuring module can also receive the reflected light reflected by the target object and obtain depth information of the target object, namely the distance from the target object to the infrared reflectivity measuring camera, and the depth measuring module can send the depth information of the target object to the infrared receiving module. The infrared receiving module can determine the infrared reflectivity of the target object according to the reflected light energy and the depth information of the target object, obtain an infrared reflectivity image of the target object and send the infrared reflectivity image to the processor.
In one embodiment of the present invention, the processor may perform target segmentation on the ir-reflectance image, determine a second target region including a specific location of the target object, and extract ir-reflectance characteristics of the target object from the second target region.
As mentioned above, the target segmentation is a process of dividing an image into a plurality of image regions having feature consistency and not overlapping each other. According to a preset segmentation algorithm, the processor can perform target segmentation on the received infrared reflectivity image, determine a second target area containing a specific position of the target object, namely an interested target area, and extract infrared reflectivity characteristics in the segmented second target area. Here, the second target region and the first target region are both regions including a specific position of the target object, and the first and second target regions are only images for distinguishing between different images.
Of course, in practical applications, the target region of interest in the visible light image and the infrared reflectivity image may also be determined manually.
It should be noted that step S110 and step S120 may be executed simultaneously, or may also be executed sequentially, for example, step S110 is executed first, and then step S120 is executed, or step S120 is executed first, and then step S110 is executed, which is not limited in this embodiment of the present invention.
S130: and comparing the extracted visible light characteristic and the infrared reflectivity characteristic with the corresponding characteristic of each material in a preset material characteristic library to determine the material of the target object.
It will be appreciated that each material has certain visible light and ir reflectance characteristics, that the visible light characteristics of different materials may be the same, and that the ir reflectance characteristics of materials having the same visible light characteristics may be different. A material can be identified by fusing the visible light characteristic and the infrared reflectivity characteristic.
In practical application, the visible light characteristics of various materials can be obtained through the measurement of the visible light camera, the infrared reflectivity characteristics of various materials can be obtained through the measurement of the infrared reflectivity measurement camera, namely the characteristics of various materials are calibrated through preprocessing and stored in a preset material characteristic library. Each material corresponds to its visible light characteristic and infrared reflectance characteristic in a predetermined library of material characteristics.
When the processor extracts the visible light features of the target object from the visible light image and the infrared reflectivity features of the target object from the infrared reflectivity image, the material identification process can be entered, that is, the extracted visible light features are compared with the visible light features of each material in the preset material feature library, the extracted infrared reflectivity features are compared with the infrared reflectivity features of each material in the preset material feature library, the matching degree of the features of the target object and the features of each material in the material feature library is judged, and the material corresponding to the features with the largest matching degree can be determined as the material of the target object. The end result of the material identification process is the output of the material properties of the target object.
In one embodiment of the present invention, step S130 may include the following steps:
the method comprises the following steps: determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
step two: calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
step three: and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
For convenience of description, the above three steps are combined for illustration.
In the material feature library, the feature corresponding to each material has a certain feature value, the feature corresponding to each material can be determined as a feature vector, the material feature library is a set formed by a plurality of feature vectors, and the extracted feature of the target object can also be used as a feature vector. And calculating to obtain the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object, and determining the material corresponding to the characteristic vector with the minimum distance between the characteristic vector of the target object and the characteristic vector in the material characteristic library as the material of the target object.
By applying the technical scheme provided by the embodiment of the invention, the processor obtains the visible light image of the target scene containing the target object through the visible light camera, obtains the infrared reflectivity image of the target scene through the infrared reflectivity measuring camera, respectively extracts the visible light characteristics of the target object from the visible light image, extracts the infrared reflectivity characteristics of the target object from the infrared reflectivity image, compares the extracted visible light characteristics and infrared reflectivity characteristics with the corresponding characteristics of each material in the preset material characteristic library to determine the material of the target object, and improves the accuracy of object material identification.
Referring to fig. 2, there is shown another implementation flowchart of the object material determination method according to the embodiment of the present invention:
the method comprises the steps that a visible light camera obtains a visible light image of a target scene containing a target object and sends the visible light image to a processor, the processor performs target segmentation on the visible light image, and visible light characteristics of the target object are extracted;
the infrared reflectivity measuring camera obtains an infrared reflectivity image of a target scene and sends the infrared reflectivity image to the processor, and the processor performs target segmentation on the infrared reflectivity image and extracts the infrared reflectivity characteristics of a target object;
corresponding characteristics of various materials, such as visible light characteristics and infrared reflectivity characteristics of the material 1, visible light characteristics and infrared reflectivity characteristics of the material 2 and the like, are stored in a material characteristic library obtained by calibrating the preprocessed material;
the processor compares the extracted visible light characteristics and infrared reflectivity characteristics with corresponding characteristics of each material in the material characteristic library by inquiring the material characteristic library to determine the material of the target object.
Corresponding to the above method embodiment, the embodiment of the present invention further provides an object material determination apparatus, which is applied to a processor, and as shown in fig. 3, the apparatus may include the following units:
a visible light feature extraction unit 310, configured to receive a visible light image of a target scene containing a target object sent by a visible light camera, and extract visible light features of the target object from the visible light image;
an ir-reflectivity feature extracting unit 320, configured to receive the ir-reflectivity image of the target scene sent by the ir-reflectivity measuring camera, and extract the ir-reflectivity feature of the target object from the ir-reflectivity image;
a material determining unit 330, configured to compare the extracted visible light characteristic and the infrared reflectivity characteristic with corresponding characteristics of each material in a preset material characteristic library, and determine the material of the target object.
By applying the device provided by the embodiment of the invention, the processor obtains the visible light image of the target scene containing the target object through the visible light camera, obtains the infrared reflectivity image of the target scene through the infrared reflectivity measuring camera, respectively extracts the visible light characteristics of the target object from the visible light image, extracts the infrared reflectivity characteristics of the target object from the infrared reflectivity image, compares the extracted visible light characteristics and infrared reflectivity characteristics with the corresponding characteristics of each material in the preset material characteristic library to determine the material of the target object, and improves the accuracy of object material identification.
In an embodiment of the invention, the visible light feature extraction unit 310 may be specifically configured to:
and carrying out target segmentation on the visible light image, determining a first target area containing a specific position of the target object, and extracting visible light characteristics of the target object from the first target area.
In an embodiment of the invention, the infrared reflectivity feature extracting unit 320 may be specifically configured to:
and carrying out target segmentation on the infrared reflectivity image, determining a second target area containing the specific position of the target object, and extracting the infrared reflectivity characteristics of the target object from the second target area.
In an embodiment of the present invention, the material determining unit 330 may be specifically configured to:
determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
Corresponding to the above method embodiment and apparatus embodiment, the present invention also provides an object material determination system, which may include a visible light camera 410, an infrared reflectance measurement camera 420, and a processor 430, as shown in fig. 4:
the visible light camera 410 is configured to obtain a visible light image of a target scene containing a target object, and send the visible light image to the processor 430;
the infrared reflectivity measuring camera 420 is configured to emit infrared light to the target object, receive reflected light reflected by the target object, obtain reflected light energy of the reflected light, depth information of the target object, and position information of the target object, determine the infrared reflectivity of the target object according to the obtained reflected light energy, depth information, and position information, obtain an infrared reflectivity image of the target scene, and send the infrared reflectivity image to the processor 430;
the processor 430 is configured to receive the visible light image sent by the visible light camera 410, extract the visible light feature of the target object from the visible light image, receive the infrared reflectance image sent by the infrared reflectance measurement camera 420, extract the infrared reflectance feature of the target object from the infrared reflectance image, compare the extracted visible light feature and infrared reflectance feature with corresponding features of each material in a preset material feature library, and determine the material of the target object.
By applying the system provided by the embodiment of the invention, the processor obtains the visible light image of the target scene containing the target object through the visible light camera, obtains the infrared reflectivity image of the target scene through the infrared reflectivity measuring camera, respectively extracts the visible light characteristics of the target object from the visible light image, extracts the infrared reflectivity characteristics of the target object from the infrared reflectivity image, compares the extracted visible light characteristics and infrared reflectivity characteristics with the corresponding characteristics of each material in the preset material characteristic library to determine the material of the target object, and improves the accuracy of object material identification.
Referring to fig. 5, in an embodiment of the present invention, the infrared reflectance measuring camera 420 may include: infrared transmitting module 421, infrared receiving module 422 and depth measuring module 423:
the infrared emitting module 421 is configured to emit infrared light to the target object;
the depth measuring module 423 is configured to receive the reflected light reflected by the target object, obtain depth information of the target object and position information of the target object, and send the depth information and the position information to the infrared receiving module 422;
the infrared receiving module 422 is configured to receive reflected light reflected by the target object, obtain reflected light energy of the reflected light, determine an infrared reflectivity of the target object according to the obtained reflected light energy, depth information, and position information, obtain an infrared reflectivity image of the target scene, and send the infrared reflectivity image to the processor 430.
The infrared emitting module 421 may emit infrared light to a target object, the infrared light may be diffusely reflected when encountering the target object, the infrared receiving module 422 may receive reflected light reflected by the target object and obtain reflected light energy, the depth measuring module 423 may also receive reflected light reflected by the target object and obtain depth information of the target object and position information of the target object, the depth information of the target object may represent a distance from the target object to the infrared reflectivity measuring camera 420, the position information of the target object is position information of the target object relative to the infrared reflectivity measuring camera 420, and the depth measuring module 423 may send the depth information and the position information of the target object to the infrared receiving module 422. The infrared receiving module 422 may determine the infrared reflectivity of the target object according to the reflected light energy, the depth information of the target object, and the position information of the target object, obtain an infrared reflectivity image of the target object, and send the infrared reflectivity image to the processor 430.
In practical applications, the infrared reflectance measurement camera may be an active depth camera, such as a ToF (Time of Flight) depth camera. The active depth camera comprises an infrared transmitting module, an infrared receiving module and a depth measuring module, and can obtain reflected light energy while calculating the depth information of a target object. That is to say, the active depth camera can directly obtain a depth image of a target scene containing a target object, and each pixel point in the depth image contains depth information and infrared reflectivity information of a corresponding position in the target scene.
The depth measurement module in the ir reflectance measurement camera may be a passive depth camera, such as a binocular stereo depth camera, which does not include an ir emitting module and an ir receiving module, and together with the ir emitting module and the ir receiving module, forms the ir reflectance measurement camera. Here, the passive depth camera may obtain a depth image of a target scene including a target object, each pixel point in the depth image includes depth information of a corresponding position in the target scene, and the infrared receiving module may obtain an infrared reflectance image of the target scene, and each pixel point in the infrared reflectance image includes an infrared reflectance feature of a corresponding position in the target scene.
It should be noted that the infrared reflectance measurement camera according to the embodiment of the present invention includes, but is not limited to, the two schemes described above.
In practical applications, the object material determination system provided in the embodiment of the present invention may further include a white light supplement device, as shown in fig. 5, for performing a light supplement process when the visible light camera obtains a visible light image of the target object.
In the embodiment of the invention, the light-emitting source of the infrared emitting module can be reasonably assumed to be an ideal point light source, and the target object is an ideal reflecting surface. The specific determination method for determining the infrared reflectivity of the target object by the infrared receiving module according to the reflected light energy and the depth information of the target object can refer to the following methods:
the first method comprises the following steps:
in the case of meeting the above assumption, the relationship between the incident light energy and the reflected light energy for any object is shown in formula (1):
wherein,is the energy of the light reflected by the object,rho is the infrared reflectivity of the object, R is the distance between the object and the infrared reflectivity measuring camera, A is the area of the infrared reflectivity measuring camera for receiving infrared light, and theta is the included angle between the reflected light and the optical axis of the infrared reflectivity measuring camera.
For the target object, the infrared receiving module receives the reflected light reflected by the target object to obtain the energy of the reflected lightIf the infrared emission module emits infrared light with fixed energy to the target object, the incident light energy of the target object can be knownThe target object can be obtained through the depth measuring moduleThe distance R from the infrared emissivity measuring camera, the area A of the infrared emissivity measuring camera for receiving infrared light can be obtained through actual measurement, and the included angle theta between reflected light and the optical axis of the infrared reflectivity measuring camera can be obtained through actual measurement. In the case where the above parameters are known, the infrared reflectance ρ of the target object is easily obtained by the formula (1).
The second method comprises the following steps:
when objects with different infrared reflectivities measure the reflected light energy at the same location, the relationship between the reflected light energies of two objects, such as object 1 and object 2, can be derived from equation (1), such as equation (2):
when the same infrared light emitting module is used to emit infrared light to two objects, the incident light energy of the object 1 and the object 2 is the same, i.e. the incident light energy is the sameWhen the infrared receiving modules are the same, A1＝A2(ii) a In the case where the measurement positions of the object 1 and the object 2 with respect to the infrared reflectance measurement camera are the same, R1＝R2，θ1＝θ2The relationship of formula (3), i.e., the basic formula of the relative measurement method, can be obtained according to formula (2).
Equation (3) illustrates that when an object (object 2) is used as a reference object for measuring the infrared reflectance of a target object (object 1), the reflected light energy thereofAnd infrared reflectance ρ2Are known conditions under which the infrared reflectance ρ for the target object is1The determination only requires knowing that the target object is under the same conditions (meaning the same measurement position as the reference object, incident onSame light energy, same infrared receiving module) of the same light energyThe infrared reflectivity rho of the target object can be determined1。
As can be seen from the formula (3), the infrared reflectivity of the target object is determined by applying the formula (3), the infrared reflectivity of the target object can be obtained only when the target object is at a relatively fixed measuring position, and the measuring position of the target object in the actual scene is random, at this time, R is1≠R2，θ1≠θ2In this case, the formula (3) cannot be applied to the calculation of the infrared reflectance, and the third method provided in the embodiment of the present invention may be applied.
The third method comprises the following steps:
when the infrared emitting modules are identical, i.e.The infrared receiving modules are identical, i.e. A1＝A2When the measurement position of the target object is different from the measurement position of the reference object, the formula (4) can be used to calculate the infrared reflectivity of the target object by simplifying the formula (2).
In which, as shown in figure 6,
after the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object to obtain the energy of the reflected light; r1After the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object, and determining the distance from the target object to the infrared reflectivity measurement camera according to the obtained depth information of the target object; theta1Is the angle between the reflected light reflected by the target object and the optical axis of the infrared reflectivity measuring camera;
R2、ρ2、θ2calibration parameters of a reference object obtained in advance for the infrared reflectance measurement camera;
is the energy of reflected light reflected by the reference object; r2Measuring a distance of the reference object to the infrared reflectance measurement camera; rho2Is the infrared reflectivity of the reference object; theta2The included angle between the reflected light reflected by the reference object and the optical axis of the infrared reflectivity measuring camera is obtained in advance.
The determination of the infrared reflectivity of the target object by applying the third method can be divided into four processes:
one is a calibration process:
the calibration process needs to be performed only once. The known IR reflectivity p needs to be measured before the determination, i.e. measurement, of the IR reflectivity of the object is made2The reference object is calibrated, and the reflected light energy of the reference object is obtained through the infrared receiving moduleObtaining the distance R between the reference object and the infrared emissivity measuring camera through a depth measuring module or actual measurement2Obtaining the included angle theta between the reflected light reflected by the reference object and the optical axis of the infrared reflectivity measuring camera through actual measurement or an image processing method2. The calibration parameters obtained in the calibration process can be stored in the memory of the infrared reflectivity measuring camera, and when the infrared reflectivity of the target object is actually measured, the parameters are substituted into the formula (4) for infrared reflectivity measurement.
In practical application, in order to make the formula (4) simpler and easier to calculate, it may be selectedReference bodies of material having a relatively high IR reflectivity, e.g. selected to have an IR reflectivity of 100%, p21, the distance between the reference object and the infrared reflectivity measuring camera is 1000mm, namely R21000 and in the middle of the target surface of the infrared reflectance measuring camera, cos θ2Substituting the above parameter values into equation (4) for 1, an equation for determining the infrared reflectance of the target object can be obtained as equation (5):
secondly, the process of obtaining the depth information and the reflected light energy of the target object is as follows:
the depth information and the reflected light energy of the target object can be obtained respectively, for example, an infrared image is obtained by a camera capable of obtaining the infrared image, and a depth image is obtained by a set of binocular depth cameras; obtaining depth information and reflected light energy of the target object may also be obtained simultaneously, such as by a ToF depth camera simultaneously acquiring a depth image and an infrared image. In the infrared image, the value of each pixel point contains infrared reflected light energy, and in the depth image, the value of each pixel point contains depth information. The parameter obtained by the process is the reflected light energy of the target objectAnd the distance R between the target object and the infrared reflectivity measuring camera1。
Thirdly, the included angle theta between the reflected light of the target object and the optical axis of the infrared reflectivity measurement camera1The acquisition process of (1):
angle theta between reflected light of target object and optical axis of infrared reflectivity measuring camera1Can be calculated by the camera pinhole imaging principle, and the calculation formula is shown as formula (6):
in the formula (6), (x, y) is the orderThe position coordinates of the object in the image (which may be a depth image or an infrared image, all described herein using the same type of image and corresponding sensor) may be obtained by a depth measurement module, (x)0,y0) Is the position coordinate of the central point of the image, delta is the pixel size of the sensor used by the corresponding image, f is the focal length of the lens of the corresponding sensor,is (x, y) to (x)0,y0) The distance of (c). The parameter diagram is shown in fig. 7.
Fourthly, calculating the infrared reflectivity:
obtaining the depth information and the reflected light energy of the target object and the included angle theta between the reflected light of the target object and the optical axis of the infrared reflectivity measuring camera1The parameter values obtained in the obtaining process and the known calibration parameters are substituted into the formula (4), and the infrared reflectivity rho of the target object at any position in the scene can be obtained1。
It should be noted that the determination method provided by the embodiment of the present invention may be applied to obtain the infrared reflectance of the object in scenes such as the infrared reflectance of the object that needs to be measured in the video image in the fields of monitoring consumer electronics and the like. Of course, the embodiments of the present invention are not limited to the above-mentioned methods for determining the ir-reflectivity characteristics of the target object, and those skilled in the art can determine the ir-reflectivity characteristics of the target object according to other methods in the prior art. In the embodiment of the present invention, the infrared reflectivity characteristic of each material in the preset material characteristic library may also be determined in the preprocessing process according to the above-mentioned exemplary method, which is not described again in the embodiment of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Those skilled in the art will appreciate that all or part of the steps in the above method embodiments may be implemented by a program to instruct relevant hardware to perform the steps, and the program may be stored in a computer-readable storage medium, which is referred to herein as a storage medium, such as: ROM/RAM, magnetic disk, optical disk, etc.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. A method for determining material of an object, applied to a processor, the method comprising:
receiving a visible light image of a target scene containing a target object sent by a visible light camera, and extracting visible light characteristics of the target object from the visible light image;
receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera, and extracting the infrared reflectivity characteristics of the target object from the infrared reflectivity image; the infrared reflectivity image is obtained after the infrared reflectivity measuring camera determines the infrared reflectivity of the target object according to the reflected light energy, the depth information and the position information of the target object;
and comparing the extracted visible light characteristic and the infrared reflectivity characteristic with the corresponding characteristic of each material in a preset material characteristic library to determine the material of the target object.
2. The method of claim 1, wherein extracting visible light features of the target object from the visible light image comprises:
and carrying out target segmentation on the visible light image, determining a first target area containing a specific position of the target object, and extracting visible light characteristics of the target object from the first target area.
3. The method of claim 1 or 2, wherein the extracting the ir-reflectance feature of the target object from the ir-reflectance image comprises:
and carrying out target segmentation on the infrared reflectivity image, determining a second target area containing the specific position of the target object, and extracting the infrared reflectivity characteristics of the target object from the second target area.
4. The method of claim 1, wherein comparing the extracted visible light and infrared reflectance signatures to corresponding signatures of each material in a predetermined library of material signatures to determine the material of the target object comprises:
determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
5. An object material determination apparatus, for use with a processor, the apparatus comprising:
the visible light feature extraction unit is used for receiving a visible light image of a target scene containing a target object and sent by a visible light camera, and extracting visible light features of the target object from the visible light image;
the infrared reflectivity characteristic extraction unit is used for receiving an infrared reflectivity image of the target scene sent by an infrared reflectivity measurement camera and extracting the infrared reflectivity characteristic of the target object from the infrared reflectivity image; the infrared reflectivity image is obtained after the infrared reflectivity measuring camera determines the infrared reflectivity of the target object according to the reflected light energy, the depth information and the position information of the target object;
and the material determining unit is used for comparing the extracted visible light characteristics and the extracted infrared reflectivity characteristics with corresponding characteristics of each material in a preset material characteristic library to determine the material of the target object.
6. The apparatus according to claim 5, wherein the visible light feature extraction unit is specifically configured to:
and carrying out target segmentation on the visible light image, determining a first target area containing a specific position of the target object, and extracting visible light characteristics of the target object from the first target area.
7. The apparatus according to claim 5 or 6, wherein the infrared reflectivity feature extraction unit is specifically configured to:
and carrying out target segmentation on the infrared reflectivity image, determining a second target area containing the specific position of the target object, and extracting the infrared reflectivity characteristics of the target object from the second target area.
8. The apparatus according to claim 5, wherein the material determination unit is specifically configured to:
determining the corresponding characteristic of each material in a preset material characteristic library as a characteristic vector;
calculating the distance between the characteristic vector of each material in the material characteristic library and the characteristic vector of the target object;
and determining the material corresponding to the characteristic vector with the minimum distance as the material of the target object.
9. An object material determination system comprising a visible light camera, an infrared reflectance measurement camera, a processor:
the visible light camera is used for obtaining a visible light image of a target scene containing a target object and sending the visible light image to the processor;
the infrared reflectivity measuring camera is used for transmitting infrared light to the target object, receiving reflected light reflected by the target object, obtaining reflected light energy of the reflected light, depth information of the target object and position information of the target object, determining the infrared reflectivity of the target object according to the obtained reflected light energy, depth information and position information, obtaining an infrared reflectivity image of the target scene, and sending the infrared reflectivity image to the processor;
the processor is configured to receive the visible light image sent by the visible light camera, extract the visible light feature of the target object from the visible light image, receive the infrared reflectance image sent by the infrared reflectance measurement camera, extract the infrared reflectance feature of the target object from the infrared reflectance image, compare the extracted visible light feature and infrared reflectance feature with corresponding features of each material in a preset material feature library, and determine the material of the target object.
10. The system of claim 9, wherein the infrared reflectance measurement camera comprises: infrared emission module, infrared receiving module and degree of depth measurement module:
the infrared emission module is used for emitting infrared light to the target object;
the depth measuring module is used for receiving reflected light reflected by the target object, obtaining depth information of the target object and position information of the target object, and sending the depth information and the position information to the infrared receiving module;
the infrared receiving module is used for receiving the reflected light reflected by the target object, obtaining the reflected light energy of the reflected light, determining the infrared reflectivity of the target object according to the obtained reflected light energy, depth information and position information, obtaining an infrared reflectivity image of the target scene, and sending the infrared reflectivity image to the processor.
11. The system of claim 9, wherein the infrared reflectance measurement camera is specifically configured to:
according to the formulaDetermining an infrared reflectance ρ of the target object1；
Wherein,
after the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object to obtain the energy of the reflected light; r1After the infrared reflectivity measurement camera emits infrared light to the target object, receiving reflected light reflected by the target object, and determining the distance from the target object to the infrared reflectivity measurement camera according to the obtained depth information of the target object; theta1Is the angle between the reflected light reflected by the target object and the optical axis of the infrared reflectivity measuring camera;
R2、ρ2、θ2calibration parameters of a reference object obtained in advance for the infrared reflectance measurement camera;
is the energy of reflected light reflected by the reference object; r2Measuring a distance of the reference object to the infrared reflectance measurement camera; rho2Is the infrared reflectivity of the reference object; theta2The included angle between the reflected light reflected by the reference object and the optical axis of the infrared reflectivity measuring camera is obtained in advance.
12. The system of claim 9, wherein the infrared reflectance measurement camera is a time-of-flight ToF depth camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610048980.1A CN106997586B (en) | 2016-01-25 | 2016-01-25 | Object material determination method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610048980.1A CN106997586B (en) | 2016-01-25 | 2016-01-25 | Object material determination method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106997586A CN106997586A (en) | 2017-08-01 |
| CN106997586B true CN106997586B (en) | 2019-12-20 |
Family
ID=59428685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610048980.1A Active CN106997586B (en) | 2016-01-25 | 2016-01-25 | Object material determination method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106997586B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3938802A1 (en) * | 2019-03-15 | 2022-01-19 | trinamiX GmbH | Detector for identifying at least one material property |
| CN110333185B (en) * | 2019-07-03 | 2022-08-02 | 香港光云科技有限公司 | Material quality detection method |
| CN110398477B (en) * | 2019-07-03 | 2022-06-24 | 香港光云科技有限公司 | Material classification and identification method |
| CN111308467A (en) * | 2020-03-10 | 2020-06-19 | 宁波飞芯电子科技有限公司 | Detection method and detection device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1087992A (en) * | 1992-10-23 | 1994-06-15 | Rwe安束根股份公司 | Differentiate the method for randomly shaped and/or flat shape material |
| CN1573742A (en) * | 2003-05-30 | 2005-02-02 | 精工爱普生株式会社 | Image retrieving system, image classifying system, image retrieving program, image classifying program, image retrieving method and image classifying method |
| CN102510461A (en) * | 2010-09-28 | 2012-06-20 | 微软公司 | Integrated low power depth camera and projection device |
| CN103901036A (en) * | 2012-12-24 | 2014-07-02 | 联想(北京)有限公司 | Method for judging material characteristics of target object and electronic device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6683681B2 (en) * | 2002-04-10 | 2004-01-27 | Baker Hughes Incorporated | Method and apparatus for a downhole refractometer and attenuated reflectance spectrometer |
-
2016
- 2016-01-25 CN CN201610048980.1A patent/CN106997586B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1087992A (en) * | 1992-10-23 | 1994-06-15 | Rwe安束根股份公司 | Differentiate the method for randomly shaped and/or flat shape material |
| CN1573742A (en) * | 2003-05-30 | 2005-02-02 | 精工爱普生株式会社 | Image retrieving system, image classifying system, image retrieving program, image classifying program, image retrieving method and image classifying method |
| CN102510461A (en) * | 2010-09-28 | 2012-06-20 | 微软公司 | Integrated low power depth camera and projection device |
| CN103901036A (en) * | 2012-12-24 | 2014-07-02 | 联想(北京)有限公司 | Method for judging material characteristics of target object and electronic device |
Non-Patent Citations (1)
| Title |
|---|
| 楔形法测量大直径反射镜的红外反射率;徐文刚 等;《应用激光》;19901231;第10卷(第2期);第80-82页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106997586A (en) | 2017-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106996922B (en) | A kind of target object infrared reflectivity measurement method and device | |
| US8718326B2 (en) | System and method for extracting three-dimensional coordinates | |
| US8121400B2 (en) | Method of comparing similarity of 3D visual objects | |
| JP6168577B2 (en) | System and method for adjusting a reference line of an imaging system having a microlens array | |
| CN109961468B (en) | Volume measurement method and device based on binocular vision and storage medium | |
| CN106997586B (en) | Object material determination method, device and system | |
| CN113592989B (en) | Three-dimensional scene reconstruction system, method, equipment and storage medium | |
| Taylor et al. | Multi‐modal sensor calibration using a gradient orientation measure | |
| US11915438B2 (en) | Method and apparatus for depth-map estimation of a scene | |
| KR20130126439A (en) | Method of geometry acquisition for specular object based on depth sensor and the device thereof | |
| JP6282377B2 (en) | Three-dimensional shape measurement system and measurement method thereof | |
| CN113099120B (en) | Depth information acquisition method and device, readable storage medium and depth camera | |
| CN111798507A (en) | Power transmission line safety distance measuring method, computer equipment and storage medium | |
| KR20150142475A (en) | Apparatus for obstacle detection and method thereof | |
| US20190313082A1 (en) | Apparatus and method for measuring position of stereo camera | |
| CA3140449A1 (en) | System and method for object recognition using 3d mapping and modeling of light | |
| Langmann | Wide area 2D/3D imaging: development, analysis and applications | |
| EP4089637B1 (en) | Hybrid feature matching between intensity image and color image | |
| US20190362517A1 (en) | Image database creation device, location and inclination estimation device, and image database creation method | |
| KR101719127B1 (en) | Apparatus for obstacle detection and method thereof | |
| US20240114119A1 (en) | Image processing device, image processing method, and program | |
| Konno et al. | Accurate plane estimation based on the error model of time-of-flight camera | |
| US20220230340A1 (en) | System and method for object recognition using 3d mapping and modeling of light | |
| JP2018063161A (en) | Information processing apparatus, method for controlling information processing apparatus, and program | |
| Ma et al. | An inverse problem approach to computational active depth from defocus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |