JPH07104491B2 - Endoscope with measuring function - Google Patents

Description

Detailed Description of the Invention [Object of the Invention] (Industrial field of application) The present invention irradiates two-dimensional regular pattern light to capture an image, and measures the unevenness of a subject from the distortion of the pattern in a non-contact manner. The present invention relates to an endoscopic device with a measuring function.

(Prior Art) In general, an endoscope apparatus has an endoscope that is introduced inside an observation target such as a living body or a mechanical device (hereinafter referred to as the inside of a body cavity), and is provided at the tip of this scope. It is used for imaging and observing a subject such as a body cavity by an imaging system.

This endoscope is provided with a means for projecting a two-dimensional regular pattern light such as a laser spot array or a deformed grating onto a subject, and the distortion of the pattern in the captured image is measured to three-dimensionally measure each portion of the subject surface. There is an endoscopic device having a measuring function for obtaining various positional information and obtaining shape information such as unevenness.

FIG. 8 shows an example of a method for obtaining a two-dimensional regular pattern light. For example, a transmission type glass fiber with a diameter of about 20 μm
26a, 27a arranged side by side in a single layer without gaps
A diffraction grating 23 is used in which the sheets 26 and 27 are laminated so that the fibers of each layer are orthogonal to each other. In this diffraction grating 23,
As shown in FIG. 9, a laser beam 29 emitted from an oscillator 28 of a laser light source is vertically incident to generate a square lattice bright spot array (hereinafter referred to as spot light) 25 in front of the laser beam 29. .

Using such a pattern light generating means, as shown in FIG. 6 and FIG. 7, the spot light 25 is projected from the endoscope scope distal end portion 21 to the subject 24, and the image pickup portion (for example, solid-state image pickup) is provided with a parallax Pa. Image is taken with a device; CCD) 22.

Although an optical system such as an objective lens is actually arranged immediately in front of the image pickup device 22, it is omitted in the drawing here for the sake of simplicity of description, and will be described as one body with the image pickup device 22. Also the 6th
In the figure, the endoscope scope distal end portion 21 has an illumination hole in which an illumination light guide and an irradiation lens (not shown) are fitted,
Equipped with forceps holes that also serve as air / water holes and suction holes.
The diffraction grating 23 and the light guide 34 that guides projection light from a laser light source oscillator or the like as a pattern projection light source in the endoscope main body to the diffraction grating are the tip of the scope using the forceps holes and passages. Up to 21 may be introduced, or may be separately provided for exclusive use.

FIG. 5 is an example of an image captured by projecting the spot light 25 on the subject 24 in this way.

In addition to such spot light, vertical stripe patterns may be generated as regular pattern light and projected onto the subject. FIG. 4 is an example of an image captured by vertical stripe pattern light.

When the subject has irregularities, the parallax Pa causes distortion in the pattern of the captured image. By measuring the degree of this distortion, the three-dimensional position of each point on the surface of the subject can be calculated. The principle of this calculation will be described later.

(Problems to be Solved by the Invention) However, in order to automatically detect the distortion of the pattern as described above, it is necessary to correctly recognize the correspondence between each point between the projected pattern and the pattern on the captured image. .

As described above, due to the parallax Pa between the pattern light projection unit (diffraction grating 23) and the imaging unit (imaging device 22 including a lens or the like),
Although the pattern light is distorted on the image, this distortion appears only in the direction parallel to the parallax Pa (left and right direction in FIG. 5).

In order to discriminate each spot on the pattern, spot rows that are in the same series in the direction perpendicular to the parallax Pa (vertical direction in FIG. 5) are called, for example, the same order or homogeneous term.

The spot sequence at the center is defined as the 0th order term, and the rightward direction in FIG. 5 is sequentially named ++ 1st order term, + 2nd order term, and the leftward direction in FIG. To do.

At this time, if the terms of any order are projected with almost the same amount of light,
When a captured image is converted into binary data having only values of 0 and 1 to detect a pattern, patterns of any orders have substantially the same size, and it becomes difficult to distinguish the orders. For this reason, in order to facilitate the determination of the order, conventionally, a method has been adopted in which only a spot sequence of a specific order, for example, a 0th order term, is projected with a larger light amount than spot lights of other orders.

However, the subject surface is naturally not flat,
The surface condition is not always suitable for photographing because there are irregularities, perspectives, and the like, and in the case of an organ inside the human body, the mucous membrane is wet. Therefore, when the pattern is detected by the binarization process, the emphasized 0th-order term imaging pattern may be exuded, and it may be difficult to separate the adjacent ± 1st-order term patterns. In addition, since light emitted from the same light source is projected so that a larger amount of light is distributed to the 0th-order term, the amount of light of the surrounding multiorder terms may be insufficient, and spot detection omission may occur. Further, at the time of the above detection, complicated image processing is required such that the binarized image data is regressed on the data area having a value of 1 to be thinned to extract the center point and the like.

As described above, in order to correctly recognize the correspondence of each point on the image, extremely complicated and difficult processing is involved. Further, it naturally takes a long time to execute such complicated and difficult processing.

The present invention is to solve this problem in view of such conventional circumstances, and without complicated image processing such as extraction of the center point of a pattern on a captured image,
An endoscopy with a measurement function that can easily and quickly calculate at least three-dimensional position information of the desired part on the subject, and can obtain information such as distance and height between multiple points based on this It is an object of the invention to provide a mirror device.

[Structure of the Invention] (Means for Solving the Problems) In order to achieve the above object, the first invention of the present application is an endoscopic device that introduces a scope inside an observation object to capture an image of a subject. A projection means for projecting a light spot on a subject with a circular projection pattern, an image pickup means for picking up the light spot projected on the subject by the projection means, and a projected image of the light spot picked up by the image pickup means. Display means,
A position designating means for designating an arbitrary light spot on the projection image displayed on the display means, and a two-dimensional coordinate computing means for obtaining two-dimensional coordinates of the light spot designated by the position designating means on the projection image. An input means for inputting an order for making the light spot designated by the position designating means correspond to the projection pattern; and the position designating means based on the two-dimensional coordinates and the order obtained by the two-dimensional coordinate computing means. And a three-dimensional coordinate calculating means for obtaining the three-dimensional coordinates of the designated light spot.

Further, a second invention of the present application is, in an endoscope apparatus for introducing a scope into an observation object to image an object, a projection means for projecting a line on the object in a vertical stripe or horizontal stripe projection pattern, An image pickup means for picking up an image of a line projected onto a subject by the projection means, a display means for displaying a projected image of the line picked up by the image pickup means, and an arbitrary point on the line displayed on the display means are designated. A position designating unit, a two-dimensional coordinate computing unit that obtains two-dimensional coordinates of the point designated by the position designating unit on the projection image, and an order that makes the point designated by the position designating unit correspond to the projection pattern. Input means for inputting, and three-dimensional coordinate calculation means for obtaining three-dimensional coordinates of the point designated by the position designation means based on the two-dimensional coordinates obtained by the two-dimensional coordinate computation means and the degree. And it said that there were pictures.

Further, a third invention of the present application is, in an endoscope apparatus for imaging a subject by introducing a scope inside an observation object, a projection means for projecting light spots on the subject in a grid-like projection pattern, and the projection means. Image pickup means for picking up the light spot projected on the subject by means of the display means, display means for displaying a projection image by the light spot picked up by the image pickup means, and a plurality of lights on the projection image displayed on the display means. The position designating means for designating a point, the two-dimensional coordinate computing means for finding the two-dimensional coordinates of each light spot designated by the position designating means on the projection image, and each light spot designated by the position designating means. An input means for inputting an order corresponding to the projection pattern, a two-dimensional coordinate obtained by the two-dimensional coordinate calculation means, and a light point designated by the position designating means on the basis of the order inputted by the input means. Characterized by comprising a calculating means for obtaining the release or height difference.

Further, a fourth invention of the present application is, in an endoscope apparatus for introducing a scope inside an observation object to image a subject, a projection means for projecting a line on the subject in a vertical-stripe or horizontal-stripe projection pattern, and An image pickup means for picking up an image of a line projected on a subject by the projection means, a display means for displaying a projected image of the line picked up by the image pickup means, and a plurality of points on the line displayed on the display means are designated. And a two-dimensional coordinate calculation means for obtaining two-dimensional coordinates on the projection image of each point designated by the position designating means,
Based on the input means for inputting the order that makes each point designated by the position designation means correspond to the projection pattern, the two-dimensional coordinates obtained by the two-dimensional coordinate calculation means and the order input by the input means, And a calculation unit for calculating a distance or a height difference between the points designated by the position designation unit.

(Operation) According to the endoscope apparatus with a measuring function including each of the above-described constituent means, the projection means has a two-dimensional regular projection pattern from the distal end of the scope with respect to the subject inside the observation object. The pattern projection image obtained by projecting the irradiation light and picking up the light spot projected on the subject by the image pickup means is displayed by the display means, and designation of any one point or a plurality of points on this projection image is accepted. To calculate the coordinate value. Further, array information on the pattern corresponding to each of the designated one point or a plurality of points is obtained, and the array information of each point and the coordinate value of each point designated by the position designating means on the projection image are obtained. From this, at least three-dimensional coordinate values for one or a plurality of points are calculated by the three-dimensional coordinate calculation means.

Based on the three-dimensional coordinate value of each point calculated in this way, it is possible to easily obtain and display the distance between a plurality of points and the height difference.

In this way, it is possible to easily obtain the measurement information such as the unevenness of the subject surface. In this way, it becomes possible to solve the above-mentioned subject.

(Embodiment) FIG. 1 is a block diagram showing a configuration of a main part of an endoscope apparatus with a measuring function according to an embodiment of the present invention. In the apparatus of this embodiment, two points are designated on the captured image of the projection pattern as shown in FIG. 2, and the distance and the height difference between the two points are obtained and displayed. .

In FIG. 1, in addition to a normal endoscope irradiation light source 1, a pattern projection light source 2 which is a light source for generating irradiation light having a two-dimensional regular pattern is provided. The pattern projection light source 2 can be configured by, for example, an oscillator that emits laser light of a specific wavelength.

In addition, a camera control unit (hereinafter, abbreviated as CCU) 4, which is a main body constituent element of an ordinary endoscope apparatus,
In addition to the decoder 5, the A / D6, the frame memory 7, the D / A8, and the monitor 9, various parts for designating points on the photographed image, inputting array information, calculating and displaying the result are provided. ing.

That is, there is a mouse 10 which is a pointing device for designating two points on a captured image of a projection pattern displayed on the monitor 9 as shown in FIG. Also, part 2
There is a keyboard 11 used for keying in the order of spots, which is the arrangement information of points. Further, there are respective processing units denoted by reference numerals 12 to 17 in FIG.

Further, FIG. 6 is a diagram showing a configuration example of a main part of a distal end portion of the endoscope. In this example, a side-view type scope for observing an object in a direction perpendicular to the longitudinal direction of the scope is shown.

A laser beam having a specific wavelength, for example, emitted from the pattern projection light source 2 in FIG. 1 is introduced into one end of the light guide 34 in FIG. Radiation is emitted from the other end of the guide 34 toward the diffraction grating 23. Then, the light passes through the diffraction grating 23 and becomes a two-dimensional regular pattern light (spot light) 25 which is projected onto the subject 24. At this time, by devising the way of applying the laser light emitted to the diffraction grating 23, it is possible to obtain the spot light 25 in which only the 0th order term is emphasized more than the other order terms.

The spot light 25 projected on the subject 24 is diffusely reflected by the surface portion of the subject 24, and the reflected light is received by the image sensor 22 via an optical system (not shown) and is imaged. An image of the subject by the spot light 25 picked up by the image pickup device 22 is sent to the endoscope main body by a signal line (not shown).

In addition to the above, the scope distal end portion 21 has a light guide for illumination that guides illumination light from the endoscope irradiation light source 1 of FIG. 1, an air / water feed hole, a suction hole, a forceps hole, which are not shown. Etc.

The image signal of the subject, which is incident on the image pickup device 22, is picked up and sent to the endoscope main body by a signal line (not shown), is sent to the camera control unit (CCU) 4 in FIG. CCU
In 4, the image signal is subjected to signal processing and output to the decoder 5 as a luminance signal Ey of an analog video signal and two color difference signals (Ey-Er) and (Ey-Ed).

The decoder 5 converts the analog video signal into, for example, R
Separation and reorganization into (red), G (green), and B (blue) color signals. And after being converted to a digital signal by A / D6,
It is stored in each of the R, G, and B frame memories 7.
Color video information stored in the frame memory 7 is D / A8
Is converted into an analog signal again and displayed on the monitor 9.

FIG. 2 is a diagram showing an example of a captured image of a projection pattern in this embodiment, and FIG. 3 is a diagram showing an optical arrangement of an irradiation system, an image pickup system and the like in the apparatus of the same example.

The coordinates of the center point (called the shooting point) of an arbitrary spot on the shooting screen of the spot light as shown in FIG. 2 and the actual point on the subject surface (called the corresponding point) corresponding to this point Three
The relationship with the dimensional coordinate value will be described. In FIG. 3, the coordinates of the photographing point on the photographing screen shown on the UV plane are (u, v), and the three-dimensional coordinates of the corresponding points in the XYZ coordinate space with the lens 20 as the coordinate center are (x, y). , z); z = d / (tan θ1 + tan θ2) ... X = u · z / 1 ... y = v · z / 1. Here, 1 is the distance between the objective lens 20 and the image sensor 22 (that is, parallax Pa), and d is the lens 20 and the diffraction grating.
23 and the distance between each center. Further, θ2 is the angle between the corresponding point viewed from the lens 20 and the Z-axis; θ2 = tan ⁻¹ (n / 1) ... θ1 is the direction in which the pattern light (spot light) is projected, depending on the order h of the pattern light; θ1 = sin ⁻¹ (h · λ / D) (where λ: laser light wavelength, D: diffraction grating interval) Is represented.

Therefore, when performing three-dimensional measurement, if the coordinates (u, v) of the center point of each pattern light on the photographing screen and the order h of that point are recognized, the three-dimensional coordinate value can be calculated. it can.

However, in order to automatically perform such processing, it is necessary to perform difficult and complicated processing such as accurate extraction of a pattern from a captured image and correspondence of the order thereof, which naturally takes a long processing time.

Therefore, in the present invention, by specifying and inputting the coordinates on the image and the corresponding degree using only a mouse or a keyboard with respect to only one or a plurality of arbitrary points on the subject image, the extraction or the like described above can be performed. The time required for complicated processing is saved, and at least the three-dimensional coordinate values of a desired point can be calculated easily and quickly.

Furthermore, in the device of this embodiment, by designating and inputting the coordinates and the order for any two points,
Each three-dimensional coordinate value of these two points is calculated, and 2 is calculated from these values.
The distance between points and the height difference are obtained and displayed. With such a configuration, the distance and height difference between any two points on the subject surface, which is the most frequently requested, can be easily and immediately obtained.

For example, it is assumed that the pattern light distribution as shown in FIG. 2 is obtained on the display screen while projecting the pattern light on the subject to be measured and observing. Then, when it is desired to measure the distance between two points A and B in FIG.
Is used to move the cursor on the screen (not shown) to specify two points A and B on the screen of the monitor 9.

In Fig. 1, "2 points A, B coordinate designation" 12 is a mouse
Follow the cursor on the screen of the monitor 9 according to the movement of 10.
To move. Then, when the two points A and B are designated by the mouse 10, the coordinates on the respective screens are read by "designation of coordinates of two points A and B" 12, and "calculation of u, v, θ2" 14 is performed. Sent. In “calculation of u, v, θ2” 14, for each of the two points A and B, the coordinates u,
v and the angle θ2 are calculated according to the above-described formula and the like.

On the other hand, the degree of each of the two points A and B is visually judged by the observer, and the value of each degree is input from the keyboard 11. For example, in the case of FIG. 2, the point A is +
Secondary, point B becomes -secondary. Then, the input value is received at “input of 2 points A and B order” 13, and “calculation of θ1” 15
Sent to. In “calculation of θ1” 15, the angle θ1 is calculated from the value of this order by the above-mentioned formula.

Based on the coordinates u, v thus obtained and the values of the angle θ2 and the angle θ1, according to the above equations, “x of two points A and B,
The “y, z calculation” 16 calculates the three-dimensional coordinate values x, y, z of the two points A and B, respectively.

From the three-dimensional coordinate values x, y, z of the two points A and B, the distance between the two points and the height difference are calculated in “calculation of height difference and distance” 17. Further, in “calculation of height difference and distance” 17, the numerical information is edited or the figure is easily recognized. The easily recognizable display image thus formed is displayed on the monitor 9 together with the original image stored in the frame memory 7.

In this way, the information on the height difference and the distance between two points on the surface of the subject such as the body cavity of the observation target can be easily measured.

Of course, instead of obtaining only the height difference and distance between two points in this way, it is also possible to calculate the three-dimensional coordinates thereof by the above equations (1) to (4) by only specifying one point and inputting the order. is there. Alternatively, a plurality of points may be designated and input at the same time, the three-dimensional coordinate values thereof may be calculated and displayed, and information such as the distance between the points may be obtained and displayed by numerical values or figures.

The present invention is not limited to the configuration of the above-described embodiment, and can be applied to, for example, a vertical striped projection pattern as shown in FIG. In this case, the method of designating points and inputting the order may be exactly the same as in the above-mentioned embodiment.

Further, the method of realizing the projection light of the two-dimensional regular pattern is not only the method of using the diffraction grating having the structure in which the sheets in which the transmissive glass fibers are arranged in parallel in one layer are stacked vertically and horizontally, Of course, it is also possible to use another method such as a modified grid.

[Effects of the Invention] As described in detail above, according to the endoscope apparatus with a measurement function according to the present invention, the coordinates on the screen of any one point or a plurality of points on the surface of the subject to be measured. Since the instruction is given and the arrangement information is obtained, it is possible to omit the difficult and complicated long-time process for extracting the pattern and detecting the arrangement information. Therefore, at least the dimensional coordinate values of any one or more points to be measured can be calculated easily and quickly, and desired information can be easily and immediately obtained. In this way, three-dimensional measurement of a subject using the endoscope can be performed very easily, and the efficiency of endoscopic diagnosis can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram of a main part of an endoscope apparatus with a measuring function according to an embodiment of the present invention, FIG. 2 is an explanatory view of an example of a captured image of a projection pattern in one embodiment, and FIG. 3 is one embodiment. Diagram showing the optical arrangement of the irradiation system and the imaging system in the device,
FIG. 4 is a diagram showing an example of an image captured by vertical stripe pattern projection light, and FIG. 5 is a diagram showing an example of an image captured by spot light.
6 and 7 are diagrams showing the relationship between the distal end of the endoscope and the subject, and FIGS. 8 and 9 are explanatory diagrams showing the structure of the fiber diffraction grating and the state of spot light generation. 1 ... Light source, 2 ... Pattern projection light source, 3 ... Scope, 4 ... Camera control unit, 5 ... Decoder, 6 ... A / D, 7 ... Frame memory, 8 ... D / A, 9
…… Monitor, 10 …… Mouse, 11 …… Keyboard, 12 ……
2 point A, B coordinate specification, 13 …… 2 point A, B order input, 14 ……
Calculation of U, V, θ2, calculation of 15 …… θ1, 16 …… of 2 points A and B
X, Y, Z calculation, 17 ... Height difference, distance calculation, 20 ... Lens, 21 ... End of endoscope scope, 22 ... Imaging element, 23
…… Diffraction grating, 24 …… Subject, 25 …… Spot light, 26,
27 …… One-dimensional fiber diffraction grating, 26a, 27a …… Glass fiber, 28 …… Oscillator, 29 …… Laser light

Claims (4)

[Claims] 1. An endoscope apparatus for capturing an image of a subject by introducing a scope inside an object to be observed, the projection unit projecting light spots on the subject in a grid-like projection pattern, and the projection unit projecting light onto the subject. Image pickup means for picking up an imaged light spot, display means for displaying a projection image of the light spot picked up by the image pickup means, and a position for designating an arbitrary light spot on the projection image displayed on the display means Designating means, two-dimensional coordinate computing means for obtaining two-dimensional coordinates of the light spot designated by the position designating means on the projection image, and order for making the light spot designated by the position designating means correspond to the projection pattern Input means for inputting, and three-dimensional coordinate calculation means for obtaining the three-dimensional coordinates of the light spot designated by the position designation means based on the two-dimensional coordinates obtained by the two-dimensional coordinate calculation means and the degree. Instrumented endoscope apparatus characterized by comprising. 2. An endoscope apparatus for capturing an image of an object by introducing a scope inside an object to be observed, wherein the object is projected by a projection means for projecting a line on the object in a vertical stripe or horizontal stripe projection pattern. Image pickup means for picking up an image of a line projected onto the line, display means for displaying a projected image of the line picked up by the image pickup means, and position specifying means for specifying an arbitrary point on the line displayed on the display means. A two-dimensional coordinate calculation means for obtaining two-dimensional coordinates of the point designated by the position designation means on the projection image; and an input means for inputting an order for making the point designated by the position designation means correspond to the projection pattern. And a three-dimensional coordinate calculating means for obtaining the three-dimensional coordinate of the point designated by the position designating means based on the two-dimensional coordinate obtained by the two-dimensional coordinate computing means and the degree. DOO instrumented endoscope apparatus according to claim. 3. An endoscope apparatus for capturing an image of a subject by introducing a scope inside an object to be observed, the projection unit projecting light spots on the subject in a lattice-shaped projection pattern, and the projection unit projecting light onto the subject. An image pickup means for picking up the projected light spot, a display means for displaying a projection image by the light spot picked up by the imaging means, and a plurality of light spots on the projection image displayed on the display means. Position designating means, two-dimensional coordinate computing means for obtaining two-dimensional coordinates of each light spot designated by the position designating means on the projection image, and each light spot designated by the position designating means on the projection pattern An input means for inputting a corresponding order; a distance between the light spots designated by the position designating means based on the two-dimensional coordinates obtained by the two-dimensional coordinate computing means and the order inputted by the input means; Is an endoscopic device with a measuring function, which is provided with a calculating means for obtaining a height difference. 4. An endoscope apparatus for capturing an image of an object by introducing a scope inside an object to be observed, wherein the object is projected by a projection means for projecting a line on the object in a vertical stripe or horizontal stripe projection pattern. Image pickup means for picking up an image of a line projected on the screen, display means for displaying a projected image of the line picked up by the image pickup means, and position specifying means for specifying a plurality of points on the line displayed on the display means And a two-dimensional coordinate calculation means for obtaining two-dimensional coordinates of each point designated by the position designation means on the projection image, and an order for making each point designated by the position designation means correspond to the projection pattern Based on the two-dimensional coordinates obtained by the two-dimensional coordinate calculation means and the order inputted by the input means, the distance or height difference between the points designated by the position designating means. An endoscope apparatus with a measuring function, comprising: