CN101264002A - Three-dimensional endoscopic measurement device and method based on grating projection - Google Patents

Abstract

A three-dimensional endoscopic measurement device and method based on grating projection, the device is composed of a fiber optic image guide bundle, an amplitude-type transmission grating, a collimating lens, a semiconductor laser light source, a micro-array CCD detector, a transmission line, an image acquisition card and a computer, and the computer has corresponding image processing and three-dimensional shape calculation and reconstruction software. The core of the method of the present invention is to project the amplitude-type transmission grating onto the surface to be measured during endoscopic observation, and the grating stripes are modulated by the surface to be measured to produce deformation. The deformed grating stripes are collected by the CCD imaging system, and after computer image processing, the three-dimensional shape information of the target to be measured is obtained through a three-dimensional shape reconstruction algorithm. The present invention can obtain three-dimensional shape information of the endoscopic object, and has the characteristics of fast measurement speed, high measurement accuracy and simple method.

Description

Three-dimensional endoscopic measurement device and method based on grating projection

technical field

This patent relates to endoscopes, especially a three-dimensional endoscopic measurement device and method based on grating projection.

Background technique

Endoscope is a precision medical instrument combining optomechanics and electronics, which is used to observe tissues and structures in the body and provide scientific diagnostic information for medical diagnosis, especially minimally invasive surgery. From the original hard tube endoscope to the current fiber endoscope and electronic endoscope, the technical development of endoscope is becoming more and more mature.

Existing endoscopes include fiber endoscopes and electronic endoscopes. An endoscope generally consists of an illumination system and an image acquisition system. The illumination system mainly transmits the illumination light generated by the light source (commonly used such as a halogen cold light source) into the body to provide imaging illumination for the tissue to be observed. Most current lighting systems use unstructured light sources for lighting, and the lighting light field itself cannot carry any coded information. The image acquisition system collects tissue images through an optical fiber image guide bundle or a CCD to obtain two-dimensional image information of the measured tissue.

The information contained in this endoscopic image reflects the two-dimensional plane information of the measured tissue, and the image processing work focuses on improving the quality of the existing plane image (such as sharpness and color image) and eliminating various aberrations caused by the system . J.F.Rey of France etc. collected the video signal output by the endoscope into the computer for image analysis and processing [see prior art 1: J.F.Rey, etc, al., Electronic Video Endoscopy: Preliminary Results of Imaging Modification, Endoscopy, Vol. 20, 1988:8-10]. S.Guadagni in Italy developed an electronic endoscope image processing and analysis system with an electronic endoscope and a 386 computer as the core [see prior art 2: S.Guadagni, etc., al., Imaging in Digestive Videoendoscopy , SPIE, Optic Fibers in Medicine, Vol.1420, 1991: 178-182]. Recently, the appearance of micro-CCD has greatly improved the quality of information contained in planar images. The development of these technologies has created good conditions for medical diagnosis.

However, due to the very little information contained in the light field of the existing endoscope system, only the two-dimensional information of the measured tissue can be obtained, and the three-dimensional shape information including the relative depth and lateral size of the object is lost. It has caused great limitations to scientific diagnosis and medical research. The way to overcome this shortcoming is to use optical three-dimensional measurement technology, which can be effectively combined with endoscopic imaging technology to measure the three-dimensional surface shape distribution of the target and provide the three-dimensional shape information of the target.

Optical three-dimensional shape measurement technology is a kind of mature measurement technology, which is widely used in various measurement fields, and has the advantages of high precision, fast speed and non-contact measurement. One of the optical three-dimensional shape measurement technologies uses the principle of active optical three-dimensional measurement to structure the illumination light field (points, lines, grating stripes, etc.), and uses structured light to illuminate the measured object. Modulation is carried out so that the light field carries the three-dimensional topography information of the surface of the measured object. Then the image of the modulated structured light field is captured by the CCD, processed by the computer, and the three-dimensional shape information of the measured target is obtained through the three-dimensional shape reconstruction algorithm. In particular, Fourier transform profilometry (FTP) using grating stripes as structured light was proposed by Takeda et al. in 1983 [see prior art 3: Takeda Mitsuo, Mutoh Kazuhiro, "Fourier transform profilometry for the automatic measurement of 3-D object shapes”, Applied Optics, Vol.22, Issue.24, 1983]. In this method, the grating stripe light field is used as a structured light source, and the three-dimensional shape information of the measurement target is obtained by performing Fourier transform, filtering, inverse Fourier transform, phase expansion and other image and information demodulation algorithms on the image intensity distribution.

Existing endoscopic three-dimensional measurement systems use laser height scanning technology. However, the control structure of this laser scanning system is complicated, the required time is long, the implementation scheme is cumbersome, and the technology is not yet mature.

Contents of the invention

The present invention aims to solve the problems of loss of three-dimensional information of the output image of the endoscope in the prior art, and provides a three-dimensional endoscopic measurement device and method based on grating projection to obtain the three-dimensional shape information of the endoscopic object, and has the ability to measure It has the characteristics of fast speed, high measurement accuracy and simple method.

Technical solution of the present invention is as follows:

A three-dimensional endoscopic measurement device based on grating projection, which is characterized in that it is composed of an optical fiber image guide bundle, an amplitude type transmission grating, a collimator lens, a semiconductor laser light source, a micro-array CCD detector, a transmission line, an image acquisition card and a computer. The computer has corresponding image processing and three-dimensional shape calculation and reconstruction software, and the connection relationship of each component is: the laser light emitted by the semiconductor laser light source is sequentially irradiated by the collimator lens, the amplitude type transmission grating and the optical fiber image guide beam On the surface of the object to be measured, the projected stripes of the amplitude-type transmission grating modulated by the three-dimensional topography of the surface of the object to be measured are photographed by the micro-array CCD detector, and then enter the computer through the transmission line and the image acquisition card.

The method for measuring with the above-mentioned three-dimensional endoscopic measurement device for grating projection comprises the following steps:

①Collect the fringe distribution pattern on the reference plane: Before measuring the three-dimensional shape of the object, first use the micro-array CCD detector and the image acquisition card to collect a grating fringe image on the reference plane and store it in the computer;

②Collect the distribution image of the deformed grating stripes on the measurement target: align the miniature area array CCD detector of the endoscopic three-dimensional measuring device with the measured object, and adjust the position so that the measured object is on the miniature area array CCD detector Clear imaging, collecting the grating fringe image deformed by the three-dimensional shape modulation of the target to be measured, and storing it in the computer;

③ The computer calculates the phase difference between the two images and reconstructs the three-dimensional shape of the measured target:

Processing of the reference plane fringe image: perform necessary image processing on the reference plane fringe image captured by the micro-array CCD detector to improve image contrast and reduce image noise, and then perform Fourier transform, fundamental frequency filter, and Fourier inverse transform ,get ${\hat{g}}_0(x,\mathrm{the\; y})=A_1{r}_0(x,\mathrm{the\; y})\exp (i2\mathrm{\π}{f}_{0}x+{\mathrm{\φ}}_0(x,\mathrm{the\; y})),$ and stored in a computer;

Processing of the deformed fringe image modulated by the measured target: Similarly, the necessary image processing is performed on the deformed fringe image modulated by the measured target captured by the micro-array CCD detector to improve the contrast of the image and reduce the image noise, and then Perform Fourier transform, fundamental frequency filter, and inverse Fourier transform to get

$\hat{g}(x,\mathrm{the\; y})=A_{1}r(x,\mathrm{the\; y})\exp (i2\mathrm{\π}{f}_{0}x+\mathrm{\φ}(x,\mathrm{the\; y})),$ stored in a computer;

Take the above processing results and the stored reference surface fringe image processing results into the following formula for calculation:

得到两幅图像的相对位相差；

Obtain the relative phase difference of the two images;

·Since the arctangent value calculated by the computer is located in [-π, π], there is a discontinuous jump in the relative phase distribution, and the phase expansion of the calculated phase difference is required;

重构出所测目标的三维形貌分布，·Last use

Reconstruct the three-dimensional shape distribution of the measured target,

(x，y)和

(x，y)表示条纹图像未经调制和经过调制两种情况下的位相分布，f₀表示投影光栅条纹的基频。In the above formula: r ₀ (x, y), r (x, y) represent the non-uniform surface reflectance of the above-mentioned reference plane and the two situations of the measured target respectively, and A _n represents the Fourier series of each level weighting factor,

(x, y) and

(x, y) represent the phase distribution of the fringe image without modulation and after modulation, and f ₀ represents the fundamental frequency of the projected grating fringes.

The period of the amplitude grating used is 100 μm, the aperture ratio is 1:1, and the diameter of the grating does not exceed 5 mm.

Technical effect of the present invention:

The invention applies Fourier transform profilometry to the method and device for endoscopic measurement, and solves technical problems such as complex scanning components, high control precision requirements, and long measurement time. This measurement method only needs one frame of deformed grating projection image to calculate the three-dimensional shape distribution of the object. Compared with the existing endoscope system, it can obtain the three-dimensional shape information of the endoscopic object, and has the characteristics of fast measurement speed, relatively high measurement accuracy and simple method.

Description of drawings

Fig. 1 is a schematic diagram of a three-dimensional endoscopic measurement device using amplitude-type transmission grating projection according to the present invention.

Fig. 2 is a schematic diagram of the principle of Fourier transform profilometry adopted in the present invention.

Detailed ways

The present invention will be further described below in conjunction with drawings and embodiments.

Please refer to FIG. 1 first. FIG. 1 is a schematic diagram of a three-dimensional endoscopic measurement device based on grating projection according to the present invention. It is also a structural schematic diagram of an embodiment of the present invention. It can be seen from the figure that the three-dimensional endoscopic measurement device based on grating projection in the present invention consists of an optical fiber image guide bundle 2, an amplitude type transmission grating 3, a collimator lens 4, a semiconductor laser light source 5, a miniature area array CCD detector 6, a transmission line 7, An image acquisition card 8 and a computer 9 are formed, and the connection relationship of each component is: the laser light emitted by the semiconductor laser light source 5 is irradiated on the surface of the object to be measured through the collimator lens 4, the amplitude type transmission grating 3 and the optical fiber image guide bundle 2 in sequence The projection fringes of the amplitude-type transmission grating modulated by the three-dimensional topography of the surface of the object to be measured are photographed by the micro-array CCD detector 6 and enter the computer 9 through the transmission line 7 and the image acquisition card 8, and the computer 9 has a corresponding image Processing and reconstruction software for 3D topography calculations. 1 in the figure shows the enlarged view of the end face of the fiber optic image guide bundle 2 and the miniature array CCD detector 6 .

The measurement principle involved in the present invention is the basic physical principle of Fourier transform profilometry as follows:

Please refer to FIG. 2 . FIG. 2 is a schematic diagram of the principle of Fourier transform profilometry adopted in the present invention. The light path structure shown in Figure 2. P1P2 represents the optical axis of the image acquisition optical path, and L1L2 represents the optical axis of the grating projection optical path. The reference plane is an imaginary plane used as a measurement reference for the surface to be measured. h(x, y) represents the height of point D on the surface of the object to be measured relative to the reference plane, and d represents the distance between the entrance pupil center of the image acquisition optical path and the optical axis of the grating projection optical path. When the system structure is determined, d is the established Quantity. L0 represents the distance between the entrance pupil of the grating projection optical path and the reference plane, which is also a known quantity in the system.

On the reference plane, that is, when h(x, y)=0, the fringe image is the original grating projection without deformation, and its light field distribution can be expressed as:

When the grating fringes are projected onto the surface to be tested, the height distribution of the surface to be tested h(x, y)≠0, the deformed fringe image is obtained, and the light field distribution can be expressed as:

(x，y)和

(x，y)表示条纹图像未经调制和经过调制两种情况下的位相分布，f₀表示投影光栅条纹的基频。Among them, r ₀ (x, y), r (x, y) respectively represent the non-uniform surface reflectance of the above two cases, A _n represents the weight factor of the Fourier series at each level,

(x, y) and

(x, y) represent the phase distribution of the fringe image without modulation and after modulation, and f ₀ represents the fundamental frequency of the projected grating fringes.

Perform a one-dimensional Fourier transform along the x-axis to the formula (1) to obtain the Fourier spectrum, select an appropriate filter function (the commonly used filter function has a rectangular window function) to filter the obtained spectrum, extract its fundamental frequency component, and then perform Fourier inverse transform , the complex signal of the unmodulated fringe image of the reference plane can be obtained:

${\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&phi;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Perform the same processing on (2) to get the complex signal of the deformed fringe image:

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&phi;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

(x，y)：Comparing equations (3) and (4), the result of the three-dimensional shape of the measured object modulating the projected grating fringes leads to a change in the phase in the complex exponential term, the magnitude of which is

(x,y):

On the other hand, it can be obtained from the geometric relationship in Figure 2:

AC/d=h/(L ₀ -h)(7)

Calculated by formulas (6) and (7), the height h(x, y) and phase difference to be measured can be obtained The relationship between (x, y):

(x，y)，然后带入(8)式得到所测物体的三维形貌分布。In the specific measurement process, the phase difference is calculated from the reference grating image and the deformation-modulated grating image by formula (5)

(x, y), and then brought into (8) to obtain the three-dimensional shape distribution of the measured object.

The amplitude type transmission grating adopted in the embodiment of the present invention is a chromium grating template with a period of 100 μm and an aperture ratio of 1:1, which can modulate the illumination light output by the semiconductor laser 5 into grating stripe-shaped structured light, from From the perspective of information theory, the projected light carrying encoded information is produced. The optical fiber image guide bundle 2 transmits and projects the structured light generated by the chromium grating onto the surface of the object to be measured. There is a slight included angle between the micro-area CCD detector 6 and the optical axis of the optical fiber image guide bundle 2, and the micro-area CCD detector 6 is controlled by a computer 9 to monitor the changes of the projected fringes in real time.

Acquire a pattern of fringe distribution on a reference plane. Before measuring the three-dimensional shape of the object, at first utilize the miniature area array CCD detector 6 and the image acquisition card 8 to collect a grating fringe image on a reference plane (height distribution is h(x, y)=0), and store it in the computer 9 As the basic reference data, in order to calculate the relative phase difference and extract the relative height distribution of the plane to be measured and the reference plane. In order to reliably extract the three-dimensional shape information, the distance between the set reference plane and the micro-array CCD detector 6 is roughly equivalent to the distance between the object to be measured and the micro-array CCD detector 6 .

The image of the deformed grating fringe distribution on the measurement target is collected. Align the miniature area array CCD detector 6 of the endoscopic three-dimensional measuring device shown in Fig. 1 on the object to be measured, adjust the position so that the object to be measured is clearly imaged on the miniature area array CCD detector 6, and collect the measured object The deformed grating fringe image generated by three-dimensional topography modulation is stored in the computer 9 to be processed in the next step.

Calculate the phase difference between the two images and reconstruct the three-dimensional shape of the measured target. Image processing is performed on the modulated grating fringe image and the reference plane fringe image. In order to improve the measurement speed, the processing of the reference plane fringe image can be processed first. Carry out necessary image processing to the image taken by the miniature area array CCD detector 6, to improve the contrast of the image, reduce image noise, then perform Fourier transform, fundamental frequency filter, and Fourier inverse transform, and store the processed results in the computer 9 . The deformed fringe image modulated by the measured target undergoes the same image processing and analysis, and then the processing result and the stored reference surface fringe image are brought into the following formula for calculation:

得到两幅图像的相对位相差。由于计算机计算的反正切值位于[-π，π]，因此相对位相分布存在不连续跃变，因此需要对计算的位相差进行相位展开。最后利用(8)式，重构出所测目标的三维形貌分布。

Get the relative phase difference of the two images. Since the arctangent value calculated by the computer is located in [-π, π], there is a discontinuous jump in the relative phase distribution, so it is necessary to perform phase unwrapping on the calculated phase difference. Finally, using formula (8), the three-dimensional shape distribution of the measured target is reconstructed.

Experiments show that the three-dimensional endoscopic measurement device and method based on grating projection of the present invention have the advantages of fast measurement speed, simple implementation method, and high measurement accuracy. Through fast processing by computer, high-speed dynamic monitoring can be performed, which has important practical value and application prospects.

Claims (3)

1, a kind of based on the amount of spying out device in the three-dimensional of optical grating projection, it is characterized in that by optical fiber image guide bundles (2), vibration amplitude type transmission grating (3), collimating lens (4), semiconductor laser light source (5), miniature area array CCD detector (6), transmission line (7), image pick-up card (8) and computer (9) constitute, the annexation of each parts is: the laser that described semiconductor laser light source (5) sends is successively through collimating lens (4), vibration amplitude type transmission grating (3) and optical fiber image guide bundles (2) are radiated at the object under test surface, by miniature area array CCD detector (6) take by the object under test three-dimensional surface shape the projected fringe of synthetic vibration amplitude type transmission grating after transmission line (7), image pick-up card (8) enters computer (9), and described computer (9) has corresponding Flame Image Process and three-dimensional appearance calculates reconstruction software.

2, the amount of spying out device in the three-dimensional of optical grating projection according to claim 1, the grating cycle that it is characterized in that described vibration amplitude type transmission grating (3) is 100 μ m, and aperture efficiency is 1: 1, and the grating diameter is no more than 5mm.

3, utilize the method that the amount of spying out device is measured in the three-dimensional of the described optical grating projection of claim 1, it is characterized in that comprising the following steps:

1. gather the striped distribution pattern on the reference plane: before the Measuring Object three-dimensional appearance, at first utilize the grating fringe image on miniature area array CCD detector (6) and image pick-up card (8) collection one width of cloth reference plane, be stored in the computer (9);

2. gather the deformation grating fringe distributed image on the measurement target: the miniature area array CCD detector (6) of peeping three-dimensional measuring apparatus in described is aimed at measurand, adjust the position, make object to be measured go up blur-free imaging at this miniature area array CCD detector (6), gather target three-dimensional appearance modulation to be measured and produce the grating fringe image of deformation, be stored in the computer (9);

3. computer (9) calculating two width of cloth image bit differ, and the Three-dimension Target pattern is surveyed in reconstruct:

The processing of reference plane stripe pattern: to the reference plane stripe pattern of miniature area array CCD detector (6) shooting, carry out necessary image processing,, reduce picture noise to improve the contrast of image, carry out Fourier transform, fundamental frequency filtering, inverse fourier transform then, obtain ${\hat{g}}_0(x,y)=A_1{r}_0(x,y)\exp (i2\mathrm{\&pi;}{f}_{0}x+{\mathrm{\&phi;}}_0(x,y)),$ And be stored in the computer (9);

Processing to the deformation stripe pattern of survey target modulation: same, the deformation stripe pattern of the target modulation of surveying that miniature area array CCD detector (6) is taken, carry out necessary image processing, to improve the contrast of image, reduce picture noise, carry out Fourier transform, fundamental frequency filtering, inverse fourier transform then, obtain

$\hat{g}(x,y)=A_{1}r(x,y)\exp (i2\mathrm{\&pi;}{f}_{0}x+\mathrm{\&phi;}(x,y)),$ Be stored in the computer (9);

Bringing the result of the result of above-mentioned processing and the plane of reference stripe pattern stored into following formula calculates:

Obtain the relative phase difference of two width of cloth images;

Because the arc-tangent value of COMPUTER CALCULATION is positioned at [π, π], so the phase para-position distributes mutually and has discontinuous transition, carry out phase unwrapping to the phase difference that calculates;

Utilize at last

Reconstruct the Three-dimension Target topographic profile of surveying,

In the above-mentioned formula: r ₀(x, y), r (x, y) show respectively reference plane show above-mentioned reference plane and the surface reflectivity heterogeneous of two kinds of situations of the target of surveying, A _nThe weight factor of representing fourier series at different levels,

(x, y) and

(x, y) expression stripe pattern non-modulated distributes mutually with position under two kinds of situations of ovennodulation, and f0 represents the fundamental frequency of projection grating striped.

Images