CN117784388B

CN117784388B - High dynamic range metallographic image generation method based on camera response curve

Info

Publication number: CN117784388B
Application number: CN202410220837.0A
Authority: CN
Inventors: 王益平; 崔志英; 郑驰; 张琦; 王方; 尤鸣宇
Original assignee: NINGBO YONGXIN OPTICS CO Ltd
Current assignee: NINGBO YONGXIN OPTICS CO Ltd
Priority date: 2024-02-28
Filing date: 2024-02-28
Publication date: 2024-05-07
Anticipated expiration: 2044-02-28
Also published as: CN117784388A

Abstract

The invention discloses a high dynamic range metallographic image generation method based on a camera response curve, under the reflection type kohler illumination condition of a traditional metallographic microscope, firstly, a group of image sequences are acquired at equal intervals in the Z-axis direction according to coarse steps, and focal plane heights with the lowest brightness, the highest brightness, moderate brightness and optimal texture are dynamically selected; then calibrating a camera response curve on the focal plane height with moderate brightness and optimal texture; then, calculating globally optimal 3 exposure times on the basis of gray scale clusters and camera response curves on the known three focal plane heights, wherein the 3 exposure times respectively correspond to the shortest, the middle and the longest exposure time; finally, a metallographic image sequence with a high dynamic range is rapidly obtained based on a depth of field fusion technology and a multithreading technology, and the metallographic image sequence with the high dynamic range has the characteristics of high dynamic, automatic and universal image acquisition and can provide a metallographic microscopic image sequence with the high dynamic range for subsequent metallographic microscopic three-dimensional modeling work.

Description

High dynamic range metallographic image generation method based on camera response curve

Technical Field

The invention relates to an image generation method of a metallographic microscope, in particular to a high dynamic range metallographic image generation method based on a camera response curve.

Background

In order to realize the three-dimensional modeling function facing a metallographic microscope, a group of microscopic image sequences under different focal planes are generally shot, and then the functions of three-dimensional modeling and super-depth-of-field measurement are completed based on a depth measurement algorithm. A group of microscopic image sequences with moderate exposure and rich details are important factors for realizing high-quality metallographic microscopic three-dimensional modeling. Because the shape, material and texture of the shooting object are different, underexposure or overexposure is easy to cause by adopting a single exposure scheme, and the exposure time difference of different focal planes is considered by adopting a multiple exposure scheme, and the shooting efficiency and the storage space are also considered. How to design a dynamic and efficient automatic exposure method is an important basic work for generating a high-quality metallographic microscopic three-dimensional modeling result.

The Chinese patent publication No. CN218995773U issued by 09/05/2023 discloses a dark field lighting device of a super-depth digital microscope, which has the characteristic of annular light and obtains more abundant object details through multiple oblique lighting; the Chinese patent publication No. CN108827184B issued 28 in 2020 discloses a structured light self-adaptive three-dimensional measurement method based on a camera response curve, and is characterized by being based on the camera response curve. The illumination modes of the two patents are different from that of single-point kohler illumination, and the automatic exposure method represented by the second patent needs to manually select one exposure reference focal plane in advance and then calibrate a camera response curve, so that the brightness and darkness of different focal planes cannot be comprehensively considered.

The traditional metallographic microscope adopts reflection type kohler illumination, belongs to single full-caliber illumination, and is easy to cause exposure abnormality. How to automatically select exposure reference standard, dynamically calculate universal multi-exposure value, and combine depth of field fusion technology of reference [1-2] to obtain a group of high dynamic range golden phase images, which is important to realize high quality metallographic microscopic three-dimensional modeling.

[1] China patent number CN116630220B issued in 2023, 11 and 21 discloses a fluorescent image depth-of-field fusion imaging method, a device and a storage medium.

[2] Mertens T, Kautz J, Reeth FV. Exposure Fusion: A simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161-171.

Disclosure of Invention

The invention aims to provide a high dynamic range metallographic image generation method based on a camera response curve, which aims to automatically, efficiently and universally generate a metallographic microscopic image sequence with a high dynamic range and is ready for subsequent metallographic microscopic three-dimensional modeling and super-depth-of-field measurement tasks.

The technical scheme adopted for solving the technical problems is as follows: a high dynamic range metallographic image generation method based on a camera response curve is characterized by comprising the following steps:

Step (1): initializing acquisition parameters, finding an interested field of a sample under a metallographic microscope, selecting a section of height range as a Z-axis range, determining stepping parameters of image acquisition according to the depth of field of an objective lens and the numerical aperture NA of the objective lens, and finally acquiring metallographic microscopic images successively along the Z-axis direction according to the stepping parameters to acquire a group of Z-sequence images of different focal planes in the Z-axis direction;

Step (2): a group of graph sequences are acquired at equal intervals along the Z-axis direction according to the coarse stepping parameters and recorded as graph sequence A;

Step (3): three focal plane heights to be exposed are selected from the graph sequence A, namely a lowest brightness focal plane height Z _Dark, a highest brightness focal plane height Z _Bright and an optimal texture focal plane height Z _Best;

step (4): calibrating a camera response curve on the optimal texture focal plane height Z _Best;

Step (5): on focal planes corresponding to the lowest brightness focal plane height Z _Dark, the highest brightness focal plane height Z _Bright and the best texture focal plane height Z _Best, gray scale clustering is firstly carried out, images are divided into three gray scale categories, and then exposure time is calculated in sequence according to a camera response curve, so that 9 exposure time to be selected are obtained in total and recorded as t _{CRF_1}、t_{CRF_2}、…、t_{CRF_9};

Step (6): the 9 exposure times t _{CRF_1~}t_{CRF_9} to be selected are arranged in an ascending order, the shortest, the middle and the longest exposure time are selected as final exposure time and are marked as t _{Exposure_n}, n epsilon [1, 2, 3], wherein the shortest exposure time is marked as t _{Exposure_1}, the middle exposure time is marked as t _{Exposure_2}, and the longest exposure time is marked as t _{Exposure_3};

Step (7): acquiring all image sequences in the range from a start image acquisition height Z _Start to an end image acquisition height Z _End according to the stepping parameters of the step (1), recording the image sequences as image sequences B, and acquiring image exposure time under three different exposure degrees on each focal plane in the range from the start image acquisition height Z _Start to the end image acquisition height Z _End;

step (8): the image sequence B is processed based on a depth of field fusion technology and a multithreading technology of a computer, and an image sequence with a high dynamic range is rapidly acquired and is recorded as the image sequence C.

Compared with the prior art, the invention has the advantages that under the reflection type kohler illumination condition of the traditional metallographic microscope, in order to automatically obtain a metallographic image sequence with high dynamic range, a group of image sequences are firstly acquired at equal intervals in the Z-axis direction according to coarse steps, and focal plane height with lowest brightness, highest brightness, moderate brightness and optimal texture is dynamically selected; then calibrating a camera response curve on the focal plane height with moderate brightness and optimal texture; then, calculating globally optimal 3 exposure times on the basis of gray scale clusters and camera response curves on the known three focal plane heights, wherein the 3 exposure times respectively correspond to the shortest, the middle and the longest exposure time; finally, a metallographic image sequence with a high dynamic range is rapidly acquired based on a depth of field fusion technology and a multithreading technology.

Experimental results show that compared with single exposure, the method provided by the invention suppresses the photographing problems of underexposure and overexposure, and the six sample results tested by the method are better than the results of single exposure in terms of four indexes, namely information entropy, average texture value, average gray value and reference gray difference. In addition, the invention does not need to manually select the exposure reference focal plane, can automatically calibrate the response curve of the camera, and can meet the exposure requirements of different bright and dark focal planes.

In conclusion, the invention has the characteristics of high dynamic, automatic and universal graph acquisition, is suitable for the metallographic microscope illumination condition of Shan Dianshi kohler illumination, and can provide a metallographic microscopic image sequence with high dynamic range for the subsequent metallographic microscopic three-dimensional modeling work.

The invention is suitable for different non-transparent materials and metallographic image shooting tasks with different shapes, can efficiently calculate three optimal global multi-exposure values without manually selecting an exposure reference focal plane, and generates a metallographic microscopic image sequence with high dynamic range by applying a depth of field fusion technology and a multithreading technology.

Drawings

FIG. 1 is a schematic diagram of a metallographic microscope of choice for use in the present invention;

FIG. 2 is a main flow chart of a high dynamic range metallographic image generation method based on a camera response curve;

FIG. 3 is a set of exposure image sequence pictures for calibrating a camera response curve according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating a response curve of a camera according to an embodiment of the present invention;

FIG. 5 is a schematic view of image clustering (gray three classification) according to an embodiment of the present invention;

FIG. 6 is a single focal plane image of a metal notch sample obtained by a single exposure method;

FIG. 7 is a single focal plane image of a metal notch sample obtained by the method of the present invention;

FIG. 8 is a single focal plane image of a PCB pad sample obtained by a single exposure method;

FIG. 9 is a single focal plane image of a PCB pad sample obtained by the method of the present invention;

FIG. 10 is a panoramic deep fusion image of a metal notch sample obtained by a single exposure method;

FIG. 11 is a panoramic deep fusion image of a metal notch sample obtained by the method of the present invention.

Detailed Description

The invention is described in further detail below with reference to the embodiments of the drawings.

Examples: the metallographic microscope structure selected by the invention is shown in figure 1, and comprises an industrial camera 1, a metallographic lighting device 2, an objective lens 3, an electric platform 4, an industrial camera data line 5, a lamp room 6, an electric objective lens turntable 7, a shooting sample 8 and a microscope bracket 9.

FIG. 2 is a main flow chart of the invention, which describes a high dynamic range metallographic image generating method based on a camera response curve, comprising the following steps:

Step (1): initializing acquisition parameters, finding an interested field of a sample under a metallographic microscope, selecting a section of height range as a Z-axis range, determining stepping parameters of image acquisition according to the field depth of an objective lens and the numerical aperture NA of the objective lens, determining the resolution of the objective lens by the numerical aperture NA of the objective lens, calculating to obtain a recommended Z-axis stepping parameter d _Step, D _Depth denotes the simplified depth of field of the objective lens and λ denotes the wavelength. In this embodiment, the objective magnification=10 times, na=0.30, λ=0.546 μm, d _Depth =3.03 μm, the selected step size d _Step =1.52 μm, and finally, sequentially acquiring metallographic microscopic images along the Z-axis direction according to the step parameter, so as to obtain a group of Z-sequence images of different focal planes in the Z-axis direction;

Step (2): a group of image sequences are acquired at equal intervals along the Z-axis direction according to the coarse stepping parameters and recorded as an image sequence A, and the size of the coarse stepping parameter d _Rough is given by the assumption that the image sequence A has N _Rough images in total and is combined with Z _Start and Z _End In this example, Z _Start=265.88μm,Z_End=427.00μm,N_Rough =22, coarse step d _Rough =7.67 μm;

step (3): three focal plane heights to be exposed are selected from the graph sequence A, namely a lowest brightness focal plane height Z _Dark, a highest brightness focal plane height Z _{Brigh t} and an optimal texture focal plane height Z _Best; the specific method comprises the following steps:

Step (3-1): firstly, selecting focal plane heights with the lowest brightness and the highest brightness from a graph sequence A, and marking the focal plane height with the lowest brightness as a focal plane height Z _Dark with the lowest brightness; the focal plane height with the highest brightness is recorded as the focal plane height Z _Bright with the highest brightness, and the average gray value of the kth picture is calculated, namely the brightness Avg _{gray_k} of the kth picture;

，

Wherein N _Pixel represents the number of pixels of each picture; k represents the sequence number of the graph sequence A; a _{ki_Red}、A_{ki_Green}、A_{ki_Blue} represents the intensity values of the red, green and blue channels of the ith pixel point in the kth picture, I (a _ki) represents the gray value of the ith pixel point in the kth picture in the picture sequence a, and in this embodiment, N _Pixel =1224×1024= 1253376 pixels of a single picture; when k=17, obtaining a focal plane image with the lowest brightness, wherein the focal plane height Z _Dark = 389.00 μm; when k=9, obtaining a focal plane image with highest brightness, wherein the focal plane height Z _Bright = 328.20 μm;

Step (3-2): the optimal texture focal plane height Z _Best under a single exposure is selected from the graph sequence A, and is specifically as follows:

Defining a luminance range as [ Threshold _GrayLow,Threshold_GrayHigh ], wherein Threshold _GrayLow and Threshold _GrayHigh respectively represent a minimum gray Threshold and a maximum gray Threshold of the pixel, in this embodiment, threshold _GrayLow=50,Threshold_GrayHigh =195, and calculating a percentage of pixels per _kLegal of the kth picture in the picture sequence a meeting the luminance range condition according to the following formula:

，

wherein u _ki represents the coefficient value of the ith pixel point in the kth picture in the sequence a, if the gray value I (a _ki) satisfies I (a _ki)∈[Threshold_GrayLow,Threshold_GrayHigh), u _ki =1 is included, otherwise u _ki =0 is included, and the focal plane height corresponding to the maximum percentage _kLegal is selected as the optimal texture focal plane height Z _Best;

in this embodiment, when k=8, the optimal texture focal plane height Z _Best = 320.60 μm;

Step (4): calibrating a camera response curve on the optimal texture focal plane height Z _Best, wherein the specific method is as follows:

Step (4-1): acquiring a group of image sequences with different exposure time on the optimal texture focal plane height Z _Best, which is marked as a graph sequence E _CRF, and setting the initial camera exposure time which is marked as t _Start; setting the ending exposure time, which is marked as t _End; setting the step size of the exposure time, recording as t _Step, and calculating the exposure time of the jth picture

，

Wherein N _Exposure represents the number of exposed pictures contained in the sequence E _CRF; t _j represents the exposure time of the j-th picture under the sequence E _CRF, j ε [1, N _Exposure ]; in this embodiment, t _Start=350μs,t_Step=500μs,N_Exposure =40, as shown in fig. 3, which is a set of exposure image sequence pictures for calibrating the response curve of the camera, that is, the sequence E _CRF, and in fig. 3, there are 40 pictures with different exposures, and the brightness is gradually changed from low to high.

Step (4-2): calibrating a camera response curve, based on a camera response curve calibration method （Debevec PE, Malik J. Recovering High Dynamic Range Radiance Maps from Photographs[J]. SIGGRAPH, 1997, 97.）, proposed by Debevec PE and combining a graph sequence E _CRF and exposure time t _j,j∈[1, 2, 3,…,N_Exposure, obtaining a camera response curve corresponding to a photographing camera, wherein y=CRF (t), t represents exposure time, y represents a gray value corresponding to the exposure time under the camera response curve, y epsilon [0, 255], creating a lookup table between a gray value and the exposure time, and marking Tabel _CRF, wherein under the condition that the gray value y is known, the corresponding exposure time t can be obtained, and the relationship of t=CRF ^-1(y),Tabel_CRF and t=CRF ^-1 (y) is an equivalent relationship, a color image generally comprises three channels, calibration work of the camera response curve can be carried out on each channel, and the camera response curve is calibrated according to gray images; in this embodiment, fig. 4 is a schematic diagram of a response curve of the camera according to the present invention, wherein the horizontal axis is a logarithmic value of exposure time, the vertical axis is a pixel gray value, and each pixel gray value corresponds to a unique exposure time;

Step (5): on focal planes corresponding to the lowest brightness focal plane height Z _Dark, the highest brightness focal plane height Z _{Brigh t} and the best texture focal plane height Z _Best, gray scale clustering is firstly carried out, images are divided into three gray scale categories, and then exposure time is calculated in sequence according to a camera response curve, so that 9 exposure time to be selected are obtained in total and recorded as t _{CRF_1}、t_{CRF_2}、…、t_{CRF_9};

The specific method comprises the following steps:

Step (5-1): shooting N _Exposure image sequences with different exposure time on a focal plane corresponding to the focal plane height Z _Dark with the lowest brightness, and marking the image sequences as a graph sequence E _Dark;

Step (5-2): the entropy of information under the graph sequence E _Dark is calculated according to the following formula,

，

Wherein Entropy _kDark represents the information entropy of the kth picture in the picture sequence E _Dark; p _kj represents the probability density value which is the same as the gray value j in the kth picture in the graph sequence E _Dark, j epsilon [0, 255]; u _ki represents a coefficient corresponding to an ith pixel point in a kth picture in the graph sequence E _Dark, I (E _kDark) represents a gray value of the kth picture in the graph sequence E _Dark, if the gray value I (E _kDark) of the ith pixel is the same as j, u _ki =1, otherwise, u _ki =0, selecting an exposure picture with the maximum information entropy in the graph sequence E _Dark, marking the exposure picture as Img _DarkBest, and obtaining exposure time corresponding to the picture Img _DarkBest, marking the exposure time as t _DarkBest; in this embodiment, t _DarkBest =350 μs;

Step (5-3): gray clustering is carried out on the pictures Img _DarkBest, gray three-classification （Reza PS, Nasser K. Exposure Bracketing via Automatic Exposure Selection[C]. 2015 IEEE International Conference on Image Processing(ICIP), 2015: 320-323.）, is realized based on an image clustering method proposed by Reza PS, and three classified gray values are arranged in ascending order and are sequentially marked as gray _{DarkCalss_1}、gray_{DarkCalss_2}、gray_{DarkCalss_3}; in this embodiment, gray _{DarkCalss_1}=18.89,gray_{DarkCalss_2}=95.28,gray_{DarkCalss_3} = 169.54; FIG. 5 is a schematic view of image clustering (gray three-class) according to the present invention, wherein gray values of the lightest first class, gray second class and darkest third class in FIG. 5 are respectively defined as 0, 125 and 255;

Step (5-4): according to the camera response curve, three exposure times t _{CRF_1}、t_{CRF_2} and t _{CRF_3} of the focal plane corresponding to the minimum brightness focal plane height Z _Dark are obtained according to the following formula,

_,

Wherein t _{CRF_p} represents the real exposure time corresponding to the gray value gray _{DarkCalss_p} under the Z _Dark focal plane; t ₀ represents the theoretical exposure time corresponding to the phase response curve CRF with a gray value of 127; t _{DarkCRF_p} represents the theoretical exposure time with a number p under the Z _Dark focal plane; gray _{DarkCalss_p_Lower} represents the result of rounding down of the p-th gray cluster value under the Z _Dark focal plane, floor () represents the rounding down operation, e.g., floor (1.7) =1, and gray _{DarkCalss_p_Upper} represents a gray value 1 greater than that of gray _{DarkCalss_p_Lower}; t _{DarkCRF_p_Lower} represents the theoretical exposure time of gray-scale value gray _{DarkCalss_p_Lower} under the Z _Dark focal plane, obtained from the lookup table Tabel _CRF; t _{DarkCRF_p_Upper} represents the theoretical exposure time of gray-scale value gray _{DarkCalss_p_Upper} under the Z _Dark focal plane, obtained from the lookup table Tabel _CRF; in this example, t _{CRF_1}=floor(5932.80)≈5932μs,t_{CRF_2}=floor(613.14)≈613μs,t_{CRF_3} =floor (182.30) ≡182 μs; since the minimum exposure time of the camera to which the present invention is applied is 200 μs, t _{CRF_3} =200 μs;

Step (5-5): replacing a focal plane corresponding to the lowest brightness focal plane height Z _Dark with a focal plane corresponding to the best texture focal plane height Z _Best, and obtaining three recommended exposure times t _{CRF_4}、t_{CRF_5} and t _{CRF_6} of the focal plane corresponding to the best texture focal plane height Z _Best by adopting the same steps from the step (5-1) to the step (5-4); in this example, t _{CRF_4}=5123μs,t_{CRF_5}=535μs,t_{CRF_6} =200 μs;

Step (5-6): replacing the focal plane corresponding to the focal plane height Z _Dark with the focal plane corresponding to the focal plane height Z _Bright with the highest brightness, and obtaining three recommended exposure times t _{CRF_7}、t_{CRF_8} and t _{CRF_9} of the focal plane corresponding to the focal plane height Z _Bright with the same steps from the step (5-1) to the step (5-4), wherein in the embodiment, t _{CRF_7}=4978μs,t_{CRF_8}=504μs,t_{CRF_9} =200 μs;

step (6): the 9 exposure times t _{CRF_1~}t_{CRF_9} to be selected are arranged in an ascending order, the shortest, the middle and the longest exposure time are selected as final exposure time and are marked as t _{Exposure_n}, n epsilon [1, 2, 3], wherein the shortest exposure time is marked as t _{Exposure_1}, the middle exposure time is marked as t _{Exposure_2}, and the longest exposure time is marked as t _{Exposure_3}; in this example, t _{Exposure_1}=200μs,t_{CRF_4}=535μs,t_{CRF_4} =5932 μs;

Step (7): acquiring all image sequences in the range from a start image acquisition height Z _Start to an end image acquisition height Z _End according to the stepping parameters of the step (1), recording the image sequences as image sequences B, and acquiring image exposure time under three different exposure degrees on each focal plane in the range from the start image acquisition height Z _Start to the end image acquisition height Z _End; the total number of image sequences in the sequence B is obtained according to the following equation:

，

Wherein N _Step represents the number of focal planes within Z _Start~Z_End, and N _B represents the total number of images in the image sequence B; in this embodiment, N _Step =107 sheets, N _B =321 sheets;

The depth of field fusion technique （Mertens T, Kautz J, Reeth FV. Exposure Fusion: A simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161-171.）, proposed by Mertens et al processes focus on the contrast, saturation and brightness information of the image.

，

Wherein W _{q_k_i} represents the weight value of the ith pixel point of the kth exposure picture under the qth focal plane; c _{q_k_i} represents the contrast of the ith pixel point of the kth exposure picture under the qth focal plane; s _{q_k_i} represents the saturation of the ith pixel point of the kth exposure picture under the qth focal plane; e _{q_k_i} represents luminance information of the image, respectively. The invention adjusts the weight formula of depth of field fusion, highlights the weight of contrast and saturation, as follows:

，

The depth of field fusion of each focal plane is regarded as an independent task (based on QtConcurrent:: run method under the Qt platform), and is submitted to multithread parallel processing, so that the calculation efficiency is improved.

(1) A processor: AMD Rayzen 9 5900hs,8 cores;

(2) Memory: 16 GB;

(3) Hard disk: 1T solid state disk;

(4) Operating system: windows 10;

Fig. 6 to 9 compare the image quality of single focal plane images obtained by the single exposure method of the prior art and the method of the present invention.

Fig. 6 and 7 are directed to a sample of one metal groove, fig. 6 is a single exposure image, and the "comparison area 1" and "comparison area 2" marked by the dashed box in fig. 6 are both under-exposed cases; fig. 7 is a high dynamic range image obtained by the method of the present invention, and is an example of the improvement effect of the present invention on the underexposed sample, and as can be seen from fig. 7, the "comparison area 1" and the "comparison area 2" of the high dynamic range image obtained by the method of the present invention have richer image quality details, and the problem of under-exposure photographing is suppressed.

Fig. 8 and 9 are directed to samples of one PCB pad, fig. 8 is a single exposure image, and the "comparison area 3" marked by the dashed box in fig. 8 is an overexposed case; fig. 9 is a high dynamic range image obtained by the method of the present invention, and is an example of the improvement effect of the present invention on the overexposure sample, and as can be seen from fig. 9, the "comparison area 3" of the high dynamic range image obtained by the method of the present invention also has more abundant image quality details, and suppresses the overexposure photographing problem.

As can be seen from the result of the image display, in the marked comparison area, the image quality (the detail richness and the brightness uniformity) of the image improved by the method is better than that of the image with single exposure, and the under-exposure and over-exposure photographing problems are restrained.

The quantitative experiment for evaluating the image quality selects 4 indexes, namely information entropy, average texture value, average gray level and reference gray level difference.

(1) In the image, the information entropy characterizes the sharpness of the image. The larger the value is, the better the definition of the image is, and the worse the definition of the image is;

(2) The average texture value can also characterize the sharpness of the image. The larger the value is, the better the definition of the image is, and the worse the definition of the image is;

(3) The average gray scale represents the brightness of the image, and the closer the numerical value is to the reference standard 125, the less likely the shot object is to be under-exposed or over-exposed;

(4) The reference gray scale difference is obtained by taking the gray scale value 125 as a reference standard, and taking the absolute value after the average gray scale is different from the reference standard value. The index is similar to the average gray scale, but the result is more intuitive. The smaller the value of the reference gray scale difference, the better the brightness of the image, and vice versa.

According to the invention, based on the Laplacian function of an OpenCV (Open Source Computer Vision Library ), texture information of a single image is obtained, and then a texture value is obtained through statistical average summation.

，

Wherein Texture (Img) represents an average Texture value of the image Img; n _Pixel denotes the number of pixels of the image Img; img _{i_Red}、Img_{i_Green}、Img_{i_Blue} represents the intensity values of the red, green, and blue channels of the I-th pixel in the image Img, respectively, and I (Img _i) represents the gray value of the I-th pixel in the image Img; l represents a laplace operator.

TABLE 1 quantitative analysis of image quality

Table 1 totally tests 6 samples, fig. 10 and 11 are image quality comparison examples of panoramic deep fusion images obtained by two different exposure methods, the sample is a metal groove, fig. 10 is a panoramic deep fusion image obtained by a single exposure method, and fig. 11 is a panoramic deep fusion image obtained by the method of the present invention. In general, panoramic deep fusion images have better global sharpness than single focal plane images. By analyzing the panoramic deep fusion image of the sample, the image quality of the single focal plane image in the whole Z-axis range can be comprehensively judged. The result shows that the four technical index results of the image obtained by the method are better than those of the single exposure method, and the purpose of automatically obtaining the metallographic image with high dynamic range is realized.

Claims

1. A high dynamic range metallographic image generation method based on a camera response curve is characterized by comprising the following steps:

step (1): initializing acquisition parameters, finding an interested field of a sample under a metallographic microscope, selecting a section of height range as a Z-axis range, determining stepping parameters of image acquisition according to the simplified depth of field of an objective lens and the numerical aperture NA of the objective lens, and finally acquiring metallographic microscopic images successively along the Z-axis direction according to the stepping parameters to obtain a group of Z-sequence images of different focal planes in the Z-axis direction;

2. The method for generating a high dynamic range metallographic image based on a camera response curve as claimed in claim 1, wherein the step parameter d _Step is:

，

Where d _Depth represents the simplified depth of field of the objective lens, λ represents the wavelength, and the step parameter d _Step is 0.5 times the simplified depth of field d _Depth of the objective lens.

3. The method for generating a metallographic image with high dynamic range based on a camera response curve as claimed in claim 2, wherein the assumed image sequence a has N _Rough images, and the coarse stepping parameter d _Rough is:

。

4. the method for generating a high dynamic range metallographic image based on a camera response curve according to claim 3, wherein the specific method for selecting three focal plane heights to be exposed from the image sequence A in the step (3) is as follows:

，

Wherein N _Pixel represents the number of pixels of each picture; k represents the sequence number of the graph sequence A; a _{ki_Red}、A_{ki_Green}、A_{ki_Blue} represents the intensity values of the red, green and blue channels of the ith pixel point in the kth picture, and I (A _ki) represents the gray value of the ith pixel point in the kth picture in the picture sequence A;

Defining a brightness range as [ Threshold _GrayLow, Threshold_GrayHigh ], wherein Threshold _GrayLow and Threshold _GrayHigh respectively represent a minimum gray Threshold and a maximum gray Threshold of the pixel point, and calculating a pixel percentage _kLegal of the kth picture in the picture sequence A, which meets the brightness range condition, according to the following formula:

，

Wherein u _ki represents the coefficient value of the ith pixel point in the kth picture in the sequence a, if the gray value I (a _ki) satisfies I (a _ki)∈[Threshold_GrayLow, Threshold_GrayHigh), u _ki =1 is included, otherwise u _ki =0 is included, and the focal plane height corresponding to the maximum percentage _kLegal is selected as the optimal texture focal plane height Z _Best.

5. The method for generating a high dynamic range metallographic image based on a camera response curve as claimed in claim 4, wherein the specific method in the step (4) is as follows:

，

Wherein N _Exposure represents the number of exposed pictures contained in the sequence E _CRF; t _j represents the exposure time of the j-th picture under the sequence E _CRF, j ε [1, N _Exposure ];

Step (4-2): the camera response curve is calibrated, based on the camera response curve calibration method proposed by Debevec PE, a camera response curve corresponding to a photographing camera is obtained by combining a graph sequence E _CRF and exposure time t _j,j∈[1, 2, 3,…, N_Exposure, the graph is marked as y=CRF (t), t represents exposure time, y represents a gray value corresponding to the exposure time under the camera response curve, y epsilon [0, 255], a lookup table between the gray value and the exposure time is created, the lookup table is marked as Tabel _CRF, and the camera response curve is calibrated according to gray images.

6. The method for generating a high dynamic range metallographic image based on a camera response curve according to claim 5, wherein the specific method in step (5) is as follows:

，

Wherein Entropy _kDark represents the information entropy of the kth picture in the picture sequence E _Dark; p _kj represents the probability density value which is the same as the gray value j in the kth picture in the graph sequence E _Dark, j epsilon [0, 255]; u _ki represents a coefficient corresponding to an ith pixel point in a kth picture in the graph sequence E _Dark, I (E _kDark) represents a gray value of the kth picture in the graph sequence E _Dark, if the gray value I (E _kDark) of the ith pixel is the same as j, u _ki =1, otherwise, u _ki =0, selecting an exposure picture with the maximum information entropy in the graph sequence E _Dark, marking the exposure picture as Img _DarkBest, and obtaining exposure time corresponding to the picture Img _DarkBest, marking the exposure time as t _DarkBest;

Step (5-3): gray clustering is carried out on the pictures Img _DarkBest, gray three classification is realized based on the image clustering method proposed by Reza PS, and three classified gray values are arranged in ascending order and are sequentially recorded as gray _{DarkCalss_1}、gray_{DarkCalss_2}、gray_{DarkCalss_3};

_,

Wherein t _{CRF_p} represents the real exposure time corresponding to the gray value gray _{DarkCalss_p} under the Z _Dark focal plane; t ₀ represents the theoretical exposure time corresponding to the phase response curve CRF with a gray value of 127; t _{DarkCRF_p} represents the theoretical exposure time with a number p under the Z _Dark focal plane; gray _{DarkCalss_p_Lower} represents a result of rounding down the p-th gray cluster value under the Z _Dark focal plane, floor () represents a rounding down operation, and gray _{DarkCalss_p_Upper} represents a gray value 1 greater than gray _{DarkCalss_p_Lower}; t _{DarkCRF_p_Lower} represents the theoretical exposure time of gray-scale value gray _{DarkCalss_p_Lower} under the Z _Dark focal plane, obtained from the lookup table Tabel _CRF; t _{DarkCRF_p_Upper} represents the theoretical exposure time of gray-scale value gray _{DarkCalss_p_Upper} under the Z _Dark focal plane, obtained from the lookup table Tabel _CRF;

Step (5-5): replacing a focal plane corresponding to the lowest brightness focal plane height Z _Dark with a focal plane corresponding to the best texture focal plane height Z _Best, and obtaining three recommended exposure times t _{CRF_4}、t_{CRF_5} and t _{CRF_6} of the focal plane corresponding to the best texture focal plane height Z _Best by adopting the same steps from the step (5-1) to the step (5-4);

step (5-6): and replacing the focal plane corresponding to the focal plane height Z _Dark with the focal plane corresponding to the focal plane height Z _Bright with the focal plane corresponding to the focal plane with the highest brightness, and obtaining three recommended exposure times t _{CRF_7}、t_{CRF_8} and t _{CRF_9} of the focal plane corresponding to the focal plane height Z _Bright with the same steps from the step (5-1) to the step (5-4).

7. The method of generating a high dynamic range metallographic image according to claim 6, wherein the total number of image sequences in the image sequence B in the step (7) is obtained according to the following formula:

，

Where N _Step represents the number of focal planes in the range of Z _Start~Z_End and N _B represents the total number of images in the sequence B.