CN114066780B - 4k endoscope image defogging method and device, electronic equipment and storage medium - Google Patents
4k endoscope image defogging method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114066780B CN114066780B CN202210047357.XA CN202210047357A CN114066780B CN 114066780 B CN114066780 B CN 114066780B CN 202210047357 A CN202210047357 A CN 202210047357A CN 114066780 B CN114066780 B CN 114066780B
- Authority
- CN
- China
- Prior art keywords
- image
- histogram
- preprocessed
- brightness value
- defogging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012545 processing Methods 0.000 claims abstract description 85
- 238000005070 sampling Methods 0.000 claims abstract description 25
- 238000001914 filtration Methods 0.000 claims description 41
- 238000013507 mapping Methods 0.000 claims description 23
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 18
- 230000006870 function Effects 0.000 description 21
- 230000005540 biological transmission Effects 0.000 description 11
- 238000012546 transfer Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 238000002324 minimally invasive surgery Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000004075 alteration Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000008280 blood Substances 0.000 description 2
- 210000004369 blood Anatomy 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000001356 surgical procedure Methods 0.000 description 2
- 230000003187 abdominal effect Effects 0.000 description 1
- 230000000740 bleeding effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 101150050759 outI gene Proteins 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The application provides a 4k endoscope image defogging method and device, electronic equipment and a storage medium, which relate to the technical field of image processing and have the technical scheme that: the method comprises the following steps: acquiring an original image acquired by an endoscope; performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image; calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image; and carrying out defogging treatment according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image. The 4k endoscope image defogging method, the device, the electronic equipment and the storage medium have the advantages of small time delay and good defogging effect.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for defogging a 4k endoscope image, an electronic device, and a storage medium.
Background
At present, the abdominal minimally invasive surgery is very popular, particularly in hepatobiliary surgery, and has the advantages of less bleeding, small tissue wound, short recovery time and the like, so most of the surgeries are performed in a minimally invasive mode. In minimally invasive surgery, the quality of endoscopic images directly affects the surgical effect, so high-quality endoscopic images are very important for minimally invasive surgery.
In minimally invasive surgery, smoke scenes are very common, when an ultrasonic knife cuts tissues, more or less white fog can be continuously generated, and if the endoscopic images can be defogged in real time in the scenes of tissue cutting, the imaging quality of the endoscope can be greatly improved. Because the defogging method is complex, image defogging method modules are rarely integrated in the current endoscope system, especially the 4K endoscope system, and the existing integrated image defogging method modules are possibly limited by resources and have insignificant defogging effect.
At present, 4K images are few in defogging method, most of the defogging methods are simple methods based on histogram optimization, and the defogging effect is not obvious.
In view of the above problems, the applicant has proposed a new solution.
Disclosure of Invention
The application aims to provide a method and a device for defogging 4k endoscope images, electronic equipment and a storage medium, which have the advantages of small time delay and good defogging effect.
In a first aspect, the present application provides a 4k endoscope image defogging method, which includes the following steps:
the method comprises the following steps:
acquiring an original image acquired by an endoscope;
performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image;
calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
and carrying out defogging treatment according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image.
The original image is subjected to down-sampling processing to reduce the calculated amount of image processing, then the down-sampled image is subjected to over-exposure noise removal processing to obtain a preprocessed image, so that the excessive negative influence on subsequent defogging caused by excessive noise after the resolution is reduced is prevented, then, calculating an atmospheric brightness value corresponding to each channel pixel of the preprocessed image, and carrying out defogging processing according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image, thereby effectively solving the problem that the traditional dark channel prior defogging method cannot adapt to the endoscopic image, the problems that the brightness of an endoscopic image is uneven, infrared scattering is more, the brightness of a red channel is higher, and the three channels use the same transmission function, such as obvious chromatic aberration and the like, can be solved, so that the method has the advantages of small time delay and good defogging effect.
Further, in this application, the step of performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image includes:
performing down-sampling processing on the original image to obtain a first image;
calculating a maximum gradient map according to the first image;
carrying out binarization processing on the maximum gradient map, carrying out expansion operation on the binarized image, and taking the result of the expansion operation as a filtering template;
and carrying out filtering calculation on the first image according to the filtering template to obtain the preprocessed image.
Further, in this application, the step of performing filtering calculation on the first image according to the filtering template to obtain the preprocessed image includes:
directly outputting pixels in the background area of the filtering template to pixels of the first image;
and outputting the filtered result to the pixels in the foreground area of the filtering template.
Further, in the present application, the step of calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image includes:
setting an atmospheric brightness value weight of each channel pixel of the preprocessed image;
and calculating the atmospheric brightness value corresponding to each channel pixel of the preprocessed image according to the preprocessed image and the atmospheric brightness value weight of each channel pixel of the preprocessed image.
Further, in the present application, the step of setting the atmospheric brightness value weight of each channel pixel of the preprocessed image includes:
acquiring the mean value and the variance of the preprocessed image;
and calculating the atmospheric brightness value weight of each channel pixel of the preprocessed image according to the mean value and the variance of the preprocessed image.
Further, in this application, after performing defogging processing according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image, the method further includes the following steps:
calculating a histogram of the fog-free image;
and optimizing the contrast of the fog-free image according to the histogram.
Further, in the present application, the step of optimizing the contrast of the fog-free image according to the histogram includes:
cutting pixels exceeding a first preset value in the histogram;
uniformly adding the cut pixels into the histogram to obtain an equalized histogram;
and iterating the equalized histogram, and outputting a histogram mapping table when the equalized histogram meets a preset condition.
Further, in the present application, the step of outputting a histogram mapping table when the equalized histogram meets a preset condition includes:
outputting the histogram mapping table when the difference value between the variance of the histogram and the variance of the equalized histogram is larger than a second preset value;
or, when the variance of the histogram is greater than a third preset value, outputting the histogram mapping table;
or outputting the histogram mapping table when the iteration times exceed a fourth preset value.
In a second aspect, the present application also provides a 4k endoscopic image defogging device comprising:
the acquisition module acquires an original image acquired by the endoscope;
the first processing module is used for performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image;
the second processing module is used for calculating an atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
and the third processing module is used for carrying out defogging processing on the original image according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image to obtain a fog-free image.
In a third aspect, the present application further provides an electronic device, comprising a processor and a memory, where the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, perform the steps of the method according to any one of the above.
In a fourth aspect, the present application also provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method according to any one of the above.
From the above, the 4k endoscope image defogging method, the device, the electronic equipment and the storage medium provided by the application reduce the calculated amount of image processing by down-sampling the original image, then obtain the preprocessed image by performing the over-exposure noise removal processing on the down-sampled image, prevent the excessive negative influence on the subsequent defogging caused by the excessive noise after the resolution is reduced, then calculate the atmospheric brightness value corresponding to each channel pixel of the preprocessed image, perform the defogging processing according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image, effectively solve the problem that the traditional dark channel prior defogging method cannot adapt to the endoscopic image, and solve the problems that the brightness of the endoscopic image is uneven, infrared scattering is more, the brightness of a red channel is higher, and the color difference is obvious when three channels use the same transmission function by calculating the atmospheric brightness value corresponding to each channel pixel, therefore, the method has the beneficial effects of small time delay and good defogging effect.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
Fig. 1 is a schematic diagram of a 4k endoscope image defogging method provided by the present application.
Fig. 2 is a schematic structural diagram of a 4k endoscope image defogging device provided by the present application.
Fig. 3 is a schematic diagram of an electronic device provided in the present application.
In the figure: 210. an acquisition module; 220. a first processing module; 230. a second processing module; 240. a third processing module; 300. an electronic device; 310. a processor; 320. a memory.
Detailed Description
The technical solutions in the present application will be described clearly and completely with reference to the drawings in the present application, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the present application, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
The dark channel prior method is to obtain a fog-free image by processing an original fog-carrying image through an atmospheric scattering model and then using an atmospheric brightness value, and specifically, the atmospheric scattering model is generally:
wherein,
is an original image,
Is a fog-free image,
Is a transfer function,
Is an atmospheric brightness value, of
Any pixel within the image is represented.
If a certain RGB image is fog-free, the minimum value of the three RGB channels of each pixel of the RGB image must be 0, the minimum value between the three RGB channels is called a dark channel, and the following formula can be used to describe:
wherein,
showing the channel,
Is shown in
A local region as a center,
It is indicated that the dark channel is,
is shown in
At any point within the central local area,
a value representing a pixel y in the c-th channel of the fog-free image;
dark channel prior means that the dark channel of the fog-free image is 0, i.e.:
the transfer function is derived as:
from the above, it can be seen that:
therefore, there are:
wherein the atmospheric brightness value A represents the brightness value of the whole scene, and is generally set to be the highest brightness value of the pixels corresponding to the original image 1% of the pixels before the image brightness of the dark channel,
representing the value of pixel y in the c-th channel of the original hazy image.
Is at the completion of
After calculation of A, can be
Find out
I.e., a fog-free image, wherein,
to avoid the use of
In the case of 0, usually
Set to 0.1.
The dark channel prior defogging method is based on the prior that the dark channel of the pure color image is 0, and is generally used for processing the natural image, when the natural image is processed, the incident light is parallel light, and the illumination is uniform, so that a good defogging effect can be obtained.
In an endoscopic image, the image resolution is very important for the operation, the higher the image resolution is, the clearer the image is, and the richer the information of the image is, so that the current endoscopic image has gradually used the resolution of 4K, however, the data size of the 4K image is large, if the dark channel defogging method is directly adopted, the higher time delay is caused, and the current defogging method for the 4K image is less, most of the defogging methods are simple methods based on histogram optimization, and the problem that the defogging effect is not obvious exists.
In this regard, applicants propose an endoscopic image defogging method based on dark channel priors and for 4k high resolution.
In a first aspect, with reference to fig. 1, the present application provides a 4k endoscope image defogging method, which specifically includes:
s110, acquiring an original image acquired by an endoscope;
s120, performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image;
s130, calculating an atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
and S140, defogging according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image.
The original image may be an image with a resolution of 4k, or may be an image with other high resolutions, such as 2k resolution, 8k resolution, and so on.
According to the technical scheme, the original image is subjected to down-sampling processing, the calculated amount of image processing is reduced, then the down-sampled image is subjected to over-exposure noise removal processing to obtain a preprocessed image, the excessive negative influence on subsequent defogging caused by excessive noise after resolution reduction is prevented, then the atmospheric brightness value corresponding to each channel pixel of the preprocessed image is calculated, defogging processing is performed according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image, the problem that the traditional dark channel prior defogging method cannot adapt to an endoscopic image is effectively solved, the problems that the brightness of the endoscopic image is uneven, infrared scattering is more, the brightness of a red channel is higher, and obvious chromatic aberration occurs when three channels use the same transmission function are solved by calculating the atmospheric brightness value corresponding to each channel pixel, so that the time delay is small, and the image processing quality is improved, The beneficial effect of good defogging effect.
In an endoscopic image, due to the influence of excessively strong reflected light of a smooth surface of blood or tissue, a part of pixel points may be overexposed, in this case, the brightest point position may be accompanied by extremely dark point noise, and an obvious gradient contrast occurs, and any one pixel of the image after down-sampling may correspond to a plurality of pixel ranges of the original image, for example, the sampling rate is 0.1, then one pixel of the image after down-sampling may correspond to 10 pixel ranges of the original image, and the dark channel image calculates a dark channel of a certain area with an area minimum value, so the overexposed point noise of the endoscopic image may significantly affect the defogging effect of the image, especially affect the defogging continuity of the video image.
Therefore, further, in some embodiments, the step of performing a down-sampling process and an over-exposure noise removing process on the original image to obtain a pre-processed image includes:
carrying out down-sampling processing on an original image to obtain a first image;
calculating a maximum gradient map according to the first image;
setting a threshold value to carry out binarization processing on the maximum gradient map, carrying out expansion operation on the binarized image, and taking the result of the expansion operation as a filtering template;
and carrying out filtering calculation on the first image according to the filtering template to obtain a preprocessed image.
By the technical scheme, the calculation amount can be obviously reduced by performing down-sampling processing on the original image, and the noise of the first image is reduced by performing overexposure noise removal processing.
Specifically, the original image may be down-sampled by a bilinear difference method at a sampling rate of 0.1, for example, the original image with a resolution of 3840 × 2160 is down-sampled to the first image of 384 × 216, which may significantly reduce the amount of calculation.
Then, in the overexposure noise removal process, two parts of filtering template calculation and filtering are mainly included, wherein firstly, a filtering template is obtained through calculation, the filtering template can be obtained through a maximum gradient map of the first image, the maximum gradient map of the first image can be obtained through the gray value of a channel pixel and eight neighborhoods, and the formula is expressed as follows:
wherein,
is a pixel
The gradient of (a) of (b) is,
、
are respectively pixels
、
Is determined by the gray-scale value of (a),
is a pixel
Eight neighborhoods.
After the maximum gradient map of the first image is calculated, the maximum gradient map may be binarized with a threshold value of 30 to obtain a binarized image, and the binarized image may be expanded using a 3 × 3 structural operator to expand the foreground region, and the expanded result may be used as a filtering template.
The boundary points of the image subjected to binarization processing can be expanded through expansion operation, so that the boundary can be expanded outwards, and the cavity can be effectively filled and the small particle noise can be eliminated.
After the filtering template is obtained, performing median filtering on the first image, specifically, directly outputting the pixels in the background region of the filtering template to the pixels of the first image, and outputting the results of median filtering to the pixels in the foreground region of the filtering template, which can be expressed as follows by using a formula:
the formula indicates that the first image is processed, wherein,
what is shown is the output pre-processed image,
a first image representing the input is displayed on the display,
showing a filtering template, i showing a pixel at an arbitrary position of the first image, and c showing a channel; if it is not
Then it means that the point does not need to be filtered, and each channel outputs
A corresponding value; if it is not
It means that the point needs to be filtered, and the value assigned to each channel is the filtered value.
Through the scheme, the over-exposure noise of the first image can be effectively reduced, and the negative influence of the over-exposure noise on defogging is prevented.
Calculating the maximum gradient image of the first image, then carrying out binarization processing on the maximum gradient image, carrying out expansion operation on the image subjected to binarization processing to obtain a filtering template, and carrying out filtering through the filtering template, which has the advantages that the whole image of the first image does not need to be processed, only partial pixel points can be subjected to overexposure due to the influence of excessively strong reflected light of a smooth surface of blood or tissue, the overexposure noise is not excessive in practice, the whole image does not need to be filtered, and if the whole image is subjected to filtering processing, a dark channel is higher, so that a transmission function is lower, and finally the image is subjected to defogging and supersaturation, and in the scheme, the overexposure position is screened out through the maximum gradient of the first image, the expansion operation is used for expanding and selecting, and only the pixels in the filtering template can be subjected to median filtering, and the pixels which are not in the template directly output the pixels of the first image, so that the processing of the whole image can be avoided, and the problem of supersaturation of the subsequent image is also avoided.
Further, in some embodiments, the step of calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image comprises:
setting the atmospheric brightness value weight of each channel pixel of the preprocessed image;
and calculating the atmospheric brightness value corresponding to each channel pixel of the preprocessed image according to the preprocessed image and the atmospheric brightness value weight of each channel pixel of the preprocessed image.
According to the technical scheme, the atmospheric brightness value of each channel pixel is calculated in a targeted manner according to the atmospheric brightness value weight of each channel pixel, and then defogging is performed according to the atmospheric brightness value of each channel pixel, so that the problem of uneven brightness of the endoscopic image is solved.
Specifically, the calculation formula of the atmospheric brightness value of each channel pixel can be expressed as:
wherein,
the atmospheric brightness value corresponding to the c-th channel pixel,
The atmospheric brightness value weight of the c channel pixel,
For pre-processing the image, a constant 255 represents the maximum luminance value.
In the conventional dark channel prior method, the atmospheric brightness value may be a fixed value of 255 or close to 255, but the image processing for the endoscope cannot be calculated in this way, because the endoscope is close to the light source and the internal tissue of the living body is complex, so the brightness of the acquired original image is not uniform, the atmospheric brightness value weight of each channel pixel of the preprocessed image is additionally designed, and the atmospheric brightness value of each channel pixel is further updated, so that the atmospheric brightness value of each channel pixel is more accurate.
Further, in some embodiments, the step of setting the atmospheric luminance value weight of each channel pixel of the preprocessed image comprises:
acquiring the mean value and the variance of the preprocessed image;
and calculating the atmospheric brightness value weight of each channel pixel of the preprocessed image according to the mean value and the variance of the preprocessed image.
Specifically, in some embodiments, the formula for calculating the atmospheric brightness value weight of each pixel of the preprocessed image according to the mean and variance of the preprocessed image is as follows:
wherein,
is the weight of the c-th channel pixel,
Is a natural constant,
Is the mean value of the pre-processed image,
Is the variance of the pre-processed image;
according to the technical scheme, the atmospheric brightness value weight of each channel pixel of the preprocessed image is obtained according to the mean value and the variance of the preprocessed image, then the atmospheric brightness value corresponding to each channel pixel of the preprocessed image is obtained according to the atmospheric brightness value weight of each channel pixel of the preprocessed image and the preprocessed image, then defogging processing is carried out according to the atmospheric brightness value corresponding to each channel pixel, and finally the fog-free image is obtained.
Further, in some embodiments, the step of performing defogging processing according to the atmospheric brightness value corresponding to each channel pixel of the preprocessed image and the original image includes:
obtaining a transmission function corresponding to each channel pixel according to the atmospheric brightness value corresponding to each channel pixel of the preprocessed image;
and carrying out defogging treatment according to the transmission function corresponding to each channel pixel, the original image and the atmospheric brightness value corresponding to each channel pixel of the preprocessed image.
Through the technical scheme, each channel pixel corresponds to one atmosphere brightness value, so that each channel pixel has a transfer function corresponding to the atmosphere brightness value.
Specifically, the transfer function corresponding to each channel pixel obtained according to the atmospheric brightness value corresponding to each channel pixel of the preprocessed image is:
wherein,
the transfer function corresponding to the c-th channel pixel,
Can be set in a user-defined way for defogging strength,
The atmospheric brightness value corresponding to the c-th channel pixel,
Is a dark channel.
Further, in some embodiments, the step of performing defogging processing according to the transfer function corresponding to each channel pixel, the original image and the atmospheric brightness value corresponding to each channel pixel of the preprocessed image further includes:
refining a transmission function corresponding to each channel pixel through guiding filtering;
and carrying out defogging treatment according to the transmission function corresponding to each refined channel pixel, the original image and the atmospheric brightness value corresponding to each channel pixel of the preprocessed image.
By the technical scheme, after the transmission functions of the three channel pixels are calculated, the transmission functions are refined through guiding filtering, so that the accuracy rate of the transmission functions is improved, and halation after defogging is avoided.
Specifically, in some embodiments, the formula for guided filtering is:
finally, performing defogging processing according to the atmospheric brightness value corresponding to each channel pixel of the preprocessed image and the original image can be expressed as:
wherein,
to the obtained fog-free image,
An original image acquired for an endoscope,
The atmospheric brightness value corresponding to the c-th channel pixel,
For the transfer function corresponding to the c-th channel pixel after the oriented filtering,
is constant and will generally be
Set to 0.1.
The improved dark channel prior defogging method can well finish defogging of the image under most conditions, but aiming at the area with serious image color cast or low image resolution, the image effect after defogging is not natural, and a certain color cast exists, so that the contrast ratio of the image in the low resolution area is adaptively improved by adopting a contrast ratio histogram equalization method on the basis, and the defogging effect is improved.
Therefore, further, in some embodiments, after performing defogging processing on the original image and the atmospheric brightness value corresponding to each channel pixel in the preprocessed image to obtain a fog-free image, the method further includes the following steps:
calculating a histogram of the fog-free image;
and optimizing the contrast of the fog-free image according to the histogram.
By the technical scheme, the haze-free image is further optimized by utilizing the histogram, and a histogram equalization method can be adopted to enhance the contrast ratio so that the defogging effect is better.
Further, in some of these embodiments, the step of optimizing the contrast of the fog-free image based on the histogram comprises:
cutting pixels exceeding a first preset value in the histogram;
uniformly adding the cut pixels into the histogram to obtain an equalized histogram;
and (4) iterating the equalized histogram, and outputting a histogram mapping table when the equalized histogram meets a preset condition.
Through the technical scheme, the improvement is made on the basis of the contrast optimization of the existing histogram, the optimization effect can be more ideal, and when the histogram is calculated, the data larger than the first preset value are redistributed to each gray level by setting the first preset value, so that the image contrast after the histogram is optimized is limited, and the defogging effect is more natural.
Specifically, it can be expressed by the formula:
wherein,
is a first preset value,
A histogram of the fog-free image,
To equalize the histogram.
And then uniformly adding the cut pixels into the histogram to obtain an equalized histogram, wherein the equalized histogram can be expressed by using a formula as follows:
wherein,
the number of pixels of the fog-free image,
the number of pixels exceeding the first preset value.
Further, in some embodiments, the step of outputting the histogram mapping table when the equalized histogram meets the preset condition includes:
outputting a histogram mapping table when the difference value between the variance of the histogram and the variance of the equalized histogram is larger than a second preset value;
or, when the variance of the histogram is greater than a third preset value, outputting a histogram mapping table;
or outputting the histogram mapping table when the iteration times exceed a fourth preset value.
The histogram mapping table is a mapping table in which a histogram is obtained after equalization calculation of the histogram, and represents a mapping between an original pixel gray level and a new pixel gray level, for example, the original gray level is 1, the new gray level is 3 after calculation of the mapping table, corresponding mapping values are all available from 0 to 255 gray levels, and a mapping table is formed by 256 levels of mapping values.
Specifically, in some embodiments, the second preset value may be set to 0.15, and the condition for outputting the histogram mapping table may be expressed as:
specifically, in some embodiments, the third preset value may be set to 40, and the condition for outputting the histogram mapping table may be expressed as:
wherein,
representing the variance of the image before histogram equalization,
representing the image variance after histogram equalization.
Specifically, in some embodiments, the maximum number of iterations may be set to 100, that is, the histogram map is output after the number of equalization times exceeds 100.
In a second aspect, referring to fig. 2, the present application also provides a 4k endoscopic image defogging device comprising:
an acquisition module 210 for acquiring an original image acquired by the endoscope;
the first processing module 220 is configured to perform downsampling processing and overexposure noise removal processing on the original image to obtain a preprocessed image;
the second processing module 230 is configured to calculate an atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
and the third processing module 240 is configured to perform defogging processing on the original image and the atmospheric brightness value corresponding to each channel pixel in the preprocessed image to obtain a fog-free image.
According to the technical scheme, the acquisition module 210 is used for acquiring an original image acquired by an endoscope, the original image is subjected to down-sampling processing through the first processing module 220 to reduce the calculated amount of image processing, then the down-sampled image is subjected to over-exposure noise removal processing to obtain a preprocessed image, so that the condition that the excessive noise after the resolution reduction has an excessive negative effect on subsequent defogging is prevented, then the second processing module 230 calculates the atmospheric brightness value corresponding to each channel pixel of the preprocessed image, and finally the third processing module 240 performs defogging processing according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image, so that the problem that the traditional dark channel prior defogging method cannot adapt to the endoscopic image is effectively solved, and the problem that the brightness of the endoscopic image is uneven can be solved through calculating the atmospheric brightness value corresponding to each channel pixel, The infrared ray scattering is more, the red channel brightness is higher, and the three channels use the same transfer function, the problems of obvious chromatic aberration and the like can occur, so that the method has the beneficial effects of small time delay and good defogging effect.
In other preferred embodiments, the apparatus is used to perform a 4k endoscopic image defogging method as described above.
In a third aspect, referring to fig. 3, the present application further provides an electronic device 300, which includes a processor 310 and a memory 320, where the memory 320 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 310, the steps in the above method are executed.
By the above technical solution, the processor 310 and the memory 320 are interconnected and communicate with each other through a communication bus and/or other form of connection mechanism (not shown), and the memory 320 stores a computer program executable by the processor 310, and when the computing device runs, the processor 310 executes the computer program to execute the method in any optional implementation manner of the foregoing embodiment to implement the following functions: acquiring an original image acquired by an endoscope; carrying out down-sampling processing and over-exposure noise removal processing on an original image to obtain a preprocessed image; calculating an atmospheric brightness value corresponding to each channel pixel in the preprocessed image; and carrying out defogging treatment according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image.
In a fourth aspect, the present application also provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to perform the steps of the above method.
Through the technical scheme, when being executed by a processor, the computer program executes the method in any optional implementation manner of the embodiment to realize the following functions: acquiring an original image acquired by an endoscope; carrying out down-sampling processing and over-exposure noise removal processing on an original image to obtain a preprocessed image; calculating an atmospheric brightness value corresponding to each channel pixel in the preprocessed image; and carrying out defogging treatment according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image.
The storage medium may be implemented by any type of volatile or nonvolatile storage device or combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic Memory, a flash Memory, a magnetic disk, or an optical disk.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (9)
1. A4 k endoscopic image defogging method comprising:
acquiring an original image acquired by an endoscope;
performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image;
calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
carrying out defogging treatment according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image;
the step of performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image comprises the following steps:
performing down-sampling processing on the original image to obtain a first image;
calculating a maximum gradient map according to the first image;
carrying out binarization processing on the maximum gradient map, carrying out expansion operation on the binarized image, and taking the result of the expansion operation as a filtering template;
and carrying out filtering calculation on the first image according to the filtering template to obtain the preprocessed image.
2. The method according to claim 1, wherein the step of performing filtering calculation on the first image according to the filtering template to obtain the preprocessed image comprises:
directly outputting the pixels in the background area of the filtering template to the pixels of the first image;
and outputting the filtered result to the pixels in the foreground area of the filtering template.
3. The method according to claim 2, wherein the step of calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image comprises:
setting an atmospheric brightness value weight of each channel pixel of the preprocessed image;
calculating an atmospheric brightness value corresponding to each channel pixel of the preprocessed image according to the preprocessed image and the atmospheric brightness value weight of each channel pixel of the preprocessed image;
the step of setting the atmospheric brightness value weight of each channel pixel of the preprocessed image comprises the following steps:
acquiring the mean value and the variance of the preprocessed image;
and calculating the atmospheric brightness value weight of each channel pixel of the preprocessed image according to the mean value and the variance of the preprocessed image.
4. The defogging method for a 4k endoscope image according to claim 1, wherein the defogging processing according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image and the original image to obtain a fog-free image further comprises the following steps:
calculating a histogram of the fog-free image;
and optimizing the contrast of the fog-free image according to the histogram.
5. The method of claim 4, wherein the step of optimizing the contrast of the haze-free image according to the histogram comprises:
cutting pixels exceeding a first preset value in the histogram;
uniformly adding the cut pixels into the histogram to obtain an equalized histogram;
and iterating the equalized histogram, and outputting a histogram mapping table when the equalized histogram meets a preset condition.
6. The defogging method for a 4k endoscope image according to claim 5, wherein the step of outputting a histogram map when the equalized histogram satisfies a preset condition comprises:
outputting the histogram mapping table when the difference value between the variance of the histogram and the variance of the equalized histogram is larger than a second preset value;
or, when the variance of the histogram is greater than a third preset value, outputting the histogram mapping table;
or outputting the histogram mapping table when the iteration times exceed a fourth preset value.
7. A 4k endoscopic image defogging device, comprising:
the acquisition module acquires an original image acquired by the endoscope;
the first processing module is used for performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image;
the second processing module is used for calculating the atmospheric brightness value corresponding to each channel pixel in the preprocessed image;
the third processing module is used for carrying out defogging processing on the original image according to the atmospheric brightness value corresponding to each channel pixel in the preprocessed image to obtain a fog-free image;
the step of performing down-sampling processing and over-exposure noise removal processing on the original image to obtain a preprocessed image comprises the following steps:
performing down-sampling processing on the original image to obtain a first image;
calculating a maximum gradient map according to the first image;
carrying out binarization processing on the maximum gradient map, carrying out expansion operation on the binarized image, and taking the result of the expansion operation as a filtering template;
and carrying out filtering calculation on the first image according to the filtering template to obtain the preprocessed image.
8. An electronic device comprising a processor and a memory, said memory storing computer readable instructions which, when executed by said processor, perform the steps of the method according to any one of claims 1 to 6.
9. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210047357.XA CN114066780B (en) | 2022-01-17 | 2022-01-17 | 4k endoscope image defogging method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210047357.XA CN114066780B (en) | 2022-01-17 | 2022-01-17 | 4k endoscope image defogging method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114066780A CN114066780A (en) | 2022-02-18 |
| CN114066780B true CN114066780B (en) | 2022-06-03 |
Family
ID=80231141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210047357.XA Active CN114066780B (en) | 2022-01-17 | 2022-01-17 | 4k endoscope image defogging method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114066780B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022397B (en) * | 2022-01-06 | 2022-04-19 | 广东欧谱曼迪科技有限公司 | Endoscope image defogging method and device, electronic equipment and storage medium |
| CN114881896B (en) * | 2022-07-12 | 2022-10-04 | 广东欧谱曼迪科技有限公司 | Endoscope image real-time defogging method and device, electronic equipment and storage medium |
| CN117649363B (en) * | 2024-01-29 | 2024-04-12 | 江苏无右微创医疗科技有限公司 | Real-time endoscope image smoke removing method |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8827899B2 (en) * | 2004-09-24 | 2014-09-09 | Vivid Medical, Inc. | Disposable endoscopic access device and portable display |
| JP2009273691A (en) * | 2008-05-15 | 2009-11-26 | Fujifilm Corp | Apparatus and method of endoscope image processing |
| KR101582479B1 (en) * | 2012-05-15 | 2016-01-19 | 에스케이 텔레콤주식회사 | Image processing apparatus for moving image haze removal and method using that |
| KR101583947B1 (en) * | 2014-06-20 | 2016-01-08 | 현대자동차주식회사 | Apparatus and method for image defogging |
| WO2016159884A1 (en) * | 2015-03-30 | 2016-10-06 | Agency For Science, Technology And Research | Method and device for image haze removal |
| JP6419019B2 (en) * | 2015-04-23 | 2018-11-07 | 富士フイルム株式会社 | Endoscopic diagnosis apparatus, operation method of endoscopic diagnosis apparatus, program, and recording medium |
| WO2017175231A1 (en) * | 2016-04-07 | 2017-10-12 | Carmel Haifa University Economic Corporation Ltd. | Image dehazing and restoration |
| CN106251296A (en) * | 2016-07-06 | 2016-12-21 | 北京元心科技有限公司 | A kind of image defogging method and system |
| CN107403421B (en) * | 2017-08-10 | 2020-05-05 | 杭州联吉技术有限公司 | Image defogging method, storage medium and terminal equipment |
| CN107610058A (en) * | 2017-08-28 | 2018-01-19 | 浙江工业大学 | High-definition picture defogging method based on down-sampling |
| CN112419162B (en) * | 2019-08-20 | 2024-04-05 | 浙江宇视科技有限公司 | Image defogging method, device, electronic equipment and readable storage medium |
| WO2021051239A1 (en) * | 2019-09-16 | 2021-03-25 | Covidien Lp | Systems and methods for real-time de-hazing in images |
| CN110766713A (en) * | 2019-10-30 | 2020-02-07 | 上海微创医疗器械（集团）有限公司 | Lung image segmentation method and device and lung lesion region identification equipment |
| CN111053523B (en) * | 2020-02-17 | 2022-04-22 | 青岛奥美克医疗科技有限公司 | Device of antifogging endoscope system |
| CN111402143B (en) * | 2020-06-03 | 2020-09-04 | 腾讯科技（深圳）有限公司 | Image processing method, device, equipment and computer readable storage medium |
| CN112580447B (en) * | 2020-12-04 | 2023-08-04 | 国网福建省电力有限公司泉州供电公司 | Edge second-order statistics and fusion-based power line detection method |
-
2022
- 2022-01-17 CN CN202210047357.XA patent/CN114066780B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114066780A (en) | 2022-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114066780B (en) | 4k endoscope image defogging method and device, electronic equipment and storage medium | |
| CN110570360B (en) | Retinex-based robust and comprehensive low-quality illumination image enhancement method | |
| CN110264424A (en) | A kind of fuzzy retinal fundus images Enhancement Method based on generation confrontation network | |
| CN110619593A (en) | Double-exposure video imaging system based on dynamic scene | |
| CN112734650A (en) | Virtual multi-exposure fusion based uneven illumination image enhancement method | |
| CN108961206B (en) | Non-reference objective evaluation method for defogging effect | |
| US12087014B2 (en) | Apparatuses, systems, and methods for managing auto-exposure of image frames depicting signal content against a darkened background | |
| CN109447912A (en) | A kind of fluorescent image of fluorescence navigation endoscopic system adaptively enhances and noise-reduction method | |
| CN115965603A (en) | Image processing method, device, terminal and readable storage medium for endoscope image | |
| CN114022397B (en) | Endoscope image defogging method and device, electronic equipment and storage medium | |
| CN115239601B (en) | Image detail enhancement method and device, electronic equipment and storage medium | |
| JP4095494B2 (en) | Ophthalmic image processing apparatus and processing method | |
| CN115861147A (en) | Endoscope dark area enhancing method and device, electronic equipment and storage medium | |
| CN111754412B (en) | Method and device for constructing data pair and terminal equipment | |
| WO2021051239A1 (en) | Systems and methods for real-time de-hazing in images | |
| CN114584675B (en) | Self-adaptive video enhancement method and device | |
| CN116228574A (en) | Gray image processing method and device | |
| CN117769720A (en) | Method, computer program and electronic device for tone mapping | |
| CN117455780B (en) | Enhancement method and device for dark field image of endoscope, electronic equipment and storage medium | |
| CN114298206B (en) | Fundus image domain conversion method and fundus image domain conversion system | |
| CN118096546B (en) | Endoscope image processing method and device and electronic equipment | |
| CN117994160B (en) | Image processing method and system | |
| Peicheng et al. | Backlit image enhancement based on illumination-reflection imaging model | |
| US11803942B2 (en) | Blended gray image enhancement | |
| Abel et al. | Automatic glare removal in endoscopic imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 528253 Room 503, Floor 5, Building A, Jingu Zhichuang Industrial Community, No. 2, Yong'an North Road, Dawu Community, Guicheng Street, Nanhai District, Foshan City, Guangdong Province (residence declaration) Patentee after: Guangdong Oupu Mandi Technology Co.,Ltd. Address before: Room B, room 504-2, floor 5, block a, Jingu photoelectric community, No. 1, Yongan North Road, Pingzhou, Guicheng Street, Nanhai District, Foshan City, Guangdong Province, 528251 Patentee before: GUANGDONG OPTOMEDIC TECHNOLOGY CO.,LTD. |
|
| CP03 | Change of name, title or address |