[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
Optimal Control of a Semi-Active Suspension System Collaborated by an Active Aerodynamic Surface Based on a Quarter-Car Model
Next Article in Special Issue
Utilizing Convolutional Neural Networks for the Effective Classification of Rice Leaf Diseases Through a Deep Learning Approach
Previous Article in Journal
Multi-Modal Fusion Network with Multi-Head Self-Attention for Injection Training Evaluation in Medical Education
Previous Article in Special Issue
Study on the Generation and Comparative Analysis of Ethnically Diverse Faces for Developing a Multiracial Face Recognition Model
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Electronics
Volume 13
Issue 19
10.3390/electronics13193883
electronics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Low-Light Image Enhancement Network Using Informative Feature Stretch and Attention

by
Sung Min Chun
,
Jun Young Park
and
Il Kyu Eom
*
Department of Electronics Engineering, Pusan National University, Busan 46241, Republic of Korea
*
Author to whom correspondence should be addressed.
Electronics 2024, 13(19), 3883; https://doi.org/10.3390/electronics13193883
Submission received: 22 August 2024 / Revised: 24 September 2024 / Accepted: 27 September 2024 / Published: 30 September 2024
(This article belongs to the Special Issue Pattern Recognition and Image Processing: Latest Advances and Prospects)
Browse Figures
Figure 1
<p>Low-light images and their respective ground-truth counterparts, along with their accompanying histograms.</p> "> Figure 2
<p>Examples of enhanced image results using different machine-learning-based methods.</p> "> Figure 3
<p>The overall structure of the proposed LIE network.</p> "> Figure 4
<p>Conventional image normalization examples with different percentile maximum and minimum values (max percentile, min percentile).</p> "> Figure 5
<p>Configuration of DRS block.</p> "> Figure 6
<p>Restored low-light images and corresponding histograms using only the DRS block.</p> "> Figure 7
<p>Configuration of IFA block.</p> "> Figure 8
<p>Configuration of FS block.</p> "> Figure 9
<p>Configuration of DR block.</p> "> Figure 10
<p>Enhanced image results for the paired synthetic LOL dataset.</p> "> Figure 11
<p>Visual comparison of enhancement results on the VV dataset (24 images).</p> "> Figure 12
<p>Visual comparison of enhancement results on the DICM dataset (44 images).</p> "> Figure 13
<p>Visual comparison of enhancement results on the LIME dataset (10 images).</p> "> Figure 14
<p>Visual comparison of enhancement results on the MEF dataset (17 images).</p> "> Figure 15
<p>Visual comparison of results using three ablation studies.</p> ">
Versions Notes

Abstract

:
Low-light images often exhibit reduced brightness, weak contrast, and color distortion. Consequently, enhancing low-light images is essential to make them suitable for computer vision tasks. Nevertheless, addressing this task is particularly challenging because of the inherent constraints posed by low-light environments. In this study, we propose a novel low-light image enhancement network using adaptive feature stretching and informative attention. The proposed network architecture mainly includes an adaptive feature stretch block designed to extend the narrow range of image features to a broader range. To achieve improved image restoration, an informative attention block is introduced to assign weight to the output features from the adaptive feature stretch block. We conduct comprehensive experiments on widely used benchmark datasets to assess the effectiveness of the proposed network. The experimental results show that the proposed low-light image enhancement network yields satisfactory results compared with existing state-of-the-art methods from both subjective and objective perspectives while maintaining acceptable network complexity.

1. Introduction

Images captured under low-light conditions often suffer from poor visibility due to insufficient incident radiation. Such images typically exhibit low contrast, faded colors, limited pixel ranges, and overall blurriness. Consequently, they are unsuitable for delivering accurate results when used in computer vision applications. To mitigate these issues, significant research has been conducted in the field of low-light image enhancement (LIE) [1,2]. Recently, machine-learning-based LIE networks [3] have gained significant attention. Despite advancements in LIE techniques and their potential to enhance computer vision performance, it remains a challenging task.
LIE methods can be broadly classified into three main categories: image-processing-based [4,5], model-based [6,7], and machine-learning-based approaches [8,9]. Image-processing-based LIE algorithms have been widely used to enhance low-light images. However, these approaches can lead to an excessive increase in contrast, ultimately resulting in a visually over-enhanced image. The most commonly used image degradation models for LIE are based on the Retinex theory [10] and the atmospheric scattering model (ASM) [11]. Although these model-based LIE approaches often produce satisfactory results, they face challenges in determining parameters and may introduce undesirable artifacts such as halos and amplified noise. Machine-learning-based LIE methods have achieved remarkable success and have gained considerable interest. Among these, convolutional neural networks (CNNs) are widely adopted for LIE tasks, while generative adversarial networks (GANs) have demonstrated effectiveness in enhancing low-light images without the requirement for supervised learning. More recently, transformer-based LIE networks [12,13] have gained significant attention for their potential in this field.
This study proposes a novel LIE network based on an adaptive feature stretching approach. Figure 1 shows examples of low-light images alongside their corresponding ground-truth images, as well as their respective histograms. In the low-light image, pixel values are predominantly concentrated in the lower pixel range, leading to minimal variation among color pixels and making it difficult to discern fine details and resulting in faded colors. Conversely, the histograms of the ground-truth images have a broader distribution, highlighting increased color variations among the corresponding pixels.
Based on these observations, this study introduces a novel LIE network based on feature stretching and attention techniques (FSANet) designed to enhance image brightness, restore color, and recover image details. The proposed FSANet consists of two main components: a dynamic range stretch (DRS) block and an informative feature attention (IFA) block. The IFA block is further divided into two submodules: the feature stretch (FS) block and the detail recovery (DR) block. The main contributions of this study are as follows:
  • To enhance the quality of low-light images, we introduced an adaptive standardization block and an adaptive normalization block [14]. We achieved successful results in enhancing underwater images by carefully and repetitively combining these blocks. Building upon these findings, we design the DRS block by sequentially connecting a single adaptive standardization block and a single normalization block. However, when the DRS block is applied directly to low-light images, it primarily enhances brightness without adequately addressing color restoration and detail enhancement.
  • We introduce an IFA block designed to complement the output of the DRS block, focusing specifically on restoring image colors and details. A histogram-equalized version of the low-light image is produced and used as input to the IFA block in conjunction with the original low-light image. The histogram-equalized image serves as a roughly reconstructed image, providing basic color and detail information. The IFA block comprises the FS and DR blocks. The FS block uses the DRS block in combination with a squeeze-and-excitation residual (SE-Res) block [15]. The DR block includes a modified multiscale block [16] and the SE-Res block.
  • Finally, the feature outputs from both the DRS and IFA blocks are multiplied to generate the restored low-light image. The features from the IFA block serve as weights for the image features produced by the DRS block. After performing a convolution operation on the multiplied features, the proposed network outputs the restored low-light image.
Figure 2 presents the enhanced image obtained using the proposed FSANet compared with the results of the following networks: unsupervised generative adversarial network (EnlightenGAN) [17], attention-guided multibranch convolutional neural network (AGLLNet) [18], two-branch exposure-fusion network (TBEFN) [19], deep unfolding network based on Retinex theory (URetinex-Net) [20], and transformer-based low-light enhancement (LLFormer) [21]. As depicted in Figure 2, the restored image obtained using EnlightenGAN shows a subdued brightness and background noise. It yields low peak signal-to-noise ratio (PSNR) and structure similarity index measure (SSIM) [22] values. AGLLNet exhibits noise and color casts, resulting in low PSNR and SSIM values. Although TBEFN achieves an acceptable SSIM score of 0.815, it exhibits limitations in the brightness of the restored image. URetinex-Net exhibits blurred images. LLFormer achieves the highest SSIM score and exhibits favorable background brightness. The enhanced image obtained using the proposed network closely resembles the ground-truth image. The proposed FSANet achieves the highest PSNR score and the second-highest SSIM score.
The rest of this paper is structured as follows. Section 2 offers a brief overview of different LIE methods. Section 3 details the proposed LIE network. The experimental results are presented in Section 4. Finally, concluding remarks are presented in Section 5.

2. Related Works

This section provides a concise overview of the most relevant LIE methods in these three categories. More detailed introductions can be found in [1,2,3].

2.1. Image-Processing-Based Method

Spatial-domain image enhancement techniques based on pixel transformation modify pixel values using mathematical functions. Nonlinear functions, such as hyperbolic tangent [23] and sigmoid transfer functions [24], are commonly used to enhance image contrast. Due to their simplicity, these methods allow for high-speed LIE. However, there is a limitation in enhancing low-light images because they do not consider the overall distribution of image pixel values.
The histogram-based image enhancement method [4] adjusts background intensity and modifies the histogram to improve visual quality. However, it typically leads to excessive contrast enhancement. To address this problem, the contextual and variational contrast enhancement method [25], multiple enhancement fusion technology [26], and bio-inspired dual-exposure fusion method [27] have been proposed. However, these approaches do not completely restore the background brightness of an image.

2.2. Model-Based Method

Two model-based approaches, Retinex theory [10] and ASM [11], are commonly used in the field of LIE. LIE methods using Retinex theory involve the estimation of the illumination and reflectance components. Previously, an effective Retinex model with robustness against significant noise in low-light images was presented [28]. A semi-decoupled Retinex image decomposition (SDD) [29] method initially estimated the illumination from the low-light image based on a Gaussian total variation method. Then, SDD jointly estimated the reflectance using the provided low-light image and estimated illumination. Guo et al. [6] introduced an illumination map estimation (LIME) method for LIE. LIME estimates the initial illumination using the pixel-wise maximum of all color channels in the low-light image. After that, it enhances the illumination map through the optimization technique. Ren et al. [30] combined a camera response model with a conventional Retinex model to enhance low-light images. In this method, enhanced images are produced by locally adjusting the exposure of low-light images.
Inverting the low-light image allows the ASM to be used for LIE. The dehazing method using the dark channel prior [31] was applied to an inverted low-light image to enhance low-light images [32]. An image degradation model for low-light images [33] was proposed. This model integrated an image prior derived from the statistical regularity observed in clear images. An inverted image normalized by atmospheric light [7] was employed with ASM for LIE. This method estimated the transmission map using the saturation values of the normal-light image. Recently, A LIE algorithm based on gamma correction prior in mixed color spaces (GCPMC) [34] was introduced to facilitate parameter determination and enhance LIE results.
Although model-based LIE schemes can achieve satisfactory results, they often result in significant computational overhead or require accurate model parameter determination. Nevertheless, the insights derived from the Retinex theory and ASM have been used in numerous machine-learning-based LIE networks.

2.3. Machine-Learning-Based Method

A patch-based image enhancement and denoising network [8] was introduced for LIE using a stacked auto-encoder. Although this network achieved satisfactory brightness recovery, it struggled to reproduce vivid colors and recover fine image details. An LIE network with a multibranch [35] based on an encoder-decoder was proposed. This network processes a diverse set of features. A deep lightening network (DLN) [36] that incorporates the lightening back-projection (LBP) module was presented for LIE. DLN uses residual images obtained from each iteration by repeatedly applying the LBP module. Various LBP modules use different features, which are combined by a feature aggregation module. TBEFN [19] produces two enhanced images through separate branches. The enhanced images are then combined using an average operation and further adjusted based on a calibration device. To enhance low-light images, a deep-stacked Laplacian restorer (DSLR) [37] was proposed for LIE. DSLR employs an image pyramid model, and the images from individual pyramid layers are subsequently merged to produce the enhanced image.
Numerous machine-learning-based LIE studies have used the Retinex theory. A Retinex network (RetinexNet) [38] based on deep learning was proposed to separately estimate illumination and reflectance maps. RetinexNet comprises a decomposition network and an enhancement network to improve the illumination map. An LIE network called KinD [39] was introduced. KinD includes feature decomposition, reflectance, and illumination estimation blocks. KinD++ [16], which is an improved version of KinD, incorporated a multiscale-based illumination block to improve the LIE performance. A three-branch convolution neural network [40] was proposed to decompose the input image into three components: illumination, reflectance, and noise. The weights of this network were updated through a zero-shot scheme, iteratively minimizing a specially designed loss function. To estimate reflectance and illumination maps, a RetinexDIP [41] conducted Retinex decomposition without relying on external images, and the estimated illumination was easily adjusted, playing a crucial role in the enhancement process. An URetinex-Net [20] unfolded an optimization problem into a trainable network to decompose a low-light image into reflectance and illumination layers. The unfolding optimization module realized noise suppression and details preservation for the final decomposition results.
Unsupervised-learning-based LIE networks have also been explored to address limitations related to training data. A zero-reference deep curve estimation network (Zero-DCE) [42] enhanced low-light images using a curve estimation method. However, images enhanced using Zero-DCE exhibit reduced brightness and lack fine details. An unsupervised EnlightenGAN [17] was designed for LIE. EnlightenGAN uses self-information extraction and addresses the problems associated with LIE.
Recently, LIE networks based on transformers have gained significant attention. A conditional normalizing flow model [12] captured the conditional distribution of normally exposed images to enhance low-light images. This model extracted a color map based on a Retinex module and exhibited acceptable qualitative and quantitative image results. A transformer-based LIE network that incorporates the SNR as a guidance (SNRA) [43] was proposed. SNRA combines a CNN for the entire image with a transformer applied to image patches. Simultaneous training of these two networks has the capability of capturing both global and local features. Yang et al. [44] presented an LIE network with a structure similar to SNRA. This network integrates a CNN and a vision transformer to extract global and local features, respectively. A transformer-based LIE network, LLFormer [21] employed axis-based multi-head self-attention and cross-layer attention fusion blocks. Systematic benchmarking studies and various comparisons were conducted using LLFormer.

3. Proposed Network

3.1. Configuration of Proposed Network

Figure 3 shows the overall structure of the proposed network. It comprises a DRS block, an IFA block, and convolution layers with various kernel sizes. The numbers along the lines represent the channel count in the feature maps. For an input low-light image L, the DRS block produces a stretched image feature as follows:
L S = D R S L ,
where LS denotes the stretched image with three channels, and DRS() denotes the function of the dynamic range stretch block. The histogram-equalized image LHE and L are concatenated and passed through the convolution layer to generate input features for the IFA block. The IFA block produces a 128-channel feature map LA to compensate for the stretched image feature LS as follows:
L A = I F A ρ C o n v 3 ( L © L H E ) ,
where IFA() denotes the function of the informative feature attention block, ρ represents the parametric rectified linear unit (PReLU), Convm() denotes an m × m convolution layer, and © denotes concatenation. The feature RF is obtained by applying LS weighted by LA as follows:
R F = ρ C o n v 3 L S ⊗ ρ C o n v 3 L A ,
where ⭙ denotes the point-wise multiplication operator. To obtain the final enhanced image R, two successive convolution operations on RF are performed as follows:
R = C o n v 1 ρ C o n v 3 R F .

3.2. Dynamic Range Stretch Block

As depicted in Figure 1, the chromatic histograms of the low-light image are concentrated toward lower brightness values. Hence, a histogram stretching method is necessary to improve the brightness of the low-light image. The classical image normalization technique employs the maximum and minimum values to adjust the dynamic range of pixel values. For a given image feature I, the normalization process is expressed as follows:
I n o = I − pmin ( I ) pmax ( I ) − pmin ( I ) .
where Ino denotes the normalized image, and pmax() and pmin() denote the percentile maximum and minimum, respectively. Typically, the percentile maximum and minimum values are used because they mitigate the influence of outliers during normalization. Figure 4 shows normalized low-light images with different percentile values. As depicted in Figure 4, normalized images are not always effective. Upon examining the results in the second row of Figure 4, proper adjustment of the percentile values ensures an appropriate level of normalization performance for a particular image. However, the results in the first low do not achieve acceptable enhancement, regardless of the different parameter selections.
Image normalization is a fundamental image processing technique used to stretch the histogram of an image and is commonly employed for image enhancement. In this study, we extend this concept to the feature level, aiming to enhance the low-light images by stretching their feature representation. A key challenge in feature normalization is determining the optimal percentile values, as they depend on the specific characteristics of the feature. To address this, we introduce the DRS block that enables adaptive feature normalization through adaptive standardization and normalization modules [14]. The critical aspect of adaptive normalization lies in determining feature-specific percentile maximum and minimum values, which is achieved by the DRS block for feature-adaptive normalization.
Figure 5 shows a detailed diagram of the DRS block. This block comprises standardization and normalization components. Before applying adaptive feature normalization, the low-light image feature is converted into a linearized Gaussian form using the batch normalization technique [45]. For a given input feature X, the normalized feature b is computed as follows:
b = α v − μ v σ v + β ,
where v = Conv3(X), and μ(v) and σ(v) denote the mean and standard deviation of v, respectively. In (6), α and β denote tunable parameters. Finally, the standardized output feature Xstd is obtained as follows:
X s t d = γ ( b ) ,
where γ() denotes the sigmoid activation function.
The existing squeeze-and-excitation (SE) block [15] mainly comprises squeeze and excitation operations. The squeeze operation uses global average pooling to capture global information. The excitation module includes two fully connected layers, a rectified linear unit (ReLU), and a sigmoid function to capture channel-wise dependencies. The SE block produces input-specific channel weights. In this study, we use two SE blocks that incorporate global maximum and minimum pooling [14] to obtain adjustable weights for two percentile values. The two tunable parameters ωmax and ωmin are computed as follows:
ω max = S E max ( X s t d ) ,
ω min = S E min ( X s t d ) ,
where SEmax() and SEmin() denote SE operation using global maximum and minimum pooling, respectively. Finally, the standardized feature Xstd is adaptively expanded using the obtained weights ωmax and ωmin as follows:
Y = DRS X = X s t d − ω min ω max − ω min ,
where Y represents the output of the DRS block. Unlike traditional normalization methods, the DRS block determines the values of A and B based on the given features, enabling adaptive, data-driven normalization. The proposed DRS block transforms a low-light image with a limited range of values into a stretched representation similar to normal light, thereby enhancing the overall brightness.

3.3. Informative Feature Attention Block

The direct application of the DRS block to low-light images enhances brightness but is insufficient for restoring color and details. Figure 6 shows the enhanced results achieved using only the DRS block. As shown in Figure 6, the enhanced image results exhibit faint colors and do not achieve full brightness, as indicated by histograms within a limited pixel range.
In this study, we propose an IFA block to recover color information and details of images. The IFA block primarily comprises three components: the FS, DR, and SE blocks, as shown in Figure 7. Inputs into the IFA block, comprising 64-channel feature maps, are generated by concatenating the histogram-equalized counterpart LHE with the low-light image L and then applying convolution, as depicted in Figure 2. LHE is used as a guide image, containing useful color information and ensuring acceptable brightness levels. Given an input feature X, the operation of the IFA block is expressed as follows:
Y = IFA X = FS X © DR X ⊕ S E max FS X © DR X ,
where ⨁ denotes the point-wise addition operator, and FS() and DR() denote the functions of the FS and DR blocks, respectively.
The IFA block is designed to calculate attentive weights to further enhance the output of the DRS block. The FS block is designed to further stretch the feature distributions under the guidance of LHE, and the DR block restores image details by analyzing the input features across multiple scales. Combining the results from the FS and DR blocks, the SE-Res block with global maximum pooling is used to generate features with improved brightness and restored details. The colorfulness of an image is primarily influenced by its saturation, which is determined by the proportion of color pixels. Therefore, effective brightness and detail restoration are essential for accurate color restoration. As the proposed IFA block efficiently recovers both brightness and details in low-light images, it can significantly improve the overall performance of color restoration.

3.3.1. Feature Stretch Block

Figure 8 illustrates the FS block, which comprises the DRS block and SE block with global average pooling. In the FS block, a point-based feature denoted as X1 is generated from the input feature X using a 1×1 convolution kernel, expressed as X1 = ρ(Conv1(X)). In a part of the FS block, the DRS block is applied to 64-channel features instead of RGB images. The output features of the DRS block, denoted as X2, are computed via two consecutive 1×1 convolution operations, and each operation is followed by a ReLU activation function, which is expressed as follows:
X 2 = δ Con v 1 δ Con v 1 DRS X ,
where δ denotes the ReLU function. The final feature Y of the FS block is obtained by employing the residual form of the SE block, along with global average pooling as follows:
Y = FS X = X 1 ⊕ ρ Con v 1 S E ave X 1 © X 2 ,
where SEave() denotes the operation of the SE block with global average pooling. The point-based features from the upper part of the FS block and the stretched features from the lower part of the FS block are weighted on the basis of their importance via the SE block.

3.3.2. Detail-Recovery Block

The FS block is designed to recover image brightness by expanding the distribution of features from a global perspective. Consequently, its operation can be limited in terms of recovering the detail and color information in the image. In this study, we introduce a DR block to restore image detail by modifying the multiscale module presented in KinD++ [16].
Figure 9 shows the DR block, which comprises the multiscale attention (MA) block and SE block with global average pooling. While the FS block employs 64-channel features, the number of channels is doubled to capture complex and diverse patterns more finely and is used as input for the DR block. In this block, X1 represents the feature vector obtained by doubling the input feature X using a 3 × 3 convolution kernel, which can be expressed as X1 = ρ(Conv3(X)). The features doubled by the convolution operation are input into the MA block, which has three resolution scales. Multiscale processing is achieved by using different convolutional strides, including 1, 2, and 4. The resulting multiscale attentive features are concatenated and subsequently undergo a 1×1 convolution followed by the application of the ReLU function to produce the output of the MA block. The output Y of the DR block is obtained using the residual form of the conventional as follows:
Y = DR X = ρ Con v 3 X 2 ⊕ ρ Con v 1 S E ave MA X 1 © X 2 ,
where X2 = ρ(Conv3 (X1)).

3.4. Loss Function

The goal of the LIE network is to generate a recovered image that closely resembles the original clean image. To achieve this, the proposed network uses SSIM [22] and total variation (TV) loss functions. The SSIM loss (LSSIM), which evaluates the structural similarity between the recovered and clean images, is defined as follows:
L S S I M = 1 − S S I M ( R , G ) ,
where G represents the ground-truth image, and SSIM(R, G) denotes the SSIM value between R and G. The TV loss LTV is defined as follows:
L T V = ∑ ( x , y ) R ( x , y + 1 ) − R ( x , y ) 2 + R ( x + 1 , y ) − R ( x , y ) 2 ,
where (x,y) denotes the spatial location of an image pixel. LTV is commonly used to recover images naturally while minimizing noise in LIE networks. The overall loss function used in this study is expressed as follows by combining the two loss functions:
L t = L S S I M + λ L T V ,
where λ denotes a hyper-parameter. In this study, λ is set to 0.001 and used for learning.

4. Experimental Results

4.1. Implementation Details

The proposed FSANet was trained on a PC equipped with a single 8 GB GeForce RTX 2080 GPU, an Intel i5-10400K CPU@ 2.90 GHz, and 16 GB of RAM. No GPU or multi-threading acceleration was employed, and the code was implemented using Pytorch (verson 1.10.1). The patch size used for training was 128 × 128, and the batch size was set to 8. To train the proposed network, the ADAM [46] optimization algorithm was used. The training process was conducted over 6000 epochs with an initial learning rate of 10−4.

4.2. Datasets and Comparison Methods

In this study, three training datasets were used: the low-light (LOL) [38] dataset, the synthetic LOL dataset, and the vision enhancement low-light (VE-LOL) [47] dataset. The LOL dataset contains 485 pairs of images for training and 15 pairs for testing. VE-LOL and the synthetic dataset consist of 900 and 240 pairs of images for training, respectively. A total of 1625 image pairs were used in the training process for our study. To assess LIE performance, we used five non-reference image datasets: DICM [48], LIME [6], Fusion [49], VV [50], and MEF [51].
To evaluate the performance of the proposed network, we compared it with the following 14 LIE methods: 3 model-based methods (LIME [6], SDD [29], and GCPMC [34]), 9 machine learning networks (KinD++ [16], EnlightenGAN [17], AGLLNet [18], TBEFN [19], URetinex-Net [20], DLN [36], DSLR [37], RetinexNet [38], and Zero-DCE [42]), and 2 transformer-based networks (LLFormer [21] and SNRA [43]). Executable codes and network models for the performance comparison were obtained from publicly accessible project sites.

4.3. Complexity

Table 1 provides the number of model parameters and runtimes for machine-learning-based LIE networks to achieve improved low-light images. The runtime was computed using all test images from the LOL dataset, and 10 runtimes were obtained and averaged to minimize deviation. Because the runtime is influenced by the performance of the processor used and the number of parameters, it is crucial to consider these two factors collectively. The proposed network, running on a low-performance GPU, exhibits a relatively low runtime of 0.388 s.

4.4. Results on Synthetic Dataset

Six metrics are used to evaluate the performance of LIE on the LOL dataset [38], comparing results with the ground truth. These metrics include PSNR, SSIM [22], the color difference measure (CIEDE2000) [52], the learned perceptual image patch similarity (LPIPS) [53], the feature similarity index measure (FSIM) [54], and the naturalness image quality evaluator (NIQE) [55]. Table 2 shows the quantitative results for various LIE methods on the LOL dataset. The symbol ↓ denotes that lower values are better, while the symbol ↑ indicates that higher values are preferable. As presented in Table 2, most model-based methods consistently achieve low rankings across all metrics except for FSIM. GCPMC only achieves a notable ranking in the NIQE metric. In contrast, recent transformer-based networks, such as SNRA, LLFormer, and the proposed network, achieve higher rankings across all subjective evaluation metrics. SNRA achieves the highest ranking in PSNR, SSIM, and CIEDE2000, secures the second position in both NIQE and FSIM, and holds the third position in LPIPS. Conversely, LLFormer ranks third in terms of PSNR, SSIM, CIEDE2000, and FSIM. In comparison, the proposed network leads in terms of NIQE and FSIM, both of which indicate naturalness, and takes the second position in terms of PSNR, SSIM, CIEDE2000, and LPIPS.
Figure 10 presents restored image results on the LOL dataset. Model-based LIE methods, including LIME, SDD, and GCPMC, show significant variations in brightness across different images, indicating that their restoration success is highly dependent on individual image characteristics. While RetinexNet achieves satisfactory brightness restoration, it introduces considerable color shifts in the restored images. DLN overly enhances brightness, leading to a notable loss of color. Zero-DCE struggles to fully restore brightness, and EnlightenGAN fails to adequately improve brightness. Although TBEFN restores brightness effectively, it shows limitations in color restoration. Images restored by AGLLNet and DSLR exhibit noticeable noise, and there is a significant color shift compared to the ground truth. URetinex-Net results in images with low contrast and color loss. In contrast, SNRA, LLFormer, and the proposed network successfully restore both brightness and color in low-light images. In summary, the proposed FSANet outperforms transformer-based methods, such as SNRA and LLFormer, in terms of overall image quality and network complexity.
To verify the generalization ability of the proposed network, we trained it using only the real-world LOL dataset and evaluated its performance on the LOL test dataset. In this case, the PSNR, SSIM, and NIQE values are 23.42, 0.82, and 2.59, respectively, indicating a decrease in performance compared to when the entire dataset is used. When trained on the real-world LOL dataset and tested on 15 LOL images, KinD++ achieves a PSNR of 21.30, SSIM of 0.82, and NIQE of 3.88, while the biologically inspired LIE method with a unified network (LA-Net) [56] achieves a PSNR of 21.71, SSIM of 0.81, and NIQE of 3.10. Compared to these two networks, the proposed network exhibits less performance degradation when trained exclusively on the real-world LOL dataset.

4.5. Results on Non-Reference Datasets

Figure 11 presents a visual comparison of various LIE methods, including the proposed FSANet, on the VV dataset. The model-based approaches LIME, SDD, and GCPMC successfully enhance brightness. However, this enhancement is excessive, resulting in nearly uniform brightening of all image areas and the introduction of noise. RetinexNet fails to recover color and detail information, resulting in an overall exaggeration of colors in the image. DLN exhibits lower brightness levels. Zero-DCE loses a significant amount of color because of excessive brightness. EnlightenGAN, KinD++, TBEFN, and AGLLNet generate over-enhanced images characterized by color distortion and an unnatural appearance. Although DSLR provides acceptable image results, it exhibits slightly reduced brightness. The transformer-based networks, including LLFormer and SNRA, exhibit excessive brightness, resulting in image blurriness. In contrast, the proposed FSANet generates an acceptably enhanced image with appropriate brightness and minimal color loss. In summary, the proposed network achieves the best balance between subjective image quality and objective metric values, demonstrating its generalization capability.
Figure 12 presents the enhanced images acquired from the DICM dataset. LIME, SDD, GCPMC, and EnlightenGAN demonstrate visually pleasing results on the DICM dataset. RetinexNet fails to achieve natural image restoration. DSLR does not fully restore image brightness. Zero-DCE produces slightly blurred results. KinD++, TBEFN, and AGLLNet produce over-enhanced images, leading to increased brightness, even in areas that are initially dark. LLFormer and SNRA generate excessive brightness, resulting in blurred images. The proposed FSANet successfully and consistently restores color and details in the enhanced image, creating natural-looking images by avoiding excessive enhancement in the dark areas.
Figure 13 illustrates the enhanced images obtained for the LIME dataset using various LIE methods. As depicted in Figure 13, the sample low-light images consist of a dark indoor scene and a partially illuminated outdoor scene. However, most methods tend to overly improve all areas of the image without considering the locations of light sources. Furthermore, the light from these sources is excessively diffused. In contrast, the proposed FSANet excessively achieves satisfactory restoration results under various illumination conditions. Moreover, the resulting image obtained by the proposed method exhibits minimal light spread caused by the small illumination source.
Figure 14 shows the results of LIE obtained on the MEF dataset. RetinexNet generates unrealistic images, whereas DLN, Zero-DEC, LLFormer, and SNRA do not fully restore the background color information. DSLR does not recover full brightness. In contrast, the model-based methods achieve acceptable brightness recovery and restore detail and color. The proposed network successfully recovers brightness, detail, and color in the restored images.
In summary, when comparing the results across non-reference image datasets, some effective methods produce varying outcomes depending on the dataset. In contrast, the proposed FSANet consistently restores brightness, color, and detail to satisfactory levels across all datasets. This consistency demonstrates the effectiveness of the proposed network. When comparing the results obtained on the LOL dataset with those obtained on the non-reference datasets, the proposed network consistently delivers high image quality in both subjective and objective evaluations. In conclusion, the proposed LIE network has significant potential to be effectively used as a pre-processing step to enhance the performance of various computer vision applications.
Table 3 lists the quantitative metric values for the five datasets: DICM, LIME, Fusion, VV, and MEF. These datasets are evaluated in terms of NIQE, which serves as an indicator of image naturalness. Among the model-based methods, SDD achieves the second position in terms of the average score ranking, whereas the other model-based methods achieve lower rankings. DSLR occupies the third and fourth positions, respectively. Conversely, the state-of-the-art transformer-based methods SNRA and LLFormer, which perform significantly well on the LOL dataset, exhibit below-average performance on the non-reference datasets. This indicates that these two transformer-based networks may result in overfitting on the LOL dataset. Consequently, they achieve high scores only on this dataset but exhibit lower average rankings across the non-reference datasets. The proposed network achieves high rankings on all five datasets and secures the top position in terms of the average value.

4.6. Ablation Experiments

For the ablation study, we conducted three experiments to evaluate the effects of the introduced DRS block and IFA block, which comprises the FS and DR blocks. The three ablation experiments are listed as follows: with DRS block + without FS and DR blocks (A-I), with DRS and FS blocks + without DR block (A-II), and with DRS and DR blocks + without FS block (A-III).
Table 4 shows the three quantitative measures, PSNR, SSIM, and FSIM, for the three ablation experiments conducted on the LOL dataset. The ablation experiment using only the DRS leads to a significant reduction in PSNR from 24.31 to 17.33 dB, along with a decrease in SSIM and FSIM values. As shown in Figure 6, the DRS block mainly contributes to increasing brightness. The results from A-I emphasize the critical role of the IFA block in LIE within the proposed network. A substantial increase of 6.97 dB in PSNR compared to A-I is observed, with noticeable improvements in SSIM and FSIM. These results highlight the significant role of the proposed FS block. In the ablactation study A-III, which combines the DRS block with only the DR block in the IFA block, significant increases in PSNR, SSIM, and FSIM are also evident.
The role of each proposed block can be observed in more detail in Figure 15. When only the DRS block is used (A-I), there is a significant increase in brightness, resulting in significant improvements in the PSNR, SSIM, and FSIM values. However, color and image details are not adequately restored. In the presence of the FS block within the IFA block, combined with the DRS block (A-II), there is a significant improvement in color restoration; however, image details are not fully recovered. In the ablation study A-III, using only the DR block within the IFA block, image detail is more pronounced in the restored image. According to these ablations studies, the proposed network, integrating all three blocks, yields outstanding results.

5. Conclusions

This study proposed a novel low-light image enhancement network using an adaptive feature stretching technique. The proposed network incorporates a dynamic range stretch block designed to restore the brightness of low-light image features. To further restore details in the initially enhanced image, an informative feature attention block was introduced, which comprises a feature stretch block and a detail-recovery block. The feature stretch block improves color information guided by the histogram-equalized image, while the detail-recovery block uses a multiscale attention method to enhance image contrast and details. Extensive experiments were conducted on various image datasets to compare the performance of state-of-the-art methods with the proposed network. The experimental results demonstrated that the proposed network outperforms existing methods, both in quantitative metrics and visual assessments.

Author Contributions

S.M.C. proposed the framework of this work, carried out the experiments, and drafted the manuscript; J.Y.P. offered useful suggestions for the color correction algorithm and helped to modify the manuscript; I.K.E. initiated the main algorithm of this work, supervised the whole work, and wrote the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by a 2-Year Research Grant from Pusan National University.

Data Availability Statement

The code is available at https://github.com/seongminim/FSANet (accessed on 26 September 2024).

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Kim, W. Low-light image enhancement: A comparative review and prospects. IEEE Access 2022, 10, 84535–84557. [Google Scholar] [CrossRef]
  2. Rasheed, M.T.; Shi, D.; Khan, H. A comprehensive experiment-based review of low-light image enhancement methods and benchmarking low-light image quality assessment. Signal Process. 2023, 204, 108821. [Google Scholar] [CrossRef]
  3. Li, C.; Guo, C.; Han, L.; Jiang, J.; Cheng, M.M.; Gu, J.; Loy, C.C. Low-light image and video enhancement using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 44, 9396–9416. [Google Scholar] [CrossRef]
  4. Singh, K.; Kapoor, R. Image enhancement using exposure based subimage histogram equalization. Pattern Recogn. Lett. 2014, 36, 10–14. [Google Scholar] [CrossRef]
  5. Kim, S.E.; Jeon, J.J.; Eom, I.K. Image contrast enhancement using entropy scaling in wavelet domain. Signal Process. 2016, 127, 1–11. [Google Scholar] [CrossRef]
  6. Guo, X.; Li, Y.; Ling, H. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 2017, 26, 982–993. [Google Scholar] [CrossRef] [PubMed]
  7. Jeon, J.J.; Eom, I.K. Low-light image enhancement using inverted image normalized by atmospheric light. Signal Process. 2022, 196, 108523. [Google Scholar] [CrossRef]
  8. Lore, K.G.; Akintayo, A.; Sarkar, S. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 2017, 61, 650–662. [Google Scholar] [CrossRef]
  9. Park, J.Y.; Park, C.W.; Eom, I.K. ULBPNet: Low-light image enhancement using U-shaped lightening back-projection. Knowl.-Based Syst. 2023, 281, 111099. [Google Scholar] [CrossRef]
  10. Land, E.H. The Retinex theory of color vision. Sci. Am. 1997, 237, 108–128. [Google Scholar] [CrossRef] [PubMed]
  11. Narasimhan, S.G.; Nayar, S.K. Contrast restoration of weather degraded images. IEEE Trans. Pattern Anal. Mach. Intell. 2003, 25, 713–724. [Google Scholar] [CrossRef]
  12. Wang, Y.; Wan, R.; Li, H.; Chau, L.P.; Kot, A. Low-light image enhancement with normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, CA, USA, 22 February–1 March 2022; pp. 2604–2612. [Google Scholar] [CrossRef]
  13. Cui, H.; Li, J.; Hua, Z.; Fan, L. TPET: Two-stage perceptual enhancement transformer network for low-light image enhancement. Eng. Appl. Artif. Intell. 2022, 116, 105411. [Google Scholar] [CrossRef]
  14. Park, C.W.; Eom, I.K. Underwater image enhancement using adaptive standardization and normalization networks. Eng.Appl. Artif. Intell. 2024, 127, 107445. [Google Scholar] [CrossRef]
  15. Hu, J.; Shen, L.; Albanie, S.; Sun, G.; Wu, E. Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 2011–2023. [Google Scholar] [CrossRef] [PubMed]
  16. Zhang, Y.; Guo, X.; Ma, J.; Liu, W.; Zhang, J. Beyond brightening low-light images. Int. J. Comput. Vis. 2021, 129, 1013–1037. [Google Scholar] [CrossRef]
  17. Jiang, Y.; Gong, X.; Liu, D.; Cheng, Y.; Shen, X.; Yang, J.; Zhou, P.; Wang, Z. EnlightenGAN: Deep light enhancement without paired supervision. IEEE Trans. Image Process. 2021, 30, 2340–2349. [Google Scholar] [CrossRef]
  18. Lv, F.; Li, Y.; Lu, F. Attention guided low-light image enhancement with a large scale low-light simulation dataset. Int. J. Comput. Vis. 2021, 29, 2175–2193. [Google Scholar] [CrossRef]
  19. Lu, K.; Zhang, L. TBEFN: A two-branch exposure-fusion network for low-light image enhancement. IEEE Trans. Multimed. 2021, 23, 4093–4105. [Google Scholar] [CrossRef]
  20. Wu, W.; Weng, J.; Zhang, P.; Wang, X.; Yang, W.; Jiang, J. URetinex-Net: Retinex-based deep unfolding network for low-light image enhancement. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 5891–5900. [Google Scholar] [CrossRef]
  21. Wang, T.; Zhang, K.; Shen, T.; Luo, W.; Stenger, B.; Lu, T. Ultra-high-definition low-light image enhancement: A benchmark and transformer-based method. In Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington, DC, USA, 7–14 February 2023; pp. 2654–2662. [Google Scholar] [CrossRef]
  22. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef]
  23. Liu, S.C.; Liu, S.; Wu, H.; Rahman, M.A.; Lin, S.C.-F.; Wong, C.Y.; Kwok, N.; Shi, H. Enhancement of low illumination images based on an optimal hyperbolic tangent profile. Comput. Electr. Eng. 2018, 70, 538–550. [Google Scholar] [CrossRef]
  24. Srinivas, K.; Bhandari, A.K. Low light image enhancement with adaptive sigmoid transfer function. IET Image Process. 2020, 14, 668–678. [Google Scholar] [CrossRef]
  25. Celik, T.; Tjahjadi, T. Contextual and variational contrast enhancement. IEEE Trans. Image Process. 2011, 20, 3431–3441. [Google Scholar] [CrossRef] [PubMed]
  26. Fu, X.; Zeng, D.; Huang, Y.; Liao, Y.; Ding, X.; Paisley, J. A fusion-based enhancing method for weakly illuminated images. Signal Process. 2016, 129, 82–96. [Google Scholar] [CrossRef]
  27. Ying, Z.; Li, G.; Gao, W. A bio-inspired multi-exposure fusion framework for low-light image enhancement. arXiv 2017, arXiv:1711.00591. [Google Scholar] [CrossRef]
  28. Li, M.; Liu, J.; Yang, W.; Sun, X.; Guo, Z. Structure-revealing low-light image enhancement via robust Retinex model. IEEE Trans. Image Process. 2018, 27, 2828–2841. [Google Scholar] [CrossRef]
  29. Hao, S.; Han, X.; Guo, Y.; Xu, X.; Wang, M. Low-light image enhancement with semi-decoupled decomposition. IEEE Trans. Multimed. 2020, 22, 3025–3038. [Google Scholar] [CrossRef]
  30. Ren, Y.; Ying, Z.; Li, T.H.; Li, G. LECARM: Low-light image enhancement using the camera response model. IEEE Trans. Circuits Syst. Video Technol. 2019, 29, 968–981. [Google Scholar] [CrossRef]
  31. He, K.M.; Sun, J.; Tang, X.O. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 23, 2341–2353. [Google Scholar] [CrossRef]
  32. Shi, Z.; Zhu, M.; Guo, B.; Zhao, M. A photographic negative imaging inspired method for low illumination night-time image enhancement. Multimed. Tools Appl. 2017, 76, 15027–15048. [Google Scholar] [CrossRef]
  33. Gu, Z.; Chen, C.; Zhang, D. A low-light image enhancement method based on image degradation model and pure pixel ratio prior. Math. Probl. Eng. 2018, 2018, 8178109. [Google Scholar] [CrossRef]
  34. Jeon, J.J.; Park, J.Y.; Eom, I.K. Low-light image enhancement using gamma correction prior in mixed color spaces. Pattern Recognit. 2024, 146, 110001. [Google Scholar] [CrossRef]
  35. Lv, F.; Lu, F.; Wu, J.; Lim, C. MBLLEN: Low-light image/video enhancement using CNNs. In Proceedings of the 29th British Machine Vision Conference (BMVC), Newcastle, UK, 3–6 September 2018; pp. 1–13. [Google Scholar]
  36. Wang, L.W.; Liu, J.S.; Siu, W.C.; Lun, D.P.K. Lightening network for low-light image enhancement. IEEE Trans. Image Process. 2020, 29, 7984–7996. [Google Scholar] [CrossRef]
  37. Lim, S.; Kim, W. DSLR: Deep stacked Laplacian restorer for low-light image enhancement. IEEE Trans. Multimed. 2021, 23, 4272–4284. [Google Scholar] [CrossRef]
  38. Wei, C.; Wang, W.; Yang, W.; Liu, J. Deep Retinex decomposition for low-light enhancement. arXiv 2018, arXiv:1808.04560. [Google Scholar] [CrossRef]
  39. Zhang, Y.; Zhang, J.; Guo, X. Kindling the darkness: A practical low-light image enhancer. In Proceedings of the 27th ACM International Conference on Multimedia, New York, NY, USA, 21–15 October 2019; pp. 1632–1640. [Google Scholar] [CrossRef]
  40. Zhu, A.; Zhang, L.; Shen, Y.; Ma, Y.; Zhao, S.; Zhou, Y. Zero-shot restoration of underexposed images via robust Retinex decomposition. In Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK, 6–10 July 2020; pp. 1–6. [Google Scholar] [CrossRef]
  41. Zhao, Z.; Xiong, B.; Wang, L.; Ou, Q.; Yu, L.; Kuang, F. RetinexDIP: A unified deep framework for low-light image enhancement. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 1076–1088. [Google Scholar] [CrossRef]
  42. Guo, C.; Li, C.; Cuo, J.; Loy, C.C.; Hou, J.; Kwong, S.; Cong, R. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 1780–1789. [Google Scholar] [CrossRef]
  43. Xu, X.; Wang, R.; Fu, C.W.; Jia, J. SNR-aware low-light image enhancement. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 17693–17703. [Google Scholar] [CrossRef]
  44. Yang, S.; Zhou, D.; Cao, J.; Guo, Y. LightingNet: An integrated learning method for low-light image enhancement. IEEE Trans. Comput. Imaging 2023, 9, 29–42. [Google Scholar] [CrossRef]
  45. Loffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
  46. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA, 7–9 May 2015; pp. 1–15. [Google Scholar] [CrossRef]
  47. Liu, J.; Xu, D.; Yang, W.; Fan, M.; Huang, H. Benchmarking low-light image enhancement and beyond. Int. J. Comput. Vis. 2021, 129, 1153–1184. [Google Scholar] [CrossRef]
  48. Lee, C.; Lee, C.; Kim, C.S. Contrast enhancement based on layered difference representation. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Orlando, FL, USA, 30 September–3 October 2012; pp. 965–968. [Google Scholar] [CrossRef]
  49. Wang, Q.; Fu, X.; Zhang, X.P.; Ding, X. A fusion-based method for single backlit image enhancement. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016; pp. 4077–4081. [Google Scholar] [CrossRef]
  50. Vonikakis, V. Dataset. Available online: https://sites.google.com/site/vonikakis/datasets (accessed on 14 June 2024).
  51. Ma, K.; Zeng, K.; Wang, Z. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 2015, 24, 3345–3356. [Google Scholar] [CrossRef] [PubMed]
  52. Sharma, G.; Wu, W.; Dalal, E.N. The ciede2000 color difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Res. Appl. 2005, 30, 21–30. [Google Scholar] [CrossRef]
  53. Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 586–595. [Google Scholar] [CrossRef]
  54. Zhang, L.; Zhang, L.; Mou, X.; Zhang, D. FSIM: A feature similarity index for image quality assessment. IEEE Trans. Image Process. 2011, 20, 2378–2386. [Google Scholar] [CrossRef] [PubMed]
  55. Mittal, A.; Soundararajan, R.; Bovik, A.C. Making a ‘completely blind’ image quality analyzer. IEEE Signal Process. Lett. 2013, 20, 209–212. [Google Scholar] [CrossRef]
  56. Yang, K.F.; Cheng, C.; Zhao, S.X.; Yan, H.M.; Zhang, X.S.; Li, Y.J. Learning to adapt to light. Int. J. Comput. Vis. 2023, 131, 1022–1041. [Google Scholar] [CrossRef]
Electronics 13 03883 g001
Figure 1. Low-light images and their respective ground-truth counterparts, along with their accompanying histograms.
Figure 1. Low-light images and their respective ground-truth counterparts, along with their accompanying histograms.
Electronics 13 03883 g001
Electronics 13 03883 g002
Figure 2. Examples of enhanced image results using different machine-learning-based methods.
Figure 2. Examples of enhanced image results using different machine-learning-based methods.
Electronics 13 03883 g002
Electronics 13 03883 g003
Figure 3. The overall structure of the proposed LIE network.
Figure 3. The overall structure of the proposed LIE network.
Electronics 13 03883 g003
Electronics 13 03883 g004
Figure 4. Conventional image normalization examples with different percentile maximum and minimum values (max percentile, min percentile).
Figure 4. Conventional image normalization examples with different percentile maximum and minimum values (max percentile, min percentile).
Electronics 13 03883 g004
Electronics 13 03883 g005
Figure 5. Configuration of DRS block.
Figure 5. Configuration of DRS block.
Electronics 13 03883 g005
Electronics 13 03883 g006
Figure 6. Restored low-light images and corresponding histograms using only the DRS block.
Figure 6. Restored low-light images and corresponding histograms using only the DRS block.
Electronics 13 03883 g006
Electronics 13 03883 g007
Figure 7. Configuration of IFA block.
Figure 7. Configuration of IFA block.
Electronics 13 03883 g007
Electronics 13 03883 g008
Figure 8. Configuration of FS block.
Figure 8. Configuration of FS block.
Electronics 13 03883 g008
Electronics 13 03883 g009
Figure 9. Configuration of DR block.
Figure 9. Configuration of DR block.
Electronics 13 03883 g009
Electronics 13 03883 g010
Figure 10. Enhanced image results for the paired synthetic LOL dataset.
Figure 10. Enhanced image results for the paired synthetic LOL dataset.
Electronics 13 03883 g010
Electronics 13 03883 g011
Figure 11. Visual comparison of enhancement results on the VV dataset (24 images).
Figure 11. Visual comparison of enhancement results on the VV dataset (24 images).
Electronics 13 03883 g011
Electronics 13 03883 g012
Figure 12. Visual comparison of enhancement results on the DICM dataset (44 images).
Figure 12. Visual comparison of enhancement results on the DICM dataset (44 images).
Electronics 13 03883 g012
Electronics 13 03883 g013
Figure 13. Visual comparison of enhancement results on the LIME dataset (10 images).
Figure 13. Visual comparison of enhancement results on the LIME dataset (10 images).
Electronics 13 03883 g013
Electronics 13 03883 g014
Figure 14. Visual comparison of enhancement results on the MEF dataset (17 images).
Figure 14. Visual comparison of enhancement results on the MEF dataset (17 images).
Electronics 13 03883 g014
Electronics 13 03883 g015
Figure 15. Visual comparison of results using three ablation studies.
Figure 15. Visual comparison of results using three ablation studies.
Electronics 13 03883 g015
Table 1. Comparisons of runtime and number of parameters between the proposed and existing networks.
Table 1. Comparisons of runtime and number of parameters between the proposed and existing networks.
NetworkNumber of Parameters (×106)Runtime (s)
RetinexNet [38]0.4449.033
DLN [36]0.7010.400
TBEFN [19]0.4860.909
Zero-DCE [42]0.0790.016
EnlightenGAN [17]8.6370.031
KinD++ [16]8.27523.45
AGLLNet [18]0.9262.800
DSLR [37]14.9314.025
URetinex-Net [20]0.3610.456
SNRA [43]39.1200.382
LLFormer [21]24.5490.454
Proposed network1.5320.388
Table 2. Comparison of values for six objective evaluation measures on the LOL dataset 1.
Table 2. Comparison of values for six objective evaluation measures on the LOL dataset 1.
MethodPSNR↑SSIM↑NIQE↓CIEDE2000↓LPIPS↓FSIM↑
LIME [6]16.420.5517.58216.6000.40350.879
SDD [29]13.340.6092.86321.8350.26090.903
GCPMC [34]17.390.5402.52214.6310.42890.876
RetinexNet [38]14.980.4058.09120.4740.46490.782
DLN [36]19.260.7286.57713.2950.29600.940
TBEFN [19]15.840.5812.62317.5100.21070.839
Zero-DCE [42]14.800.5836.92518.9200.32990.927
EnlightenGAN [17]15.310.4985.05619.1280.31920.823
KinD++ [16]15.720.5972.82417.2600.19770.780
AGLLNet [18]17.530.6593.15713.9450.21810.919
DSLR [37]14.080.5142.96819.6710.32750.833
URetinex-Net [20]17.280.6182.83815.2320.12750.849
SNRA [43]24.610.8432.5166.8490.16100.962
LLFormer [21]23.650.8102.6647.9380.16400.959
Proposed network24.310.8202.3047.5090.13800.965
1 The top-performing, second-best, and third-best methods are emphasized in bold, italic, and underlined, respectively.
Table 3. Comparison of NIQE values for various enhancement methods.
Table 3. Comparison of NIQE values for various enhancement methods.
MethodDICMLIMEFusionVVMEFAverage
LIME [6]3.3383.5812.7642.4582.9973.028
SDD [29]2.3212.8562.3922.0192.2272.363
GCPMC [34]3.3083.6302.7282.3063.0022.995
RetinexNet [38]4.0764.1433.0812.9073.8323.608
DLN [36]2.0233.2132.4631.8942.5522.429
TBEFN [19]2.3413.2152.2081.9162.1672.369
Zero-DCE [42]2.6063.3932.5962.1602.8462.720
EnlightenGAN [17]2.7312.9572.2462.8842.3702.638
KinD++ [16]2.2573.5632.4141.9992.2802.503
AGLLNet [18]2.7893.6302.6522.5342.6092.843
DSLR [37]2.5792.8532.3321.7602.4112.387
SNRA [43]2.5013.5232.9354.2252.3733.111
LLFormer [21]2.9673.4173.2402.3892.5622.915
Proposed network2.2043.0342.4071.8542.4042.331
Table 4. Three quantitative measures for three ablation experiments conducted on the LOL dataset.
Table 4. Three quantitative measures for three ablation experiments conducted on the LOL dataset.
DRSIFAPSNR↑SSIM↑FSIM↑
FSDR
√√√24.310.8200.965
√ 17.340.5370.914
√√ 22.410.7760.953
√ √22.160.7860.946
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Chun, S.M.; Park, J.Y.; Eom, I.K. Low-Light Image Enhancement Network Using Informative Feature Stretch and Attention. Electronics 2024, 13, 3883. https://doi.org/10.3390/electronics13193883

AMA Style

Chun SM, Park JY, Eom IK. Low-Light Image Enhancement Network Using Informative Feature Stretch and Attention. Electronics. 2024; 13(19):3883. https://doi.org/10.3390/electronics13193883

Chicago/Turabian Style

Chun, Sung Min, Jun Young Park, and Il Kyu Eom. 2024. "Low-Light Image Enhancement Network Using Informative Feature Stretch and Attention" Electronics 13, no. 19: 3883. https://doi.org/10.3390/electronics13193883

APA Style

Chun, S. M., Park, J. Y., & Eom, I. K. (2024). Low-Light Image Enhancement Network Using Informative Feature Stretch and Attention. Electronics, 13(19), 3883. https://doi.org/10.3390/electronics13193883

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop