Open AccessArticle

The Identification and Assessment of Rail Corrugation Based on Computer Vision

Dehua Wei

Xiukun Wei

^2,*,

Yuxin Liu

¹,

Limin Jia

² and

Wenqiang Zhang

School of Traffic and Transportation, Beijing Jiaotong University, Beijing 100044, China

State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing 100044, China

Beijing Mass Transit Railway Operation Corporation LTD, Beijing 100044, China

Author to whom correspondence should be addressed.

Appl. Sci. 2019, 9(18), 3913; https://doi.org/10.3390/app9183913

Submission received: 29 August 2019 / Revised: 15 September 2019 / Accepted: 16 September 2019 / Published: 18 September 2019

(This article belongs to the Section Applied Industrial Technologies)

Download

Browse Figures

Figure 1
The corrugation on a railway track. "> Figure 2
The procedure of the two-stage image pre-processing method. (a) original image; (b) image de-noised by bilateral filter; (c) illumination extraction result; (d) corrected image of the proposed two-stage method. "> Figure 3
The process diagram of rail surface localization method. (a) original corrected image; (b) image processed by vertical edge enhanced; (c) the result of calculating weighted gradient sum value; (d) the result of rail surface localization. "> Figure 4
Sketch map of SPM model. "> Figure 5
The general structure diagram of Spatial Pyramid Matching (SPM). "> Figure 6
Comparison of two coding methods. (a) Vector quantization (VQ); (b) Locality constrained linear coding (LLC). "> Figure 7
The general structure diagram of integrated multi-features and locality constrained linear coding (IMFLLC). "> Figure 8
The diagram of corrugation interval localization. (a) Visualization of the localization process and results of the corrugation interval; (b) the result of the horizontal gray gradient statistical analysis. "> Figure 9
The structure diagram of fast Fourier transform normalized distance matching function (FFT-NDMF). "> Figure 10
The frequency-amplitude curves of some corrugation interval columns. "> Figure 11
Visualized examples of measurement results: sample image (a–f). "> Figure 12
The pipeline of corrugation severity assessment method. "> Figure 13
The visual result of the used mathematical morphology algorithm. "> Figure 14
The level membership function used in this paper. "> Figure 15
The visualization process and severity assessment result. (a) the original corrugation image; (b) the result rustless corrugation image after using horizontal gray gradeint statistical (HGGS); (c) the corrugation saliency map obtained by normalized frequency tuned (NFT); (d) the result of image binarization; (e) the result of each corrugation localization and the wear salliency (WS) of them; and (f) the final severity assessment result of each corrugation area. ">

Versions Notes

Abstract

The identification and assessment of rail corrugation are two of the essential tasks of daily railway inspection to guarantee the safety of train operation and promote the development of an efficient maintenance strategy. In view of the requirements for automatic identification and smart decision-making, computer vision-based rail corrugation identification and assessment methods are presented in this paper. Firstly, an improved Spatial Pyramid Matching (SPM) model, integrating multi-features and locality-constrained linear coding (IMFLLC), is proposed for rail corrugation identification. After that, an innovative period estimation method for rail corrugation is proposed based on the frequency domain analysis of each column in the corrugation region. Finally, the severity of the rail corrugation is assessed with the help of the wear saliency calculation and fuzzy theory. The experiment results demonstrate that the proposed corrugation identification method achieves a higher precision rate and recall rate than those of traditional methods, reaching 99.67% and 98.34%, respectively. Besides, the validity and feasibility of the proposed methods for the rail corrugation period estimation and severity assessment are also investigated.

Keywords:

rail corrugation identification; period estimation; severity assessment; computer vision

1. Introduction

Rail corrugation refers to the periodic non-uniform plastic deformation and wave-shaped irregularities along the longitudinal surface of rail heads [1]. In recent years, the development of urban rail transit has been severely affected by the problem of corrugation on the metro operation tracks. Due to the occurrence and evolution of rail corrugation, not only will the sudden changes in wheel-rail forces shorten the service life of the critical components of the rail line, which increase the maintenance costs but also the violent vibration between wheels and rails will generate great noise, reduce the passenger comfort and the safety of vehicle operation [2]. Corrugation detection has attracted more and more attention in recent years [1,3,4]. Therefore, it is of great significance to inspect corrugation in time and alleviate the development of corrugation effectively, which can reduce the cost of railway maintenance and ensure the safety of railway traffic.

For a long time, the detection methods for rail corrugation mainly based on manual inspection with a particular calliper or a corrugation inspection car. However, the fixed detection cycle cannot inspect the track state in time and this results in low detection efficiency [3]. Therefore, a growing number of research studies have been focused on improving the detection rate and accuracy. Nowadays, researchers have proposed a variety of new non-contact corrugation detection methods, including the chord surveying method, inertial reference method, computer vision method and so forth [5,6,7]. Among them, the first two methods are mostly based on mechanical and physical mechanisms, which are easy to calculate and handle but there are various factors to be considered, the anti-interference ability is insufficient and they are not efficient enough for a routine inspection. Conversely, with the rise of computer vision technology, the image processing technique has made significant achievements in the track inspection device, which has the advantages of being non-contact, low cost and fast. In addition, it can achieve multiple target detections simultaneously and analyze the images for a track field to figure out rail defects with advanced pattern recognition algorithms. Hence, it is remarkably suitable to performing long-term, high precision and repetitive tasks under complex environmental conditions [8,9,10,11].

At present, the vehicle track inspection system based on computer vision, mainly for rail fasteners and surface defects, has been used in some countries and the corresponding identification methods have been investigated and applied. Nevertheless, there are only a few reports on rail corrugation identification based on computer vision. Mandriota et al. proposed a rail corrugation identification method based on spatial filter characteristics and a machine learning algorithm [12]. The Gabor texture features are first extracted from the entire rail image and then the K-nearest neighbor method is used for corrugation recognition, in which a high recognition precision is obtained. It has become a widely used corrugation identification method based on computer vision [13] but it is sensitive to disturbances such as rust on the rail surface. Li et al. proposed an identification method for rail corrugation based on rail image features in the frequency domain, including the image acquisition subsystem and the corrugation identification subsystem installed under the vehicle body [14,15]. Instead of global texture features, local frequency features used in this method can reduce the time-consuming problem of traditional image methods and further improve the identification accuracy. However, it is still only able to detect the existence of corrugation and cannot analyze the periodicity and severity of the corrugation. Therefore, it is necessary to make some further efforts on this issue. These motivate the following works.

In this paper, the rail corrugation identification is converted into an image classification problem. Specifically, a learned classifier is used to determine whether the captured rail image is a corrugation rail image or not. At present, image classification methods based on local image descriptors are widely applied, which mainly includes Bag of Features (BOF) based method [16] and a Spatial Pyramid Matching (SPM) based method [17]. What is crucial to this kind of classification method is visual feature extraction and feature encoding process [18]. However, the traditional BOF or SPM only extracts a single feature of the image and the obtained feature information is not sufficient to adequately represent the rail image in many cases. In addition, by adopting the vector quantization (VQ) encoding method, the reconstruction error is large. These result in low image classification accuracy [19]. Therefore, taking the characteristics of corrugation rail image into account, the integrated idea is adopted to improve rail image classification performance. At first, to extract the image features more effectively, the Uniform LBP [20] and dense_SIFT [21] are used simultaneously to construct the feature set of the images. After that, to express the image information more specifically, codebooks for the two feature sets are created separately by using k-means clustering [22] to improve the robustness of the results. Finally, to reduce the loss of image information while improving the accuracy of the calculation, the LLC (locality-constrained linear coding) [23] is used to encode each image feature and further merged as the final image feature vector. In summary, an improved SPM model based on integrated multi-features and locality-constrained linear coding (IMFLLC) for rail corrugation identification is proposed. This method is expected to achieve a better identification result.

In addition, as an important visual property of rail corrugation, period plays a vital role in further corrugation analyzing. Inspired by References [24,25], a corrugation period estimation method in combination with image Fourier analysis and normalized distance matching function [25] (FFT-NDMF) is presented to achieve the automatic estimation of the corrugation period along the rail direction. It is useful to analyze these factors which result in the corrugation of the rail. To our best knowledge, this is the first paper handling the rail corrugation period estimation problem based on computer vision.

Furthermore, valid and reliable corrugation severity assessment is helpful to make maintenance strategies for a lower maintenance cost and to extend the service life of the rails. Therefore, given the successful application of visual saliency in target detection and content analysis [26], a novel method for rail corrugation severity assessment based on synthesizing the visual saliency and fuzzy theory is proposed in this paper.

The remainder of this paper is organized as follows. In Section 2, the problem solved in this paper is stated. In Section 3, the proposed rail corrugation identification method is presented in detail. In Section 4, the steps of the rail corrugation period estimation using the FFT-NDMF algorithm is investigated. After that, the method of the rail corrugation severity assessment is provided in Section 5. Finally, some conclusions of this paper are given in Section 6.

2. Problem Statement

For quite some time, the occurrence and evolution of rail corrugation have seriously affected the development of urban rail transit. The rail corrugation not only causes strong vibration of the vehicle-track structure but also strengthens the wheel-rail rolling noise, which reduces the passenger comfort and increases the track maintenance cost. Therefore, it is critically important to identify and assess rail corrugation automatically and intelligently. The corrugation is a periodic irregular wear phenomenon on the rail surface and a rail image contains corrugation is shown as in Figure 1, which is collected from the track line between the Nanluoguxiang Station and the Dongsi Station of the Beijing Metro Line 6.

In this paper, for real application, three issues will be investigated. First, the identification method of rail corrugation. Second, the method of estimating the rail corrugation period. Finally, the severity assessment method of rail corrugation. The ideas how to solve these problems are introduced in the following.

Rail corrugation identification based on improved SPM model. First of all, to reduce the impact of noise and uneven illumination, a two-stage pre-processing method is considered to improve the original image quality. After that, rail head surface positioning is achieved by enhancing the rail its edge followed by weighted gradient projection analysis. Finally, according to the local texture and gradient features of the corrugation, an improved SPM model is proposed to identify rail corrugation.

Estimation of rail corrugation period. To estimate the rail corrugation period more efficiently, the horizontal gray gradient statistical analysis and mathematic morphology algorithm are first used to localize the corrugation interval. After that, the proposed method by integrating the fast Fourier transform and normalized distance matching function is applied to estimate the rail corrugation period.

The severity assessment of the rail corrugation. Based on the visual significance analysis for the rail corrugation image, the concept of wear saliency is introduced to assess the corrugation severity. In detail, the normalized frequency-tuned model is used to calculate the corrugation saliency map. Subsequently, each corrugation area is localized by employing the adaptive threshold and mathematic morphology algorithm. In the end, after calculating the wear saliency, each corrugation area severity is determined with the help of fuzzy theory.

3. Rail Corrugation Identification Based on Improved SPM Model

In this section, the whole frame and relevant details of the rail corrugation identification method proposed in this paper are presented.

3.1. Image Pre-Processing

Owing to the complex operating environment of the metro railway system, the rail images collected from the field are susceptible to noise and uneven illumination, which would degrade the image quality, affect the ultimate rail corrugation identification precision and impact the period estimation results of the rail surface images. To alleviate this problem and improve the image quality, a two-stage pre-processing method by Bilateral Filter (BF) [27] and Rolling Guidance Filter [28] with Parameter Adaptive Gamma Function [29] (RGF-PAGF) is considered, which are reviewed in the following.

3.1.1. Image De-Noising

The first stage of pre-processing is noise reduction, which is implemented by using image filters. The bilateral filter [27] is a non-linear filter that can achieve noise reduction and edge-preserving simultaneously. In addition, in comparison with other conventional filter-based methods, it can commendably preserve the rail surface texture characteristics. Hence, it is suitable for image de-noising. The bilateral filter is developed from the traditional Gaussian filter, which can be described as follows

I_{o u t} (x) = \frac{1}{W_{p}} \sum_{x_{i} \in Ω} I (x_{i}) f_{r} (∥ I (x_{i}) - I (x) ∥) g_{s} (∥ x_{i} - x ∥)

(1)

where I and

I_{o u t}

represent the original and filtered image, respectively. x is the coordinates of the pixels to be processed while

Ω

is the neighborhood window.

f_{r}

and

g_{s}

denote the distance and intensity filter, respectively.

W_{p}

stands for the normalization function. More details about BF can be found in Reference [27].

3.1.2. Uneven Illumination Correction

According to Retinex theory [30], a visible image consists of two components: illumination and reflectance. That is, the visible image

I (x, y)

can be described as the product of illumination component and reflectance component as follows [30]

I (x, y) = L (x, y) • R (x, y)

(2)

where

(x, y)

denote the pixel coordinates.

L (x, y)

and

R (x, y)

represent the illumination and reflectance, respectively. In general, the illumination components exist in the low-frequency part of the image and varies smoothly. Therefore, to correct the uneven illumination phenomenon, it is crucial to extract the illumination component efficiently from the observed image.

There are several filter based methods for extracting illumination component [31,32,33]. In this paper, given the outstanding edge-preserving capabilities and the low computation complexity, the rolling guidance filter (RGF) [28] is used to extract the illumination. It is a new type of edge-preserving filter based on a scale structure, which can be formulated as follows

J^{t + 1} (p) = \frac{1}{K_{p}} \sum_{q \in N (p)} exp (- \frac{(∥ p - {q ∥)}^{2}}{2 σ_{s}^{2}} - \frac{(∥ J^{t} (p) - J^{t} (q) {∥)}^{2}}{2 σ_{r}^{2}}) I (q)

(3)

where

J^{t + 1}

denotes the

t + 1

iteration result which is the extracted illumination component of the input image I (also the

J^{1}

K_{p}

stands for the normalization function. Other parameters are the same as the one mentioned above.

After that, an uneven illumination correction function is constructed. In the light of the parameter adaptive gamma function (PAGF) mentioned in References [29,34], the constructed correction function is given by

O (x, y) = I {(x, y)}^{γ}, γ = 2^{\frac{m - L (x, y)}{m}}

(4)

where

O (x, y)

stands for the enhanced intensity value.

γ

denotes the exponential value including the dynamic local characteristic of the intensity enhancement. m represents the mean value of the intensity for illumination component. The whole procedure of the two-stage pre-processing method is shown as in Figure 2.

3.2. Rail Head Surface Localization

The rail images collected from the field usually contain irrelevant components such as fasteners, sleepers and subgrade on both sides of the track, which would cause unnecessary interference. Therefore, the exact rail head surface area should be first extracted to reduce this interference. It is observed that the gray level of the rail area is mainly continuous along the longitudinal direction of the rail, while in the other non-rail regions, the gray scale of the pixels has apparent discontinuity both on the horizontal and vertical direction. And, along with the transverse direction, there are distinct gray scale discontinuities between the rail area and the other regions. Furthermore, the rail is usually located in the middle of the obtained rail image.

Based on these observations, the vertical edge of the rail image is enhanced by using horizontal gradient operator to highlight gray level change along the horizontal direction and sharpen the boundaries of the rail head surface. From Reference [35], the operator templates used in this paper are stated as follows

p = [\begin{matrix} - 1 & 2 & - 1 \\ - 2 & 4 & - 2 \\ - 1 & 2 & - 1 \end{matrix}], q = [\begin{matrix} - 1 & 0 & 1 \\ - 2 & 0 & 2 \\ - 1 & 0 & 1 \end{matrix}]

(5)

hence, the enhanced rail image

E (x, y)

is computed as

E (x, y) = | p * O (x, y) | + | q * O (x, y) |

(6)

where

(x, y)

denote the pixel coordinates.

O (x, y)

is the corrected rail image. * represents the convolution operation.

The enhanced rail boundaries indicate significant gradient information along the transversal direction, as shown in Figure 3b. Therefore, the horizontal gradient for each pixel of the enhanced rail image is first computed and then the sum of gradient value (SG) for each column of the rail image is calculated, the SG is defined as

S G (y) = \sum_{x = 1}^{h} g r a d (x, y), y = 1, 2, \dots, w .

(7)

where w and h stand for the width and height of the rail image, respectively.

g r a d (x, y)

represent the gradient value at the coordinates

(x, y)

Subsequently, in terms of the horizontal position of each column in the rail image, the SG is further multiplied with an adaptive weight. The adaptive weight

A W (y)

of position y is defined as

A W (y) = \frac{1}{\sqrt{2 π} σ} e^{- \frac{{(y - μ)}^{2}}{2 σ^{2}}}, y = 1, 2, \dots, w .

(8)

where

μ

is set with

w / 2

in this study. Therefore, the weighted gradient SUM value is computed as

W G (y) = S G (y) \times A W (y), y = 1, 2, \dots, w .

(9)

Finally, a weighted gradient projection profile for rail locating is formed by using all of these values. In the light of practical experience, when the setup of image acquisition equipment is fixed, the rail width

w_{r}

in a rail image will be a constant. Therefore, the starting position of the rail is obtained based on the following rules [15]

p_{r} = arg max_{x} S W G (y), y = 1, 2, \dots, w - w_{r} .

(10)

with the sum of weighted gradient value

S W G (y) = \sum_{i = y}^{i + w_{r} - 1} W G (i)

(11)

where y stands for the starting position with the length of

w_{r}

and then the exact rail location would be determined by them. The procedure of rail surface localization is shown as in Figure 3. Brief pseudocode about the method for rail surface localization is described in Appendix A.

3.3. Improved Spatial Pyramid Matching (SPM) Model

The traditional SPM (Spatial Pyramid Matching) model [17] has developed from the BOF (Bag of Features) method [16]. To overcome the inherent demerits of BOF, SPM introduces the spatial location information between local feature points by calculating the distribution of image feature points at a different resolution [17], as shown in Figure 4. That is, the image is divided into progressively refiner spatial sub-regions and then local feature histograms are computed from each region. The diagram of SPM is shown in Figure 5.

Although SPM exhibits excellent performance in many image recognition and classification tasks, it only extracts a single feature of an image leading to the obtained feature information is insufficient to represent the rail corrugation image adequately. Moreover, the vector quantization (VQ) encoding method used in the SPM has a large reconstruction error and will result in low image classification accuracy. Therefore, taking the characteristics of rail corrugation image into account, an improved SPM model based on integrated multi-features and locality-constrained linear coding (IMFLLC) for rail corrugation identification is proposed, which is reviewed briefly as follows.

3.3.1. Multi-Feature Extraction

In comparison with normal rails without corrugation, typical corrugation rails exhibit distinctive textural features. A widely-used method for texture feature extraction is the local binary pattern (LBP). To extract the textural characteristics of rail image more effectually, the rotation invariant Uniform LBP [20] that the number of patterns decreased from 256 to 59 is used in this paper. It significantly compresses the dimension of the eigenvector and reduces the effect of high-frequency noise, allowing for better representation of image information under a variety of complex conditions.

Another local feature descriptor with excellent performance is SIFT, which has been widely applied since was first reported [36]. In this paper, the dense_SIFT proposed by Liu et al. [21] is applied. It takes a fixed-size mask and extracts the dense_SIFT patches from left to right and top to bottom on the image with a certain step size and then extracts 128-dimensional SIFT descriptors from each patch as the local feature descriptors. Compared to the traditional SIFT, dense_SIFT reduces the computation complexity and extracts dense_SIFT features that can be better used for linear classification.

In view of the well-characterized ability and complementarity of the two local feature descriptors mentioned above, the image features are extracted by using Uniform LBP and dense_SIFT, respectively, which can reflect the different properties of the image data effectively. After that, these two features are used in combination to train the classifier. This can take full advantages of the two methods, therefore, a better classifier performance can be obtained.

3.3.2. Vector Quantization of Image Features

The low-level features extracted above may contain some redundancy and noise. In this paper, to improve the robustness and distinctiveness of feature representation, an efficient feature vector quantization method is needed to encode the low-level features. In general, the vector quantization method of image features can be divided into two main steps [18]. First of all, the visual codebook is obtained by clustering the descriptors of the image training set and then the features extracted from the image are encoded according to the obtained visual codebook. Different visual codebook or different feature encoding methods will lead to different performance during the quantization procedure.

In general, there are several clustering algorithms for obtaining sparse or dense visual codebook from the image training set, of which the most widely used is the K-means clustering algorithm. It is fast, straightforward and still can maintain high efficiency and scalability when the data volume is large [22]. Therefore, the most classical partition-based K-means clustering method is first used to generate different visual codebooks for the two types of features extracted from the image training set, which can be used to comprehensively express different types of image information, therefore, improving the quality and robustness of the results.

Furthermore, to effectively organize the feature vectors and reduce the reconstruction error of image features, the LLC (locality-constrained linear coding) is used to encode local image feature descriptor. In comparison with VQ used in traditional SPM, LLC encodes and reconstructs low-level features on a local manifold by adding locality constraints [23], as shown in Figure 6. In addition, the output image feature representation based on LLC is nonlinear and satisfactory results can be obtained by using a linear classifier.

3.4. Rail Corrugation Identification Based on IMFLLC

In this paper, to achieve better identification accuracy, an improved SPM model for rail corrugation identification is proposed, which integrates multi-features and locality-constrained linear coding, named IMFLLC. The diagram of IMFLLC is shown as in Figure 7.

More specifically, the Uniform LBP and dense_SIFT of all training rail images are first extracted to construct two corresponding feature sets. Secondly, the K-means clustering algorithm is used to cluster the two feature sets of the training image, respectively. As a result, two visual codebooks are obtained which can represent different aspects of the image information. After that, based on the two obtained visual codebooks, LLC is used to encode each image, followed by SPM pooling to receive two different image representations of each image. Then, different image representations are integrated to form the final image representation of each image. Finally, the final image representations with artificial labels are used to train a linear SVM classifier [8].

For new or test rail images, the Uniform LBP and dense_SIFT feature sets are first extracted. Subsequently, based on the corresponding two different visual codebooks, each image feature is encoded by using LLC and the final feature representation of each image is obtained after SPM pooling and feature integration. Finally, the linear SVM classifier obtained from the training procedure is used to test the image and get the final class label to judge whether it is a corrugation rail image or not.

3.5. Identification Experiments and Results

3.5.1. Introduction to Experiment Dataset

The dataset used in this paper consists of images taken from Beijing Metro Line 6. These images are captured by the handheld DSLR camera. Additionally, the collected images are characterized in that the track is located in the middle of the image and each image is an RGB image with a resolution of 3456 × 4608 pixels. The dataset contains 200 typical corrugation rail images as positive samples and 200 non-corrugation images as negative samples. In the experiment, 70% of the images of each class are used as the training images and the rest is used as the test images.

3.5.2. Performance Evaluation for Corrugation Identification

Corrugation identification is one of the critical parts of this work and this subsection will evaluate the performance of the proposed method for corrugation identification. The proposed identification method is compared with three baselines including Gabor+SVM [13], Accumulate Energy Thresholding (AET) [14] and Maximum Energy (ME)+SVM [15]. More precisely, the global 8-dimensional Gabor filtering features and SVM classifier are adopted in Gabor+SVM. AET uses the local dominant frequency features and fixed thresholding. And the local dominant frequency features and SVM classifier are applied in ME+SVM. In addition, all parameters are consistent with the original literature and 10-fold cross validation method for SVM training and prediction is used.

In this subsection, to assess the proposed method quantitatively, indicators such as precision (P), recall (R) and accuracy (ACC) are used, which are widely adopted in many research fields [37]. The formula for calculating these indicators as follows:

P = \frac{T P}{T P + F P}

(12)

R = \frac{T P}{T P + F N}

(13)

A C C = \frac{T P + T N}{T P + F P + T N + F N}

(14)

where

T P

represents the number of corrugation images that are correctly detected.

F N

indicates the number of corrugation images for missed inspection.

F P

is the number of images that are falsely reported as corrugation.

T N

stands for the number of non-corrugation images that are correctly excluded.

Table 1 shows the comparison result between the traditional methods and the proposed method. Note that the size of the two visual codebooks is adaptively set to

p a t c h N u m_s i f t * 0.7

and

p a t c h N u m_l b p * 0.7

, respectively (

p a t c h N u m_s i f t

is the number of patches extracted according to dense_SIFT and

p a t c h N u m_l b p

refers to the number of patches extracted according to Uniform LBP). It can be seen that the algorithm proposed in this paper achieves a sound performance, especially the recall rate is significantly improved. The recall rate is the most critical indicator among the three indicators and the high recall rate means that the number of undetected corrugation images is small. In summary, in the light of the most distinguishable texture features and spatial structure relationship of the corrugation, the multi-features extracted in this paper can represent the corrugation image more comprehensively than those mentioned in the baselines. Moreover, the results demonstrate that the LLC method can facilitate precise and efficient encoding of the corrugation image features, which benefits from the degradation of image feature reconstruction error and the effective organization of image feature vectors.

3.5.3. Analysis of Computational Complexity

Computational complexity is considered to be another significant indicator of the corrugation identification algorithms. Therefore, in this subsection, the computational time complexity of the proposed IMFLLC algorithm is theoretically analyzed at first and then the practical identification speed is given.

In general, computational complexity consists of space complexity and time complexity. The reason why only time complexity is considered here is that the time source is more important than memory space for the corrugation identification algorithms. The time complexity of an algorithm is a function that qualitatively describes the running time of the algorithm. In addition, the complexity function is usually represented by

O (.)

notation, which excludes the low-order terms and the first coefficient of this function [38].

At first, the computational time complexity function of IMFLLC is analyzed. As discussed in Section 3.4, IMFLLC-based corrugation identification involves three key steps: (1) rail image multi-feature extraction; (2) vector quantization of extracted features; and (3) rail corrugation identification. In the first step, the Uniform LBP and dense_SIFT feature of the input rail image are extracted, respectively. In terms of the feature extraction principle of Uniform LBP [20] and dense_SIFT [21], the time complexity of the first step is approximately equal to

O (N)

, where N is the pixel number of the input rail image. Secondly, by using LLC method, the computational complexity of vector quantization procedure is

O (M + K^{2})

[23], in which M denotes the codebook size and K stands for the nearest neighbors number. In the last step, based on the linear SVM classifier obtained from the training procedure, the input rail image is judged whether it is a corrugation rail image or not. This procedure using simple linear SVM obtains a constant complexity, so the computational time complexity of rail corrugation identification is

O (1)

. As a whole, the time complexity of IMFLLC is

(O (N) + O (M + K^{2}) + O (1))

After that, the actual time of a corrugation rail image identification is tested. This experiment was carried on a desktop with Intel(R) Core(TM)2 Duo CPU (2.93 GHz) and 4-GB memory. The proposed IMFLLC algorithm is implemented with MATLAB. Furthermore, the program is performed on the test images and the mean of the run time is recorded. The final average identification time of the corrugation rail image is 0.27 s, which indicates that an acceptable corrugation identification speed based on the IMFLLC can be achieved.

4. Estimation of Rail Corrugation Period

The corrugation is a type of wave-like rail surface defect that appears as a periodic texture pattern in the rail surface image. In general, the wavelength of corrugation is used to further analysis corrugation and mathematically, the wavelength is proportional to the period. Hence, it is meaningful to estimate the period of the corrugation. However, the corrugation rail image obtained from the field includes not only the corrugation area but also other regions (such as edge rust and normal surface area), as shown in Figure 8a. Therefore, the corrugation interval is first localized and then the corrugation period is estimated based on the combination of one-dimensional fast Fourier transform [39] and normalized distance matching function (FFT-NDMF) in the corrugation interval.

4.1. Corrugation Interval Localization

Due to the complex operation environment of the vehicle and the wheel-rail interaction, there is rust on the edges of the rail surface. Besides, it is observed that there is a significant gray scale difference between the rusted edge and the remaining rail surface area along the horizontal direction. Therefore, the rusted edge is first cropped from rail surface image based on the horizontal gray gradient statistical (briefly as HGGS) analysis. The starting and ending coordinates of the remaining rail surface area are computed as follows

m g r a d (x_{i}, y) = \frac{1}{h} \sum_{y = 1}^{h} [I (x_{i + 1}, y) - I (x_{i - 1}, y)], i = 2, 3, \dots, w - 1 .

(15)

R_{l} = min_{i} [p e a k s (m g r a d_{2 \leq i \leq w / 2} (x_{i}, y))]

(16)

R_{r} = max_{i} [p e a k s (m g r a d_{w / 2 \leq i \leq w - 1} (x_{i}, y))]

(17)

where

m g r a d (x_{i}, y)

denotes the average horizontal gradient value of the i-th column of the image.

I (x_{i}, y)

is the gray value of the i-th column of the image. w and h stand for the width and height of the rail image, respectively.

p e a k s

represents the calculation of the extreme values of

m g r a d

R_{l}

and

R_{r}

are the starting and ending coordinates of the remaining rail surface area, respectively. The result of the horizontal gray gradient statistical analysis are shown as in Figure 8b. Brief pseudocode for HGGS is shown in Appendix B.

After eliminating the interference of rust edge, the Otsu algorithm [40] is used to achieve the segmentation of corrugation area and the background area, resulting in a binary image B. After that, the mathematic morphological reconstruction and opening operators are used to remove small, bright pixels. Finally, the corrugation interval is localized based on the foreground pixel statistical analysis of each column of the image. The starting and ending coordinates of the corrugation interval are computed as follows

C_{l} = {arg}_{i} min [S B_{i} | S B_{i} > δ], i = 1, 2, \dots, m .

(18)

C_{r} = {arg}_{i} max [S B_{i} | S B_{i} > δ], i = 1, 2, \dots, m .

(19)

where

S B_{i}

denotes the sum binary value of the i-th column of the obtained binary image. m represents the width of the remaining rail surface area.

C_{l}

and

C_{r}

are the starting and ending coordinates, respectively.

δ

stands for the threshold of the sum binary value. The result of the corrugation interval localization is shown as in Figure 8a.

4.2. Corrugation Period Estimation Based on FFT-NDMF

After the corrugation interval is localized, Fourier transformation analysis is performed on each column of the corrugation interval image. Subsequently, the simplified NDMF is used to obtain the period of each column. Finally, the results of the entire interval are statistically analyzed to determine the period of the corrugation. The diagram of FFT-NDMF is shown as in Figure 9.

Specifically, let L be a set of H-dimensional image columns obtained from the corrugation interval, that is,

L = [l_{1}, l_{2}, \dots, l_{N}] \in ℜ^{H \times N}

where

l_{i}

represents a column of the corrugation interval image at position i, H denotes the height of the corrugation interval image and N is the width of the corrugation interval. Firstly, one-dimensional fast Fourier transformation is conducted to each column of interval image, that is

f_{m} = \sum_{k = 0}^{H - 1} l_{i k} e^{\frac{- j 2 π m k}{H}}, 1 \leq i \leq N

(20)

where

l_{i k}

denotes the gray value of the image column

l_{i}

at position k.

f_{m}

for

0 \leq m \leq H - 1

represents the Fourier coefficients with frequency m. As a result, an amplitude-frequency curve for each column is obtained. In this section, to better introduce subsequent steps, the amplitude-frequency curve of some interval image columns randomly selected in corrugation interval is shown as in Figure 10.

Each curve has an amplitude peak at the same frequency, as observed in Figure 10. In most cases, the peak is the maximum but in a few cases, the peak is the second maximum due to the interference of the sub-period or the multiple of the main period. Therefore, when locating the peak value of each amplitude-frequency curve, the frequency corresponding to the first two maximum peaks are extracted simultaneously and then the corresponding period is calculated as the possible period of each image column. On this basis, NDMF is calculated to determine the final period of each image column.

The traditional distance matching function (DMF) proposed by Oh et al. [24] is the most used method for period estimation, which is based on the gray level co-occurrence matrix (GLCM) inertial characteristics [24]. In practice, to improve its robustness to the data differences, enhance the stability of the algorithm and reduce the amount of computation, the normalized distance matching function (NDMF) is often used [25]. To meet the period estimation demand of the corrugation interval image, a simplified NDMF of a one-dimensional image function

l_{j}

with h pixels in corrugation interval is defined as

N D M F (d) = \frac{1}{H - d} \sum_{i = 1}^{H - d} | l_{j} (i) - l_{j} (i + d) |

(21)

where H represents the height of the corrugation interval image.

l_{j}

stands for a column of the corrugation interval image at position j.

First of all, let

d_{i 1}

and

d_{i 2}

be the potential periods corresponding to the maximum and the second maximum peak value of the amplitude-frequency curve of image column

l_{i}

, respectively. Then, the NDMF values of

d_{i 1}

and

d_{i 2}

are calculated to determine the exact period, which is given by

N D M F_{i 1} (d_{i 1}) = \frac{1}{H - d_{i 1}} \sum_{j = 1}^{H - d_{i 1}} | l_{i j} - l_{i (j + d_{i 1})} |

(22)

N D M F_{i 2} (d_{i 2}) = \frac{1}{H - d_{i 2}} \sum_{j = 1}^{H - d_{i 2}} | l_{i j} - l_{i (j + d_{i 2})} |

(23)

where H denotes the height of the corrugation interval image. Given the definition of NDMF, the smaller the NDMF value calculated by Equation (21) is, the greater the probability that it is the real period is. Hence, the final period of the image column

l_{i}

p_{i} = min (N D M F_{i 1}, N D M F_{i 2})

Finally, the period value of each image column is statistically analyzed in the corrugation interval and the mode of all period values is calculated as the final period of the corrugation rail image, that is

P_{C} = m o d (P)

(24)

where

P = [P_{1}, P_{2}, \dots, P_{N}]

denotes the period value set of the corrugation interval. N is the width of the corrugation interval.

4.3. Performance Evaluation for Corrugation Period Estimation

In this subsection, to verify the validity and accuracy of the proposed method in estimating rail corrugation period, experiments are performed on images selected from the corrugation image dataset mentioned above. Figure 11 shows the visual period estimation results of some samples, with the red boxes indicating the period estimated by the method in this paper. The results show that the proposed method of period estimation can be applied to rail corrugation images, which is consistent with the artificial visual result.

In addition, Table 2 presents the corresponding concrete results of the corrugation periodicity in Figure 11, note that the height of the images used in this experiment is 512 pixels. The comparison between the estimated period and the manual measurement shown in Table 2 demonstrate that the estimation method proposed in this paper can estimate the rail corrugation period effectively.

5. The Severity Assessment of the Corrugation

The evolution of the rail corrugation not only affects the security state of the rail line but also increases the maintenance costs. Therefore, it is critically important to assess the severity of corrugation to provide auxiliary maintenance strategy. Generally, the severity of the corrugation is related to the wave depth of corrugation, while the depth variation of the wear is reflected in the change of the gray level of the image [41], that is, the difference in grayscale of the corrugation image can be used to assess the severity of the corrugation. In this paper, it is considered as the issue of visual saliency calculation. For a specific wear state, the concept of wear saliency (WS) is introduced to assess the severity of corrugation, in which

0 < W S < 1

. When the wear saliency is close to 0, it means that there is no corrugation. Correspondingly, when the wear saliency is close to 1, it means that the corrugation is severer. Concretely, the rail corrugation is classified into three levels of wear severity including level 1 (slight wear), level 2 (medium wear) and level 3 (severe wear).

5.1. Visual Significance Analysis for Corrugation Image

At present, visual saliency has been most applied in many fields such as target detection and tracking, image retrieval and recognition, scene analysis and so forth [26]. Saliency identifies a particular area of a scene that has unique properties and can attract more visual attention and the area is considered to be visually significant. For humans, the most eye-catching is often the goal with different colors, because the human visual system (HSV) is most sensitive to color. In other words, the targets which are different from the surrounding colors will most likely attract attention. Moreover, as can be seen in Figure 1, the color of the defect area and the background area are significantly different in the corrugation image. There are also significant differences between the corrugation defect areas of varying severity. Hence, compared with other low-level visual features, the color characteristics of the defect area of the corrugation image are more significant.

5.2. Visual Saliency Feature Extraction and Saliency Map Generation

Nowadays, there are several ways to extract significant visual features and further generate saliency maps, mainly including spatial-based and frequency-based methods. Among them, the frequency domain-based saliency detection method is increasingly investigated owing to its simple implementation and high real-time performance. In this paper, in terms of the characteristic of corrugation images, the significant visual features are extracted by the frequency-tuned (FT) model [42] and the full resolution saliency map is further obtained. In the first step, the input RGB image is converted into the CIELab color space closer to human vision. After that, the arithmetic mean of the image features is calculated and Gaussian smoothing is performed on the image. Finally, the difference of each pixel is calculated and the saliency map is obtained after the value is normalized. The saliency map S for a corrugation image I can be written as

S (x, y) = N (∥ L_{G} (x, y) - L_{m} ∥)

(25)

where

(x, y)

denote the pixel coordinates.

L_{m}

stands for the average image feature vector in CIELab color space.

L_{G} (x, y)

is the relevant image pixel vector value in the intermediate image blurred by a 5 × 5 Gaussian kernel.

∥ \cdot ∥

is the

L_{2}

norm.

N (\cdot)

represents the min-max normalization operation.

5.3. Severity Assessment Based on Fuzzy Theory

After the saliency map is obtained, the rail corrugation severity is assessed based on fuzzy theory [43]. Fuzzy theory is an effective tool to deal with uncertain information. Hence, it is suitable for solving the problem of corrugation severity assessment with ambiguity. The whole procedure of corrugation severity assessment is shown as in Figure 12.

5.3.1. Image Binarization and Corrugation Area Localization

In general, image binarization can reduce the amount of computation and can simplify the localizing process of each corrugation area. Therefore, the adaptive threshold algorithm, mentioned in Reference [42], is used to get the binarized image of the corrugation saliency map, in which the adaptive threshold

T_{a}

is computed as follows

T_{a} = \frac{2}{w \times h} \sum_{x = 0}^{w - 1} \sum_{y = 0}^{h - 1} S (x, y)

(26)

where w and h stand for the width and height of the corrugation saliency map, respectively.

S (x, y)

denote the saliency value at the coordinates

(x, y)

After that, with the intention of localizing each corrugation area more accurately, the mathematical morphology algorithm is used. Firstly, the morphological closing operation is used to merge smaller regions without changing the overall region, that is

S_{m} = (S_{b} \oplus b) Θ b

(27)

where

S_{b}

denotes the binarized saliency map. b is the structure element used in this paper. ⊕ and

Θ

represent dilation operation and erosion operation, respectively. Subsequently, the label matrix method is used to decide the position coordinates of each corrugation area, which is given by

S_{c_{i}} = S_{m} (L_{s} = = i), i = 1, 2, \dots, k

(28)

where

S_{c_{i}}

represents the i-th corrugation area.

L_{S}

stands for the label matrix of

S_{m}

. k is the number of corrugation area. The visual results of the used mathematical morphology algorithm are shown in Figure 13, where the scope of each corrugation area is accurately located without interference.

5.3.2. Corrugation Severity Assessment

First of all, taking the relationship between visual saliency and wear state into account comprehensively, the concept of wear saliency (WS) is introduced to assess the severity of corrugation, which is defined as

W S_{c_{i}} = m e a n (S_{c_{i}}), i = 1, 2, \dots, k

(29)

where

W S_{c_{i}}

is the wear saliency of the i-th corrugation area.

After that, three level membership functions are designed for the slight wear, medium wear and severe wear, respectively, which are shown as in Figure 14.

Finally, the maximum subordination principle is used to determine the severity level of each corrugation area, which is defined as

max (μ_{s l i g h t}, μ_{m e d i u m}, μ_{s e v e r e}) = \{\begin{matrix} μ_{s l i g h t} & l e v e l 1 \\ μ_{m e d i u m} & l e v e l 2 \\ μ_{s e v e r e} & l e v e l 3 \end{matrix}

(30)

5.4. Case Study for Rail Corrugation Severity Assessment

In this subsection, the visualization process and feasibility of the proposed corrugation severity assessment method will be demonstrated by a case study. The image used in this case, with a slight corrugation, is collected from the small radius curve between the Nanluoguxiang Station and the Dongsi Station of the Beijing Metro Line 6.

In the first step, the horizontal gray gradient statistical (HGGS) analysis method is applied to crop the rusted edge of the corrugation image (see (b) in Figure 15). Then, the rustless corrugation image is converted to the CIELab color space and the saliency map of it is obtained by using the normalized frequency-tuned (NFT) model (see (c) in Figure 15). Next, the adaptive threshold method is used to get the binarized corrugation saliency map (see (d) in Figure 15) and then, the morphological closing operation and label matrix method are adopted to localize each corrugation area in it. After that, the wear saliency (WS) of each corrugation area is calculated according to Equation (29) (see (e) in Figure 15). Finally, the severity level of each corrugation is determined in terms of the level membership function and maximum subordination principle defined in Equation (30) (see (f) in Figure 15). The visualization process and results are shown as in Figure 15.

In summary, according to the site actual conditions, the visualization process and severity assessment results of this case demonstrate that the proposed method is a feasible approach for assessing the rail corrugation severity.

6. Conclusions

In this paper, the rail corrugation identification and assessment issues are concerned. Firstly, the improved SPM model for rail corrugation identification is introduced, in which the multi-features (including Uniform LBP and dense_SIFT) and locality-constrained linear coding are used. The quantitative experimental results indicate that the proposed identification method achieves markedly better performance. Specifically, the precision and recall of the proposed corrugation identification method are 99.67% and 98.34%. After that, to further analysis the corrugation, an innovative estimation method for rail corrugation period based on one-dimensional fast Fourier transform and normalized distance matching function is proposed. The experimental results demonstrate that the proposed method can be applied to estimate the rail corrugation period efficiently. Finally, the wear saliency (WS) based on the visual significance analysis of the corrugation image is calculated to assess the rail corrugation severity. To the best knowledge of the authors, this method is the first one that introduces the concept of wear saliency (WS) to assess the severity of the rail corrugation in terms of the human visual system.

In the future, our research work will mainly focus on the following aspects. On the one hand, the investigation and construction of an intelligent visual inspection system for rail corrugation, including an image acquisition subsystem installed at the bottom of the operating vehicle and a background corrugation identification and analysis system. The methods proposed in this paper will be further validated and the relevant results will be published. On the other hand, other different types of signals (e.g., vibration acceleration and sound) will be carefully investigated and combined in the future researches to assess the severity of the rail corrugation more accurately.

Author Contributions

D.W. collected and analyzed the data, made charts and diagrams, conceived and performed the experiments and wrote the paper; X.W. conceived the structure and provided guidance; Y.L. collected and processed the data; L.J. modified the manuscript; W.Z. arranged the rail image acquisition time and line.

Funding

This work is partly supported by Chinese National Key Project of Research and Development (Contract No. 2016YFB1200402).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

The rail surface localization algorithm used in this paper is shown in Algorithm A1, of which the detailed procedure is discussed in Section 3.2.

Algorithm A1 The Algorithm for rail surface localization.

Input: The enhanced rail image

e n I m g

; The rail width

w_{r}

;
Output: The starting position of the rail

p_{r}

;

1:: procedureAlgorithm 1( $e n I m g, w_{r}$ )
2:: $h \leftarrow s i z e (e n I m g, 1)$
3:: $w \leftarrow s i z e (e n I m g, 2)$
4:: for $i \leftarrow 2, w - 1$ do
5:: $A W (i) \leftarrow a d a p t i v e w e i g h t o f i - t h c o l u m n$
6:: $G (i) \leftarrow e n I m a g e (\cdot, i + 1) - e n I m a g e (\cdot, i)$
7:: $S G (i) \leftarrow s u m o f G (i)$
8:: $W G (i) \leftarrow S G (i) * A W (i)$
9:: end for
10:: $m a x V a l u e \leftarrow 0$
11:: $p_{r} \leftarrow 1$
12:: for $i \leftarrow 1, w - w_{r} + 1$ do
13:: $i n t e r V a l \leftarrow W G (i : i + w_{r} - 1)$
14:: $s I n t e r v a l \leftarrow s u m o f i n t e r V a l$
15:: if $s I n t e r V a l > m a x V a l u e$ then
16:: $m a x V a l u e \leftarrow s I n t e r V a l$
17:: $p_{r} \leftarrow i$
18:: end if
19:: end for
20:: return $p_{r}$
21:: end procedure

Appendix B

The HGGS algorithm proposed in this paper is used to crop the rusted edges from the rail surface image, of which the procedure is discussed in Section 4.1. For a better understanding, the HGGS algorithm is described in Algorithm A2.

Algorithm A2 HGGS.

Input: The rail surface image

s u r I m g

;
Output: The starting coordinate of the remaining rail surface area

R_{l}

; The ending coordinate of the remaining rail surface area

R_{r}

;

1:: procedureAlgorithm 2( $s u r I m g$ )
2:: $r o w \leftarrow s i z e (s u r I m g, 1)$
3:: $c o l \leftarrow s i z e (s u r I m g, 2)$
4:: for $i \leftarrow 2, c o l$ do
5:: $G r a d (\cdot, i - 1) \leftarrow s u r I m g (\cdot, i) - s u r I m g (\cdot, i - 1)$
6:: end for
7:: $m G r a d \leftarrow m e a n o f G r a d$
8:: $p k s 1, l o c s 1 \leftarrow f i n d p e a k s (- m G r a d)$ // get the minimum peak
9:: $p k s 2, l o c s 2 \leftarrow f i n d p e a k s (m G r a d)$ // get the maximum peak
10:: $m i n V \leftarrow m i n [p k s 1^{T}, l o c s 1^{T}], m a x V \leftarrow m a x [p k s 2^{T}, l o c s 2^{T}]$
11:: $s m i n V \leftarrow s i z e (m i n V, 1), s m a x V \leftarrow s i z e (m a x V, 1)$
12:: $R_{l} \leftarrow 1, R_{r} \leftarrow c o l$
13:: $m i n I \leftarrow 1, m a x I \leftarrow 1$
14:: while $m i n I < s m i n V$ do
15:: if $s m i n V (m i n I, 2) < c o l / 2$ then
16:: $R_{l} \leftarrow s m i n V (m i n I, 2)$
17:: $break$
18:: end if
19:: $m i n I \leftarrow m i n I + 1$
20:: end while
21:: while $m a x I < s m a x V$ do
22:: if $s m a x V (m a x I, 2) > c o l / 2$ then
23:: $R_{r} \leftarrow s m a x V (m a x I, 2)$
24:: $break$
25:: end if
26:: $m a x I \leftarrow m a x I + 1$
27:: end while
28:: return $R_{l}, R_{r}$
29:: end procedure

References

Oostermeijer, K.H. Review on short pitch rail corrugation studies. Wear 2008, 265, 1231–1237. [Google Scholar] [CrossRef]
Jin, X.; Xia, L.I.; Wei, L.I.; Wen, Z. Review of Rail Corrugation Progress. J. Southwest Jiaotong Univ. 2016, 51, 264–273. [Google Scholar]
Zhang, H.; Liu, W.; Liu, W.; Wu, Z. Study on the cause and treatment of rail corrugation for Beijing metro. Wear 2014, 317, 120–128. [Google Scholar] [CrossRef]
Nicholson, G.L.; Davis, C.L. Modelling of the response of an ACFM sensor to rail and rail wheel RCF cracks. NDT E Int. 2012, 46, 107–114. [Google Scholar] [CrossRef]
Ekberg, A.; Akesson, B.; Kabo, E. Wheel/rail rolling contact fatigue—Probe, predict, prevent. Wear 2014, 314, 2–12. [Google Scholar] [CrossRef]
Grassie, S.L. Rail corrugation: Advances in measurement, understanding and treatment. Wear 2005, 258, 1224–1234. [Google Scholar] [CrossRef]
Wang, J.; Xu, Y.J.; Wang, L.; Wang, P. The application Research of Machine Vision in Rail Wear Detection. Railw. Stand. Des. 2014, 58, 36–39. [Google Scholar]
Resendiz, E.; Hart, J.M.; Ahuja, N. Automated Visual Inspection of Railroad Tracks. IEEE Trans. Intell. Transp. Syst. 2013, 14, 751–760. [Google Scholar] [CrossRef] [Green Version]
Dou, Y.; Huang, Y.; Li, Q.; Luo, S. A fast template matching-based algorithm for railway bolts detection. Int. J. Mach. Learn. Cybern. 2014, 5, 835–844. [Google Scholar] [CrossRef]
Marino, F.; Distante, A.; Mazzeo, P.L.; Stella, E. A real-time visual inspection system for railway maintenance: Automatic hexagonal-headed bolts detection. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 2007, 37, 418–428. [Google Scholar] [CrossRef]
Li, Q.; Ren, S. A real-time visual inspection system for discrete surface defects of rail heads. IEEE Trans. Instrum. Meas. 2012, 61, 2189–2199. [Google Scholar] [CrossRef]
Mandriota, C.; Nitti, M.; Ancona, N.; Stella, E.; Distante, A. Filter-based feature selection for rail defect detection. Mach. Vis. Appl. 2004, 15, 179–185. [Google Scholar] [CrossRef]
Marino, F.; Stella, E. ViSyR: A vision system for real-time infrastructure inspection. In Vision Systems: Applications; InTech: Shanghai, China, 2007. [Google Scholar]
Qingyong, L.I.; Zhang, H.; Ren, S.; Peng, D.; Weiyi, L.I. Detection Method for Rail Corrugation Based on Rail Image Feature in Frequency Domain. China Railw. Sci. 2016, 37, 24–30. [Google Scholar]
Li, Q.; Shi, Z.; Zhang, H.; Tan, Y.; Ren, S.; Dai, P.; Li, W. A cyber-enabled visual inspection system for rail corrugation. Future Gener. Comput. Syst. 2018, 79, 374–382. [Google Scholar] [CrossRef]
Csurka, G.; Dance, C.; Fan, L.; Willamowski, J.; Bray, C. Visual categorization with bags of keypoints. In Proceedings of the Workshop on Statistical Learning in Computer Vision, ECCV, Prague, Czech Republic, 11–14 May 2004; Volume 1, pp. 1–2. [Google Scholar]
Lazebnik, S.; Schmid, C.; Ponce, J. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA, 17–22 June 2006; pp. 2169–2178. [Google Scholar]
Luo, H.L.; Guo, M.J.; Kong, F.S. Image Classification Method by Combining Multi-features and Sparse Coding. Pattern Recognit. Artif. Intell. 2014, 27, 345–355. (In Chinese) [Google Scholar]
Huang, K.Q.; Ren, W.Q.; Tan, T.N. A Review on Image Object Classification and Detection. Chin. J. Comput. 2014, 37, 1225–1240. [Google Scholar]
Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24, 971–987. [Google Scholar] [CrossRef]
Liu, C.; Yuen, J.; Torralba, A.; Sivic, J.; Freeman, W.T. Sift flow: Dense correspondence across different scenes. In Proceedings of the European Conference on Computer Vision, Marseille, France, 12–18 October 2008; pp. 28–42. [Google Scholar]
Kanungo, T.; Mount, D.M.; Netanyahu, N.S.; Piatko, C.D.; Silverman, R.; Wu, A.Y. An efficient k-means clustering algorithm: Analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24, 881–892. [Google Scholar] [CrossRef]
Wang, J.; Yang, J.; Yu, K.; Lv, F.; Huang, T.; Gong, Y. Locality-constrained linear coding for image classification. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 3360–3367. [Google Scholar]
Asha, V.; Nagabhushan, P.; Bhajantri, N.U. Automatic extraction of texture-periodicity using superposition of distance matching functions and their forward differences. Pattern Recognit. Lett. 2012, 33, 629–640. [Google Scholar] [CrossRef]
Sheng, J.; Tang, G.A.; Yang, T. Automatic Extraction Method for Texture Periodicity Based on Improved Normalized Distance Matching Function. Pattern Recognit. Artif. Intell. 2014, 27, 1098–1104. [Google Scholar]
Borji, A.; Cheng, M.M.; Jiang, H.; Li, J. Salient object detection: A survey. arXiv 2014, arXiv:1411.5878. [Google Scholar] [CrossRef]
Paris, S.; Kornprobst, P.; Tumblin, J. Bilateral Filtering. Int. J. Numer. Methods Eng. 2009, 63, 1911–1938. [Google Scholar]
Zhang, Q.; Shen, X.; Xu, L.; Jia, J. Rolling guidance filter. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014; pp. 815–830. [Google Scholar]
Lee, S.; Kwon, H.; Han, H.; Lee, G.; Kang, B. A space-variant luminance map based color image enhancement. IEEE Trans. Consum. Electron. 2010, 56, 2636–2643. [Google Scholar] [CrossRef]
Land, E.H. An alternative technique for the computation of the designator in the retinex theory of color vision. Proc. Natl. Acad. Sci. USA 1986, 83, 3078–3080. [Google Scholar] [CrossRef] [PubMed]
Li, X.; Pan, J.; He, Y.; Liu, C. Bilateral filtering inspired locality preserving projections for hyperspectral images. Neurocomputing 2015, 164, 300–306. [Google Scholar] [CrossRef]
Han, H.; Sohn, K. Automatic illumination and color compensation using mean shift and sigma filter. IEEE Trans. Consum. Electron. 2009, 55, 978–986. [Google Scholar] [CrossRef]
Banić, N.; Lončarić, S. Light random sprays Retinex: Exploiting the noisy illumination estimation. IEEE Signal Process. Lett. 2013, 20, 1240–1243. [Google Scholar] [CrossRef]
Wang, D.; Yan, W.; Zhu, T.; Xie, Y.; Song, H.; Hu, X. An Adaptive Correction Algorithm for Non-Uniform Illumination Panoramic Images Based on the Improved Bilateral Gamma Function. In Proceedings of the 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Sydney, Australia, 29 November–1 December 2017; pp. 1–6. [Google Scholar]
Liu, Y.; Wei, X. Track Surface Defect Detection Based on Image Processing. In Proceedings of the International Conference on Electrical and Information Technologies for Rail Transportation, Changsha, China, 20–22 October 2017; pp. 225–232. [Google Scholar]
Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef]
Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 2006, 27, 861–874. [Google Scholar] [CrossRef]
Sipser, M. Introduction to the Theory of Computation. ACM Sigact News 2008, 27, 27–29. [Google Scholar] [CrossRef]
Zhou, J.; Wang, J.; Pan, R.; Gao, W. Periodicity Measurement for Fabric Texture by Using Frequency Domain Analysis and Distance Matching Function. J. Donghua Univ. 2017, 43, 629–633. [Google Scholar]
Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef]
Wu, L.; Zhang, C.; Wan, C.; Shi, H. Study on Optical Model for Rail Surface Detection Based on Machine Vision. Railw. Stand. Des. 2017, 61, 50–53. [Google Scholar]
Achanta, R.; Hemami, S.; Estrada, F.; Susstrunk, S. Frequency-tuned salient region detection. In Proceedings of the Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 1597–1604. [Google Scholar]
Lee, K.H. First Course on Fuzzy Theory and Applications; Springer Science & Business Media: Berlin, Germany, 2006; Volume 27. [Google Scholar]

Figure 1. The corrugation on a railway track.

Figure 2. The procedure of the two-stage image pre-processing method. (a) original image; (b) image de-noised by bilateral filter; (c) illumination extraction result; (d) corrected image of the proposed two-stage method.

Figure 3. The process diagram of rail surface localization method. (a) original corrected image; (b) image processed by vertical edge enhanced; (c) the result of calculating weighted gradient sum value; (d) the result of rail surface localization.

Figure 4. Sketch map of SPM model.

Figure 5. The general structure diagram of Spatial Pyramid Matching (SPM).

Figure 6. Comparison of two coding methods. (a) Vector quantization (VQ); (b) Locality constrained linear coding (LLC).

Figure 7. The general structure diagram of integrated multi-features and locality constrained linear coding (IMFLLC).

Figure 8. The diagram of corrugation interval localization. (a) Visualization of the localization process and results of the corrugation interval; (b) the result of the horizontal gray gradient statistical analysis.

Figure 9. The structure diagram of fast Fourier transform normalized distance matching function (FFT-NDMF).

Figure 10. The frequency-amplitude curves of some corrugation interval columns.

Figure 11. Visualized examples of measurement results: sample image (a–f).

Figure 12. The pipeline of corrugation severity assessment method.

Figure 13. The visual result of the used mathematical morphology algorithm.

Figure 14. The level membership function used in this paper.

Figure 15. The visualization process and severity assessment result. (a) the original corrugation image; (b) the result rustless corrugation image after using horizontal gray gradeint statistical (HGGS); (c) the corrugation saliency map obtained by normalized frequency tuned (NFT); (d) the result of image binarization; (e) the result of each corrugation localization and the wear salliency (WS) of them; and (f) the final severity assessment result of each corrugation area.

Table 1. The comparison of detection performance between the proposed method and the baselines.

Method	Precision (%)	Recall (%)	Accuracy (%)
Gabor + SVM	91.80	93.33	92.50
AET	97.00	96.50	96.80
ME + SVM	98.97	97.00	98.00
The proposed	99.67	98.34	99.00

Table 2. The period estimation results of corrugation displayed in Figure 11.

Sample Image	(a)	(b)	(c)	(d)	(e)	(f)
Frequency (num)	5	4	4	3	5	6
Period (pixel)	102.4	128	128	170.7	102.4	85.3
Manual result (pixel)	102	128	128	172	102	85

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Wei, D.; Wei, X.; Liu, Y.; Jia, L.; Zhang, W. The Identification and Assessment of Rail Corrugation Based on Computer Vision. Appl. Sci. 2019, 9, 3913. https://doi.org/10.3390/app9183913

AMA Style

Wei D, Wei X, Liu Y, Jia L, Zhang W. The Identification and Assessment of Rail Corrugation Based on Computer Vision. Applied Sciences. 2019; 9(18):3913. https://doi.org/10.3390/app9183913

Chicago/Turabian Style

Wei, Dehua, Xiukun Wei, Yuxin Liu, Limin Jia, and Wenqiang Zhang. 2019. "The Identification and Assessment of Rail Corrugation Based on Computer Vision" Applied Sciences 9, no. 18: 3913. https://doi.org/10.3390/app9183913

APA Style

Wei, D., Wei, X., Liu, Y., Jia, L., & Zhang, W. (2019). The Identification and Assessment of Rail Corrugation Based on Computer Vision. Applied Sciences, 9(18), 3913. https://doi.org/10.3390/app9183913

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

The Identification and Assessment of Rail Corrugation Based on Computer Vision

Abstract

1. Introduction

2. Problem Statement

3. Rail Corrugation Identification Based on Improved SPM Model

3.1. Image Pre-Processing

3.1.1. Image De-Noising

3.1.2. Uneven Illumination Correction

3.2. Rail Head Surface Localization

3.3. Improved Spatial Pyramid Matching (SPM) Model

3.3.1. Multi-Feature Extraction

3.3.2. Vector Quantization of Image Features

3.4. Rail Corrugation Identification Based on IMFLLC

3.5. Identification Experiments and Results

3.5.1. Introduction to Experiment Dataset

3.5.2. Performance Evaluation for Corrugation Identification

3.5.3. Analysis of Computational Complexity

4. Estimation of Rail Corrugation Period

4.1. Corrugation Interval Localization

4.2. Corrugation Period Estimation Based on FFT-NDMF

4.3. Performance Evaluation for Corrugation Period Estimation

5. The Severity Assessment of the Corrugation

5.1. Visual Significance Analysis for Corrugation Image

5.2. Visual Saliency Feature Extraction and Saliency Map Generation

5.3. Severity Assessment Based on Fuzzy Theory

5.3.1. Image Binarization and Corrugation Area Localization

5.3.2. Corrugation Severity Assessment

5.4. Case Study for Rail Corrugation Severity Assessment

6. Conclusions

Author Contributions

Funding

Conflicts of Interest

Appendix A

Appendix B

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI