A Vision Based Condition Monitoring Approach for Rail Switch and Level Crossing using Hierarchical SVM in Railways

Rail track, turnout and level crossing are critical parts of railway components. So faults of rail surface are directly related to the operation safety. The defects occurring in switches can lead to accidents and the derailment of the train. Therefore, it is necessary to utilize advanced technology to monitor rail damage of the switch and level crossing zones. In this study, the switch and level crossing detection are performed with vision based contactless image processing techniques. Edge detection algorithm, image processing filters, morphological feature extraction, Hough transform and hierarchical SVM classifier are utilized to detect turnout and level crossing regions in the railway images. While switch points are detected, images of the area in front of the train are continuously taken from camera placed on the train. These images contain foreign objects such as buildings, trees, etc. The foreign objects existing in the railway image make it difficult to detect accurately the rail track. The turnout and level crossing detection cannot also be performed using the information of the rail track that cannot be correctly detected. The several processes, such as some image processing filters and morphological feature extraction, have been performed to solve this problem. With destroying the foreign objects from the railway images, both the accuracy rate of the proposed method has been increased and real-time implementation of the method is provided. The Hough transform is utilized to detect rail track, switch and level crossing in the proposed method. The turnout detection is performed by applying SVM classifier to railway image. The proposed method provides real-time rail track, level crossing and turnout detection. Furthermore, this method has gained dynamic feature due to not be utilized any template image for switch detection. When evaluated from these aspects, both the proposed method is superior to other methods in literature and results were obtained in a shorter time thanks to applying ROI Segmentation to the railway image.


Introduction
Railway transport is becoming common all over the world.Transportation system is one of critical infrastructures in a society [1].It is important to ensure the safety of railway transportation [2].As the railway traffic are developing at a dramatically speed all over the world, the safety and comfort of the railway system are paid more attention than ever before.It is required that the railways are examined in more detail with the enhanced railroad technology.With being manually done of this control process, both railway transportation is temporarily disabled and rail track can damage.Safety can be enhanced using intelligent systems that supply additional information about the exact location of the train, its speed and upcoming obstacles [3].Manual inspection of rails is slow and contact inspection methods harm the rails, rail analysis is performed with machine vision via non-contact, high accuracy and high-speed methods [4].Besides, manual inspection is expensive and cause to temporarily disable of railway [5].A turnout is used to transfer vehicles from one track to another track.As an important part of railway system, the turnouts have a strong relation to the comfort of the railway vehicles and severe turnout defects can lead to accidents [6].Therefore, it is important to detect turnout defects timely.Switch is one of the three outdoor railway equipment [7].It has many characteristics such as large quantity, complex structure, short using life, limiting the train speed, low driving safety, big investment for maintenance etc. [7].To locate railway vehicles exact and reliably in railroad network, collision avoidance systems are required.In this systems it is vital to determine the direction a railroad vehicle turns at switches.Most of common driver assistant systems work on unstructured environments for detection of obstacles [8].These environments include many nonplanar surfaces that pose a big challenge for vision systems [8].Alike problems exist for railroad environments that frequently contain complex shapes and surfaces like hills and vegetation along railroad tracks [8].In railroad transportation, main task of train driver is carefully to focus on rail track.Therefore, vision area of train driver is area between two rails in front of train and right and left sides of these rails.Detection of switches can be seen as possible accidentals prevention tool and driver assistance system.The developed methods related with based contactless image processing switch detection are available in literature.Espino et al. [3] offered a method that detects railway track and turnout.In addition, this method was not based on an empirical threshold.Railway track extraction was not based on an edge detection using the width of the rolling pads.This edge detection scheme was used as an input to the RANSAC algorithm to determine the model of the rails.The turnout detection scheme was based on the Histogram of Oriented Gradient (HOG) and Template Matching (TM) [3].To detect railway turnout, Support Vector Machine (SVM) was also used.Block diagram of this switch detection in literature is shown in Fig. 1.Li et al. [9], enhanced line detection method for railway switch machine monitoring system.This method was based on image processing techniques.It was used to Canny edge detection, Zhang Suen Thinning method to reduce the thickness of the edges and Probabilistic Hough Transform to detect the lines in this study [9].Qi et al. [10] presented a novel railway tracks detection and turnouts recognition method by using HOG (Histogram of Oriented Gradients) features.Firstly, this method computes HOG features and defines integral images, and then extracts railway tracks by region-growing algorithm [10].Wohlfeil [11] recommended a vision-based method that provides to determine the direction a railroad vehicle turns at switches.A camera observed area in front of railroad vehicle and images of this area were used to detect rail tracks in real-time.Switches were detected by tracking rail tracks in these images.It was shown that the followed track can be determined at branching switches [11].Kaleli and Akgun [12] offered an algorithm to extract railroad track area in front of the train by utilizing Dynamic Programming.They utilized dynamic programming to compute the optimal path which gives the minimum cost to extract the railroad track space.This algorithm extracted the left and right rails through utilizing dynamic programming simultaneously.Espino and Stanciulescu [13] improved a vision-based algorithm that detects railway track and turnout and classifies turnout.In this algorithm, railway track detection was performed with edge detection using the width of rolling pads [13].Then this edge detection scheme was used as an input to the RANSAC algorithm to determine model of rails knowing their gauge.Turnout detection process was based on Histogram of Oriented Gradient (HOG) and Template Matching (TM) in this study.Ross [14] proposed the utilization of a monofocal video camera to enhance the localization quality.Ross's algorithm estimated the track recursively in camera images.Ying et al. [15] recommended a vision based algorithm to determine and evaluate the railway components.This method is running on a train with 16km/h speed.Rail component is detected by applying Sobel edge detection and Hough transform methods.The block diagram of the proposed railway inspection system is demonstrated in Fig. 2.
In this study, a vision based switch detection method was proposed using image processing algorithms.SVM classifier and Hough transform were applied to images that are taken from a camera on the train.In this study, a vision based switch and level crossing detection method were proposed using image processing algorithms.SVM classifier and Hough transform were applied to images that are taken from a camera on the train.

The Structure and Types of Railway Switches and Level Crossing
Switches, also called turnouts or points, are an important part of railroad.This component enables trains to orient from one infrastructure, given that they enable the creation of a real network.Turnouts are placed in the infrastructure not only to connect different lines, but also to connect parallel tracks of the same line [16].Therefore, this process give flexibility to track.
Components on the railway line should be monitored at regular intervals to ensure continuous transport security [17].Switches must be kept in good condition in order to guarantee an enough running of the train through switch.If this situation is not properly maintained, the wheel-rail interface will be negatively affected.Because of safety and operational issues, switches are intensively examined and protected.Each switch has a right track and a diverted track [18].The radius of the curve located on the diverted track that supplies passing to secondary roads is R=300 m, the maximum length standard of single switch is 34.20 m.The sleepers of the switches on highspeed trains are concrete, the radius of the curve located on the diverted track that provides transition to secondary roads is R=1500 m, the maximum length standard of single turnout is 72 m.An example of turnout components is given in Fig. 3.As it is seen in Fig. 3, the turnout system has many elements.The switch crossings are very important on the railway line.Major defects may happen because of the defects of the elements on the switch crossings.Therefore, the railway line particularly the switch crossings should consistently be examined.It is required to make the necessary maintenance even considering the small defects.
There are a number of standard layouts or types of turnouts [19][20][21].These types are given in Fig. 4.  The level crossing is a crossing on one level of a railway line by a road, path, or another railroad [23].Level crossings are high-risk areas where rail traffic crosses paths with road and pedestrian traffic at the same level.In many countries, level crossings on less important roads and railway lines are often open or uncontrolled [23].An example of level crossing is shown in Fig. 5.

Switch Detection Method
The rail tracks, switches and level crossings are vital components in railway.To be safety transportation in railroad, these components should be controlled and examined whether defects or not.Determination of these components with contactless image processing techniques provides additional information such as exact location of train, its speed and upcoming obstacles.
A vision based image processing approach is proposed in order to detect rail track, switch and level crossing.Vanishing point term refers to the intersection of the track lines defined as in Liang [33].It is firstly performed color conversion from RGB to HSV color space in this algorithm.HSV color space In gradient computing step, the gradient of input image is computed by using Sobel operator and thereby gradient magnitude image Gradient(i,j) is obtained.Then a given threshold value is applied to the bottom 1/3 of the gradient image due to the fact that this part of the image includes sufficient information to extract the features.Furthermore rail tracks are straight and their gradient magnitudes are high in the bottom of the image.This threshold process eliminates the points that have lower gradient magnitude from the threshold value T. In the next step, a binary image Binary(i,j) is constituted from gradient image Gradient(i,j) by Equation 1.

𝐵𝑖𝑛𝑎𝑟𝑦(𝑖, 𝑗) = { 1 0 𝑖𝑓 𝐺𝑟𝑎𝑑𝑖𝑒𝑛𝑡(𝑖, 𝑗) > 𝑇 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
(1) In the next step, Hough transform is carried out to binary image Binary(i,j)in order to detect rail lines.Hough transform returns lines that have different lengths.Hough transform is very helpful in order to detect straight lines.Lines returned include their endpoint coordinates without line length information.The length of a straight line between its two end points ( 1 ,  1 ) and ( 2 ,  2 ) is computed by Equation 2.
This operation specifies many lines including rail track lines.But the lines expect for rail track lines are also available.Thus we extract the rail lines which are the longest lines on the left and right.Then the longest line on the left and right can be founded by calculating line lengths by Equation 3.
After rail track lines are detected by using Hough transform [28,29], the intersection of two straight rail lines in the image can be computed.To compute vanishing point, common form of a linear equation can be used.This equation is described in Equation 3.This equation is slope-intercept form, where m is the slope and c is the y-intercept of the line.The slope equation of a line which passes through two different points  1 ( 1 ,  1 ) and  2 ( 2 ,  2 ).This equation is shown in Equation 4.
The y intercept of a line which passes through two different points  1 ( 1 ,  1 ) and  2 ( 2 ,  2 ).This equation is shown in Equation 5.
The equation 6 computes the y coordinate of vanishing point p.
In Equation 6,   is the y coordinate of the intersection of lines.  can be computed by subtracting x coordinates of the rail lines at bottom of the image.Therefore, region of interest area in image is detected with   and   obtained.Then this area is extracted from image by cropping.So ROI process facilitates in order to detect of rail track in image.
The image obtained from ROI segmentation method is given to switch detection algorithm.Block diagram of switch crossing detection approach is as shown in Fig. 7 The steps of switch detection algorithm are respectively acquiring the railway images, grayscale color conversion, edge-preserving smoothing, removing noise by Adaptive filtering, image enhancement, labelling connected components, finding lines with Hough transform and applying Hierarchical SVM classifier.
The image that are given to the proposed method as input parameters are taken from a camera placed on the train.The viewing point of the camera on the train comprises rail track on which the train travels.Rail images taken from the camera are in RGB color format.In the next step of the proposed method, RGB-Grayscale color conversion is applied to these images.Then edge-preserving smoothing filter is enforced to these images.The edge-preserving smoothing is much more suitable for feature extraction [25].Edge-preserving smoothing algorithm is calculating the filtered gray value in dependence of the content of a defined neighbourhood.The effect of this smoothing strategy is to preserve edges [26].Noisy regions in image are vanished by using Adaptive filtering [27].With being used this filter, switch crossings on image have been more prominent.In the image enhancement step, accurate detection of rail track on image is provided.The connected components have been detected to extract rail track from the railway image.After rail track is labelled on image, rail lines have been founded by utilizing Hough transform [28,29].Hough transform returns lines that have different lengths.Hough transform is very helpful in order to detect straight lines [30].With being detected rail lines, switch points are used in Hierarchical SVM classifier [31,32] and so switch crossings have been detected.

Level Crossing Detection Method
Level crossings that are located between the rails in a vertical position are horizontal position.To determine level crossings some of image processing techniques such as preprocessing, feature extraction, Hough transform, etc. are applied to image.
The pseudo code of level crossing detection algorithm is shown in Fig. 8.In the first step of level crossing detection algorithm, the image in RGB color space is convert to XYZ color space.XYZ color space is expressed such that all visible colors can be defined using only positive values.And the Y value is luminance [29].Red, green, and blue values are to be unwanted for creating a standardized color model that is suitable for all devices [29].
Because of this reason XYZ is used a mathematical formula to convert the RGB data.X, Y, and Z values can be obtained as shown in the following Equation 10.
In the next step of level crossing detection algorithm, Y channel image is obtained and this channel image is utilized in later stages of proposed algorithm.The reason for using this channel in later stages of the proposed algorithm, is to express luminance value in image.Then, gradient magnitude and x and y direction gradients of this image are calculated [34][35][36].
In the next step, edge detection process is made on image.Prewitt edge detection algorithm is used to this.Prewitt mask is a discrete differentiation operator.This operator uses two 3×3 masks for calculating approximate derivative values in horizontal and vertical directions [32,33].Prewitt masks are shown in Fig. 9. (a) X direction mask Prewitt masks [37][38][39].
The Prewitt edge detection mask gives equal weightage to all pixels when averaging [39].As seen in the mask the coefficients in the matrix kernel are all 1 [39].The key idea is that of central differences.This mask when convolved with an image, it performs a two dimensional spatial gradient [39].
After done edge detection step, two different images are obtained by applying two filter which 0 and 90 degrees in terms of size.And difference process is performed by subtracting these images obtained from each other.In this way the level crossing that is horizontal position in the image, has become sharper.Then Hough transform is applied to detect line in image.The Hough transform is utilized in order to determine the straight lines and parametric curves in an image [40].The pseudo code of Hough transform algorithm is shown in Fig. 10.

Experimental Results
A computer which has Intel (R) Core (TM) i7-2400 M CPU, 2.5 GHz, 6.00 GB RAM under Windows seven 64 bits is used to perform the proposed method.The proposed methods are written in Matlab R2014b.In this study, switch and level crossing detection in railway are developed through using contactless image processing techniques.ROI segmentation is made in the proposed algorithm to detect region of interest in the image i.e. rail track area.Images which are given to proposed ROI method as input and results of the proposed ROI method are respectively shown in Fig. 11.As seen in Fig. 11, with ROI segmentation process railway image is reduced to rail track image.And thus rail track has been identified more easily.In addition to this, success rate of rail track detection has been increased.Experimental result of ROI segmentation is shown in Table 1.Switch detection is performed in the proposed algorithm to extract switch points in the image.Switch crossing is identified in the image that is taken from ROI segmentation algorithm as an output image.Input and output images of Switch Detection algorithm are given in Fig. 12.And in addition to this, experimental result of Rail Track Extraction algorithm is shown in Table 2.
(a) (b)  Level crossing detection has been made in the railway image.And edge detection algorithms such as Sobel, Canny, Roberts, and Prewitt have been tried to detect level crossing.It has been observed that Prewitt edge detection algorithm has produced better results than other edge detection algorithms.Images which are given to proposed level crossing detection method as input and results of the proposed level crossing detection method are respectively shown in Fig. 13 and Fig. 14.Experimental result of level crossing detection algorithm is shown in Table 3.

Conclusions
An algorithm based contactless image processing has been committed to determine switches and level crossings in railroad The switch detection has been performed by using ROI segmentation, Hough transform and hierarchical SVM classifier.
With being performed ROI segmentation, both rail track has been extracted more easily and accurate rate of switch detection has been increased.So expressed step is very important for switch detection process.And also image used for switch detection is free of foreign objects such as trees, buildings, etc. by applying this step.To perform ROI segmentation, the image has been converted to HSV color space.V channel of this image has been used during the proposed method.With used this channel image, rail object could appear in clearer way and that has facilitated other image processing steps.Hough transform has been used to detect rail track line.The use of Hough transform has played an important role in determining the rail line.The proposed methods have been written in Matlab 2014b.Avarage elapsed time of the proposed switch detection method has been 0.41 seconds and accuracy rate of the proposed switch detection method has been 90.3 %.Thanks to level crossing detection, while train travel it can be found that whether there are obstacles on level crossing.And thus train accidents can be prevented.With detection of level crossing, obstacle detection process can be done in the image.Avarage elapsed time of the proposed level crossing detection method has been 0.3 seconds and accuracy rate of the proposed level crossing detection method has been 88 %.

Figure 3 .
Figure 3. Example of a turnout system and components.

Figure 4 .
Figure 4. Turnout types a) Single left switch b) Single right switch c) Symmetric switch d) Compound switch e) Cross switch f) Crossover [19 -21].
Determining the region of interest is performed with ROI segmentation.Region of interest consists of rail track area in front of train.Using ROI segmentation has led to easier detection of the track.The flowchart of ROI segmentation algorithm is shown in Fig. 6 To extract region of interest area, we compute to the railroad vanishing point p and track width parameter W_max from an input image.Rail tracks are aligned parallel to each other and their width decrease linearly in the image from bottom to up.

Figure 6 .
Figure 6.The flowchart of ROI Segmentation [30].consists of 3 channels.These channels are Hue (H), Saturation (S), and Value (V).Hue channel defines the dominant wavelength of the color such as green, blue, yellow, and etc.And this channel has a value between 0-360 angular.Saturation channel identifies the vividness of the color.High saturation reasons to vivid colors and low saturation ensures that the color approaches to shades of gray.While value channel defines white rate of within color.So it identifies the bright of color.In this study, ROI segmentation steps are performed on V channel image.In gradient computing step, the gradient of input image is computed by using Sobel operator and thereby gradient magnitude image Gradient(i,j) is obtained.Then a given threshold value is applied to the bottom 1/3 of the gradient image due to the fact that this part of the image includes sufficient information to extract the features.Furthermore rail tracks are straight and their gradient magnitudes are high in the bottom of the image.This threshold process eliminates the points that have lower gradient magnitude from the threshold value T. In the next step, a binary image Binary(i,j) is constituted from gradient image Gradient(i,j) by Equation1.

Figure 8 .
Figure 8.The algorithm of level crossing detection.

Figure 7 .
Figure 7. Block diagram of switch detection proposed method.

Figure 11 .
Figure 11.(a) Example railway input image (b) output image for ROI segmentation method

Figure 12 .
Figure 12.Input (a) and output (b) images of Switch Detection.

Figure 13 .
Figure 13.An railway input image of level crossing detection method.

Table 1 .
Example result of ROI segmentation

Table 2 .
Example result of switch detection algorithm

Table 3 .
Example result of level crossing algorithm