Page 23 - Read Online

Ji et al. Intell Robot 2021;1(2):151-75 https://dx.doi.org/10.20517/ir.2021.14 Page 167

Image data have been widely used with useful results. However, they tend to be used in detection and
classification, which normally correspond to corrective or preventive maintenance strategies. In order for
the industry to advance to predictive maintenance, the causes of defects need to be investigated and
corresponding signals can be used for deep learning models’ training and testing. Thus, deep learning
application in rail track condition monitoring need to cater to more varieties of data types; for example, the
vibration signal is normally a sequential signal and vibration tends to cause wear on the rails over time.

The performance of deep learning models such as accuracy, response time, and precision tend to be
influenced by the data used. Therefore, a common dataset of rail tracks will be helpful for researchers to use
and validate the performance of deep learning models developed around the world. True performance
comparisons could be made as well.

There are always more sections of normal rail tracks than defective sections; therefore, the number of
normal track images tend to be much more than the defective ones. When considering the different classes
of defects, the number of defect images will be even lower, which can cause class imbalance issues. GAN is a
deep learning-based generative model. With the application of GAN to generate more images of defects, the
training of deep learning models for detecting and classifying rail defects could be more efficient and
accurate.

Transfer learning focuses on storing knowledge gained while solving one problem and applying it to a
different but related problem. As rail track condition monitoring task is shared by researchers around the
world, a “more” related problem and the knowledge gained will be more useful when it is transferred to
another problem but in the same industry. This could be a promising research area for future work.

Reinforcement learning is the training of machine learning models to make a sequence of decisions. The
agent learns to achieve a goal in an uncertain, potentially complex environment. This trial-and-error
approach suits well the complex situation of deciding when it is the right time to perform the maintenance
operations. This research area could generate meaningful results for the industry.

5. ILLUSTRATIVE CASE STUDIES
In this section, we use two application examples to illustrate the implementation of deep learning models to
support rail track condition monitoring and rail defect detection and classification.

5.1. Data acquisition and preparations
Data acquisition equipment or devices could be installed on the rail inspection vehicles or passenger trains
at different positions. We use rail vision systems here to record videos of the rail tracks for both head and
rail checks. Lights are usually required to further enhance the quality of images taken; we use halogen
floodlights to support visibility. The train speed might affect the image quality; therefore, a maximum speed
may be set. For us, it is 100 km/h. Figure 8 illustrates a typical setup.

Various types of input data (image, sound, vibration, etc.) could be adopted for deep learning tasks. For
image data, different formats and sizes might be used. Original greyscale images are captured from the rail
track videos and then used without segmenting the tracks from the ballast, the sleepers, or other
background textures around the rail tracks so as to minimize the image pre-processing efforts and maximize
the utility of deep learning models. Pre-processing of input images is optional. Figure 9 shows a sample
image that was used for training and test purpose.