One of the main issues with training data-driven models such as machine learning algorithms is the quality and quantity of the available data. In this case study, for example, devising ML-based models that conduct adequate fault classification on in-orbit satellites was infeasible due to the operational data scarcity inherent to these systems. The other issue with the existing satellites' telemetry datasets was that they did not include all the possible operational modes, such as different fail states, and were heavily imbalanced.
In this project, two methods to overcome the data scarcity issue and to effectively train neural networks to perform fault classification on satellite telemetry data (1D signals) were explored: 1) Wasserstein Generative Adversarial Networks (WGANs), and 2) Dynamic Time Warping (DTW)-based oversampling.
The devised WGAN consists of a generator and discriminator (critic) that are pitted against each other in order for the generator to produce more realistic signals and the critic to get better at distinguishing real signals from generated signals. The devised generator and critic's inner architecture in this project was based on 1D convolutional upsampling and downsampling layers. Figure 1, illustrates how the training is carried out in the proposed WGAN model.
Figure 1. The architecture of the generative AI model or Wasserstein Generative Adversarial Network (WGAN) for satellite Reaction Wheel signal generation
Solving the inadequate data quality and quantity issues in training neural networks using neural networks themselves, might seem counter-intuitive. However, therein lies an important distinction between the objectives of a fault classification neural network and a generative neural network. The fault classifiers are designed to learn the features of different fault classes and classify them accordingly, whereas the generative models (WGAN in this case) are responsible for receiving a random signal and modifying it so it resembles the initial imbalanced dataset's signals. In this way, there is added variability due to building up from a random signal that helps cover all the different fail states of a satellite.
In this project, another oversampling method known as Dynamic Time Warping (DTW) oversampling is also explored. The way this technique works is that it measures the distance between each signal within an overarching dataset and another signal in the same dataset. Then, the closest pair are "averaged" to arrive at an interpolated signal for augmenting an imbalanced dataset. The interpolation techniques implemented in this project include one-to-one interpolation, linear interpolation, weighted interpolation, enhanced weighted interpolation, and sinusoidally weighted interpolation. Some of these techniques contain one-to-one, one-to-many, and many-to-one interpolation capabilities.
Figure 2. Imbalanced dataset augmentation using the generative AI model (WGAN) and performing fault diagnosis using the Long Short-Term Memory (LSTM) network
Testing oversampling techniques and generative AI on the realness of their generated instances is often plagued with uncertainties. Qualitative and quantitative testing has been carried out to address the testing of our developed methodologies in an unbiased way. Qualitative testing was conducted to measure the diversity of our generated signals and quantitative testing was carried out to measure the similarity of the generated signals to the real dataset.
Different instances of a Long Short Term Memory (LSTM) model were developed that were trained on the oversampled datasets and tested on real signals. If the WGAN model managed to generate signals that resembled the real datasets, it would improve the LSTM model's performance in classifying different fault-indicator characteristics in signals and enhance the fault classification process of satellite telemetry data. Figure 2 depicts this process.
Figure 3. Average testing accuracies of the LSTM model as it is trained on different datasets
Figure 3 contains a comparison between the three instances of an LSTM model that was trained on an imbalanced dataset, a WGAN-augmented dataset, and a naturally balanced dataset. It can be observed that the WGAN-generated dataset builds upon the accuracy rates obtained by training on the imbalanced dataset and closes in on the naturally balanced dataset's obtained accuracy.
The contents of this study have been published here. For more information regarding this study and to view the complete topic outline, results, and discussions, please consult the following work:
Hedayati, M.; Barzegar, A.; Rahimi, A. Mitigating Data Scarcity for Satellite Reaction Wheel Fault Diagnosis with Wasserstein Generative Adversarial Networks. In Proceedings of the 2024 IEEE International Conference on Prognostics and Health Management (ICPHM), Washington, DC, USA, 11–13 October 2024; pp. 367–376.