Multimodal data imputation and fusion for trustworthy fault diagnosis of mechanical systems

The presence of missing values in the collected data due to sensor failure, communication interruption, or environmental interference can greatly diminishes the trustworthiness of fault diagnosis for mechanical systems. Therefore, this study proposes and evaluates a novel multimodal data imputation and fusion method to perform the trustworthy fault diagnosis of mechanical systems. First, a generative adversarial imputation network, termed as the L2 regularization temporal–spatial generative adversarial imputation network (L2-TSGAIN), is developed. This L2-TSGAIN network, based on a temporal–spatial feature extraction module and L2 regularization loss function, can comprehensively extract data features from both temporal and spatial perspectives, thus achieving high-quality imputation of anomalous sensor data. Subsequently, a multi-input single-output autoencoder (MISO-AE) is designed to extract a universal representation of the imputed data from different modalities and recover features in the fusion data. Finally, the fusion data from different health states of mechanical systems are input into a convolutional neural network classifier to perform fault diagnosis. Experiment validations, considering the presence of missing values in sensor data, have been carried out on the planetary transmission system and gearbox test bench. Compared with several mainstream data imputation methods for fault diagnosis, the optimal diagnostic accuracy of 99.68 % and 100 % on these two datasets can be obtained using the proposed method, respectively, confirming its superior performance and reliability. Thus, the proposed method can provide a trustworthy fault diagnosis tool for mechanical systems in industrial scenarios considering anomalous sensor data.