Representation learning using a multi-branch transformer for industrial time series anomaly detection

In recent years, due to the rapid expansion of the Industrial Internet of Things (IIoT), substantial amounts of high-dimensional industrial time series data have been generated. Anomaly detection in such industrial time series data is a challenging task due to complex temporal dynamics. In this paper, we propose multi-branch transformer with Gaussian mixture model (MBTGMM), a novel transformer-based framework to address some of these challenges. In the framework, normal representations are learned with a multi-branch transformer architecture that comprises of a convolution branch and a multi-head attention branch in order to learn both short- and long-term temporal dependencies in the time series data. These representations are then fed into a Gaussian mixture model for density estimation and anomaly detection task. Experimental results on public industrial datasets show the effectiveness of our proposed framework, and the ablation studies clearly demonstrate the efficacy of our design choices.