Multivariate Time Series Anomaly Detection Model Selection based on Rank Aggregation of Performance Metrics

Multivariate time series anomaly detection MTAD has significant practical relevance in various applications. Despite the recent proposals of numerous MTAD models, none have demonstrated consistent optimal performance across various scenarios. Hence, there is an urgent need to investigate the accurate selection of the most appropriate MTAD model for a specific dataset. Most studies on model selection depend on extensive pre-trained models. Nevertheless, in real-world situations, labels for time series anomaly data are seldom accessible, and the training cost of pre-trained models is significant. This paper presents an unsupervised method for selecting multivariate time series anomaly detection model based on rank aggregation of performance metrics. We create a reliable performance ranking by aggregating rankings from various unsupervised evaluation metrics. Subsequently, an early-stopping mechanism is applied to minimize computational expenses by identifying the Top-K models that consistently maintain their ranking in robust performance throughout the epochs. Extensive experiments on six real-world datasets demonstrates that our proposed unsupervised model selection method is comparably effective to the supervised method in selecting the optimal MTAD model.