Towards a Universal, Data-Agnostic Framework for Automatic Anomaly Detection in Complex Multivariate Time-Series

Precise real-time detection of irregular behavior in highly complex systems is a cross-cutting challenge in accelerator physics, high-energy experiments, and large-scale industrial processes. Yet, most data streams collected at research facilities such as DESY and larger economic entities are unlabeled, noisy, and heterogeneous. This project attempts to develop and evaluate a data-agnostic, automatic anomaly-detection framework. It aims to unify state-of-the-art forecasting, reconstruction, and representation paradigms. Furthermore, it will deliver actionable uncertainty estimates and can adapt seamlessly to other domains.

Three contrasting data sets will anchor the research and development in increasing order of difficulty: (1) Optical Synchronisation System (OSS) sensor traces from DESY (rich understanding, labeled and part of prior DASHH projects); (2) SuperKEKB beam-diagnostics data from KEK, Japan (partial labels, language & coordination barriers); and (3) Economic transactional data (heterogeneous, exogenous shocks, non-physical process). Dataset 1  serves as the basis for development; Datasets 2–3 function as harder additional datasets that may reveal transfer limits and guide the design of a genuinely agnostic framework. The results will feed back into control-room tools at DESY and may be transferred to industry partners when appropriate (e.g., with Dataset 3).