Configurable data processing pipelines in the Baikal-GVD experiment: architecture of sustainable scientific software

Abstract

The article is devoted to the development of an automated data processing system for the Baikal-GVD neutrino telescope. The key step was the separation of the logic of individual tasks from the structure of their interconnections: the dependency graph was moved out of the task code and into a configuration file. At the same time, the computational logic of the existing tasks remained unchanged, previous processing pipelines were preserved without disruption, and new ones were added through configuration. The updated architecture leverages the capabilities of the Law framework — an extension of Luigi that provides ready‑to‑use interfaces for working with the EOS remote storage and the HTCondor and SLURM batch processing systems. The resulting modularity, reusability of tasks, and robustness against errors embody the principles of sustainable scientific software in practice. This approach can be applied in other scientific experiments.