Conveners
APL2 - Bridging the Gap: Reproducibility, Deployment, Operation, Updates, and Monitoring of Machine Learning-Based Solar Flare Forecast Models: Orals
- Sophie Murray (Dublin Institute for Advanced Studies)
- Anna Maria Massone (Dipartimento di Matematica, Università degli Studi di Genova)
- Paul Wright (University of Exeter)
- Shane Maloney (Dublin Institute for Advanced Studies)
Description
Machine learning (ML) has shown promise in solar flare forecasting, yet major challenges remain in moving beyond single studies often published without the associated code and datasets. Are forecast models truly reproducible, deployable, operable, updatable, and monitorable? It has been nearly a decade since the influential work of Bobra & Couvidat (2015), where Support Vector Machine (SVM) was used to predict solar flares using features derived from solar magnetic field observations, what progress has been made? While the field has made strides in improving flare forecasting, little to no progress has been made in ensuring the reproducibility, deployment, and sustained operation of ML models. As models become increasingly complex, how can we guarantee that their results are reproducible and transparent? How do we ensure that models remain accurate as new data becomes available and do not degrade over time? The traditional approach of publishing research papers—often without the accompanying code, data, or reproducibility frameworks—is no longer sufficient. To truly advance the field, we must move beyond current academic practices and adopt best practices from ML and meteorology, which have well-established methodologies for real-time prediction systems. This includes:
Reproducibility: Establishing standardized benchmarks, dataset and model versioning, and open-source implementations.
Deployment: Models must be easily deployed from zero to a running deployment with as little human intervention as possible.
Operation: Ensuring once deployed models can easily be run in research or operational environments and they are robust to missing data, latency, and changing solar cycle conditions.
Updates: Implementing retraining strategies to include new data and prevent model degradation over time.
Monitoring: Developing frameworks for continuous evaluation, explainability, and reliability of forecasts.
We must leverage platforms like Hugging Face, Kaggle, Comet, Neptune, WandB and open source solutions like MLFlow to facilitate transparent and collaborative development. The space weather community must also engage MLops, automated retraining pipelines, and robust monitoring tools to transition ML-based forecasting from one off publications to an operational reality.
This session invites submission focusing on the use or implementation of any of the above aspects and interdisciplinary approaches to move ML-based space weather forecasting from promise to practice.
We present an ongoing effort dedicated to the development of reproducible and operationally viable artificial intelligence models for solar flare forecasting. Our approach leverages the extensive archive of multi-wavelength solar images captured by the Solar Dynamics Observatory (SDO) using the Atmospheric Imaging Assembly (AIA). Specifically, we employ a self-supervised learning strategy,...
The CORonal mass ejection, solar eNERgetic particle and flare forecaSTing from phOtospheric sigNaturEs (CORNERSTONE) project focuses on the prediction of intense solar events through the application of machine learning (ML) techniques to real observational data. This domain poses significant challenges to reproducibility, primarily due to the heterogeneous nature of the data and the complexity...
Segmentation and characterization of solar coronal structures are essential for advancing our understanding of the solar atmosphere and accurately identifying key regions such as active regions and coronal holes which are precursors to phenomena like solar flares and coronal mass ejections (CMEs). In parallel, it is crucial to incorporate onboard such artificial intelligence (AI) algorithms...
We develop two artificial intelligence-based models (Models A and B) to predict time-evolving photospheric magnetic fields with an adjustable timestep, ranging from a few seconds to one solar rotation ahead or behind. Model A predicts future magnetic field data using three consecutive radial magnetic field datasets with a 12-hour cadence. Model B reconstructs evolving magnetic fields over the...
The aim of the ARCAFF (Active Region Classification and Flare Forecasting) project is to develop a beyond state-of-the-art flare forecasting system leveraging end-to-end deep learning models to significantly improve upon current flare forecasting capabilities. However, transitioning these models from research environments to robust, operational systems presents significant challenges,...
