ADABEL
Research Highlights & Outcomes
WP1
Research outcomes about machine learning-based forecasting models can be summarized as follows:
-
New privacy-preserving forecasting models are developed to capture dependencies among grid users and hedge against local data scarcity. Outcomes highlight that local models (using only their private information) achieve a lower accuracy than the collaborative privacy-enhancing model.
-
Irrelevant data (that can be, e.g., aberrant or inaccurate) can be filtered out through the implementation of additional neural layers based on attention mechanisms. Applied to the Belgian system imbalance data, the outcomes suggest that these additional layers do not hinder the prediction performance of the forecasting model, while providing interpretable insights to the user.
-
A generative methodology based on the nearest neighbour algorithm is developed for generating representative minute-wise imbalance system scenarios. This approach, with its detailed temporal granularity, enables the monitoring of various ramping behaviors of balancing products within the economic dispatch of frequency reserves.
-
Taking advantage of recent developments in formal explainable AI using boosted regression trees, the usefulness of generating abductive explanations have been demonstrated for the forecasting task of Belgian imbalance prices. Formally explaining these forecasts can provide valuable insights into the factors that drive the imbalance price regime, while increasing the user’s trust on the prediction.
WP2
Novel approaches and models are developed based on the latest advances in the field of Machine Learning and Data Science to improve the modelling of the cross-border exchange capacities in the European Resource Adequacy Assessments. Focusing on the two-step Clustering-Correlation methodology currently used by the European TSOs to model the Flow-Based domains (representing the cross-border exchange capacities) in adequacy assessments, the following methodological suggestions and improvements are developed within the WP2:
-
New clustering algorithms (from K-Medoids or Fuzzy clustering family) are proposed to improve the performance of the classical Partitioning Around Medoids method used by the TSOs.
-
A new distance measure tailored for the clustering of Flow-Based domains for adequacy assessments is developed that focuses on the extreme vertices of the Flow-Based domains. It is shown that the so-called Goal-Oriented clustering of Fow-Based domains can enhance the accuracy, scalability, and calculation burden of the classical Flow-Based clustering technique used by the TSOs that relies on the overall shape of the Flow-Based domains.
-
A novel Supervised Learning-based model is proposed that can link several input features of the Flow-Based market (for instance, zonal load and generation data) to the appropriate cluster prototype. In an extensive numerical simulation, it is confirmed that the proposed classification-based model (integrating several different techniques such as Support Vector Machines, Random Forest, K-Nearest Neighbours, etc.) can outperform the classical correlation study utilized by the TSOs thanks to its capability to handle several input features and by capturing the important trends and correlations present in the Flow-Based market.
-
An innovative alternative to the classical two-step methodology utilized by the TSOs is proposed. Relying on the K-Nearest Neighbours algorithm, it finds the most similar Flow-Based domain from the available dataset without performing the clustering study. The simulation results confirm the superior performance of the proposed single-step approach in terms of accuracy, scalability, and computational complexity, compared to all combinations of the two-step techniques.
-
A salient feature of the proposed single-step method resides in the fact that it does require to extract the vertices of the Flow-Based domains (while the vertices of the Flow-Based domains are needed for clustering study within the two-step methodology). Given the extension of the Flow-Based Market Coupling in Europe, the increased dimensionality of Flow-Based domains poses extreme difficulties for vertex enumeration operation to extract the vertices of Flow-Based domains.
WP3
Given the growing complexity of keeping real-time balance in renewable-dominated power systems, the need for innovative balancing solutions is becoming more critical. Within this framework, WP3 has made the following major contributions:
-
Innovative machine learning architectures have been developed for making probabilistic predictions of imbalances within power systems. In particular, by integrating downstream decision-making processes directly into the training phase, we have significantly enhanced the prediction accuracy. This Decision-Focused Learning approach ensures model quality is optimized in areas where it most substantially influences decision-making outcomes.
-
A stochastic Model Predictive Control methodology has been defined, enabling a strategic and potentially risk-averse market actor to adopt real-time out-of-balance positions. This approach leverages imbalance predictions within a European-style balancing market, enhancing the participant's ability to make informed decisions that support system stability.
-
A new policy for the proactive activation of manual reserves in balancing markets has been proposed, aiming at finding an optimal trade-off between different (aFRR and mFRR) products via a stochastic optimization method. This formulation enables lowering the average costs of balancing activation, a crucial advantage in today's context where the demand for balancing actions is rising.
WP4
The growing integration of distributed energy resources has led to an increased volatility in active and reactive power exchanges at the interfaces between the transmission and distribution systems, rendering these power exchanges hardly predictable, which implies financial (e.g., penalties) and technical (e.g., voltage and congestion issues) consequences for both TSOs and DSOs. In a long-term perspective, the reduced predictability of power exchanges can lead to nonoptimal investments on reactive power/voltage control assets.
Novel algorithms and techniques are firstly developed in WP4 to characterize and model the behavior and future evolution of power exchanges at the interface between the transmission and distribution systems (VARLearn). The extracted characteristics and captured trends by the developed VARLearn model are then utilized in order to define the optimal investment plan of the future power grid (VARPlan).
-
With regard to the VARLearn task, a Physics-Informed Machine Learning (PIML) model is designed to enhance the prediction of power exchanges at transmission-distribution interfaces. The PIML model leverages the proposed Inverse Load Flow formulation (defining an equivalent model of the distribution network), providing valuable physical insights to complement the employed data-driven module. The proposed PIML model leads to superior results in terms of the statistical metrics (e.g., NRMSE) as well as the technical indices (such as the redispatch costs of the power grid), when compared to the classical fully data-driven methods.
-
Concerning the VARPlan task, a novel probabilistic framework relying on the Optimal Power Flow algorithm and Monte Carlo simulations is developed in order to determine the optimal investment plan of the future power grid taking into account the uncertainty and the future evolution of the power exchanges at the interfaces between the transmission and distribution systems. Through a comprehensive study, it is concluded that an enhanced level of coordination between the TSO and DSO can lead to reduced investment costs thanks to a more optimal allocation of control assets within both distribution and transmission networks.