End-to-end solar nowcasting embedded pipeline within the Girasol Machine, based on computer vision to fuse multi-sensor data for very short-term solar radiation forecasting
This project develops an end-to-end AI/ML framework for solar nowcasting (very short-term forecasting) using a real-time stream of infrared sky images from the Girasol system, enabling continuous extraction of cloud features and dynamic updating of intra-hour forecasts.
End-to-end ML/AI pipeline for solar nowcasting using real-time infrared sky imaging and multi-sensor data fusion.
The embedded architecture comprises three learning modules.
Module 1 (Physics-informed learning): Estimates and visualizes wind velocity fields from image streams, enabling prediction of cloud motion and Sun-occlusion events (Terrén-Serrano & Martínez-Ramón, 2021).
Module 2 (Bayesian learning): Detects clouds and associates each cloud to an estimated wind flow using mixture models in a Hidden Markov Chain (Terrén-Serrano & Martínez-Ramón, 2023).
Module 3 (Probabilistic deep learning): Fuses multi-sensor data—including sky images, weather station measurements, and pyranometer data to generate probabilistic forecasts of solar irradiance over 1–15 minute horizons (Terrén-Serrano & Martínez-Ramón, 2023).
Forecasting event (bright green), very short-term probabilistic functional forecast (green), partially observed solar radiation (black), ground-truth unobserved radiation (red), persistence forecast (gray), and deterministic functional forecast orange.
Together, these modules form a unified computational framework for accurate, real-time solar forecasting to support reliable power system operations under uncertainty.
This work is part of my Ph.D. dissertation at the University of New Mexico.
References
2023
JOURNAL
Detection of clouds in multiple wind velocity fields using ground-based infrared sky images
To improve the performance of an intra-hour global solar irradiance forecasting algorithm, it is important to detect multiple layers of clouds. Horizontal atmospheric wind shear causes wind velocity fields to have different directions and speeds. In images of clouds acquired using ground-based sky imagers, clouds may be moving in different wind layers. The information provided by a solar forecasting algorithm is necessary to optimize and schedule the solar generation resources and storage devices in a smart grid. This investigation studies the performance of unsupervised learning techniques when detecting the number of cloud layers in infrared sky images. The images are acquired using an innovative infrared sky imager mounted on a solar tracker. Different mixture models are used to infer the distribution of the cloud features. Multiple Bayesian metrics and a sequential hidden Markov model are implemented to find the optimal number of clusters in the mixture models, and their performances are compared. The motion vectors are computed using a probabilistic implementation of the Lucas-Kanade algorithm. The correlations between the cloud motion vectors and temperatures are analyzed to discover the method that leads to the most accurate results. The findings point that a sequential hidden Markov model outperforms the detection accuracy of standard Bayesian model selection metrics.
@article{TERRENSERRANO2023110628,title={Detection of clouds in multiple wind velocity fields using ground-based infrared sky images},journal={Knowledge-Based Systems},volume={274},pages={110628},year={2023},issn={0950-7051},doi={https://doi.org/10.1016/j.knosys.2023.110628},author={Terrén-Serrano, Guillermo and Martínez-Ramón, Manel},keywords={Cloud detection, Hidden Markov model, Mixture models, Sky imaging, Weighted Lucas-Kanade},}
JOURNAL
Deep learning for intra-hour solar forecasting with fusion of features extracted from infrared sky images
The increasing penetration of solar energy leaves power grids vulnerable to fluctuations in the solar radiation that reaches the surface of the Earth due to the projection of cloud shadows. Therefore, an intra-hour solar forecasting algorithm is necessary to reduce power instabilities caused by the impact of moving clouds on energy generation. The most accurate intra-hour solar forecasting methods apply convolutional neural networks to a series of visible light sky images. Instead, this investigation uses data acquired by a novel infrared sky imager on a solar tracker, which is capable of maintaining the Sun in the center of the images throughout the day and, at the same time, reducing the scattering effect produced by the Sun’s direct radiation. In addition, infrared sky images allow the derivation and extraction of physical cloud features. The cloud dynamics are analyzed in sequences of images to compute the probability of the Sun intercepting air parcels in the sky images (i.e., voxels). The method introduced in this investigation fuses sky condition information from multiple sensors (i.e., pyranometer, sky imager, solar tracker, weather station) and feature sources using a multi-task deep learning architecture based on recurrent neural networks. The proposed deterministic and Bayesian architectures reduce computation time by avoiding convolutional filters. The proposed intra-hour solar forecasting algorithm reached a forecast skill of 18.6% with a forecasting horizon of 8 min. Consequently, the proposed intra-hour solar forecasting method can potentially reduce the operational costs of power grids with high participation of solar energy.
@article{TERRENSERRANO202342,title={Deep learning for intra-hour solar forecasting with fusion of features extracted from infrared sky images},journal={Information Fusion},volume={95},pages={42-61},year={2023},issn={1566-2535},doi={https://doi.org/10.1016/j.inffus.2023.02.006},author={Terrén-Serrano, Guillermo and Martínez-Ramón, Manel},keywords={Bayesian networks, Bayesian optimization, Deep learning, Girasol dataset, Solar forecasting, Sky imaging},}
2021
JOURNAL
Multi-layer wind velocity field visualization in infrared images of clouds for solar irradiance forecasting
The energy available in a solar energy powered grid is uncertain due to the weather conditions at the time of generation. Forecasting global solar irradiance could address this problem by providing the power grid with the capability of scheduling the storage and dispatch of energy. The occlusion of the Sun by clouds is the main cause of instabilities in the generation of solar energy. This investigation proposes a method to visualize the wind velocity field in sequences of longwave infrared images of clouds when there are multiple wind velocity fields in an image. This method can be used to forecast the occlusion of the Sun by clouds, providing stability in the generation of solar energy. Unsupervised learning is implemented to infer the distribution of the clouds’ velocity vectors and heights in multiple wind velocity fields in an infrared image. A multi-output weighted support vector machine with flow constraints is used to extrapolate the wind velocity fields to the entire frame, visualizing the path of the clouds. The proposed method is capable of approximating the wind velocity field in a small air parcel using the velocity vectors and physical features of clouds extracted from infrared images. Assuming that the streamlines are pathlines, the visualization of the wind velocity field can be used for forecasting cloud occlusions of the Sun. This is of importance when considering ways of increasing the stability of solar energy generation.
@article{TERRENSERRANO2021116656,title={Multi-layer wind velocity field visualization in infrared images of clouds for solar irradiance forecasting},journal={Applied Energy},volume={288},pages={116656},year={2021},issn={0306-2619},doi={https://doi.org/10.1016/j.apenergy.2021.116656},author={Terrén-Serrano, Guillermo and Martínez-Ramón, Manel},keywords={Cloud tracking, Machine learning, Flow visualization, Beta mixture model, Sky imaging, Solar forecasting},}
