KI:STE Research

Earth Science meets AI: Research within KI:STE

Cloud systems | Renewable Energy | Regional Climate - Dwaipayan Chatterjee

The 21st century grapples with crucial challenges—transitioning to renewable energy and combating climate change. A responsible society anticipates sustainable solutions for both. Another term in the discourse is Artificial Intelligence (AI), with considerable speculation. Delving into all three areas, Dwaipayan Chatterjee, while working in Prof. Crewell’s AWARES team, uncovered significant potential for future interdisciplinary research.

Cloud systems, crucial for solar energy, modulate the solar radiation budget. They impact heat and moisture distribution in climate change. However, uncertainties persist about their physical properties and connections at different scales.

Satellites capture cloud patterns, but labeling for traditional neural networks is challenging. From an AI perspective, my research prioritizes unlabeled learning, allowing the neural network learns to from scratch. This open up understanding the underlying representation of satellite observations and allows extracting features for a data-driven comprehension of regional climate and its application in solar energy production.

Classification by the self-supervised deep neural network at k = 7 for 128 x 128 configuration. Thirty random samples were selected over central Europe and visualized for each class. Each image is assigned a unique class through a colored frame. The bar charts in the lower part of a) and b) represent the number of images in relative percentage and cloud fraction in each regime. To better associate the class number with the cloud regime, the centroid image is shown as well. 
Feature space with wet-dry conditions defined based on 80/20th percentile of integrated water vapor and w-500 wind speed from ERA-5. Grey defines dry conditions, blue is wet conditions, and light grey is the intermediate state, dry/wet intrusion as decided based on the 30 nearest neighbors. Average profiles of vertical wind speed W, and sea surface temperature (SST) for the reference states, intrusions, and their 30 nearest opposite neighbors.

Terrestrial Protection - Timo Stomberg

Anthropogenic land use has a huge impact on (vegetational) ecology, mostly with a negative effect on plant health and development. However, there is also an increasing effort to protect naturalness by protecting areas and managing them to slow down climate change, conserve local biodiversity, and retain habitats. Therefore, mapping naturalness and human influence is an important task and we developed a machine learning technique to do so. Neural networks consist of neurons that are activated during calculation. These activations can be analyzed to understand the decision process of the neural network. We have evaluated their connection to natural and anthropogenic characteristics in landscapes and established a linkage between activations and attributions. An attribution describes the effect an activation has on the model’s prediction. Our method allows us to recognize complex patterns in unseen test data and evaluate their influence on the model’s decision. Harmonizing the attributions, large-scale scenes and scenes at different points in time become comparable and can be evaluated (Stomberg et al., 2023). We further invented a neural network architecture that allows us to get high-level features with the resolution of the input image. With standard convolutional neural network architectures the user has to decide between resolution and complexity of the features when applying attribution methods. 

 

From left to right: (1) For demonstration, the satellite image has been split into four separate images. (2) An attribution method applied to the input layer gives rarely meaningful results. (3) Applied to the last convolutional layer, the result is meaningful, however, the resolution is low. (4) With our neural network architecture, we generate meaningful attributions in high-resolution. (5) Harmonizing these attributions, all images become comparable. (6) Data not well represented in the training data can be grayed out.

Timo Stomberg is pursuing his PhD at the Institute of Geodesy and Geoinformation, University of Bonn, under the supervision of Prof. Dr.-Ing. Ribana Roscher. Throughout his involvement in the KI:STE project, he has authored two peer-reviewed articles as the lead author and collaborated on four other peer-reviewed publications. He has showcased his research findings at three conferences.

Model Error Correction Kaveh P. Yousefi

This study employs Convolutional Neural Networks (CNNs), to correct errors in precipitation simulations of atmospheric models, crucial for assessing hydrological extremes. FIGURE 4 displays significant improvements achieved through DL-based corrections on high-resolution (H-RES) precipitation data using H-SAF satellite observations across seasons. Positive enhancements in mean error (ME), root mean squared error (RMSE), and Pearson correlation coefficient (COR) are observed compared to reference data (H-SAF), offering insights into model performance under diverse weather conditions. Detailed in our forthcoming publication, our methodology involves training CNNs, correcting model-based simulations, and evaluating impacts on soil moisture and runoff simulations. This research contributes to advancing atmospheric simulations with implications for weather forecasting, hydrology, and Earth systems modeling.

Figure 4

Landcover and Crop Classification - Ankit Patnala

The KISTE project is driven by a core vision to explore the application of machine learning in Earth Science. Our team focused on investigating relationship between plant condition and air quality using existing self-supervised techniques i.e. to learn without labels. Recognizing the limitations of these techniques in detecting vegetation details and changes over time, we narrowed our focus to mapping crops from Earth Observation (EO) imagery.

Contrastive learning, a self-supervised learning method, has emerged as a superior approach for natural images. Drawing inspiration from it, our earlier work concentrated on developing an approach to efficiently apply contrastive learning to remote sensing images. Contrastive methods relies on contrastive losses and meaningful transformations. The exisitng methods developed for natural images such as in Imagenet, uses color jittering and grayscaling as one of the important transformation. The naive extension of color jittering and grayscaling to multiple channels of multispectral Sentinel2 image is physically insignificant. However, these channels contain vital information about the Earth’s surface, such as NIR reflections that aid in identifying vegetation states. Our research focused on developing a meaningful alternative transformation. Our research focused on developing atmospheric transformation, a transformation method based on well established atmpspheric correction method. We interpolated between atmospheric corrected and uncorrected images to obtain multiple views of the same image. In principle, this methods can be extended to all the channels of the multi-spectral image but we restricted our anaysis only on 4 channels i.e. NIR channel along with RGB channels. Further details can be found in our paper published in IEEE GRSL.

The subsequent task, aligning with our primary goal of crop mapping, involves extending the concept of contrastive learning to crops. Crops are typically analyzed with the temporal signature of different channels. Crop data are associated either with a field parcel or set of pixels from the same field parcel. The task of developing a meaningful transformation for pixels is non-trivial, and rather than relying solely on transformations, we employed multiple sources, specifically reflections measured by publicly available Sentinel2 and commercial Planetscope satellite missions. We designed a setup for obtaining representation using multi-modal contrastive learning and defined how to use the obtained representation to enhance crop mapping. Our self-supervised learning strategy involves aligning the spatial components of both satellite missions. The preprint of our work is currently available for further reference.

Multi-modal contrastive learning using SimCLR loss. The left selected pixels in red boxes show the corresponding data pair selected for pre-training. SCARF is the feature transformation applied on these data samples. On the right side, it shows how the alignment and uniformity is learning by optimizing SimCLR loss.

Shallow Landslides - Ann-Kathrin Edrich

The project conducted at the Chair of Methods for Model-based Development in Computational Engineering at RWTH Aachen University, focused on developing a generic, open-access framework for landslide susceptibility and hazard mapping (https://doi.org/mf3t). Built around a Random Forest classifier, this framework offers generic implementations of key mapping steps, from conceptualisation and study design to input dataset generation and map creation. Each step is accompanied by appropriate validation methods, ensuring accuracy and reliability.

The workflow aims to simplify and standardise future mapping studies while allowing for flexibility and customisation. A test scenario in Graubünden, Switzerland, highlighted the importance of quality training data for reliable mapping results, with findings set to be shared in an upcoming publication (https://doi.org/mf2f).

Furthermore, the project addressed time-dependent hazard mapping by analysing decades of rainfall data. Time-dependent mapping allows for real-time assessment of susceptibility, adaptability to predicted triggering events, accommodating variations in spatial distribution of landslide susceptibility based on event type and magnitude.

As the project nears completion, efforts are focused on finalising the time-dependent mapping effort and publishing the insights gained.

Edrich, A. K., Yildiz, A., Roscher, R., & Kowalski, J. (2023). A modular framework for FAIR shallow landslide susceptibility mapping based on machine learning [PREPRINT]. https://doi.org/10.21203/rs.3.rs-3254996/v2

Universality in Deep Learnning - research by Thomas Seidler

The Ising model from statistical physics is a powerful tool for modeling complex systems with interacting components. We apply the Ising model to the training of deep neural networks to explore the use of the Ising model for estimating the amount of data needed to train a certain architecture and finding universality in deep learning. One of the challenges in training deep neural networks is determining the optimal number of samples needed to achieve good performance. By using the Ising model, the research will attempt to estimate the minimum number of samples needed to train a given architecture. This will help to reduce the amount of time and resources needed for training, as well as improve the efficiency of deep learning algorithms. In addition, the research will explore the concept of universality in deep learning, which refers to the idea that certain properties of deep neural networks are independent of the specific architecture or dataset. The Ising model provides a framework for investigating universality in deep learning by modeling the interactions between neurons in a deep neural network. Overall, this research aims to contribute to the development of more efficient and effective deep learning algorithms by leveraging the Ising model from statistical physics.