Earth Science meets AI: Research within KI:STE
Cloud systems | Renewable Energy | Regional Climate - Dwaipayan Chatterjee
The 21st century grapples with crucial challenges—transitioning to renewable energy and combating climate change. A responsible society anticipates sustainable solutions for both. Another term in the discourse is Artificial Intelligence (AI), with considerable speculation. Delving into all three areas, Dwaipayan Chatterjee, while working in Prof. Crewell’s AWARES team, uncovered significant potential for future interdisciplinary research.
Cloud systems, crucial for solar energy, modulate the solar radiation budget. They impact heat and moisture distribution in climate change. However, uncertainties persist about their physical properties and connections at different scales.
Satellites capture cloud patterns, but labeling for traditional neural networks is challenging. From an AI perspective, my research prioritizes unlabeled learning, allowing the neural network learns to from scratch. This open up understanding the underlying representation of satellite observations and allows extracting features for a data-driven comprehension of regional climate and its application in solar energy production.
Terrestrial Protection - Timo Stomberg
Anthropogenic land use has a huge impact on (vegetational) ecology, mostly with a negative effect on plant health and development. However, there is also an increasing effort to protect naturalness by protecting areas and managing them to slow down climate change, conserve local biodiversity, and retain habitats. Therefore, mapping naturalness and human influence is an important task and we developed a machine learning technique to do so. Neural networks consist of neurons that are activated during calculation. These activations can be analyzed to understand the decision process of the neural network. We have evaluated their connection to natural and anthropogenic characteristics in landscapes and established a linkage between activations and attributions. An attribution describes the effect an activation has on the model’s prediction. Our method allows us to recognize complex patterns in unseen test data and evaluate their influence on the model’s decision. Harmonizing the attributions, large-scale scenes and scenes at different points in time become comparable and can be evaluated (Stomberg et al., 2023). We further invented a neural network architecture that allows us to get high-level features with the resolution of the input image. With standard convolutional neural network architectures the user has to decide between resolution and complexity of the features when applying attribution methods.
Timo Stomberg is pursuing his PhD at the Institute of Geodesy and Geoinformation, University of Bonn, under the supervision of Prof. Dr.-Ing. Ribana Roscher. Throughout his involvement in the KI:STE project, he has authored two peer-reviewed articles as the lead author and collaborated on four other peer-reviewed publications. He has showcased his research findings at three conferences.
Model Error Correction Kaveh P. Yousefi
This study employs Convolutional Neural Networks (CNNs), to correct errors in precipitation simulations of atmospheric models, crucial for assessing hydrological extremes. FIGURE 4 displays significant improvements achieved through DL-based corrections on high-resolution (H-RES) precipitation data using H-SAF satellite observations across seasons. Positive enhancements in mean error (ME), root mean squared error (RMSE), and Pearson correlation coefficient (COR) are observed compared to reference data (H-SAF), offering insights into model performance under diverse weather conditions. Detailed in our forthcoming publication, our methodology involves training CNNs, correcting model-based simulations, and evaluating impacts on soil moisture and runoff simulations. This research contributes to advancing atmospheric simulations with implications for weather forecasting, hydrology, and Earth systems modeling.
Landcover and Crop Classification - Ankit Patnala
The KISTE project is driven by a core vision to explore the application of machine learning in Earth Science. Our team focused on investigating relationship between plant condition and air quality using existing self-supervised techniques i.e. to learn without labels. Recognizing the limitations of these techniques in detecting vegetation details and changes over time, we narrowed our focus to mapping crops from Earth Observation (EO) imagery.
Contrastive learning, a self-supervised learning method, has emerged as a superior approach for natural images. Drawing inspiration from it, our earlier work concentrated on developing an approach to efficiently apply contrastive learning to remote sensing images. Contrastive methods relies on contrastive losses and meaningful transformations. The exisitng methods developed for natural images such as in Imagenet, uses color jittering and grayscaling as one of the important transformation. The naive extension of color jittering and grayscaling to multiple channels of multispectral Sentinel2 image is physically insignificant. However, these channels contain vital information about the Earth’s surface, such as NIR reflections that aid in identifying vegetation states. Our research focused on developing a meaningful alternative transformation. Our research focused on developing atmospheric transformation, a transformation method based on well established atmpspheric correction method. We interpolated between atmospheric corrected and uncorrected images to obtain multiple views of the same image. In principle, this methods can be extended to all the channels of the multi-spectral image but we restricted our anaysis only on 4 channels i.e. NIR channel along with RGB channels. Further details can be found in our paper published in IEEE GRSL.
The subsequent task, aligning with our primary goal of crop mapping, involves extending the concept of contrastive learning to crops. Crops are typically analyzed with the temporal signature of different channels. Crop data are associated either with a field parcel or set of pixels from the same field parcel. The task of developing a meaningful transformation for pixels is non-trivial, and rather than relying solely on transformations, we employed multiple sources, specifically reflections measured by publicly available Sentinel2 and commercial Planetscope satellite missions. We designed a setup for obtaining representation using multi-modal contrastive learning and defined how to use the obtained representation to enhance crop mapping. Our self-supervised learning strategy involves aligning the spatial components of both satellite missions. The preprint of our work is currently available for further reference.
Shallow Landslides - Ann-Kathrin Edrich
The project conducted at the Chair of Methods for Model-based Development in Computational Engineering at RWTH Aachen University, focused on developing a generic, open-access framework for landslide susceptibility and hazard mapping (https://doi.org/mf3t). Built around a Random Forest classifier, this framework offers generic implementations of key mapping steps, from conceptualisation and study design to input dataset generation and map creation. Each step is accompanied by appropriate validation methods, ensuring accuracy and reliability.
The workflow aims to simplify and standardise future mapping studies while allowing for flexibility and customisation. A test scenario in Graubünden, Switzerland, highlighted the importance of quality training data for reliable mapping results, with findings set to be shared in an upcoming publication (https://doi.org/mf2f).
Furthermore, the project addressed time-dependent hazard mapping by analysing decades of rainfall data. Time-dependent mapping allows for real-time assessment of susceptibility, adaptability to predicted triggering events, accommodating variations in spatial distribution of landslide susceptibility based on event type and magnitude.
As the project nears completion, efforts are focused on finalising the time-dependent mapping effort and publishing the insights gained.
Universality in Deep Learnning - research by Thomas Seidler
The Ising model from statistical physics is a powerful tool for modeling complex systems with interacting components. We apply the Ising model to the training of deep neural networks to explore the use of the Ising model for estimating the amount of data needed to train a certain architecture and finding universality in deep learning. One of the challenges in training deep neural networks is determining the optimal number of samples needed to achieve good performance. By using the Ising model, the research will attempt to estimate the minimum number of samples needed to train a given architecture. This will help to reduce the amount of time and resources needed for training, as well as improve the efficiency of deep learning algorithms. In addition, the research will explore the concept of universality in deep learning, which refers to the idea that certain properties of deep neural networks are independent of the specific architecture or dataset. The Ising model provides a framework for investigating universality in deep learning by modeling the interactions between neurons in a deep neural network. Overall, this research aims to contribute to the development of more efficient and effective deep learning algorithms by leveraging the Ising model from statistical physics.