Earth Science meets AI: Research within KI:STE
Cloud systems | Renewable Energy | Regional Climate - Dwaipayan Chatterjee
From an energy perspective, cloud systems play a significant role in modulating the solar radiation budget and the availability of solar radiation on the ground. They thus are of high interest for renewable energy applications. From a climate change perspective, they influence the distribution of heat and moisture around the planet. Significant uncertainty exists about their physical characteristics and missing links across various spatial and temporal scales. Satellites capture a field of clouds, and very often similar, looking individual clouds form neighbors of each other, giving rise to spatial patterns. But from a satellite and AI perspective, we may have many cloud system observations; still, they need to be labeled/segmented to train a conventional supervised neural network. Therefore, from an AI point of view, my research focuses on learning without labels and letting the neural network learn from scratch, understand the underlying representation of satellite observations, and focus on extracting meaningful features which can be used further for data-driven process understanding of regional climate and exploit its usage for solar energy power production.
Left: Classification by the self-supervised deep neural network at k = 7 for 128 x 128 configuration. Thirty random samples were selected over central Europe and visualized for each class. Each image is assigned a unique class through a colored frame. The bar charts in the lower part of a) and b) represent the number of images in relative percentage and cloud fraction in each regime. To better associate the class number with the cloud regime, the centroid image is shown as well.
Right: Feature space with wet-dry conditions defined based on 80/20th percentile of integrated water vapor and w-500 wind speed from ERA-5. Grey defines dry conditions, blue is wet conditions, and light grey is the intermediate state, dry/wet intrusion as decided based on the 30 nearest neighbors. Average profiles of vertical wind speed W, and sea surface temperature (SST) for the reference states, intrusions, and their 30 nearest opposite neighbors.
Terrestrial Protection - Timo Stomberg
Anthropogenic land use has a huge impact on (vegetational) ecology, mostly with a negative effect on plant health and development. However, there is also an increasing effort to protect naturalness by protecting areas and managing them to slow down climate change, conserve local biodiversity, and retain habitats. Therefore, mapping naturalness and human influence is an important task and we developed a machine learning technique to do so. Neural networks consist of neurons that are activated during calculation. These activations can be analyzed to understand the decision process of the neural network. We have evaluated their connection to natural and anthropogenic characteristics in landscapes and established a linkage between activations and attributions. An attribution describes the effect an activation has on the model’s prediction. Our method allows us to recognize complex patterns in unseen test data and evaluate their influence on the model’s decision. Harmonizing the attributions, large-scale scenes and scenes at different points in time become comparable and can be evaluated (Stomberg et al., 2023). We further invented a neural network architecture that allows us to get high-level features with the resolution of the input image. With standard convolutional neural network architectures the user has to decide between resolution and complexity of the features when applying attribution methods.
Model Error Correction Kaveh P. Yousefi
This study employs Convolutional Neural Networks (CNNs), to correct errors in precipitation simulations of atmospheric models, crucial for assessing hydrological extremes. FIGURE 4 displays significant improvements achieved through DL-based corrections on high-resolution (H-RES) precipitation data using H-SAF satellite observations across seasons. Positive enhancements in mean error (ME), root mean squared error (RMSE), and Pearson correlation coefficient (COR) are observed compared to reference data (H-SAF), offering insights into model performance under diverse weather conditions. Detailed in our forthcoming publication, our methodology involves training CNNs, correcting model-based simulations, and evaluating impacts on soil moisture and runoff simulations. This research contributes to advancing atmospheric simulations with implications for weather forecasting, hydrology, and Earth systems modeling.
Landcover and Crop Classification - Ankit Patnala
The KISTE project is driven by a core vision to explore the application of machine learning in Earth Science. Our team focused on investigating relationship between plant condition and air quality using existing self-supervised techniques i.e. to learn without labels. Recognizing the limitations of these techniques in detecting vegetation details and changes over time, we narrowed our focus to mapping crops from Earth Observation (EO) imagery.
Contrastive learning, a self-supervised learning method, has emerged as a superior approach for natural images. Drawing inspiration from it, our earlier work concentrated on developing an approach to efficiently apply contrastive learning to remote sensing images. Contrastive methods relies on contrastive losses and meaningful transformations. The exisitng methods developed for natural images such as in Imagenet, uses color jittering and grayscaling as one of the important transformation. The naive extension of color jittering and grayscaling to multiple channels of multispectral Sentinel2 image is physically insignificant. However, these channels contain vital information about the Earth’s surface, such as NIR reflections that aid in identifying vegetation states. Our research focused on developing a meaningful alternative transformation. Our research focused on developing atmospheric transformation, a transformation method based on well established atmpspheric correction method. We interpolated between amtospheric corrected and uncorrected images to obtain multiple views of the same image. In principle, this methods can be extended to all the channels of the multi-spectral image but we restricted our anaysis only on 4 channels i.e. NIR channel along with RGB channels. Further details can be found in our paper published in IEEE GRSL.
The subsequent task, aligning with our primary goal of crop mapping, involves extending the concept of contrastive learning to crops. Crops are typically analyzed with the temporal signature of different channels. Crop data are associated either with a field parcel or set of pixels from the same field parcel. The task of developing a meaningful transformation for pixels is non-trivial, and rather than relying solely on transformations, we employed multiple sources, specifically reflections measured by publicly available Sentinel2 and commercial Planetscope satellite missions. We designed a setup for obtaining representation using multi-modal contrastive learning and defined how to use the obtained representation to enhance crop mapping. Our self-supervised learning strategy involves aligning the spatial components of both satellite missions. The preprint of our work is currently available for further reference.
Shallow Landslides - Ann-Kathrin Edrich
Shallow landslides are a common natural hazard in mountainous regions, and their prediction is crucial for mitigating risks to infrastructure and communities. We aim to create static and dynamic hazard maps for shallow landslides in Switzerland using only openly available data and a random forest machine learning algorithm. The research will leverage a range of openly available data sources, including digital elevation models, soil information, and climate data, to create a comprehensive dataset for training a random forest model. The model will be trained using historical landslide data from Switzerland and validated using cross-validation techniques. Static hazard maps will identify areas that are susceptible to landslides based on a range of static factors, such as topography and soil properties. Dynamic hazard maps will take into account additional factors such as rainfall patterns and soil moisture to identify areas that are at higher risk of landslides in real-time. To validate the accuracy of the hazard maps, the research will compare the predicted hazard zones with historical landslide events in Switzerland. The research will also investigate the sensitivity of the hazard maps to different factors, such as the spatial resolution of the input data and the number of trees in the random forest model. The resulting maps will provide valuable information for decision-makers and stakeholders involved in land use planning and disaster risk reduction.
Universality in Deep Learnning - research by Thomas Seidler
The Ising model from statistical physics is a powerful tool for modeling complex systems with interacting components. We apply the Ising model to the training of deep neural networks to explore the use of the Ising model for estimating the amount of data needed to train a certain architecture and finding universality in deep learning. One of the challenges in training deep neural networks is determining the optimal number of samples needed to achieve good performance. By using the Ising model, the research will attempt to estimate the minimum number of samples needed to train a given architecture. This will help to reduce the amount of time and resources needed for training, as well as improve the efficiency of deep learning algorithms. In addition, the research will explore the concept of universality in deep learning, which refers to the idea that certain properties of deep neural networks are independent of the specific architecture or dataset. The Ising model provides a framework for investigating universality in deep learning by modeling the interactions between neurons in a deep neural network. Overall, this research aims to contribute to the development of more efficient and effective deep learning algorithms by leveraging the Ising model from statistical physics.
