This website uses cookies that are necessary to deliver an enjoyable experience and ensure
its correct functionality and cannot be turned off. Optional cookies are used to improve the page with analytics, by
clicking “Yes, I accept” you consent to this use of cookies. Learn more
Elefridge.jl: Compressing atmospheric data into its real information content
Weather and climate forecasting centres worldwide produce very large amounts of data that has to be stored and shared with users. Data compression is essential to reduce file sizes sent over the internet and the demand on data archive capacity.
The previously completed challenge within the ESoWC 2020 developed the concept of information-preserving compression by analysing the real information content in data from the Copernicus Atmospheric Monitoring Service (CAMS). Separating the false and hardly compressible information from the real information was shown to allow for high compression factors without significant information loss.
Here, we focus on further details in the implementation of information-preserving compression for CAMS. Readily available in the current GRIB2 compression are different precision and accuracy options that can be translated to preserved information for a given data set. To implement this successfully and in an automated fashion, further improvements are necessary and the best lossless compressor available in GRIB2 that satisfies both speed and size requirements has to be found.
This project aims to successfully implement information-preserving compression for CAMS to put this advanced compression technique into practice.
At its core, ECMFW is a data organization that produces and distributes essential weather data to its member states and outside businesses. They also provide various other services such as global forecasting, supercomputing facilities, environmental services, meteorological services.
Many users around the world use these services. In this project, I aim to improve the user experience with ECMWF by providing individual users with their own dashboard showcasing valuable data, favourite charts, and a high-level overview of their relationship with ECMWF and its services.
Nowadays it is possible to obtain atmospheric composition datasets for the same locations from different sources. However, most datasets are not easily comparable due to their file formats and structure.
In this regard, Atmospheric Datasets Comparison (ADC) Toolbox is aimed to have a set of tools that allows unit conversion, side-by-side visual comparison, regridding, time and geographic data aggregation and statistics visualization to show how similar the datasets are among them.
The toolbox will consist of different scripts written in Jupyter Notebooks with the tools:
- Transformation: The datasets to be compared will be transformed into a common format.
- Merge: The files will be regridded and, if needed, its units will be converted, to combine them.
- Comparison: Statistics methods will be used to show information about the datasets.
- Visualization: The merge output will be seen side-by-side in tables and maps.
- File format change: The files in a common format will be given in any desired file format.
Bias correction of CAMS model forecasts for air quality variables by using in-situ observations. The bias correction algorithm will mainly be based on machine-learning / deep-learning techniques.
ML4Land: Using Earth's observation data, Climate reanalysis
& Machine Learning to detect Earth’s heating patterns
Skin temperature has been pivotal in identifying the heating and land-use patterns of Earth. The project aims to learn a mapping from model simulations (using ERA5) to satellite observations of skin temperature. Various works have shown how Machine Learning based models can efficiently recognize and learn useful patterns from complex datasets. We thus aim to use Machine Learning algorithms to learn the mapping between ERA5 variables and satellite observations of maximal skin temperature. These solutions will provide predictions at higher resolutions and offer valuable insights into the relationships between skin temperature and various ERA5 variables.
MaLePoM (Machine Learning for Pollution Monitoring)
The project aims to build a Machine Learning model to estimate emissions using suitable proxy data due to anthropogenic activities. Initially, we will model the concentrations of NOx in Europe. Therefore, proxy data should frame these activities exploiting databases such as Land cover maps, Dynamic traffic data, lighthing data and others.
Subsequently, different approach will be tested in order to capture both spatial and temporal variability at high resolution and eventually allow accurate emissions estimates at global scales.
CliMetLab - Machine Learning on weather and climate data
CliMetLab is a Python package aiming at simplifying access to climate and meteorological datasets, allowing users to focus on science instead of technical issues such as data access and data formats.
This project aims at handling the data loading as well as interpreting the output from the machine learning models with the use of plots, graphs, etc. This will remove the overhead of manual data retrieval, writing specific data loaders per dataset.
The plugin architecture in CliMetLab aims at easy addition of data sources, datasets, plotting styles and data formats.
Specific goals of the project:
1) extend CliMetLab so that it offers the user with high-level Matplotlib-based plotting functions to produce graphs and plot which are relevant to weather and climate applications.
2) Python package Intake is a lightweight set of tools for loading and sharing data in data science projects. Extend CliMetLab so that it seamlessly interfaces with Intake and allows users to access all intake-compatible datasets.
3) Xarray uses the data format Zarr to allow parallel read and parallel write. Convert large already available datasets to xarray-readable zarr format, define appropriate configuration (chunking/compression/other) according to domain use cases, develop tools to benchmark when used on a cloud-platform, compare to other formats (N5, GRIB, netCDF, geoTIFF, etc.).
Project Meeresvogel seeks to make it easier to incorporate weather visualisations into multimedia presentations. We will design and develop a Python module which enables users to create interactive Google Earth presentations which are enhanced with weather data and visualisations from MetView.
Using this module, we aim to create three examples to demonstrate how this could be useful to diverse audiences wanting to explore various aspects of the 2020/21 Vendée Globe Race, a sporting event in which 33 skippers set out to race their 60 foot yachts solo non-stop around the world.
We will explore ways in which weather visualisations can provide insights for the public following the race, the race teams wanting to analyse performance data, and scientists analysing the oceanmet observations which were collected by a number of the boats during the race.
