GitLocker: The Coding Marketplace

Description:

gribmagic 0.2.2

GribMagic - generic weather forecast downloader

Simulations are believed by no one except those
who conducted them.
Experimental results are believed by everyone except
those who conducted them.
ANONYMOUS

About
The goal of the GribMagic project is to unify the download process of
public GRIB1/GRIB2 and netCDF data from numerical weather prediction
models originating from different organizations and data providers.
This is a work in progress, as such GribMagic is currently considered to
be beta software. As this is an early-stage project, contributions are
highly appreciated.
Details
The software package includes different code bases. Both provide their features
by means of a gribmagic subcommand.

GribMagic Unity program: gribmagic unity --help
DWD GRIB Downloader program: gribmagic dwd --help

You can find corresponding invocation examples later in this document.
Setup
Install the ecCodes package by ECMWF.
# Debian Linux
apt-get install --yes libeccodes0 cdo

# macOS/Homebrew
brew install eccodes cdo

Install GribMagic Python package.
pip install gribmagic --upgrade

Run GribMagic Unity program
Ad hoc usage
# List labels of available models.
gribmagic unity list

# Acquire data.
gribmagic unity acquire --model=dwd-icon-eu --timestamp=2021-10-03T00:00:00Z --target=.gribmagic-data

Configuration
To optionally adjust configuration settings, you can define the following environment variables:
GM_DATA_PATH={PATH_TO_PROJECT}/data

GM_MODEL_CONFIG={PATH_TO_PROJECT}/config/model_config.yml"
GM_MODEL_VARIABLES_MAPPING={PATH_TO_PROJECT}/config/model_variables_mapping.yml"
GM_MODEL_VARIABLES_LEVELS_MAPPING={PATH_TO_PROJECT}/config/model_variables_levels_mapping.yml"

ECCODES_DEFINITION_PATH=/usr/share/eccodes/definitions:/usr/local/opt/eccodes/share/eccodes/definitions

Run DWD GRIB Downloader program
# Install DWD GRIB Downloader program.
gribmagic install dwd-grib-downloader

# Acquire wind-specific parameters from ICON-D2.
wget https://raw.githubusercontent.com/earthobservations/gribmagic/98da3fd4f/examples/dwd/recipe_d2_wind.py
gribmagic dwd acquire --recipe=recipe_d2_wind.py --timestamp="2021101800" --output=.gribmagic-data/raw

# Acquire assorted parameters from ICON-GLOBAL.
wget https://raw.githubusercontent.com/earthobservations/gribmagic/98da3fd4f/examples/dwd/recipe_global_assorted.py
gribmagic dwd acquire --recipe=recipe_global_assorted.py --timestamp="2021101800" --output=.gribmagic-data/raw

When omitting the --timestamp parameter, the most recent modelrun is automatically selected.
When omitting the --output parameter, it can be supplied using the GM_DATA_PATH environment variable.
In this manner, the most compact form to invoke gribmagic dwd would be something like:
export GM_DATA_PATH=.gribmagic-data/raw
gribmagic dwd acquire --recipe=recipe_d2_wind.py

Run bbox tool
Extract area of interest from GRIB files using a bounding box.
Extract subset by coordinates, a space-separated list of lat_min lat_max lon_min lon_max.
gribmagic smith bbox \
".gribmagic-data/raw/icon-d2/**/*regular-lat-lon*.grib2" \
--output=.gribmagic-data/subgrid/icon-d2 \
--bbox=46.0 47.5 14.5 16.8

Extract subset by 2-letter country name.
gribmagic smith bbox \
".gribmagic-data/raw/icon-d2/**/*regular-lat-lon*.grib2" \
--output=.gribmagic-data/subgrid/icon-d2 \
--country=AT

For plotting the result on a map, install Magics:
# Linux
apt-get install --yes libmagplus3v5

# macOS
make magics-install
make magics-info
export MAGPLUS_HOME=/usr/local/opt/magics-4.9.3

# Install GribMagic with Magics bindings
pip install gribmagic[plotting]

and use the --plot option:
gribmagic smith bbox \
".gribmagic-data/raw/icon-d2/**/*regular-lat-lon*.grib2" \
--output=.gribmagic-data/subgrid \
--country=AT \
--plot

Run regridding tool
Acquire grid information files.
gribmagic install dwd-grid-information

Process regridding on a single file. The default is to assume ICON GLOBAL files and the
regridding will take place with 0.250° resolution.
gribmagic smith regrid \
".gribmagic-data/raw/icon/**/*icon_global_icosahedral*.grib2" \
--output=.gribmagic-data/regridded/icon-global

When aiming to regrid with 0.125° resolution, please specify the --resolution=0.125 argument, like:
gribmagic smith regrid \
".gribmagic-data/raw/icon/**/*icon_global_icosahedral*.grib2" \
--output=.gribmagic-data/regridded/icon-global \
--resolution=0.125

Run program in Docker
To use GribMagic in a Docker container, you have to build the Docker image like
docker build --tag gribmagic .

and then invoke it like
docker run -it --volume=$PWD/.gribmagic-data:/var/spool/gribmagic gribmagic:latest gribmagic unity acquire --model=dwd-icon-eu --timestamp=2021-10-03T00:00:00Z

Development
Acquire source code
git clone https://github.com/earthobservations/gribmagic
cd gribmagic

Run software tests
# Run all tests.
make test

# All tests, with code coverage report.
make test-coverage

# Run specific tests, with coverage report.
.venv/bin/pytest -vvv --cov-report=term-missing --cov=gribmagic.smith.regrid -k test_regrid

# Run tests in parallel.
make test-parallel
make test-coverage-parallel

Content attributions
The copyright of data sets, images and pictograms are held by their respective owners, unless otherwise noted.
Banner image

2 metre temperature from ECMWF, rendered using Magics.

Source: MARS labelling or ensemble forecast data (2013-03-25)
URL: http://download.ecmwf.int/test-data/magics/2m_temperature.grib