raster-loader 0.7.1

Description:

rasterloader 0.7.1

raster-loader




Python library for loading GIS raster data to standard cloud-based data warehouses that
don't natively support raster data.
Raster Loader is currently tested on Python 3.8, 3.9, 3.10, and 3.11.
Documentation
The Raster Loader documentation is available at raster-loader.readthedocs.io.
Install
pip install -U raster-loader

pip install -U raster-loader"[bigquery]"
pip install -U raster-loader"[snowflake]"

Installing from source
git clone https://github.com/cartodb/raster-loader
cd raster-loader
pip install .

Usage
There are two ways you can use Raster Loader:

Using the CLI by running carto in your terminal
Using Raster Loader as a Python library (import raster_loader)

CLI
After installing Raster Loader, you can run the CLI by typing carto in your terminal.
Currently, Raster Loader supports uploading raster data to BigQuery.
Accessing BigQuery with Raster Loader requires the
GOOGLE_APPLICATION_CREDENTIALS environment variable to be set to the path of a JSON
file containing your BigQuery credentials. See the
GCP documentation
for more information.
Two commands are available:
Uploading to BigQuery
carto bigquery upload loads raster data from a local file to a BigQuery table.
At a minimum, the carto bigquery upload command requires a file_path to a local
raster file that can be read by GDAL and processed with rasterio. It also requires
the project (the GCP project name)
and dataset (the BigQuery dataset name)
parameters. There are also additional parameters, such as table (BigQuery table
name) and overwrite (to
overwrite existing data).
For example:
carto bigquery upload \
--file_path /path/to/my/raster/file.tif \
--project my-gcp-project \
--dataset my-bigquery-dataset \
--table my-bigquery-table \
--overwrite

This command uploads the TIFF file from /path/to/my/raster/file.tif to a BigQuery
project named my-gcp-project, a dataset named my-bigquery-dataset, and a table
named my-bigquery-table. If the table already contains data, this data will be
overwritten because the --overwrite flag is set.
Inspecting a raster file on BigQuery
Use the carto bigquery describe command to retrieve information about a raster file
stored in a BigQuery table.
At a minimum, this command requires a
GCP project name,
a BigQuery dataset name, and a
BigQuery table name.
For example:
carto bigquery describe \
--project my-gcp-project \
--dataset my-bigquery-dataset \
--table my-bigquery-table

Using Raster Loader as a Python library
After installing Raster Loader, you can import the package into your Python project. For
example:
from raster_loader import rasterio_to_bigquery, bigquery_to_records

Currently, Raster Loader supports uploading raster data to BigQuery. Accessing BigQuery with Raster Loader requires the
GOOGLE_APPLICATION_CREDENTIALS environment variable to be set to the path of a JSON
file containing your BigQuery credentials. See the
GCP documentation
for more information.
You can use Raster Loader to upload a local raster file to an existing
BigQuery table using the rasterio_to_bigquery() function:
rasterio_to_bigquery(
file_path = 'path/to/raster.tif',
project_id = 'my-project',
dataset_id = 'my_dataset',
table_id = 'my_table',
)

This function returns True if the upload was successful.
You can also access and inspect a raster file from a BigQuery table using the
bigquery_to_records() function:
records_df = bigquery_to_records(
project_id = 'my-project',
dataset_id = 'my_dataset',
table_id = 'my_table',
)

This function returns a DataFrame with some samples from the raster table on BigQuery
(10 rows by default).
Development
See CONTRIBUTING.md for information on how to contribute to this
project.
ROADMAP.md contains a list of features and improvements planned for future
versions of Raster Loader.
Releasing
1. Create and merge a release PR updating the CHANGELOG

Branch: release/X.Y.Z
Title: Release vX.Y.Z
Description: CHANGELOG release notes

Example:
## [0.7.0] - 2024-06-02

### Added
- Support raster overviews (#140)

### Enhancements
- increase chunk-size to 10000 (#142)

### Bug Fixes
- fix: make the gdalwarp examples consistent (#143)

2. Create and push a tag vX.Y.Z
This will trigger an automatic workflow that will publish the package at https://pypi.org/project/raster-loader.
3. Create the GitHub release
Go to the tags page (https://github.com/CartoDB/raster-loader/tags), select the release tag and click on "Create a new release"

Title: vX.Y.Z
Description: CHANGELOG release notes

Example:
### Added
- Support raster overviews (#140)

### Enhancements
- increase chunk-size to 10000 (#142)

### Bug Fixes
- fix: make the gdalwarp examples consistent (#143)