datacatalogtagtemplateprocessor 0.2.0
Datacatalog Tag Template Processor
A package to manage Google Cloud Data Catalog Tag Template scripts.
Disclaimer: This is not an officially supported Google product.
Table of Contents
Executing in Cloud Shell
1. Environment setup
1.1. Python + virtualenv
1.1.1. Install Python 3.6+
1.1.2. Get the source code
1.1.3. Create and activate an isolated Python environment
1.1.4. Install the package
1.2. Docker
1.3. Auth credentials
1.3.1. Create a service account and grant it below roles
1.3.2. Download a JSON key and save it as
1.3.3. Set the environment variables
2. Load Templates from CSV file
2.1. Create a CSV file representing the Templates to be created
2.2. Run the ddatacatalog-tag-template-processor script - Create the Tag Templates
2.3. Run the datacatalog-tag-template-processor script - Delete the Tag Templates
Executing in Cloud Shell
# Set your SERVICE ACCOUNT, for instructions go to 1.3. Auth credentials
# This name is just a suggestion, feel free to name it following your naming conventions
export GOOGLE_APPLICATION_CREDENTIALS=~/datacatalog-tag-template-processor-sa.json
# Install datacatalog-tag-template-processor
pip3 install datacatalog-tag-template-processor --user
# Add to your PATH
export PATH=~/.local/bin:$PATH
# Look for available commands
datacatalog-tag-template-processor --help
1. Environment setup
1.1. Python + virtualenv
Using virtualenv is optional, but strongly recommended unless you use Docker.
1.1.1. Install Python 3.6+
1.1.2. Get the source code
git clone https://github.com/mesmacosta/datacatalog-tag-template-processor
cd ./datacatalog-tag-template-processor
All paths starting with ./ in the next steps are relative to the datacatalog-tag-template-processor
folder.
1.1.3. Create and activate an isolated Python environment
pip install --upgrade virtualenv
python3 -m virtualenv --python python3 env
source ./env/bin/activate
1.1.4. Install the package
pip install --upgrade .
1.2. Docker
Docker may be used as an alternative to run the script. In this case, please disregard the
Virtualenv setup instructions.
1.3. Auth credentials
1.3.1. Create a service account and grant it below roles
Data Catalog Admin
1.3.2. Download a JSON key and save it as
This name is just a suggestion, feel free to name it following your naming conventions
./credentials/datacatalog-tag-template-processor-sa.json
1.3.3. Set the environment variables
This step may be skipped if you're using Docker.
export GOOGLE_APPLICATION_CREDENTIALS=~/credentials/datacatalog-tag-template-processor-sa.json
2. Load Templates from CSV file
2.1. Create a CSV file representing the Templates to be created
Templates are composed of as many lines as required to represent all of their fields. The columns are
described as follows:
Column
Description
Mandatory
template_name
Resource name of the Tag Template for the Tag.
Y
display_name
Resource name of the Tag Template for the Tag.
Y
field_id
Id of the Tag Template field.
Y
field_display_name
Display name of the Tag Template field.
Y
field_type
Type of the Tag Template field.
Y
enum_values
Values for the Enum field.
N
2.2. Run the ddatacatalog-tag-template-processor script - Create the Tag Templates
Python + virtualenv
datacatalog-tag-template-processor tag-templates create --csv-file CSV_FILE_PATH
2.3. Run the datacatalog-tag-template-processor script - Delete the Tag Templates
Python + virtualenv
datacatalog-tag-template-processor tag-templates delete --csv-file CSV_FILE_PATH
TIPS
sample-input/create-tag-templates for reference;
History
0.1.0 (2020-04-14)
First release on PyPI.
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
There are no reviews.