Description:
dagsesh 0.1.0
Dagsesh: Airflow DAG Session Manager
Overview
Prerequisites
Getting Started
(macOS Users) Upgrading GNU Make
Creating the Local Environment
Local Environment Maintenance
Help
Running the Test Harness
Using dagsesh Plugin in your Project's Test Suite
FAQs
Overview
An Apache Airflow session context manager that overrides the default AIRFLOW_HOME path with a random, ephemeral alternate.
Why is this useful? As per the Airflow configuration docs:
The first time you run Airflow, it will create a file called airflow.cfg in your $AIRFLOW_HOME directory (~/airflow by default).
Dagsesh delays the creation of AIRFLOW_HOME using a lazy-loading facility whilst injecting it with a random alternate. Great if you want to create a pristine Airflow environment for repeatable testing.
Dagsesh can be used as a convenient pytest plugin to prime an Airflow environment for testing.
top
Prerequisites
GNU make
Python 3 Interpreter. We recommend installing pyenv
Docker
top
Getting Started
Makester is used as the Integrated Developer Platform.
(macOS Users) Upgrading GNU Make
Follow these notes to get GNU make.
Creating the Local Environment
Get the code and change into the top level git project directory:
git clone git@github.com:loum/dagsesh.git && cd dagsesh
NOTE: Run all commands from the top-level directory of the git repository.
For first-time setup, get the Makester project:
git submodule update --init
Initialise the environment:
make init-dev
Local Environment Maintenance
Keep Makester project up-to-date with:
git submodule update --remote --merge
top
Help
There should be a make target to get most things done. Check the help for more information:
make help
top
Running the Test Harness
We use pytest. To run the tests:
make tests
top
Using dagsesh Plugin in your Project's Test Suite
Add the dagsesh package as a dependency to your project's development environment so that the plugin is installed and visible in your PYTHONPATH. dagsesh takes care of the distribution's entry point so that pytest automatically finds the plugin module. Nothing else needs to be done.
NOTE: See Making your plugin installable by others for more information.
top
License
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
Files:
Customer Reviews
There are no reviews.
