Description:
pipeshell 0.3.0
pipeshell
pipeshell is a simple pipeline execution library allowing you to define and run pipelines of shell commands with dependencies, retries, and other features.
Features
Define steps with shell commands
Set dependencies between steps
Retry steps on failure
Run steps in the background
Capture and display step outputs
Set environment variables for steps
Installation
To install pipeshell, you can use pip:
pip install pipeshell
Usage
Here's a quick example of how to define and run a pipeline with pipeshell:
from pipeshell import pipeline, Step
# Define the steps
prepare_environment = Step(
name="prepare_environment",
command="echo 'Preparing environment...'"
)
fetch_data = Step(
name="fetch_data",
command="curl -o data.txt https://example.com/data.txt",
depends_on=[prepare_environment]
)
process_data = Step(
name="process_data",
command="python process_data.py data.txt processed_data.txt",
depends_on=[fetch_data]
)
analyze_data = Step(
name="analyze_data",
command="python analyze_data.py processed_data.txt results.txt",
depends_on=[process_data]
)
cleanup = Step(
name="cleanup",
command="rm data.txt processed_data.txt",
depends_on=[analyze_data],
allow_failure=True # Allow cleanup to fail without affecting pipeline
)
pipeline("example_pipeline", prepare_environment, fetch_data, process_data, analyze_data, cleanup)
Contributing
Contributions are welcome! Please open an issue or submit a pull request on GitHub.
License
This project is licensed under the MIT License.
License
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
Customer Reviews
There are no reviews.
